Memory leak caused by inability to notify Julia GC about memory allocated by C++

[archit120]

I was tracking the cause behind a memory leak related to a CxxWrap based project. https://stackoverflow.com/questions/64978891/how-do-i-fix-a-julia-opencv-cxx-memory-leak-in-image-capturing

It seems that Julia's garbage collector does not realize the actual size of objects allocated by C++ and this can cause a memory leak like scenario because the garbage collector isn't destroying objects whose references have already been lost. The underlying issue is that to Julia's garbage collector all C++ allocated objects appear as the size of a pointer. A manual GC run is able to free lost references correctly so the only issue is of having the GC run automatically when needed.

The solution probably involves some way of making Julia GC aware of the real size of objects being allocated by C++ so that the GC can run automatically at appropriate times.

[fingolfin]

I think the title of this issue is not quite correct; there is no leak here, just a failure to trigger GC automatically, which means memory is collected later than desired or even "too late" because some C++ allocation in the meantime fail.

In general it is a difficult problem to handle external memory pressure in a garbage collector. One trick one can use is to replace allocation operations (e.g. in C++ code, use custom new/delete operators). The replacements can either go through the Julia allocator; or they can call the regular allocator, but catch any allocation errors, checking if they are of the "out of memory" kind; if so, run a GC, and try again (and if it fails again, then throw for real).

It's not clear to me whether either could be done in a fully generic fashion, though, esp. since the linked C++ code might also use alternate new/delete implementations...

[archit120]

Yeah, you're right about the naming. Even though my problem could be mostly resolved with the help of the alternate allocation functions, the checking if "out of memory" method seems somewhat inefficient though because Julia would be sitting on a large pile of unused memory. Another workable fix would be if there was some way to inform Julia about the memory pressure without having to allocate memory through Julia.

[PallHaraldsson]

There was some way, a PR in Julia merged some time ago (then reverted), to feel the memory pressure of other processes (same as done in JavaScript VM), to make independet GCs of different processes to "cooperate", so there is a way to feel even fully external memory pressure. This seems simpler. I think, but I'm not sure, that you could ask libc somehow how much memory has been allocated by your own process (occasionally, i.e. not do it for each managed GC-tracked allocation), not just rely on Julia's collected stats, on those it does bypassing malloc.

If there's no way to ask libc, then there must be a way to look up memory use of you (or any) process, by asking to OS?

I feel like this is Julia's job, and this issue should be moved there, or another one made there to mirror this one.