MemoryCleaner#
- public final class MemoryCleaner#
ColumnVectors may store data off heap, and because of complicated processing the life time of an individual vector can vary a lot.
Typically a java finalizer could be used for this but they can cause a number of performance issues related to gc, and in some cases may effectively leak resources if the heap is large and GC’s end up being delayed.
To address these issues the primary way to releasing the resources of a ColumnVector that is stored off of the java heap should be through reference counting. Because memory leaks are really bad for long lived daemons this is intended to be a backup.
When a ColumnVector first allocates off heap resources it should register itself with this along with a Cleaner instance. The Cleaner instance should have no direct links to the ColumnVector that would prevent the ColumnVector from being garbage collected. This will use WeakReferences internally to know when the resources have been leaked. A ColumnVector may keep a reference to the Cleaner instance and either update it as new resources are allocated or use it to release the resources it is holding. Once the ColumnVector’s reference count reaches 0 and the resources are released. At some point later the Cleaner itself will be released.
Method Summary
This is used to close all Rmm resources to check leaks.
Check if configured the shutdown hook which checks leaks at shutdown time.
De-register the default shutdown hook from Java default Runtime, then return the corresponding shutdown runnable.
Static Methods
- public static boolean configuredDefaultShutdownHook()#
Check if configured the shutdown hook which checks leaks at shutdown time.
- Returns:
true if configured, false otherwise.
- public static void cleanAllRmmBlockers()#
This is used to close all Rmm resources to check leaks.
Should invoke this before closing Rmm.
- public static Runnable removeDefaultShutdownHook()#
De-register the default shutdown hook from Java default Runtime, then return the corresponding shutdown runnable.
If you want to register the default shutdown runnable in a custom shutdown hook manager instead of Java default Runtime, should first remove it using this method and then add it
- Returns:
the default shutdown runnable
- public static void register(CompiledExpression expr, Cleaner cleaner)#
- public static void register(AstJitProgram program, Cleaner cleaner)#
- public static abstract class Cleaner#
API that can be used to clean up the resources for a vector, even if there was a leak.
Subclassed by ai.rapids.cudf.ColumnVector.OffHeapState, ai.rapids.cudf.CuFileResourceCleaner, ai.rapids.cudf.HostColumnVectorCore.OffHeapState, ai.rapids.cudf.MemoryBuffer.MemoryBufferCleaner
Field Summary
Constructor Summary
Method Summary
Clean up any resources not previously released.
Clean up any resources not previously released.
Check if the underlying memory has been cleaned up or not.
Return true if a leak is expected for this object else false.
Fields
- public final long id#
Constructors
- public Cleaner()#
Methods
- public final void addRef()#
- public final void delRef()#
- public final boolean clean(boolean logErrorIfNotClean)#
Clean up any resources not previously released.
- Parameters:
logErrorIfNotClean – if true we should log a leak unless it is expected.
- Returns:
true if resources were cleaned up else false.
- public final boolean isLeakExpected()#
Return true if a leak is expected for this object else false.
- public void noWarnLeakExpected()#
- public abstract boolean isClean()#
Check if the underlying memory has been cleaned up or not.
- Returns:
true this is clean else false.
Protected Methods
- protected abstract boolean cleanImpl(boolean logErrorIfNotClean)#
Clean up any resources not previously released.
- Parameters:
logErrorIfNotClean – if true and there are resources to clean up a leak has happened so log it.
- Returns:
true if resources were cleaned up else false.