`core.inference.unified_memory`#

Module Contents#

Enum to distinguish between unified memory (UVM) compilation states.

`_compile_timeout`	Context manager to timeout compilation.
`compile_allocator`	Attempt to compile UVM allocator.
`create_unified_mempool`	Create a unified memory mempool using CUDA managed memory.

class core.inference.unified_memory.CompilationState(*args, **kwds)#

Bases: enum.Enum

Enum to distinguish between unified memory (UVM) compilation states.

Initialization

exception core.inference.unified_memory.UnifiedMemoryUnsupportedError#

Bases: Exception

Unified memory is not supported on this system.

Initialization

Initialize self. See help(type(self)) for accurate signature.

exception core.inference.unified_memory.UnifiedMemoryCompileTimeoutError#

Unified memory compilation timed out.

Initialization

Initialize self. See help(type(self)) for accurate signature.

core.inference.unified_memory._compile_timeout(timeout_s: int)#

Context manager to timeout compilation.

core.inference.unified_memory.compile_allocator()#: Attempt to compile UVM allocator.

core.inference.unified_memory.create_unified_mempool() → torch.cuda.memory.MemPool#

Create a unified memory mempool using CUDA managed memory.