API Reference¶

VideoFrameGeneration¶

class nvvfx.VideoFrameGeneration(mode: Mode = Mode.MEDIUM, automatic_shot_change_detection_enabled: bool = True, device: int = 0, image_encoding: ImageEncoding = ImageEncoding.RGB8)¶

Bases: Effect

AI-powered video frame generation effect.

Accepts DLPack-compatible CUDA input arrays. The output is a raw DLPack capsule that references SDK-owned memory; framework compatibility varies by consumer.

Generates intermediate frames from a previous/current frame pair. Set input dimensions and call load() before run(). Use run() for uniform frame-rate multiplication or run_at_timestep() for an explicit temporal position.

Parameters:

mode – Generation quality mode (default: MEDIUM)
automatic_shot_change_detection_enabled – Enable automatic shot-change detection (default: True)
device – CUDA device index (default: 0)
image_encoding – Image encoding mode (default: RGB8). RGB8 accepts two (3, H, W) float32 CUDA inputs and returns (3, H, W) float32. RGB10A2 accepts two (H, W) uint32 CUDA inputs with packed R10G10B10A2 pixels and returns (H, W) uint32 packed pixels.

property input_width: int | None¶: Input frame width in pixels.

property input_height: int | None¶: Input frame height in pixels.

property frame_multiplier: int¶: Uniform frame-rate multiplier.

property mode: Mode¶: Generation quality mode.

property automatic_shot_change_detection_enabled: bool¶: Whether automatic shot-change detection is enabled.

property image_encoding: ImageEncoding¶: Input and output image encoding mode.

load() → None¶: Load the effect for the configured input dimensions.

run(previous_frame: Any, current_frame: Any, *, frame_index: int = 1, shot_change: bool = False, non_blocking: bool = False, stream_ptr: int = 0) → VideoFrameGenerationOutput¶

Generate a uniformly spaced intermediate frame.

Parameters:

previous_frame – Previous CUDA frame. RGB8 uses (3, H, W) float32; RGB10A2 uses (H, W) packed uint32.
current_frame – Current CUDA frame with the same shape and encoding as previous_frame.
frame_index – One-based intermediate frame index.
shot_change – Mark this input pair as a shot change.
non_blocking – If True, return immediately (async). Default False.
stream_ptr – CUDA stream pointer as int (0 for default stream).

Returns:

VideoFrameGenerationOutput containing a DLPack output capsule. Ensure producer work has completed before importing it, then clone/copy it immediately and synchronize that copy before the next run(), output-size or encoding change, or close(). For RGB8, avoid non_blocking=True with a nonzero stream_ptr.

run_at_timestep(previous_frame: Any, current_frame: Any, timestep: float, *, shot_change: bool = False, non_blocking: bool = False, stream_ptr: int = 0) → VideoFrameGenerationOutput¶

Generate an intermediate frame at an explicit temporal position.

Parameters:

previous_frame – Previous CUDA frame. RGB8 uses (3, H, W) float32; RGB10A2 uses (H, W) packed uint32.
current_frame – Current CUDA frame with the same shape and encoding as previous_frame.
timestep – Temporal position in the open interval (0.0, 1.0), where 0.0 is the previous frame and 1.0 is the current frame.
shot_change – Mark this input pair as a shot change.
non_blocking – If True, return immediately (async). Default False.
stream_ptr – CUDA stream pointer as int (0 for default stream).

Returns:

VideoFrameGenerationOutput containing a DLPack output capsule that references SDK-owned memory. Ensure producer work has completed before importing it, then clone/copy it immediately and synchronize that copy before the next run(), output-size or encoding change, or close(). For RGB8, avoid non_blocking=True with a nonzero stream_ptr.

Mode¶

class nvvfx.VideoFrameGeneration.Mode(*values)

Bases: IntEnum

Quality modes for Video Frame Generation.

LOW = 0

MEDIUM = 1

HIGH = 2

VideoSuperRes¶

class nvvfx.VideoSuperRes(quality: QualityLevel = QualityLevel.HIGH, strength: float = 1.0, device: int = 0, image_encoding: ImageEncoding = ImageEncoding.RGB8)¶

Bases: Effect

AI-powered video super resolution effect.

Accepts DLPack-compatible CUDA input arrays. The output is a raw DLPack capsule that references SDK-owned memory; framework compatibility varies by consumer.

Set output dimensions via properties, then call load() before run(). Input dimensions are inferred automatically from the input array. Quality level, strength, and dimensions can be changed dynamically between calls.

Parameters:

quality – Quality level for enhancement (default: HIGH)
strength – Effect strength in [0.0, 1.0] (default: 1.0)
device – CUDA device index (default: 0)
image_encoding – Image encoding mode (default: RGB8). RGB8 accepts (3, H, W) float32 CUDA input in range [0, 1] and returns (3, output_height, output_width) float32. RGB10A2 accepts (H, W) uint32 CUDA input with packed R10G10B10A2 pixels and returns (output_height, output_width) uint32 packed pixels.

Example with PyTorch:

>>> import torch
>>> from nvvfx import VideoSuperRes
>>> input_frame = torch.rand(3, 540, 960, device="cuda")
>>> with VideoSuperRes() as sr:
...     sr.output_width = 1920
...     sr.output_height = 1080
...     sr.load()
...     output = torch.from_dlpack(sr.run(input_frame).image).clone()
>>> output.shape
torch.Size([3, 1080, 1920])

property output_width: int | None¶: Output width in pixels.

property output_height: int | None¶: Output height in pixels.

property quality: QualityLevel¶: Quality level for enhancement.

property strength: float¶: Strength in [0.0, 1.0].

property image_encoding: ImageEncoding¶: Input and output image encoding mode.

run(input_array: Any, *, non_blocking: bool = False, stream_ptr: int = 0) → VideoSuperResOutput¶

Apply super resolution to input array.

Parameters:

input_array – Input image array supporting DLPack protocol - RGB8 shape: (3, H, W), RGB channels first - RGB8 dtype/range: float32 in [0, 1] - RGB10A2 shape: (H, W) - RGB10A2 dtype/range: uint32 packed R10G10B10A2 - Device: CUDA
non_blocking – If True, return immediately (async). Default False.
stream_ptr – CUDA stream pointer as int (0 for default stream).

Returns:

VideoSuperResOutput containing the upscaled image as a DLPack output capsule. Ensure producer work has completed before importing it, then clone/copy it immediately and synchronize that copy before the next run(), output-size or encoding change, or close(). For RGB8, avoid non_blocking=True with a nonzero stream_ptr.

Raises:

NvVFXError – If load() was not called, output dimensions not set, or inference fails.
TypeError – If input doesn’t support DLPack.
ValueError – If input is not on CUDA device.

QualityLevel¶

class nvvfx.VideoSuperRes.QualityLevel(*values)

Bases: IntEnum

Quality levels for Video Super Resolution.

Selects the AI model and processing strategy. Modes are grouped by operation type; choose the group that matches your use case, then choose an intensity level within it (LOW → ULTRA trades speed for quality).

Values 5-7, 20 and 22 are reserved.

Bicubic mode (0): Bicubic is a mathematical interpolation method that estimates new pixels from neighboring pixels.

BICUBIC: Non-AI bicubic interpolation (fastest, baseline quality)

Standard upscaling modes (1-4): Upscaling modes optimized for typical compressed video. They combine compression-artifact reduction with upscaling.

LOW: AI upscaling, speed-optimized
MEDIUM: AI upscaling, balanced speed/quality
HIGH: AI upscaling, quality-favoring (default)
ULTRA: AI upscaling, maximum detail preservation

Denoise modes (8-11): Same-resolution processing. Removes noise and compression artifacts (macro-blocking, mosquito noise) while preserving resolution. Higher intensities remove more noise but may soften fine texture.

DENOISE_LOW: Light cleanup, maximum texture preservation
DENOISE_MEDIUM: Moderate noise/artifact removal
DENOISE_HIGH: Aggressive noise/artifact removal
DENOISE_ULTRA: Maximum denoising strength

Deblur modes (12-15): Same-resolution processing. Sharpens soft or blurry footage rather than targeting noise. Best for out-of-focus or motion-blurred sources.

DEBLUR_LOW: Light sharpening
DEBLUR_MEDIUM: Moderate sharpening
DEBLUR_HIGH: Aggressive sharpening
DEBLUR_ULTRA: Maximum sharpening strength

High-bitrate modes (16-19): Upscaling modes optimized for high-quality compressed sources, such as high-bitrate H.264/H.265, ProRes, or other lightly compressed video with few compression artifacts. Unlike standard modes, these skip artifact suppression to avoid degrading already-clean detail.

HIGHBITRATE_LOW: AI upscaling, speed-optimized (clean source)
HIGHBITRATE_MEDIUM: AI upscaling, balanced (clean source)
HIGHBITRATE_HIGH: AI upscaling, quality-favoring (clean source)
HIGHBITRATE_ULTRA: AI upscaling, maximum detail (clean source)

Streaming modes (21, 23): Upscaling modes optimized for streaming or webcam use cases. The input is typically lossless or has little to no compression artifacts, though camera noise may still be present. These modes reduce camera noise, deblur, sharpen, and upscale.

STREAMING_MEDIUM: Balanced streaming upscaling
STREAMING_ULTRA: Maximum-quality streaming upscaling

BICUBIC = 0

LOW = 1

MEDIUM = 2

HIGH = 3

ULTRA = 4

DENOISE_LOW = 8

DENOISE_MEDIUM = 9

DENOISE_HIGH = 10

DENOISE_ULTRA = 11

DEBLUR_LOW = 12

DEBLUR_MEDIUM = 13

DEBLUR_HIGH = 14

DEBLUR_ULTRA = 15

HIGHBITRATE_LOW = 16

HIGHBITRATE_MEDIUM = 17

HIGHBITRATE_HIGH = 18

HIGHBITRATE_ULTRA = 19

STREAMING_MEDIUM = 21

STREAMING_ULTRA = 23

ImageEncoding¶

class nvvfx.VideoSuperRes.ImageEncoding(*values)

Bases: IntEnum

Input and output image encoding for NVIDIA VFX effects.

RGB8 uses normalized float32 RGB input and float32 RGB output. RGB10A2 uses packed uint32 R10G10B10A2 input and output.

RGB8 = 0

RGB10A2 = 1

TrueHDR¶

class nvvfx.TrueHDR(*, contrast: int = 100, saturation: int = 100, middle_gray: int = 50, luminance: int = 650, debanding_off: int = 0, device: int = 0)¶

Bases: Effect

AI-powered SDR-to-HDR10 conversion effect with PQ-encoded output.

The output uses the Rec. 2020 (BT.2020) color space and the PQ (SMPTE ST 2084) transfer function. Pixels are packed as RGB10A2 (10 bits per RGB channel in a 32-bit word), with a configurable target peak luminance of 400 to 2000 nits.

Input is full-range Rec. 709/D65 RGB with gamma 2.2 encoding. The Python API accepts planar float32 RGB in [0, 1].

Accepts DLPack-compatible CUDA input arrays. The output is a raw DLPack capsule that references SDK-owned memory; framework compatibility varies by consumer.

Same-resolution: output dimensions are pinned to the input dimensions on each call. Input dimensions are inferred from the input array; just call run() and load() will run lazily. Optionally call load() explicitly to prewarm the NGX feature ahead of the first run() (dimensions are not required at load time).

Tunables (contrast, saturation, middle_gray, luminance, debanding_off) can be set at init or changed dynamically between calls; the SDK re-reads them per run().

Parameters:

contrast – Contrast tunable, [0..200] (default: 100)
saturation – Saturation tunable, [0..200] (default: 100)
middle_gray – Middle-grey reference, [10..100] (default: 50)
luminance – Target HDR display peak in nits, [400..2000] (default: 650)
debanding_off – Set to 1 to skip the DL-Debander pass (default: 0)
device – CUDA device index (default: 0)

Example with PyTorch:

>>> import torch
>>> from nvvfx import TrueHDR
>>> input_frame = torch.rand(3, 540, 960, device="cuda")
>>> with TrueHDR(luminance=1000) as thdr:
...     result = thdr.run(input_frame)
...     hdr10 = torch.from_dlpack(result.image).clone()
>>> hdr10.shape, hdr10.dtype
(torch.Size([540, 960]), torch.uint32)

property contrast: int¶: Contrast tunable, [0..200].

property saturation: int¶: Saturation tunable, [0..200].

property middle_gray: int¶: Middle-grey reference, [10..100].

property luminance: int¶: Target HDR display peak luminance in nits, [400..2000].

property debanding_off: int¶: Skip DL-Debander pass, {0, 1}.

run(input_array: Any, *, non_blocking: bool = False, stream_ptr: int = 0) → TrueHDROutput¶

Convert an SDR frame to HDR10.

Parameters:

input_array – Input image array supporting DLPack protocol - Shape: (3, H, W) - RGB channels first - Dtype: float32 - Device: CUDA - Range: [0, 1]
non_blocking – If True, return immediately (async). Default False.
stream_ptr – CUDA stream pointer as int (0 for default stream).

Returns:

TrueHDROutput containing an (H, W) uint32 DLPack output capsule of packed R10G10B10A2 words. Ensure producer work has completed before importing it, then clone/copy it immediately and synchronize that copy before the next run(), output-size or encoding change, or close(). The capsule does not implement consumer-stream handoff.

Raises:

NvVFXError – If the SDK call fails.
TypeError – If input doesn’t support DLPack.
ValueError – If input is not on CUDA device.

property input_width: int | None¶: Input/output width in pixels.

property input_height: int | None¶: Input/output height in pixels.

Effect¶

class nvvfx.effects.base.Effect(device: int = 0)

Bases: object

Base class for all NVVFX effects.

Provides a thin wrapper around the C++ _Effect bindings with state tracking and context manager support. Effect-specific classes inherit from this and implement load() and run().

is_loaded: Whether the effect has been loaded.

needs_reload: Whether load() must be called again due to config changes.

device: CUDA device index.

selector: Effect selector string.

property is_loaded: bool: Check if the effect is loaded.

property needs_reload: bool

property device: int: CUDA device index.

property selector: str: Effect selector string.

load() → None

Load the effect.

Must be called after setting all configuration parameters and before calling run(). Some parameters can be changed after load without requiring a reload.

Raises:: NvVFXError – If loading fails.

run(input_array: Any, **kwargs: Any) → Any

Run inference. Subclasses must implement.

Raises:: NotImplementedError – If not implemented by subclass.

close() → None: Release resources. Effect should not be used after this.