Load, Run, and Destroy an Audio Effect#

Load an Audio Effect#

Loading an effect involves validating the parameters that were set for the effect and loading the specified model into GPU memory.

To load an audio effect, set the parameters for the effect (as described in Set the Parameters of an Audio Effect) and call NvAFX_Load() with the effect handle:

NvAFX_Status err = NvAFX_Load(handle);

If the function succeeds, NVAFX_STATUS_SUCCESS is returned. Otherwise, an error code is returned.

Run an Audio Effect#

After the effect is loaded, it can be applied to input audio by calling the NvAFX_Run() function. When the effect is run, the contents of the input memory buffer are read, the audio effect is applied, and the output is written to the output memory buffer.

Note

Input/output memory buffers are in CPU memory. Copying to or from GPU memory is handled internally by the SDK.

To run an audio effect, call the NvAFX_Run() function with the following parameters:

  • Previously created effect handle.

  • The input memory buffer.

    For the AEC effect, specify two channels, where the first channel is the batched near-end audio, and the second channel is the batched far-end audio.

  • The output memory buffer.

    For the Super Resolution effect, the size of input and output memory buffer will differ and should be queried by the user using NVAFX_PARAM_NUM_SAMPLES_PER_OUTPUT_FRAME and NVAFX_PARAM_NUM_SAMPLES_PER_INPUT_FRAME.

  • The number of samples per frame per stream of input data.

  • The number of channels in input audio.

For more information, refer to Get the Parameters of an Audio Effect.

The following example runs an audio effect:

NvAFX_Status err = NvAFX_Run(handle, input, output, num_samples, num_channels);

Run Multiple Audio Effects in a Chain#

The following example demonstrates running two effects in a chain:

NvAFX_Status err;
err = NvAFX_CreateChainedEffect(NVAFX_CHAINED_EFFECT_DENOISER_16k_SUPERRES_16k_TO_48k, &effect);

// Set effect parameters & load effect
NvAFX_Status err = NvAFX_Run(chained_handle, input_audio, intermediate_output, num_samples, num_channels);

Note

Running effects in a chain might impact the performance and latency of the audio pipeline.

Run an Audio Effect on Delayed Audio Streams on Linux#

The Linux SDK supports cases where some streams do not arrive at the expected time, such as when the audio is being processed in real-time. These streams are referred to as delayed streams. To support handling these streams, the SDK allows applications to specify a list that indicates whether the corresponding stream is currently active or delayed/inactive.

The list can be set by calling NvAFX_SetBoolList() with the following function parameters:

  • Previously created effect handle.

  • The selector string NVAFX_PARAM_ACTIVE_STREAMS for the parameter that will be set.

  • An array of type NVAFX_BOOL in which each element represents the status of the corresponding audio stream, with NVAFX_TRUE indicating an active stream and NVAFX_FALSE indicating an inactive stream.

  • The length of the preceding array (equal to the number of audio streams).

Note

This functionality is not currently supported by Speaker Focus or Studio Voice.

For delayed audio streams, the effect can be initially applied on all delayed audio streams by setting them as active and setting the on-time audio streams as inactive. This should be followed by one or more NvAFX_Run() calls to apply the effect on the delayed audio streams. After the delayed audio streams are processed, the on-time audio streams are set to active, and NvAFX_Run() is executed once to apply the effect.

The following example demonstrates the procedure for processing four streams:

  1. Consider an effect that accepts 10ms audio inputs.

  2. Audio streams 1 and 3 are delayed by 10ms each and arrive with 20ms worth of data.

  3. Audio streams 2 and 4 are on time and arrive with 10ms of data.

  4. Streams can be processed in either of the following ways:

    • Option 1: Process the extra 10ms only in the delayed streams and then process on-time 10ms data for all streams. Initially, by using NvAFX_SetBoolList, streams 1 and 3 are set as active, and 2 and 4 are set as inactive:

      1. An NvAFX_Run call is executed where 10ms of data from streams 1 and 3 is populated in the input while the rest of the input is set to 0. This step processes the extra 10ms of data in streams 1 and 3.

      2. A second NvAFX_SetBoolList call is executed to set all streams (1, 2, 3, and 4) as active.

      3. An NvAFX_Run call is executed with the real-time 10ms data from all four streams.

    • Option 2: Process 10ms in all streams and then process the extra 10ms data only in delayed streams:

      1. Process 10ms of data from all streams (stale data from stream 1 and 3 and new data from stream 2 and 4) by calling NvAFX_Run.

      2. Set streams 1 and 3 to active and 2 and 4 to inactive by calling NvAFX_SetBoolList.

      3. Process the extra 10ms from stream 1 and 3 by calling NvAFX_Run.

The following example runs an audio effect after setting some of the audio streams as inactive:

NvAFX_Status err = NvAFX_SetBoolList(handle, NVAFX_PARAM_ACTIVE_STREAMS, stream_active_list, num_streams);

NvAFX_Status err = NvAFX_Run(handle, input, output, num_samples, num_channels);

The internal state of each stream is updated during each NvAFX_Run call only for active streams. Setting a stream to inactive disables updating this state. If required, this state can also be reset by using NvAFX_Reset.

Destroy an Audio Effect#

When an audio effect is no longer required, it should be destroyed to free the resources and memory used by the effect.

To destroy an audio effect, call NvAFX_DestroyEffect() and specify the handle to the effect to be destroyed:

NvAFX_Status err = NvAFX_DestroyEffect(handle);