ReferenceAPIsRust Library Referencenemo-relay-ffiapi

Function nemo_relay_llm_stream_call_execute

View as Markdown

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

#[unsafe(no_mangle)]pub unsafe extern "C" fn nemo_relay_llm_stream_call_execute(
    name: *const c_char,
    native_json: *const c_char,
    func: NemoRelayLlmExecCb,
    func_user_data: *mut c_void,
    func_free: NemoRelayFreeFn,
    collector: Option<NemoRelayCollectorCb>,
    finalizer: Option<NemoRelayFinalizerCb>,
    parent: *const FfiScopeHandle,
    attributes: u32,
    data_json: *const c_char,
    metadata_json: *const c_char,
    model_name: *const c_char,
    codec_decode: NemoRelayCodecDecodeFn,
    codec_encode: NemoRelayCodecEncodeFn,
    codec_user_data: *mut c_void,
    codec_free_fn: NemoRelayFreeFn,
    response_codec: *const FfiCodecHandle,
    out: *mut *mut FfiStream,
) -> NemoRelayStatus

Execute a streaming LLM call end-to-end. Conditional-execution guardrails run first on the raw request. Returns a stream handle that can be polled with nemo_relay_stream_next. Blocks until the stream is set up.

Parameters

  • name: Null-terminated LLM provider name.
  • native_json: The request payload as a JSON C string representing an LlmRequest ({"headers": {...}, "content": {...}}).
  • func: C callback that performs the actual LLM call.
  • func_user_data: Opaque pointer passed to func.
  • func_free: Optional destructor for func_user_data.
  • collector: Callback invoked with each intercepted chunk as a JSON string. May be null, in which case chunks are not collected.
  • finalizer: Callback invoked once when the stream is exhausted to produce the aggregated response as a JSON C string. May be null, in which case the finalizer returns Json::Null.
  • parent: Optional parent scope handle, or null.
  • attributes: Bitfield of LLM attributes.
  • data_json: Optional JSON data, or null.
  • metadata_json: Optional JSON metadata, or null.
  • model_name: Optional LLM model identifier, or null.
  • out: On success, receives a heap-allocated FfiStream.

Safety

name, native_json, and out must be valid, non-null pointers. collector and finalizer may be null.