Struct OpenAI Cache Plugin | NVIDIA NeMo Relay

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-ffi.

1 pub struct OpenAICachePlugin;

OpenAI-specific provider plugin for deterministic JSON serialization.

Ensures that semantically identical request prefixes produce byte-identical JSON output, maximizing OpenAI’s automatic prefix cache hit rate. Stateless - no constructor arguments needed.

Usage

1 let plugin = OpenAICachePlugin;
2 let output = plugin.translate(&input)?;

Trait Implementations

`impl ProviderPlugin for OpenAICachePlugin`

impl ProviderPlugin for OpenAICachePlugin

`plugin_id`

fn plugin_id(&self) -> &str

`plugin_name`

fn plugin_name(&self) -> &str

`translate`

fn translate(&self, input: &PluginInput<'_>) -> Result<PluginOutput>

`capabilities`

fn capabilities(&self) -> BackendCapabilities

1	let plugin = OpenAICachePlugin;
2	let output = plugin.translate(&input)?;