ReferenceAPIsRust Library Referencenemo-relay-adaptive

Module response_cache

View as Markdown

Generated from cargo doc --no-deps -p nemo-relay -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

Opt-in LLM response cache (exact-match). Opt-in LLM response cache (exact-match): a feature of the adaptive plugin, configured through crate::config::AdaptiveConfig::response_cache.

[intercept] holds the execution intercepts and storage rules, [key] the cache-key derivation, [store] the backends, [replay] the streaming replay, and [mark] the observability surface.

Re-exports

  • pub use crate::config::ResponseCacheConfig;
  • pub use crate::response_cache::config::BackendConfig;
  • pub use crate::response_cache::config::KEY_STRATEGY_EXACT_REQUEST;

Modules

  • config: Backend selection for the adaptive plugin’s response_cache feature.

Constants