nemo-relay-model-pricing

This page is for NeMo Relay, version dev. For other versions, use one of these documentation indexes: - Latest (v0.11.1) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt - v0.11.1: https://docs.nvidia.com/nemo/relay/v0.11.1/llms.txt - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt

Generated from cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

Data-driven LLM model pricing used to layer cost estimates onto usage.

Model pricing is deliberately separate from response normalization so adding providers, aliases, or cache-accounting rules does not require editing [AnnotatedLlmResponse].

Modules

• catalog: Catalog loading and source-aware resolution. Catalog source loading and resolution with caller-owned file access.
• plugin: Built-in component implementation. Built-in model pricing plugin component module.

Structs

• ModelPricing: Per-token model pricing expressed in USD per one million tokens.
• PricingCatalog: Collection of model pricing entries.
• PricingConfig: Runtime model pricing resolver configuration.
• PricingResolver: Ordered model pricing lookup chain.
• PromptCachePricing: Prompt-cache accounting rules for a model pricing entry.
• TokenPricingRates: Token rates expressed as USD per one million tokens.
• TokenRateTier: A model pricing tier for a range of input tokens, including cache tokens.

Enums

• CacheReadAccounting: How cache-read tokens relate to prompt token counts in provider usage.
• PricingCatalogError: Errors produced while parsing or validating a model pricing catalog.
• PricingSourceConfig: Declarative model pricing source supported by Relay configuration.
• PricingUnit: Billing unit represented by a model pricing entry.
• RateScheduleApplication: How a selected rate-schedule tier applies to billable usage.
• TokenRateSchedule: Data-driven token rate schedule for model pricing with request thresholds.

Traits

• PricingSource: Pluggable model pricing source interface.

Functions

• active_pricing_resolver: Returns the active process-wide model pricing resolver.
• attach_estimated_cost: Adds a model pricing estimate to a normalized response when cost is missing.
• attach_estimated_cost_for_provider: Adds a provider-aware model pricing estimate to a normalized response when cost is missing.
• estimate_cost: Estimates USD cost for a model/usage pair when model pricing is known.
• estimate_cost_for_provider: Estimates USD cost for a provider/model pair when model pricing is known.
• estimate_cost_with_catalog: Estimates USD cost using the provided catalog.
• estimate_cost_with_provider: Estimates USD cost using the provided catalog and provider/model pair.
• infer_model_provider: Infers a provider/route value for a decoded model.
• pricing_for_model: Returns known model pricing for a model ID.
• pricing_for_provider: Returns known model pricing for a provider/model pair.
• reset_active_pricing_resolver: Restores the active process-wide model pricing resolver to an empty resolver.
• set_active_pricing_resolver: Replaces the active process-wide model pricing resolver.