Model Pricing

Use the built-in pricing component to configure model pricing catalogs for cost estimates on managed or manually observed LLM responses. Manual lifecycle end events must supply a response codec or normalized response annotation with model and token usage. Relay leaves cost absent when no configured source matches the provider, model, and token usage.

Configuration

Add the pricing component to plugins.toml. The following configuration loads a JSON catalog and defines an inline catalog. Relay searches sources in the listed order, so place overrides before their fallbacks:

1
version = 1
2
3
[[components]]
4
kind = "pricing"
5
enabled = true
6
7
[[components.config.sources]]
8
type = "file"
9
path = "./pricing.json"
10
11
[[components.config.sources]]
12
type = "inline"
13
14
[components.config.sources.catalog]
15
version = 1
16
17
[[components.config.sources.catalog.entries]]
18
provider = "openai"
19
model_id = "gpt-4o-mini"
20
aliases = ["openai/openai/gpt-4o-mini"]
21
currency = "USD"
22
unit = "per_token"
23
pricing_as_of = "2026-06-06"
24
pricing_source = "internal-pricing-snapshot"
25
26
[components.config.sources.catalog.entries.rates]
27
input_per_million = 0.15
28
output_per_million = 0.60
29
cache_read_per_million = 0.075
30
31
[components.config.sources.catalog.entries.prompt_cache]
32
read_accounting = "included_in_prompt_tokens"

Use type = "file" with a JSON catalog path or type = "inline" with the catalog in plugins.toml. Relay checks sources and catalog entries in listed order, and it uses the first entry that matches the provider and model.

Relay publishes a numeric total only when every nonzero usage category has a configured rate. If a response reports nonzero cache-read or cache-write usage and the matching catalog entry omits cache_read_per_million or cache_write_per_million, Relay keeps the priced components but omits the aggregate total. Set a rate to 0.0 when a category is intentionally free.

When Relay merges explicit-or-user and system configuration files, it prepends higher-priority sources instead of replacing lower-priority sources. The effective order is system, then explicit-or-user. This lets an enterprise catalog override narrower catalogs while preserving those catalogs as fallbacks.

Manage Catalog Sources with the CLI

Run the following commands to validate a catalog and add it to user configuration:

$
nemo-relay model-pricing validate <path/to/pricing.json>
$
nemo-relay model-pricing init --user
$
nemo-relay model-pricing add-source <path/to/pricing.json> --user

Replace <path/to/pricing.json> with the path to your pricing catalog before running either command. Use --user for the user configuration file or --global for system configuration at /etc/nemo-relay/plugins.toml on Unix or %ProgramData%\nemo-relay\plugins.toml on Windows. add-source places a new source ahead of existing sources by default. Use --append to keep it as a fallback.

Verify the Active Configuration

Run the following command to resolve a model against the configured catalogs:

$
nemo-relay model-pricing resolve gpt-4o-mini \
>
  --provider openai \
>
  --prompt-tokens 1000 \
>
  --completion-tokens 500

Use nemo-relay doctor to validate configured model pricing sources with the rest of the gateway configuration. Refer to Add Model Pricing for Cost Estimates for the catalog schema, source precedence, and complete CLI workflow.