Model Pricing

Use the built-in pricing component to configure model pricing catalogs for cost estimates on managed LLM responses. Relay leaves cost absent when no configured source matches the provider, model, and token usage.

Configuration

Add the pricing component to plugins.toml. The following configuration loads a JSON catalog and defines an inline catalog. Relay searches sources in the listed order, so place overrides before their fallbacks:

1
version = 1
2
3
[[components]]
4
kind = "pricing"
5
enabled = true
6
7
[[components.config.sources]]
8
type = "file"
9
path = "./pricing.json"
10
11
[[components.config.sources]]
12
type = "inline"
13
14
[components.config.sources.catalog]
15
version = 1
16
17
[[components.config.sources.catalog.entries]]
18
provider = "openai"
19
model_id = "gpt-4o-mini"
20
aliases = ["openai/openai/gpt-4o-mini"]
21
currency = "USD"
22
unit = "per_token"
23
pricing_as_of = "2026-06-06"
24
pricing_source = "internal-pricing-snapshot"
25
26
[components.config.sources.catalog.entries.rates]
27
input_per_million = 0.15
28
output_per_million = 0.60
29
cache_read_per_million = 0.075
30
31
[components.config.sources.catalog.entries.prompt_cache]
32
read_accounting = "included_in_prompt_tokens"

Use type = "file" with a JSON catalog path or type = "inline" with the catalog in plugins.toml. Relay checks sources and catalog entries in listed order, and it uses the first entry that matches the provider and model.

When Relay merges explicit-or-user, project, and system configuration files, it prepends higher-priority sources instead of replacing lower-priority sources. The effective order is system, project, then explicit-or-user. This lets an enterprise catalog override narrower catalogs while preserving those catalogs as fallbacks.

Manage Catalog Sources with the CLI

Run the following commands to validate a catalog and add it to project configuration:

$
nemo-relay model-pricing validate /path/to/pricing.json
$
nemo-relay model-pricing init --project
$
nemo-relay model-pricing add-source /path/to/pricing.json --project

Use --user for the user configuration file or --global for /etc/nemo-relay/plugins.toml. add-source places a new source ahead of existing sources by default. Use --append to keep it as a fallback.

Verify the Active Configuration

Run the following command to resolve a model against the configured catalogs:

$
nemo-relay model-pricing resolve gpt-4o-mini \
>
  --provider openai \
>
  --prompt-tokens 1000 \
>
  --completion-tokens 500

Use nemo-relay doctor to validate configured model pricing sources with the rest of the gateway configuration. Refer to Add Model Pricing for Cost Estimates for the catalog schema, source precedence, and complete CLI workflow.