Adaptive Hints

Use Adaptive Hints when downstream model calls or provider adapters can safely receive guidance metadata from the adaptive runtime.

Adaptive hints register as LLM request intercepts. Lower numeric priority values run earlier in the intercept chain. The default priority is chosen relative to other middleware rather than as a standalone importance score.

plugins.toml Example

1
version = 1
2
3
[[components]]
4
kind = "adaptive"
5
enabled = true
6
7
[components.config]
8
version = 1
9
agent_id = "planner"
10
11
[components.config.state.backend]
12
kind = "in_memory"
13
14
[components.config.telemetry]
15
subscriber_name = "adaptive.telemetry"
16
learners = ["tool_parallelism"]
17
18
[components.config.adaptive_hints]
19
priority = 100
20
break_chain = false
21
inject_header = true
22
inject_body_path = "nvext.agent_hints"

This configuration injects adaptive guidance into outgoing model requests while allowing later request intercepts to continue running.

Plugin Configuration

Use plugin configuration when the application should let NeMo Relay own the Adaptive Hints request-intercept lifecycle.

1
import nemo_relay
2
3
adaptive_config = nemo_relay.adaptive.AdaptiveConfig(
4
    agent_id="planner",
5
    state=nemo_relay.adaptive.StateConfig(
6
        backend=nemo_relay.adaptive.BackendSpec.in_memory(),
7
    ),
8
    telemetry=nemo_relay.adaptive.TelemetryConfig(learners=["tool_parallelism"]),
9
    adaptive_hints=nemo_relay.adaptive.AdaptiveHintsConfig(
10
        inject_body_path="nvext.agent_hints",
11
    ),
12
)
13
14
plugin_config = nemo_relay.plugin.PluginConfig(
15
    components=[nemo_relay.adaptive.ComponentSpec(adaptive_config)]
16
)
17
18
report = nemo_relay.plugin.validate(plugin_config)
19
if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]):
20
    raise RuntimeError(report["diagnostics"])
21
22
await nemo_relay.plugin.initialize(plugin_config)
23
try:
24
    # Run instrumented application work here.
25
    pass
26
finally:
27
    nemo_relay.plugin.clear()

Manual API

Use the manual runtime API when an integration needs to own adaptive lifecycle directly instead of activating the top-level plugin component.

1
import nemo_relay
2
3
adaptive_config = nemo_relay.adaptive.AdaptiveConfig(
4
    agent_id="planner",
5
    state=nemo_relay.adaptive.StateConfig(
6
        backend=nemo_relay.adaptive.BackendSpec.in_memory(),
7
    ),
8
    telemetry=nemo_relay.adaptive.TelemetryConfig(learners=["tool_parallelism"]),
9
    adaptive_hints=nemo_relay.adaptive.AdaptiveHintsConfig(
10
        inject_body_path="nvext.agent_hints",
11
    ),
12
)
13
14
runtime = nemo_relay.adaptive.AdaptiveRuntime(adaptive_config.to_dict())
15
await runtime.register()
16
try:
17
    # Run instrumented application work here.
18
    nemo_relay.adaptive.set_latency_sensitivity(8)
19
finally:
20
    await runtime.shutdown()

Fields

Disable break_chain unless the adaptive hint should be the final request transform. Adjust priority only when adaptive hints need to run before or after known application middleware.

Expected Output

Outgoing managed LLM requests receive adaptive hint metadata in the configured header and body location. The hints do not replace the application callback or change the returned value by themselves. Downstream code must explicitly interpret the metadata before behavior changes.

Common Validation Failures

• Unknown adaptive hint fields when unknown fields are treated as errors.
• inject_body_path does not match the request shape expected by downstream provider adapters.
• Hint injection is enabled before downstream model paths can consume or ignore the metadata safely.