Adaptive Hints

Use Adaptive Hints when downstream model calls or provider adapters can safely receive guidance metadata from the adaptive runtime.

Adaptive hints register as LLM request intercepts. Lower numeric priority values run earlier in the intercept chain. The plugin sets the default priority relative to other middleware rather than as a standalone importance score.

plugins.toml Example

Add the following Adaptive Hints configuration to plugins.toml:

1
version = 1
2
3
[[components]]
4
kind = "adaptive"
5
enabled = true
6
7
[components.config]
8
version = 1
9
agent_id = "planner"
10
11
[components.config.state.backend]
12
kind = "in_memory"
13
14
[components.config.telemetry]
15
subscriber_name = "adaptive.telemetry"
16
learners = ["tool_parallelism"]
17
18
[components.config.adaptive_hints]
19
priority = 100
20
break_chain = false
21
inject_header = true
22
inject_body_path = "nvext.agent_hints"

This configuration injects adaptive guidance into outgoing model requests while allowing later request intercepts to continue running.

Plugin Configuration

Use plugin configuration when the application should let NeMo Relay own the Adaptive Hints request-intercept lifecycle. The following examples configure and activate Adaptive Hints through each supported language binding.

validate() checks only the supplied in-memory object. initialize() also layers discovered plugins.toml configuration. For effective file-backed validation, refer to Plugin Configuration Files and run the gateway with the same configuration path that production uses.

1
import asyncio
2
3
import nemo_relay
4
5
adaptive_config = nemo_relay.adaptive.AdaptiveConfig(
6
    agent_id="planner",
7
    state=nemo_relay.adaptive.StateConfig(
8
        backend=nemo_relay.adaptive.BackendSpec.in_memory(),
9
    ),
10
    telemetry=nemo_relay.adaptive.TelemetryConfig(learners=["tool_parallelism"]),
11
    adaptive_hints=nemo_relay.adaptive.AdaptiveHintsConfig(
12
        inject_body_path="nvext.agent_hints",
13
    ),
14
)
15
16
plugin_config = nemo_relay.plugin.PluginConfig(
17
    components=[nemo_relay.adaptive.ComponentSpec(adaptive_config)]
18
)
19
20
report = nemo_relay.plugin.validate(plugin_config)
21
if any(diagnostic["level"] == "error" for diagnostic in report["diagnostics"]):
22
    raise RuntimeError(report["diagnostics"])
23
24
async def main():
25
    await nemo_relay.plugin.initialize(plugin_config)
26
    try:
27
        # Run instrumented application work here.
28
        pass
29
    finally:
30
        nemo_relay.plugin.clear()
31
32
asyncio.run(main())

Manual API

Use the manual runtime API when an integration needs to own adaptive lifecycle directly instead of activating the top-level plugin component.

1
import asyncio
2
3
import nemo_relay
4
5
adaptive_config = nemo_relay.adaptive.AdaptiveConfig(
6
    agent_id="planner",
7
    state=nemo_relay.adaptive.StateConfig(
8
        backend=nemo_relay.adaptive.BackendSpec.in_memory(),
9
    ),
10
    telemetry=nemo_relay.adaptive.TelemetryConfig(learners=["tool_parallelism"]),
11
    adaptive_hints=nemo_relay.adaptive.AdaptiveHintsConfig(
12
        inject_body_path="nvext.agent_hints",
13
    ),
14
)
15
16
runtime = nemo_relay.adaptive.AdaptiveRuntime(adaptive_config.to_dict())
17
asyncio.run(runtime.register())
18
try:
19
    # Run instrumented application work here.
20
    nemo_relay.adaptive.set_latency_sensitivity(8)
21
finally:
22
    asyncio.run(runtime.shutdown())

Fields

The following table describes Adaptive Hints settings:

Disable break_chain unless the adaptive hint should be the final request transform. Adjust priority only when adaptive hints need to run before or after known application middleware.

Expected Output

Outgoing managed LLM requests receive adaptive hint metadata in the configured header and body location. The hints do not replace the application callback or change the returned value by themselves. Downstream code must explicitly interpret the metadata before behavior changes.

Common Configuration and Runtime Issues

• Unknown adaptive hint fields when unknown fields are treated as errors.
• inject_body_path does not match the request shape expected by downstream provider adapters.
• Hint injection is enabled before downstream model paths can consume or ignore the metadata safely.