Using Codecs

Use this guide when a framework integration needs typed application values at its public boundary while NeMo Relay still records JSON-compatible payloads.

What You Build

You will choose typed value codecs for framework-facing wrappers so that:

• Application code can pass native objects to framework callbacks
• NeMo Relay can emit JSON-compatible lifecycle payloads
• Middleware and subscribers receive predictable serialized values
• The framework callback still receives the application type it expects

For provider-native LLM payload normalization, use Provider Codecs.

Before You Start

You need:

• A stable framework callback boundary.
• Application request and response types that can be projected into JSON.
• A managed wrapper that accepts codecs, such as the typed tool or typed LLM helpers.
• A validation path that confirms the returned value still matches framework expectations.

What Codecs Are

A typed value codec is a pure data translator at the NeMo Relay boundary. It converts application-facing values to JSON before NeMo Relay emits events or runs JSON-based middleware, then converts JSON back into the type expected by the framework callback or caller.

Typed value codecs are different from provider codecs:

Use this page for typed value codecs. Use Provider Codecs when request intercepts or request-side middleware need normalized LLM messages, tools, model names, and generation parameters, or when subscribers and exporters need provider response annotations.

How Typed Value Codecs Work

When a managed typed wrapper receives a codec:

1. The wrapper converts the application input into JSON before entering the NeMo Relay runtime.
2. NeMo Relay emits lifecycle events and runs middleware against JSON-compatible payloads.
3. The wrapper converts JSON back into the callback type before invoking framework-owned code when needed.
4. The wrapper converts only ToolExecutionResult.result back through the result codec and preserves the optional opaque annotation unchanged.

Keep codecs deterministic and side-effect free. A codec should not open network connections, mutate framework state, or hide provider I/O.

Typed Value Codecs

Python exposes:

• JsonPassthrough
• DataclassCodec
• PydanticCodec
• BestEffortAnyCodec

Node.js exposes JsonPassthrough and custom Codec<T> implementations.

from dataclasses import dataclass
from pydantic import BaseModel
from nemo_relay.typed import DataclassCodec, JsonPassthrough, PydanticCodec
@dataclass
class SearchArgs:
    query: str
class SearchResult(BaseModel):
    hits: int
args_codec = DataclassCodec(SearchArgs)
result_codec = PydanticCodec(SearchResult)
passthrough = JsonPassthrough()

Use BestEffortAnyCodec only at boundary code where strict schemas are unavailable. Prefer dataclass, Pydantic, or explicit Node.js codecs when the framework owns a stable schema.

For a tool that returns several result types, set allowed_types and tool_result=True on the same codec. The options serve different purposes: allowed_types lists the classes the codec can restore, and tool_result=True adds the JSON wrapper described in Normalized Tool Results.

For example, a search tool can return a SearchResult object with matches or plain text when it finds none. Both results use the same codec:

from dataclasses import dataclass
from nemo_relay.typed import BestEffortAnyCodec
@dataclass
class SearchResult:
    titles: list[str]
result_codec = BestEffortAnyCodec(
    allowed_types=[SearchResult],
    tool_result=True,
)
matches = SearchResult(titles=["Relay codec guide"])
encoded = result_codec.to_json(matches)
assert encoded["schema"] == "nemo.relay.tool_result@1"
assert encoded["content"] is None
assert result_codec.from_json(encoded) == matches
encoded = result_codec.to_json("No matches found.")
assert encoded["content"] == "No matches found."
assert result_codec.from_json(encoded) == "No matches found."

Pass result_codec to the result_codec argument of a typed tool helper such as tool_execute. The JSON wrapper stores the encoded value in result. The codec also fills content for strings and arrays of text or image parts. For other values, including the SearchResult object above, content is null.

The codec sets is_error to false. It does not treat an application field named status or error as proof of failure. If you supply a valid wrapper, the codec keeps its content and failure flag.

Without allowed_types, tagged dataclass and Pydantic values stay as JSON dictionaries during decoding. Each class must be listed on its own. Adding a base class does not allow its subclasses. Calling to_json() does not add classes to the list.

By default, encoding leaves out the tool-result wrapper. With either setting, from_json reads a valid wrapper’s result field and applies the same allowed_types rules. It also accepts older encoded values. It leaves wrappers with unknown versions or invalid fields as JSON.

During decoding, the codec never imports modules. It ignores old __nv_runtime_type__ tokens. Types outside the list remain dictionaries. Pickle decoding supports only a small set of Python built-in containers. The allowed_types option does not change these pickle rules.

Restoring an allowed type calls its constructor or runs Pydantic validation on the input data. Choose classes that can safely handle the data you expect. With allowed_types, the codec also supports data saved earlier or encoded in another process.

For one known schema, use DataclassCodec or PydanticCodec. Use a framework’s own codec to restore its messages or commands. Use JsonPassthrough for model-controlled tool arguments, as the LangChain and LangGraph integrations do.

Example: Typed Tool Boundary

Use typed value codecs when the framework wants native objects but NeMo Relay should emit JSON payloads.

from dataclasses import dataclass
import nemo_relay
from nemo_relay.typed import DataclassCodec, JsonPassthrough, tool_execute
@dataclass
class SearchArgs:
    query: str
async def invoke(args: SearchArgs) -> nemo_relay.ToolExecutionResult[dict[str, str]]:
    return nemo_relay.ToolExecutionResult({"echo": args.query})
result = await tool_execute(
    "search",
    SearchArgs("weather"),
    invoke,
    args_codec=DataclassCodec(SearchArgs),
    result_codec=JsonPassthrough(),
)
search_result = result.result

Validation Checklist

Use this checklist to confirm the implementation preserves the expected runtime contract.

• Codec output is JSON-compatible.
• Required fields are preserved by toJson and fromJson.
• Only the application-owned result passes through the result codec; an opaque tool result annotation is preserved unchanged.
• Middleware sees the expected serialized shape.
• The framework callback receives the expected application type.
• Error paths report conversion failures close to the framework boundary.

Next Steps

Use these links to continue from this workflow into the next related task.

• Use Provider Codecs when provider payloads need normalized request or response annotations.
• Use Handle Non-Serializable Data when framework objects cannot pass through JSON payloads.
• Use Integrate into Frameworks Code Examples for explicit fallback APIs.