Skip to navigation

This page is for framework authors and application integrators who extend the gateway. Read the architecture overview for system context and the request lifecycle for execution behavior.

Application Assembly

The YAML launcher parses a manifest into ProxyConfig. Python can supply that configuration or explicit ProviderIntegration declarations. Both paths call create_proxy_application and use the same route assembly and lifespan owners.

Construction validates declarations and routes. Lifespan startup creates network resources and preloads the Rails catalog. Requests then use those ready owners. Keeping these phases separate prevents request-time configuration loading and ensures initialization failures prevent readiness.

The following artifacts define application behavior and review coverage:

ArtifactAuthority and Use
Manifest or Python application configurationSelect provider instances, upstreams, operations, dialects, limits, and Rails entries. Python also supplies callbacks and explicit integrations.
Typed projections, bindings, and endpointsDefine field coverage, selected content, capability profiles, stream events, and provider operation semantics.
Rails configurationsDefine enabled checks, their dependencies, and streaming settings. All registered entries load at startup.
Exported guard contractsDerived from typed declarations for review. The runtime does not load these exports as policy.

Guard-contract derivation runs from typed declarations to exported YAML. Editing an export does not change execution. Stateful hooks, arbitrary Python validation, and transport behavior require review beyond the export. Refer to guard contracts for the exact boundary.

Execution Dependencies

This dependency view is scoped to one gateway process. Rectangles are components or declarations; hexagons are internal callable or protocol interfaces. Solid arrows show invocation; dotted arrows describe declaration, protocol, and implementation dependencies. Interfaces describe dependencies, not extra runtime services.

Bind projection, event, anderror behavior Requires checker methods Implements Invoke input/output checks Call check_async Invoke supplied dispatch Supplies bound dispatch methods Forward unrelated operations Build and send outbound request Provider declarations:projections, endpoints, streambinding, errors Guarded HTTP pipeline: bufferedand streaming execution ContentChecker protocol Rails adapter NVIDIA NeMo Guardrails library Injected HTTP dispatch callable ProviderTransport Shared HTTPX client and outboundtransport Pass-through router

The execution kernels receive a checker and dispatch callable instead of importing a provider implementation or constructing the Rails engine. Route assembly supplies those dependencies. Provider declarations carry the API semantics; reusable kernels control check outcomes and byte release. This lets tests exercise execution with deterministic checkers and dispatch functions.

The HTTP proxy layer uses ProviderTransport to adapt outbound requests and apply forwarding and representation rules. The HTTPX transport performs the final exchange. A custom HTTP transport does not select a new provider binding or Rails policy after inspection.

Composition Hooks and Internal Interfaces

Distinguish documented application composition from the private interfaces used to implement it:

BoundaryIntended Use
create_proxy_application, ProxyConfig, ProviderConfig, ConfigEntryAssemble an application through nemoguardrails.server.experimental.application.
Explicit ProviderIntegration declarationsCompose reviewed endpoints through Python using the factory’s direct mode.
config_resolverSelect one registered Rails ID from trusted context. It does not return a path or load configuration.
http_transport_factorySupply a new application-wide HTTPX transport for each lifespan. The runtime closes the returned transport; a custom transport owns any child transports it creates.
Private ContentChecker and dispatch interfacesMaintainer boundaries used by execution and its tests. The application factory does not expose a custom-checker plugin argument.

Use the Python integration guide for supported composition examples and the transport reference for streaming and cleanup requirements.

Trusted Rails Selection

This view shows per-request selection for declarative ProxyConfig providers against a catalog that is already ready. Rectangles represent selection steps or inputs; the diamond branches on application configuration. Solid arrows describe selection. The dotted arrow supplies the preloaded checker catalog. Authentication is application-owned; receiving a request header does not make that value an authorized policy choice.

Read application configuration Yes Fixed ID No: dynamic selection Trusted facts when identityaffects policy Selected ID Failure or invalid result Ready checkers keyed by ID Unknown ID Registered ID Guarded request andserver-supplied operationidentity Provider has fixed Rails ID? Use configured ID and bypassuser resolver Application authenticationsupplies trusted identity orstate Trusted Python resolver selects an ID Verify registered ID and look upready checker All Rails catalog entriesinitialized at startup Provider-shaped resolution errorbefore dispatch Continue guarded execution withthe ready checker

YAML requires fixed per-provider IDs. ProxyConfig providers without fixed IDs require a resolver, including with a one-entry catalog. Direct Python composition can infer static selection from one entry when no resolver is supplied; an explicit resolver still runs in direct mode. These assembly rules differ, while request-time lookup uses the same preloaded repository.

The resolver receives the original buffered request and canonical operation identity. Its cooperative deadline defaults to five seconds, independently of provider HTTP deadlines. Async callbacks and sync callbacks returning awaitables are supported; a blocking synchronous callback cannot be preempted by this deadline. Cancellation must propagate without selecting a fallback configuration.

The callback owns authorization for its selection. Use trusted identity or deliberately populated application state rather than raw caller headers. The catalog contains trusted Rails directories and is fixed for a lifespan. Model routing remains independent of Rails selection. Refer to configuration resolution for examples and failure behavior.

State and Resource Ownership

The following scopes own declarations and runtime resources:

ScopeOwner and Invariant
Bound declarationsReusable projection models, endpoint declarations, and event-classification definitions describe coverage.
Application lifespanThe runtime owns one shared HTTP client, its transport, and the Rails repository. Different Rails IDs own independent engines, even for the same directory.
Per requestA stream binding creates a fresh adapter and protocol hooks; the kernel keeps snapshot history and pending windows per execution. Ordering, snapshots, terminal state, and error framing must not leak between requests.
Upstream responseDispatch owns acquired response resources until handoff; the returned response owner handles completion, disconnect, and failure before body iteration.

Classifiers describe events; hooks enforce stateful protocol behavior. Factories must return independent hook instances for each request. The stream binding encodes that boundary, and the provider composition tests check request-local state. Refer to streaming errors and cleanup for response ownership and cooperative cleanup limits.

Where to Make a Change

Use these implementation and test entry points for each change:

ChangeStart HereVerify With
Provider operation or reviewed dialectChat declarations, Responses declarations, and integration assembly.Buffered integration tests, composition tests, and contract exports.
Event selection and streaming protocol behaviorEvent classifier declarations and request-local hooks.Stream classifier tests and Chat streaming tests.
New content capabilityMatching projection, buffered execution, streaming execution, and checker semantics.Accepted/rejected payloads, both execution modes, byte preservation, and representable exports. Profile metadata alone cannot extend support.
Egress, TLS, or trusted destination routingHTTP transport composition and the routing example.Application transport tests and routing transport tests.

Keep changes aligned with the current content profiles. Follow Development for validation commands and release compatibility requirements.