Module nvidia_worker_v1

This page is for NeMo Relay, version dev. For other versions, use one of these documentation indexes: - Latest (v0.11.1) (default): https://docs.nvidia.com/nemo/relay/latest/llms.txt - dev: https://docs.nvidia.com/nemo/relay/dev/llms.txt - v0.11.1: https://docs.nvidia.com/nemo/relay/v0.11.1/llms.txt - v0.10.4: https://docs.nvidia.com/nemo/relay/v0.10.4/llms.txt - v0.9.4: https://docs.nvidia.com/nemo/relay/v0.9.4/llms.txt - v0.8.4: https://docs.nvidia.com/nemo/relay/v0.8.4/llms.txt - v0.7.3: https://docs.nvidia.com/nemo/relay/v0.7.3/llms.txt

Generated from cargo doc --no-deps -p nemo-relay-builtins -p nemo-relay-diagnostics -p nemo-relay-logging -p nemo-relay-plugin-host -p nemo-relay-codecs -p nemo-relay -p nemo-relay-runtime -p nemo-relay-model-pricing -p nemo-relay-resource-metrics -p nemo-relay-observability -p nemo-relay-adaptive -p nemo-relay-pii-redaction -p nemo-relay-ffi -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay-worker-proto -p nemo-relay-worker.

Typed worker contract introduced alongside the legacy API.

Modules

• content_part: Nested message and enum types in ContentPart.
• decode_llm_codec_request_response: Nested message and enum types in DecodeLlmCodecRequestResponse.
• decode_llm_codec_response_response: Nested message and enum types in DecodeLlmCodecResponseResponse.
• diagnostic_code_definition: Nested message and enum types in DiagnosticCodeDefinition.
• encode_llm_codec_request_response: Nested message and enum types in EncodeLlmCodecRequestResponse.
• event: Nested message and enum types in Event.
• finish_reason: Nested message and enum types in FinishReason.
• get_current_scope_stack_snapshot_response: Nested message and enum types in GetCurrentScopeStackSnapshotResponse.
• get_diagnostic_discovery_response: Nested message and enum types in GetDiagnosticDiscoveryResponse.
• guardrail_decision: Nested message and enum types in GuardrailDecision.
• inspect_diagnostic_memory_response: Nested message and enum types in InspectDiagnosticMemoryResponse.
• invoke_request: Nested message and enum types in InvokeRequest.
• invoke_response: Nested message and enum types in InvokeResponse.
• invoke_stream_response: Nested message and enum types in InvokeStreamResponse.
• llm_message: Nested message and enum types in LlmMessage.
• llm_next_response: Nested message and enum types in LlmNextResponse.
• llm_stream_next_response: Nested message and enum types in LlmStreamNextResponse.
• memory_limit_event_slot: Nested message and enum types in MemoryLimitEventSlot.
• message_content: Nested message and enum types in MessageContent.
• plugin_worker_service_client: Generated client implementations.
• plugin_worker_service_server: Generated server implementations.
• read_diagnostic_page_response: Nested message and enum types in ReadDiagnosticPageResponse.
• register_response: Nested message and enum types in RegisterResponse.
• relay_diagnostics_service_client: Generated client implementations.
• relay_diagnostics_service_server: Generated server implementations.
• relay_host_runtime_service_client: Generated client implementations.
• relay_host_runtime_service_server: Generated server implementations.
• resource_memory_number: Nested message and enum types in ResourceMemoryNumber.
• resource_memory_unit: Nested message and enum types in ResourceMemoryUnit.
• runtime_registration_owner: Nested message and enum types in RuntimeRegistrationOwner.
• submit_diagnostic_request: Nested message and enum types in SubmitDiagnosticRequest.
• tool_choice: Nested message and enum types in ToolChoice.
• tool_definition: Nested message and enum types in ToolDefinition.
• tool_next_response: Nested message and enum types in ToolNextResponse.
• validate_response: Nested message and enum types in ValidateResponse.

Structs

• AnnotatedLlmRequest: Normalized request structure with explicit provider-specific and extension fields.
• AnnotatedLlmResponse: Normalized completed response, usage, pricing, and optimization evidence.
• AssistantMessage: Assistant output with independently optional content and tool calls.
• BaseEvent: Common ATOF identity, propagation, timing, and application payload fields.
• CacheWriteTokenCounts: Prompt-cache write tokens retained for one cache duration.
• CacheWriteTokenCountsList: Presence-bearing duration-aware prompt-cache write counts.
• CancelInvocationRequest: Requests cancellation of one active invocation without replaying work.
• CancelInvocationResponse: Idempotent: unknown/completed IDs return accepted=false, without resurrecting work.
• CategoryProfile: Known category-specific fields and lossless extensions attached to an event.
• ComponentMemoryFootprint: Typed component memory footprint preserving the shared runtime fields.
• ComponentMemoryUsage: Typed component memory usage preserving the shared runtime fields.
• ConditionalMiddlewareGuardrailRegistration: Declares a static or callback-based filter over runtime registrations.
• ConditionalMiddlewareInvocation: Asks a conditional callback whether selected middleware registrations may execute.
• ConfigDiagnostic: Structured configuration warning or error with optional component and field location.
• ConfigDiagnostics: Present validation result; an empty list is successful validation without diagnostics.
• ContentPart: Selects a normalized content block or a lossless provider-native block.
• ContentPartList: Ordered multimodal content; a present empty list remains an explicit variant.
• CostEstimate: Normalized monetary cost with provenance for estimates and provider-reported values.
• CreateScopeStackRequest: Creates an isolated host-owned scope stack for this activation.
• CreateScopeStackResponse: Returns the new isolated stack’s opaque binding identity.
• Credentials: Required on authenticated requests. Both strings must match the activation.
• DataSchema: Application-owned schema identity; its version is not the worker version.
• DecodeLlmCodecRequestRequest: Requests normalized decoding through an invocation-bound request codec.
• DecodeLlmCodecRequestResponse: Selects the normalized request or the codec’s domain failure.
• DecodeLlmCodecResponseRequest: Requests normalized decoding through an invocation-bound response codec.
• DecodeLlmCodecResponseResponse: Selects the normalized completed response or the codec’s domain failure.
• DeregisterConditionalMiddlewareGuardrailRequest: Removes a conditional guardrail using its activation-owned handle.
• DeregisterConditionalMiddlewareGuardrailResponse: Reports whether an owned conditional guardrail was removed.
• DiagnosticCapabilities: Host-owned limits and contract identities required for diagnostic admission.
• DiagnosticCapability: Availability and optional reason for one named diagnostic feature.
• DiagnosticClassification: Preserves catalogue and failure-class names, including unknown future names.
• DiagnosticCodeDefinition: Selects an existing catalogue name or an activation-local extension definition.
• DiagnosticComponentClose: Closes a component handle without deleting its retained accounting.
• DiagnosticComponentIdentity: Identifies a logical component and one registration incarnation.
• DiagnosticComponentOwnership: Correlates component responsibility without creating additional counter dimensions.
• DiagnosticComponentRegistration: Registers a bounded diagnostic component and an ordered list of code handles.
• DiagnosticComponentSnapshot: A retained component row with lifetime, refresh, counters, and windows.
• DiagnosticCorrelation: Optional trajectory correlation, excluded from aggregation keys.
• DiagnosticCount: Unsigned count that stops at its maximum. After that limit, it is a lower bound.
• DiagnosticCounter: Lifetime occurrence count and latest observation time for one admitted code.
• DiagnosticCoverage: Describes snapshot availability, aggregation scope, retention, and evidence loss.
• DiagnosticCursor: Canonical snapshot position; it does not grant transport read authority.
• DiagnosticDetail: Retained diagnostic text with explicit truncation state.
• DiagnosticDiscovery: Typed diagnostic discovery preserving the shared runtime fields.
• DiagnosticEvent: One retained observation, its attribution, occurrence delta, and optional evidence.
• DiagnosticExtensionDefinition: Declares one plugin-defined diagnostic classification and its allowed causes.
• DiagnosticLifetimeSummary: Lifetime totals and monotonic elapsed time for a source or component.
• DiagnosticLoss: Explicit counters for bounded-storage, admission, and projection losses.
• DiagnosticMemoryPage: Typed diagnostic memory page preserving the shared runtime fields.
• DiagnosticMemoryUsage: Typed diagnostic memory usage preserving the shared runtime fields.
• DiagnosticObservation: Records a positive occurrence delta using admitted component and code handles.
• DiagnosticOrigin: Captures the refresh generation and ownership at observation time.
• DiagnosticPluginContext: Carries independently optional plugin execution and lifecycle phase names.
• DiagnosticProducerSupport: Typed diagnostic producer support preserving the shared runtime fields.
• DiagnosticQuery: Typed diagnostic query preserving the shared runtime fields.
• DiagnosticReceipt: Reports sequence consumption, admission outcome, and retained evidence disposition.
• DiagnosticRefreshSummary: Accounting for the component’s current refresh generation and ownership.
• DiagnosticRetention: Reports retained bounds and whether the requested page was truncated.
• DiagnosticSnapshot: JSON output represents every uint64 diagnostic counter as a decimal string.
• DiagnosticSourceIdentity: Identifies the process incarnation that owns diagnostic accounting.
• DiagnosticTotals: Outcome and recording totals for an explicitly identified accounting scope.
• DiagnosticWindow: Bounded outcome history at one configured time resolution.
• DiagnosticWindowBucket: Outcome counts for one source-local elapsed-time interval.
• DiagnosticWindowCoverage: Reports the temporal extent and completeness of a diagnostic window.
• DropScopeStackRequest: Releases an activation-owned isolated scope stack.
• DropScopeStackResponse: Confirms stack release; failures use gRPC status.
• EmitMarkRequest: Requests host materialization of a mark in the selected scope context.
• EmitMarkResponse: Confirms the mark operation completed; failures use gRPC status.
• EncodeLlmCodecRequestRequest: Requests provider encoding from an annotation and original request baseline.
• EncodeLlmCodecRequestResponse: Selects the encoded provider request or the codec’s domain failure.
• EntryMemoryFootprint: Typed entry memory footprint preserving the shared runtime fields.
• EntryMemoryUsage: Typed entry memory usage preserving the shared runtime fields.
• Event: Exactly one kind is required. UUID strings use canonical hyphenated UUIDs.
• EventSanitizeFields: Sanitizer output replaces these fields; absence clears the corresponding field.
• FinishReason: Normalized completion reason with an explicit lossless unknown-name alternative.
• FunctionCall: Function name and original argument text, which need not be complete JSON.
• FunctionDefinition: Function name, optional description and JSON Schema, strictness, and extensions.
• FunctionMessage: Legacy function result; absent content represents explicit null in the runtime DTO.
• FunctionToolDefinition: Function tool definition plus provider fields outside the normalized function.
• GenerationParams: Portable generation controls with presence preserved for every optional value.
• GetCurrentScopeStackSnapshotRequest: Queries the scope stack bound to an invocation or selected scope context.
• GetCurrentScopeStackSnapshotResponse: Selects a present snapshot or an explicit absence of bound context.
• GetDiagnosticCapabilitiesRequest: Queries diagnostic policy using activation credentials, without opening a session.
• GetDiagnosticCapabilitiesResponse: Returns the host’s bounded diagnostic delivery and read policy.
• GetDiagnosticDiscoveryRequest: Requests discovery evidence only under the activation’s separately granted read authority.
• GetDiagnosticDiscoveryResponse: Returns discovery evidence or a portable domain read failure; transport failures use gRPC status.
• GetRuntimeDiagnosticsRequest: Queries bounded operational counters for the active host.
• GetRuntimeDiagnosticsResponse: Returns bounded operational counters, independent of restricted evidence reads.
• GuardrailDecision: Explicit allow or nonempty block reason; missing selection is malformed.
• HandshakeRequest: Authenticates an activation and proposes the exact worker contract and plugin identity.
• HandshakeResponse: Confirms worker identity, component multiplicity, and supported registration surfaces.
• HealthRequest: Queries readiness using activation credentials; it may precede handshake.
• HealthResponse: Reports readiness and worker identity without implying registration success.
• ImageUrl: Image URL and optional provider detail preference.
• ImageUrlContentPart: Image URL input with portable URL options and provider extensions.
• InspectDiagnosticMemoryRequest: Requests memory evidence only under the activation’s separately granted read authority.
• InspectDiagnosticMemoryResponse: Returns memory evidence or a portable domain read failure; transport failures use gRPC status.
• InvokeRequest: Requires an invocation ID, a registration name, and one invocation alternative. The alternative must match the registered callback type. The request has no separate surface enum that could conflict with that alternative.
• InvokeResponse: One surface-specific success alternative or one callback/domain failure.
• InvokeStreamRequest: Streaming is a distinct method with a request-only codec context.
• InvokeStreamResponse: Exactly one item required. Error is terminal; clean EOF is successful closure.
• JsonArray: Presence-bearing lists preserve None versus Some([]) in runtime DTOs.
• JsonContentPart: Provider-neutral image/audio/file payload with open provider fields.
• JsonObject: Open object with structurally enforced string keys; each value is required.
• JsonValue: Exactly one UTF-8 RFC 8259 value, without NaN/Infinity or trailing content. Required payloads must contain bytes, including the bytes “null” for JSON null. Numbers are never converted through google.protobuf.Value/double.
• ListRuntimeRegistrationsRequest: Lists runtime registrations, optionally restricted to selected surfaces.
• ListRuntimeRegistrationsResponse: Returns the matching visible runtime registrations.
• LlmCodecContext: Codec reference and optional invocation-scoped host access capability.
• LlmCodecReference: Uniform codec reference metadata, independent of implementation or provider. The host selects the actual codec through its invocation-scoped capability.
• LlmConditionalInvocation: Provider request supplied to an LLM execution condition.
• LlmExecutionInvocation: Unary LLM execution input with request and response codec access.
• LlmMessage: The role is the alternative; contradictory roles cannot be represented.
• LlmNextRequest: Invokes a live unary LLM continuation with a rewritten provider request.
• LlmNextResponse: Selects the downstream provider response or its original domain failure.
• LlmOptimizationContribution: Plugin-neutral optimization evidence with an open kind and optional custom payload.
• LlmOptimizationModel: Model and optional provider used in optimization accounting.
• LlmOptimizationModelTransition: Optional baseline and effective model identities for an optimization.
• LlmOptimizationSummary: Host-computed optimization accounting at completed LLM scope closure.
• LlmOptimizationTokenImpact: Baseline, effective, and saved token evidence with its quality and method.
• LlmOptimizationTokens: Optional token evidence; missing categories remain unknown rather than zero.
• LlmRequest: Headers and content are required; content remains provider-owned JSON.
• LlmRequestInterceptOutcome: Rewritten request, optional authoritative annotation, pending marks, and evidence.
• LlmRequestInvocation: Request-interceptor input with optional normalized annotation.
• LlmSanitizeRequestInvocation: Provider request and codec context supplied to an LLM request sanitizer.
• LlmSanitizeResponseInvocation: Provider response and codec context supplied to an LLM response sanitizer.
• LlmStreamNextRequest: Opens a live streaming LLM continuation with a rewritten provider request.
• LlmStreamNextResponse: One provider chunk or one terminal downstream domain failure.
• LogRequest: Submits an authenticated process-log record to the host-owned sink and threshold.
• LogResponse: Confirms the host accepted the log operation; failures use gRPC status.
• MarkEvent: An ATOF point-in-time event with optional semantic categorization.
• MemoryLimitEvent: Typed memory limit event preserving the shared runtime fields.
• MemoryLimitEventSlot: One fixed resource limit-event slot, preserving the runtime’s null positions.
• MemorySelectionTotals: Typed memory selection totals preserving the shared runtime fields.
• MessageContent: Selects plain text or a multimodal parts list, including explicit empty content.
• NamedMessage: System, user, or developer message with required content and optional sender name.
• OpenDiagnosticSessionRequest: Requests a session with exactly the diagnostic contracts advertised by the host.
• OpenDiagnosticSessionResponse: Returns the activation’s session identity and the confirmed host policy.
• PendingMarkSpec: Requests a mark whose UUID, timestamp, and parent are assigned by the host.
• PluginRegistrationOwner: Identifies the plugin kind and optional component ordinal owning a registration.
• PopScopeRequest: Closes a previously pushed scope with optional output, metadata, and time.
• PopScopeResponse: Confirms scope closure; failures use gRPC status.
• ProtocolIdentity: Protocol name nvidia.nemo.relay.worker.v1, contract version 1.0.0. Package/SDK releases are independent.
• ProviderNativeComponent: Preserves provider-owned content with its provider and native discriminator.
• PushScopeRequest: Creates and binds a scope under the selected parent context.
• PushScopeResponse: Returns the host-owned handle needed to close the newly pushed scope.
• ReadDiagnosticPageRequest: Requests snapshot evidence only under the activation’s separately granted read authority.
• ReadDiagnosticPageResponse: Returns snapshot evidence or a portable domain read failure; transport failures use gRPC status.
• RegisterConditionalMiddlewareGuardrailRequest: Installs an activation-owned conditional middleware guardrail.
• RegisterConditionalMiddlewareGuardrailResponse: Returns the opaque handle for the newly installed conditional guardrail.
• RegisterRequest: Registration is idempotent for the same semantic config within an activation. Concurrent repeats install once; changed config fails with FAILED_PRECONDITION.
• RegisterResponse: Selects the installed registration plan or a registration callback failure.
• Registration: One callback’s local identity, surface, priority, and chain behavior.
• RegistrationPlan: Declarative callback and initial conditional-guardrail registrations for one component.
• RemoteDiagnosticOrigin: Preserves authenticated worker provenance and the host’s receipt timing.
• ResourceMemoryMeasurement: Typed resource memory measurement preserving the shared runtime fields.
• ResourceMemoryMetrics: Typed resource memory metrics preserving the shared runtime fields.
• ResourceMemoryNumber: Exact integer, finite decimal, or explicit non-finite collector result.
• ResourceMemorySample: Typed resource memory sample preserving the shared runtime fields.
• ResourceMemoryUnit: Original measurement dimension and canonical unit, without conversion.
• ResourceMemoryView: Typed resource memory view preserving the shared runtime fields.
• ResourceProcessIdentity: Typed resource process identity preserving the shared runtime fields.
• ResponseToolCall: Completed-response tool call with parsed JSON arguments.
• ResponseToolCallList: Presence-bearing completed-response tool calls.
• RuntimeDiagnostic: One operational diagnostic code, safe description, and occurrence count.
• RuntimeRegistrationIdentity: Describes local/effective callback identity, surface, and ownership.
• RuntimeRegistrationOwner: Selects core, global API, or plugin ownership of a runtime registration.
• ScopeContext: Opaque host-issued stack/parent bindings, not public scope UUIDs.
• ScopeEvent: An ATOF scope lifecycle event with an open semantic category.
• ScopeStackEntry: Owned public scope identity, distinct from an opaque host binding.
• ScopeStackSnapshot: Owned scope identities ordered from root to the current top.
• ShutdownRequest: Requests bounded worker drain and resource cleanup for the activation.
• ShutdownResponse: Confirms successful admission of the worker shutdown request.
• StringList: Presence-bearing string list; a present empty list differs from absence.
• SubmitDiagnosticRequest: Sends one ordered operation per request. Identical retries keep the sequence and content. A rejection before admission does not consume the sequence. An admitted policy rejection returns REJECTED and consumes the sequence. This service never retries execution callbacks.
• SubmitDiagnosticResponse: Acknowledges one ordered diagnostic submission with its admission receipt.
• TextContentPart: Text or refusal content with provider-specific extension fields.
• ToolCall: Normalized request-side tool call; argument text remains provider-issued text.
• ToolCallItem: Portable top-level provider tool-call item with parsed arguments.
• ToolCallList: Presence-bearing tool-call list, preserving absent versus explicitly empty.
• ToolChoice: Explicit automatic, disabled, required, named, or provider-native tool selection.
• ToolChoiceFunction: Provider function-choice discriminator and the selected function identity.
• ToolChoiceFunctionName: Identifies the function selected by a specific tool-choice rule.
• ToolDefinition: Selects a normalized function definition or a provider-native tool definition.
• ToolDefinitionList: Presence-bearing tool-definition list, preserving absent versus explicitly empty.
• ToolExecutionInterceptOutcome: Canonical tool result plus ordered marks for host lifecycle materialization.
• ToolExecutionInvocation: Managed tool execution input and its invocation-scoped continuation.
• ToolExecutionResult: Application-visible tool result and optional opaque annotation.
• ToolMessage: Tool result content correlated to a provider-issued tool call.
• ToolNextRequest: Invokes a live tool continuation using a rewritten application value.
• ToolNextResponse: Selects the downstream tool result or its original domain failure.
• ToolResultContentPart: Tool result embedded in a provider content-block array.
• ToolResultItem: Portable top-level provider tool-result item with an open result payload.
• ToolUseContentPart: Tool invocation embedded in a provider content-block array.
• ToolValueInvocation: Tool request/response value supplied to a sanitizer, condition, or request interceptor.
• Usage: Optional provider token counts and cost; absent counts never imply zero.
• ValidateRequest: Validates one plugin configuration without installing callbacks.
• ValidateResponse: Selects completed validation diagnostics or a validation callback failure.
• WorkerError: Domain failure, never combined with a success value. RPC admission/auth/shape failures use gRPC status. Retryable never authorizes replay of execution.
• WorkerIdentity: Worker, SDK, runtime, and protocol identities reported during activation.

Enums

• CapacityUnit: Canonical capacity unit values from the shared runtime types.
• ConfigDiagnosticLevel: Configuration diagnostic severity; unspecified is invalid for a reported diagnostic.
• CostSource: Origin of a cost value, separating pricing estimates from provider reports.
• CountUnit: Canonical count unit values from the shared runtime types.
• DiagnosticAvailability: Availability of a diagnostic feature, window, or snapshot.
• DiagnosticDomain: Canonical diagnostic domain values from the shared runtime types.
• DiagnosticEvidenceRetention: Evidence retention disposition, independent of occurrence accounting.
• DiagnosticObservationOutcome: Observed operation outcome, independent of diagnostic severity.
• DiagnosticObservationSeverity: Evidence severity accepted by diagnostic delivery.
• DiagnosticReadReason: Canonical diagnostic read reason values from the shared runtime types.
• DiagnosticReceiptStatus: Terminal admission outcome for an ordered diagnostic submission.
• DiagnosticSummaryScope: Aggregation scope of the reported lifetime summaries.
• DurationUnit: Canonical duration unit values from the shared runtime types.
• LlmOptimizationEvidenceQuality: Whether token evidence was observed or estimated.
• LlmOptimizationSummaryStatus: Whether optimization accounting is complete or limited by missing evidence.
• LogSeverity: Log evidence severity; an unspecified submitted severity is invalid.
• MemoryFieldScope: Canonical memory field scope values from the shared runtime types.
• PluginDiagnosticFamily: Canonical plugin diagnostic family values from the shared runtime types.
• RegistrationSurface: Closed registration surfaces; unspecified and unknown values cannot be invoked.
• ResourceAssociation: Canonical resource association values from the shared runtime types.
• ResourceLimitEventKind: Canonical resource limit event kind values from the shared runtime types.
• ResourceLimitResource: Canonical resource limit resource values from the shared runtime types.
• ResourceMeasurementScope: Canonical resource measurement scope values from the shared runtime types.
• ResourceMemoryAvailability: Canonical resource memory availability values from the shared runtime types.
• ResourceOperatingSystem: Canonical resource operating system values from the shared runtime types.
• ResourceSampleStream: Canonical resource sample stream values from the shared runtime types.
• ScopeCategory: Scope lifecycle phase; unspecified is invalid for emitted scope events.
• ScopeType: Recognized runtime scope types, including the explicit unknown-category fallback.
• WorkerHealthStatus: Worker readiness state; unspecified is not a readiness assertion.

Constants

• CONTRACT_VERSION: Initial stable worker/schema contract baseline, independent of crate versions.
• DIAGNOSTIC_PROVENANCE_CONTRACT_VERSION: Diagnostic provenance contract negotiated independently of the worker protocol.
• DIAGNOSTIC_SNAPSHOT_CONTRACT_VERSION: Diagnostic snapshot contract negotiated independently of the worker protocol.
• WORKER_PROTOCOL: Negotiation identifier for this contract, incompatible with legacy grpc-v1.