For AI agents: a documentation index is available at the root level at /llms.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
LogoLogo
  • About NVIDIA NeMo Relay
    • Overview
    • Architecture
    • Ecosystem
    • Concepts
    • Release Notes
  • Getting Started
    • Agent Runtime Primer
    • Prerequisites
    • Installation
    • Configuration / Setup
    • Quick Start
  • NVIDIA NeMo Relay CLI
    • About
    • Basic Usage
    • Plugin Installation
    • Claude Code
    • Codex
    • Hermes Agent
  • Supported Integrations
    • About
    • OpenClaw Plugin Guide
    • LangChain Integration Guide
    • LangGraph Integration Guide
    • Deep Agents Integration Guide
  • Instrument Applications
    • About
    • Adding Scopes and Marks
    • Instrument a Tool Call
    • Instrument an LLM Call
    • Add Middleware
    • Code Examples
  • Configure Plugins
    • About
    • Model Pricing
    • Configure Discoverable Plugins
    • Plugin Configuration Files
  • Integrate into Frameworks
    • About
    • Adding Scopes
    • Wrap Tool Calls
    • Wrap LLM Calls
    • Handle Non-Serializable Data
    • Using Codecs
    • Provider Codecs
    • Provider Response Codecs and Model Pricing
    • Code Examples
  • Build Plugins
    • About
  • Contribute
    • About
    • Development Setup
    • Workflow and Reviews
    • Testing and Documentation
    • Runtime Contract Documentation
  • Reference
    • APIs
      • Python Library Reference
        • nemo-relay
        • nemo-relay-plugin
      • Node.js Library Reference
      • Rust Library Reference
    • Performance
    • LLM Request Intercept Outcomes
    • Tool Execution Intercept Outcomes
  • Resources
    • Support and FAQs
    • Glossary
    • Troubleshooting Guide
    • Community
    • Legal
  • Overview
  • Architecture
  • Ecosystem
  • Concepts
  • Scopes
  • Middleware
  • Plugins
  • Events
  • Subscribers
  • Framework Integrations
  • Codecs
  • Release Notes
  • Highlights
  • Known Issues
  • Agent Runtime Primer
  • Prerequisites
  • Installation
  • Configuration / Setup
  • Quick Start
  • Python Quick Start
  • Node.js Quick Start
  • Rust Quick Start
  • About
  • Basic Usage
  • Plugin Installation
  • Claude Code
  • Codex
  • Hermes Agent
  • About
  • OpenClaw Plugin Guide
  • LangChain Integration Guide
  • LangGraph Integration Guide
  • Deep Agents Integration Guide
  • About
  • Adding Scopes and Marks
  • Instrument a Tool Call
  • Instrument an LLM Call
  • Add Middleware
  • Code Examples
  • About
  • About
  • Configuration
  • Agent Trajectory Interchange Format (ATIF)
  • Agent Trajectory Observability Format (ATOF)
  • OpenTelemetry
  • OpenInference
  • About
  • Configuration
  • Adaptive Cache Governor (ACG)
  • Adaptive Hints
  • About
  • Configuration
  • About
  • Configuration
  • Model Pricing
  • Configure Discoverable Plugins
  • Plugin Configuration Files
  • About
  • Adding Scopes
  • Wrap Tool Calls
  • Wrap LLM Calls
  • Handle Non-Serializable Data
  • Using Codecs
  • Provider Codecs
  • Provider Response Codecs and Model Pricing
  • Code Examples
  • About
  • Language Binding Plugins
  • Validate Plugin Configuration
  • Register Plugin Behavior
  • Design Plugin Configuration
  • Code Examples
  • Discoverable Plugins
  • Native Dynamic Plugins (Rust)
  • Build a Rust Native Plugin
  • gRPC Worker Plugin Concepts
  • gRPC Worker Protocol Overview
  • gRPC Worker Plugins (Python)
  • gRPC Worker Plugins (Rust)
  • About
  • Development Setup
  • Workflow and Reviews
  • Testing and Documentation
  • Runtime Contract Documentation
  • APIs
  • Python Library Reference
  • nemo-relay
  • adaptive
  • codecs
  • guardrails
  • integrations
  • deepagents
  • callbacks
  • middleware
  • langchain
  • callbacks
  • middleware
  • langgraph
  • callbacks
  • intercepts
  • llm
  • model_pricing
  • observability
  • pii_redaction
  • plugin
  • scope
  • scope_local
  • subscribers
  • tools
  • typed
  • utils
  • nemo-relay-plugin
  • Node.js Library Reference
  • Runtime
  • Typed Helpers
  • Plugin Helpers
  • Adaptive Helpers
  • Observability Helpers
  • nemo-relay-node/pii_redaction
  • nemo-relay-node/model_pricing
  • Rust Library Reference
  • nemo-relay
  • api
  • event
  • BaseEvent
  • CategoryProfile
  • DataSchema
  • EventCategory
  • MarkEvent
  • PendingMarkSpec
  • ScopeEvent
  • Event
  • ScopeCategory
  • ATOF_VERSION
  • EventNormalizationExt
  • attributes_from_handle
  • llm_attributes_to_strings
  • scope_attributes_to_strings
  • tool_attributes_to_strings
  • llm
  • CreateLlmHandleParams
  • EndLlmHandleParams
  • LlmAttributes
  • LlmCallEndParams
  • LlmCallExecuteParams
  • LlmCallParams
  • LlmHandle
  • LlmRequest
  • LlmRequestInterceptOutcome
  • LlmStreamCallExecuteParams
  • llm_call
  • llm_call_end
  • llm_call_execute
  • llm_conditional_execution
  • llm_request_intercepts
  • llm_stream_call_execute
  • registry
  • deregister_llm_conditional_execution_guardrail
  • deregister_llm_execution_intercept
  • deregister_llm_request_intercept
  • deregister_llm_sanitize_request_guardrail
  • deregister_llm_sanitize_response_guardrail
  • deregister_llm_stream_execution_intercept
  • deregister_tool_conditional_execution_guardrail
  • deregister_tool_execution_intercept
  • deregister_tool_request_intercept
  • deregister_tool_sanitize_request_guardrail
  • deregister_tool_sanitize_response_guardrail
  • register_llm_conditional_execution_guardrail
  • register_llm_execution_intercept
  • register_llm_request_intercept
  • register_llm_sanitize_request_guardrail
  • register_llm_sanitize_response_guardrail
  • register_llm_stream_execution_intercept
  • register_tool_conditional_execution_guardrail
  • register_tool_execution_intercept
  • register_tool_request_intercept
  • register_tool_sanitize_request_guardrail
  • register_tool_sanitize_response_guardrail
  • scope_deregister_llm_conditional_execution_guardrail
  • scope_deregister_llm_execution_intercept
  • scope_deregister_llm_request_intercept
  • scope_deregister_llm_sanitize_request_guardrail
  • scope_deregister_llm_sanitize_response_guardrail
  • scope_deregister_llm_stream_execution_intercept
  • scope_deregister_tool_conditional_execution_guardrail
  • scope_deregister_tool_execution_intercept
  • scope_deregister_tool_request_intercept
  • scope_deregister_tool_sanitize_request_guardrail
  • scope_deregister_tool_sanitize_response_guardrail
  • scope_register_llm_conditional_execution_guardrail
  • scope_register_llm_execution_intercept
  • scope_register_llm_request_intercept
  • scope_register_llm_sanitize_request_guardrail
  • scope_register_llm_sanitize_response_guardrail
  • scope_register_llm_stream_execution_intercept
  • scope_register_tool_conditional_execution_guardrail
  • scope_register_tool_execution_intercept
  • scope_register_tool_request_intercept
  • scope_register_tool_sanitize_request_guardrail
  • scope_register_tool_sanitize_response_guardrail
  • runtime
  • callbacks
  • EventSubscriberFn
  • LlmCollectorFn
  • LlmConditionalFn
  • LlmExecutionFn
  • LlmExecutionNextFn
  • LlmFinalizerFn
  • LlmJsonStream
  • LlmRequestInterceptFn
  • LlmSanitizeRequestFn
  • LlmSanitizeResponseFn
  • LlmStreamExecutionFn
  • LlmStreamExecutionNextFn
  • ToolConditionalFn
  • ToolExecutionFn
  • ToolExecutionNextFn
  • ToolInterceptFn
  • ToolSanitizeFn
  • global
  • global_context
  • scope_stack
  • ScopeStack
  • ThreadScopeStackBinding
  • TASK_SCOPE_STACK
  • capture_thread_scope_stack
  • create_scope_stack
  • current_scope_stack
  • propagate_scope_to_thread
  • restore_thread_scope_stack
  • scope_stack_active
  • set_thread_scope_stack
  • sync_thread_scope_stack
  • task_scope_push
  • task_scope_remove
  • task_scope_top
  • with_scope_stack
  • ScopeStackHandle
  • state
  • NemoRelayContextState
  • subscriber_dispatcher
  • flush_subscribers
  • scope
  • CreateScopeHandleParams
  • EmitMarkEventParams
  • EndScopeHandleParams
  • PopScopeParams
  • PushScopeParams
  • ScopeAttributes
  • ScopeHandle
  • HandleAttributes
  • ScopeType
  • event
  • get_handle
  • pop_scope
  • push_scope
  • subscriber
  • deregister_subscriber
  • flush_subscribers
  • register_subscriber
  • scope_deregister_subscriber
  • scope_register_subscriber
  • tool
  • CreateToolHandleParams
  • EndToolHandleParams
  • ToolAttributes
  • ToolCallEndParams
  • ToolCallExecuteParams
  • ToolCallParams
  • ToolExecutionInterceptOutcome
  • ToolHandle
  • tool_call
  • tool_call_end
  • tool_call_execute
  • tool_conditional_execution
  • tool_request_intercepts
  • codec
  • anthropic
  • AnthropicMessagesCodec
  • AnthropicMessagesStreamingCodec
  • model_pricing
  • CacheReadAccounting
  • ModelPricing
  • PricingCatalog
  • PricingCatalogError
  • PricingConfig
  • PricingResolver
  • TokenRateTier
  • PricingSource
  • PricingSourceConfig
  • PricingUnit
  • PromptCachePricing
  • TokenPricingRates
  • RateScheduleApplication
  • active_pricing_resolver
  • TokenRateSchedule
  • attach_estimated_cost
  • attach_estimated_cost_for_provider
  • estimate_cost
  • estimate_cost_for_provider
  • estimate_cost_with_catalog
  • estimate_cost_with_provider
  • infer_model_provider
  • pricing_for_model
  • pricing_for_provider
  • reset_active_pricing_resolver
  • set_active_pricing_resolver
  • openai_chat
  • OpenAIChatCodec
  • OpenAIChatStreamingCodec
  • openai_responses
  • OpenAIResponsesCodec
  • OpenAIResponsesStreamingCodec
  • request
  • AnnotatedLlmRequest
  • FunctionCall
  • FunctionDefinition
  • GenerationParams
  • OpenAiImageUrl
  • ToolCall
  • ToolChoiceFunction
  • ToolChoiceFunctionName
  • ToolDefinition
  • ContentPart
  • Message
  • MessageContent
  • ToolChoice
  • resolve
  • ProviderSurface
  • detect_request_surface
  • detect_request_surface_with_hint
  • detect_response_surface
  • normalize_request
  • normalize_request_with_hint
  • normalize_response
  • response
  • AnnotatedLlmResponse
  • CostEstimate
  • ResponseToolCall
  • Usage
  • ApiSpecificResponse
  • CostSource
  • FinishReason
  • streaming
  • SseEvent
  • SseEventDecoder
  • StreamingCodec
  • traits
  • LlmCodec
  • LlmResponseCodec
  • config_editor
  • EditorFieldSpec
  • EditorSchema
  • EditorFieldKind
  • EditorConfig
  • error
  • FlowError
  • Result
  • json
  • merge_json
  • Json
  • observability
  • atif
  • AtifAgentInfo
  • AtifAncestry
  • AtifExporter
  • AtifFinalMetrics
  • AtifInvocationInfo
  • AtifMetrics
  • AtifObservation
  • AtifObservationResult
  • AtifStep
  • AtifStepExtra
  • AtifSubagentTrajectoryRef
  • AtifToolCall
  • AtifTrajectory
  • ATIF_SCHEMA_VERSION
  • atof
  • AtofEndpointConfig
  • AtofExporter
  • AtofExporterConfig
  • AtofEndpointFieldNamePolicy
  • AtofEndpointTransport
  • AtofExporterError
  • AtofExporterMode
  • Result
  • openinference
  • OpenInferenceConfig
  • OpenInferenceSubscriber
  • OpenInferenceError
  • OtlpTransport
  • Result
  • otel
  • OpenTelemetryConfig
  • OpenTelemetrySubscriber
  • OpenTelemetryError
  • OtlpTransport
  • Result
  • plugin_component
  • AtifSectionConfig
  • AtofEndpointSectionConfig
  • AtofSectionConfig
  • ComponentSpec
  • HttpStorageConfig
  • ObservabilityConfig
  • OtlpSectionConfig
  • S3StorageConfig
  • AtifStorageConfig
  • OBSERVABILITY_PLUGIN_KIND
  • deregister_observability_component
  • register_observability_component
  • plugin
  • dynamic
  • DynamicPluginFailure
  • DynamicPluginManifest
  • DynamicPluginManifestCapabilities
  • DynamicPluginManifestCompat
  • DynamicPluginManifestConfigSchema
  • DynamicPluginManifestDefaults
  • DynamicPluginManifestIntegrity
  • DynamicPluginManifestPlugin
  • DynamicPluginManifestRustDynamicLoad
  • DynamicPluginManifestSource
  • DynamicPluginManifestWorkerLoad
  • DynamicPluginMetadata
  • DynamicPluginRecord
  • DynamicPluginRegistry
  • DynamicPluginRuntimeStatus
  • DynamicPluginRustCompatibility
  • DynamicPluginRustLoadContract
  • DynamicPluginSource
  • DynamicPluginSpec
  • DynamicPluginStatus
  • DynamicPluginValidationStatus
  • DynamicPluginWorkerCompatibility
  • DynamicPluginWorkerLoadContract
  • NativePluginActivation
  • NativePluginLoadSpec
  • DynamicPluginAttestationMode
  • DynamicPluginCapability
  • DynamicPluginCheckState
  • DynamicPluginCompatibility
  • DynamicPluginFailurePhase
  • DynamicPluginKind
  • DynamicPluginLoadContract
  • DynamicPluginManifestLoad
  • DynamicPluginRuntimeState
  • DynamicPluginStartupClass
  • WorkerRuntime
  • DYNAMIC_PLUGIN_MANIFEST_FILENAME
  • load_native_plugins
  • DynamicPluginId
  • ConfigDiagnostic
  • ConfigPolicy
  • ConfigReport
  • PluginComponentSpec
  • PluginConfig
  • PluginRegistration
  • PluginRegistrationContext
  • DiagnosticLevel
  • PluginError
  • UnsupportedBehavior
  • Plugin
  • active_plugin_report
  • clear_plugin_configuration
  • deregister_plugin
  • ensure_builtin_plugins_registered
  • initialize_plugins
  • list_plugin_kinds
  • lookup_plugin
  • register_plugin
  • rollback_registrations
  • validate_plugin_config
  • Result
  • plugins
  • model_pricing
  • PRICING_PLUGIN_KIND
  • register_pricing_component
  • nemo_guardrails
  • component
  • ComponentSpec
  • LocalBackendConfig
  • NeMoGuardrailsConfig
  • RemoteBackendConfig
  • RequestDefaultsConfig
  • RequestRailsConfig
  • RailSelector
  • NEMO_GUARDRAILS_PLUGIN_KIND
  • deregister_nemo_guardrails_component
  • register_nemo_guardrails_component
  • stream
  • LlmStreamWrapper
  • editor_config
  • nemo-relay-adaptive
  • acg
  • anthropic_plugin
  • AnthropicCachePlugin
  • canonicalize
  • canonicalize_json
  • canonicalize_value
  • normalize_whitespace
  • sha256_hex
  • capability
  • BackendCapabilities
  • CacheEconomics
  • CapabilityRegistry
  • ModelFamilyCapabilities
  • ProviderFeature
  • ir_builder
  • build_prompt_ir
  • openai_plugin
  • OpenAICachePlugin
  • passthrough
  • PassthroughPlugin
  • plugin
  • PluginInput
  • PluginOutput
  • ProviderPlugin
  • plugin_registry
  • PluginRegistry
  • policy
  • ArchetypeRoutingOverride
  • CachePolicy
  • PolicyEnvelope
  • RetentionPolicy
  • RewritePolicy
  • RoutingPolicy
  • ScopeRetentionOverride
  • TransformationClass
  • profile
  • BehavioralProfile
  • BlockStabilityScore
  • DistributionSummary
  • ParallelismPattern
  • ToolUsagePhase
  • SessionArchetype
  • StabilityClass
  • prompt_ir
  • PromptBlock
  • PromptIR
  • SpanId
  • TokenizationMetadata
  • ToolSchemaHash
  • BlockContentType
  • PromptRole
  • ProvenanceLabel
  • SensitivityLabel
  • retention
  • RetentionThresholds
  • generate_retention_intent
  • generate_retention_intent_default
  • stability
  • StabilityAnalysisResult
  • StabilityThresholds
  • analyze_stability
  • find_stable_prefix_length
  • telemetry
  • CacheHitRate
  • CacheMissDiagnosis
  • CacheRequestFacts
  • CacheTelemetryEvent
  • CacheMissEvidence
  • CacheMissReason
  • CacheTelemetryProvider
  • types
  • AgentIdentity
  • CacheStabilityIntent
  • CompressionIntent
  • ContentExtractionIntent
  • IntentOutcome
  • ModelRoutingIntent
  • OptimizationIntentBundle
  • PlacementIntent
  • PriorityIntent
  • RetentionIntent
  • SerializationIntent
  • ToolScopeIntent
  • TranslationReport
  • IntentType
  • ModelClass
  • OptimizationIntent
  • PlacementTarget
  • ReasonCode
  • RetentionTier
  • SharingScope
  • TranslationStatus
  • variable_extractor
  • ExtractedVariable
  • ExtractionResult
  • VariablePattern
  • VariableCategory
  • default_variable_patterns
  • extract_variables
  • extract_variables_from_blocks
  • error
  • AcgError
  • Result
  • MIN_ACG_OBSERVATIONS
  • acg_component
  • acg_learner
  • AcgLearner
  • acg_profile
  • adaptive_hints_intercept
  • AdaptiveHintsIntercept
  • cache_diagnostics
  • CacheDiagnosticsTracker
  • build_cache_request_facts
  • config
  • AcgComponentConfig
  • AdaptiveConfig
  • AdaptiveHintsComponentConfig
  • BackendSpec
  • StateConfig
  • TelemetryComponentConfig
  • ToolParallelismComponentConfig
  • context_helpers
  • SharedParentScopeIdentity
  • LATENCY_SENSITIVITY_POINTER
  • extract_scope_path
  • read_manual_latency_sensitivity
  • resolve_agent_id
  • resolve_shared_parent_scope_identity
  • set_latency_sensitivity
  • drain
  • error
  • AdaptiveError
  • Result
  • intercepts
  • AGENT_HINTS_HEADER_KEY
  • learner
  • latency
  • LatencySensitivityLearner
  • compute_default_hints
  • traits
  • Learner
  • plugin_component
  • ComponentSpec
  • ADAPTIVE_PLUGIN_KIND
  • deregister_adaptive_component
  • register_adaptive_component
  • redis
  • RedisBackend
  • storage
  • erased
  • AnyBackend
  • memory
  • InMemoryBackend
  • traits
  • StorageBackend
  • StorageBackendDyn
  • subscriber
  • tool_parallelism_learner
  • ToolParallelismLearner
  • trie
  • accumulator
  • AccumulatorState
  • NodeAccumulators
  • RunningStats
  • builder
  • PredictionTrieBuilder
  • SensitivityConfig
  • data_models
  • LlmCallPrediction
  • PredictionMetrics
  • PredictionTrieNode
  • lookup
  • PredictionTrieLookup
  • serialization
  • TrieEnvelope
  • CURRENT_VERSION
  • types
  • cache
  • HotCache
  • metadata
  • AgentHints
  • MetadataEnvelope
  • ParallelHint
  • Json
  • plan
  • ExecutionPlan
  • ParallelGroup
  • records
  • CallRecord
  • RunRecord
  • CallKind
  • AdaptiveRuntime
  • nemo-relay-pii-redaction
  • component
  • BuiltinBackendConfig
  • ComponentSpec
  • LocalBackendConfig
  • PiiRedactionConfig
  • PII_REDACTION_PLUGIN_KIND
  • deregister_pii_redaction_component
  • register_pii_redaction_component
  • nemo-relay-ffi
  • api
  • FfiStream
  • nemo_relay_active_plugin_report_json
  • nemo_relay_adaptive_build_cache_telemetry_event
  • nemo_relay_adaptive_runtime_bind_scope
  • nemo_relay_adaptive_runtime_build_cache_request_facts
  • nemo_relay_adaptive_runtime_create
  • nemo_relay_adaptive_runtime_deregister
  • nemo_relay_adaptive_runtime_register
  • nemo_relay_adaptive_runtime_report_json
  • nemo_relay_adaptive_runtime_shutdown
  • nemo_relay_adaptive_runtime_wait_for_idle
  • nemo_relay_adaptive_set_latency_sensitivity
  • nemo_relay_adaptive_validate_config
  • nemo_relay_anthropic_messages_codec_new
  • nemo_relay_atif_exporter_clear
  • nemo_relay_atif_exporter_create
  • nemo_relay_atif_exporter_deregister
  • nemo_relay_atif_exporter_export
  • nemo_relay_atif_exporter_register
  • nemo_relay_atof_exporter_create
  • nemo_relay_atof_exporter_create_from_json
  • nemo_relay_atof_exporter_deregister
  • nemo_relay_atof_exporter_force_flush
  • nemo_relay_atof_exporter_path
  • nemo_relay_atof_exporter_register
  • nemo_relay_atof_exporter_shutdown
  • nemo_relay_clear_plugin_configuration
  • nemo_relay_deregister_llm_conditional_execution_guardrail
  • nemo_relay_deregister_llm_execution_intercept
  • nemo_relay_deregister_llm_request_intercept
  • nemo_relay_deregister_llm_sanitize_request_guardrail
  • nemo_relay_deregister_llm_sanitize_response_guardrail
  • nemo_relay_deregister_llm_stream_execution_intercept
  • nemo_relay_deregister_plugin
  • nemo_relay_deregister_subscriber
  • nemo_relay_deregister_tool_conditional_execution_guardrail
  • nemo_relay_deregister_tool_execution_intercept
  • nemo_relay_deregister_tool_request_intercept
  • nemo_relay_deregister_tool_sanitize_request_guardrail
  • nemo_relay_deregister_tool_sanitize_response_guardrail
  • nemo_relay_event
  • nemo_relay_flush_subscribers
  • nemo_relay_get_handle
  • nemo_relay_initialize_plugins
  • nemo_relay_list_plugin_kinds_json
  • nemo_relay_llm_call
  • nemo_relay_llm_call_end
  • nemo_relay_llm_call_execute
  • nemo_relay_llm_conditional_execution
  • nemo_relay_llm_request_intercept_outcome_json_new
  • nemo_relay_llm_request_intercepts
  • nemo_relay_llm_stream_call_execute
  • nemo_relay_observability_component_spec_json
  • nemo_relay_observability_default_config_json
  • nemo_relay_observability_plugin_kind
  • nemo_relay_openai_chat_codec_new
  • nemo_relay_openai_responses_codec_new
  • nemo_relay_openinference_subscriber_create
  • nemo_relay_openinference_subscriber_deregister
  • nemo_relay_openinference_subscriber_force_flush
  • nemo_relay_openinference_subscriber_register
  • nemo_relay_openinference_subscriber_shutdown
  • nemo_relay_otel_subscriber_create
  • nemo_relay_otel_subscriber_deregister
  • nemo_relay_otel_subscriber_force_flush
  • nemo_relay_otel_subscriber_register
  • nemo_relay_otel_subscriber_shutdown
  • nemo_relay_plugin_context_register_llm_conditional_execution_guardrail
  • nemo_relay_plugin_context_register_llm_execution_intercept
  • nemo_relay_plugin_context_register_llm_request_intercept
  • nemo_relay_plugin_context_register_llm_sanitize_request_guardrail
  • nemo_relay_plugin_context_register_llm_sanitize_response_guardrail
  • nemo_relay_plugin_context_register_llm_stream_execution_intercept
  • nemo_relay_plugin_context_register_subscriber
  • nemo_relay_plugin_context_register_tool_conditional_execution_guardrail
  • nemo_relay_plugin_context_register_tool_execution_intercept
  • nemo_relay_plugin_context_register_tool_request_intercept
  • nemo_relay_plugin_context_register_tool_sanitize_request_guardrail
  • nemo_relay_plugin_context_register_tool_sanitize_response_guardrail
  • nemo_relay_pop_scope
  • nemo_relay_push_scope
  • nemo_relay_register_llm_conditional_execution_guardrail
  • nemo_relay_register_llm_execution_intercept
  • nemo_relay_register_llm_request_intercept
  • nemo_relay_register_llm_sanitize_request_guardrail
  • nemo_relay_register_llm_sanitize_response_guardrail
  • nemo_relay_register_llm_stream_execution_intercept
  • nemo_relay_register_plugin
  • nemo_relay_register_subscriber
  • nemo_relay_register_tool_conditional_execution_guardrail
  • nemo_relay_register_tool_execution_intercept
  • nemo_relay_register_tool_request_intercept
  • nemo_relay_register_tool_sanitize_request_guardrail
  • nemo_relay_register_tool_sanitize_response_guardrail
  • nemo_relay_scope_deregister_llm_conditional_execution_guardrail
  • nemo_relay_scope_deregister_llm_execution_intercept
  • nemo_relay_scope_deregister_llm_request_intercept
  • nemo_relay_scope_deregister_llm_sanitize_request_guardrail
  • nemo_relay_scope_deregister_llm_sanitize_response_guardrail
  • nemo_relay_scope_deregister_llm_stream_execution_intercept
  • nemo_relay_scope_deregister_subscriber
  • nemo_relay_scope_deregister_tool_conditional_execution_guardrail
  • nemo_relay_scope_deregister_tool_execution_intercept
  • nemo_relay_scope_deregister_tool_request_intercept
  • nemo_relay_scope_deregister_tool_sanitize_request_guardrail
  • nemo_relay_scope_deregister_tool_sanitize_response_guardrail
  • nemo_relay_scope_register_llm_conditional_execution_guardrail
  • nemo_relay_scope_register_llm_execution_intercept
  • nemo_relay_scope_register_llm_request_intercept
  • nemo_relay_scope_register_llm_sanitize_request_guardrail
  • nemo_relay_scope_register_llm_sanitize_response_guardrail
  • nemo_relay_scope_register_llm_stream_execution_intercept
  • nemo_relay_scope_register_subscriber
  • nemo_relay_scope_register_tool_conditional_execution_guardrail
  • nemo_relay_scope_register_tool_execution_intercept
  • nemo_relay_scope_register_tool_request_intercept
  • nemo_relay_scope_register_tool_sanitize_request_guardrail
  • nemo_relay_scope_register_tool_sanitize_response_guardrail
  • nemo_relay_scope_stack_active
  • nemo_relay_scope_stack_capture_thread
  • nemo_relay_scope_stack_create
  • nemo_relay_scope_stack_restore_thread
  • nemo_relay_scope_stack_set_thread
  • nemo_relay_stream_free
  • nemo_relay_stream_next
  • nemo_relay_tool_call
  • nemo_relay_tool_call_end
  • nemo_relay_tool_call_execute
  • nemo_relay_tool_conditional_execution
  • nemo_relay_tool_request_intercepts
  • nemo_relay_validate_plugin_config
  • callable
  • wrap_codec_fn
  • wrap_collector_fn
  • wrap_event_subscriber
  • wrap_finalizer_fn
  • wrap_json_fn
  • wrap_llm_conditional_fn
  • wrap_llm_exec_fn
  • wrap_llm_exec_intercept_fn
  • wrap_llm_request_intercept_fn
  • wrap_llm_response_fn
  • wrap_llm_sanitize_request_fn
  • wrap_llm_stream_exec_fn
  • wrap_llm_stream_exec_intercept_fn
  • wrap_tool_conditional_fn
  • wrap_tool_exec_fn
  • wrap_tool_exec_intercept_fn
  • wrap_tool_request_intercept_fn
  • wrap_tool_sanitize_fn
  • NemoRelayCodecDecodeCb
  • NemoRelayCodecDecodeFn
  • NemoRelayCodecEncodeCb
  • NemoRelayCodecEncodeFn
  • NemoRelayCollectorCb
  • NemoRelayEventSubscriberCb
  • NemoRelayFinalizerCb
  • NemoRelayFreeFn
  • NemoRelayJsonCb
  • NemoRelayLlmConditionalCb
  • NemoRelayLlmExecCb
  • NemoRelayLlmExecInterceptCb
  • NemoRelayLlmExecNextFn
  • NemoRelayLlmRequestCb
  • NemoRelayLlmRequestInterceptCb
  • NemoRelayPluginRegisterCb
  • NemoRelayPluginValidateCb
  • NemoRelayToolConditionalCb
  • NemoRelayToolExecCb
  • NemoRelayToolExecInterceptCb
  • NemoRelayToolExecNextFn
  • NemoRelayToolSanitizeCb
  • convert
  • c_str_to_json
  • c_str_to_opt_json
  • c_str_to_string
  • json_to_c_string
  • nemo_relay_string_free
  • str_to_c_string
  • unix_micros_to_opt_timestamp
  • error
  • NemoRelayStatus
  • clear_last_error
  • last_error_message
  • nemo_relay_last_error
  • nemo_relay_set_last_error_message
  • set_last_error
  • status_from_error
  • status_from_plugin_error
  • types
  • FfiAdaptiveRuntime
  • FfiAtifExporter
  • FfiAtofExporter
  • FfiCodecHandle
  • FfiEvent
  • FfiLLMHandle
  • FfiLLMRequest
  • FfiOpenInferenceSubscriber
  • FfiOpenTelemetrySubscriber
  • FfiPluginContext
  • FfiScopeHandle
  • FfiScopeStack
  • FfiThreadScopeStackBinding
  • FfiToolHandle
  • NemoRelayScopeType
  • nemo_relay_adaptive_runtime_free
  • nemo_relay_atif_exporter_free
  • nemo_relay_atof_exporter_free
  • nemo_relay_codec_free
  • nemo_relay_event_annotated_request
  • nemo_relay_event_annotated_response
  • nemo_relay_event_atof_version
  • nemo_relay_event_attributes
  • nemo_relay_event_attributes_json
  • nemo_relay_event_category
  • nemo_relay_event_category_profile
  • nemo_relay_event_data
  • nemo_relay_event_data_schema
  • nemo_relay_event_free
  • nemo_relay_event_input
  • nemo_relay_event_json
  • nemo_relay_event_kind
  • nemo_relay_event_metadata
  • nemo_relay_event_model_name
  • nemo_relay_event_name
  • nemo_relay_event_output
  • nemo_relay_event_parent_uuid
  • nemo_relay_event_scope_category
  • nemo_relay_event_scope_type
  • nemo_relay_event_timestamp
  • nemo_relay_event_tool_call_id
  • nemo_relay_event_uuid
  • nemo_relay_llm_handle_attributes
  • nemo_relay_llm_handle_free
  • nemo_relay_llm_handle_name
  • nemo_relay_llm_handle_parent_uuid
  • nemo_relay_llm_handle_uuid
  • nemo_relay_llm_request_content
  • nemo_relay_llm_request_free
  • nemo_relay_llm_request_headers
  • nemo_relay_llm_request_new
  • nemo_relay_openinference_subscriber_free
  • nemo_relay_otel_subscriber_free
  • nemo_relay_scope_handle_attributes
  • nemo_relay_scope_handle_data
  • nemo_relay_scope_handle_free
  • nemo_relay_scope_handle_metadata
  • nemo_relay_scope_handle_name
  • nemo_relay_scope_handle_parent_uuid
  • nemo_relay_scope_handle_scope_type
  • nemo_relay_scope_handle_uuid
  • nemo_relay_scope_stack_free
  • nemo_relay_tool_handle_attributes
  • nemo_relay_tool_handle_free
  • nemo_relay_tool_handle_name
  • nemo_relay_tool_handle_parent_uuid
  • nemo_relay_tool_handle_uuid
  • nemo-relay-types
  • api
  • event
  • BaseEvent
  • CategoryProfile
  • DataSchema
  • EventCategory
  • MarkEvent
  • PendingMarkSpec
  • ScopeEvent
  • Event
  • ScopeCategory
  • ATOF_VERSION
  • attributes_from_handle
  • llm_attributes_to_strings
  • scope_attributes_to_strings
  • tool_attributes_to_strings
  • llm
  • LlmAttributes
  • LlmRequest
  • LlmRequestInterceptOutcome
  • scope
  • ScopeAttributes
  • HandleAttributes
  • ScopeType
  • tool
  • ToolAttributes
  • ToolExecutionInterceptOutcome
  • codec
  • request
  • AnnotatedLlmRequest
  • FunctionCall
  • FunctionDefinition
  • GenerationParams
  • OpenAiImageUrl
  • ToolCall
  • ToolChoiceFunction
  • ToolChoiceFunctionName
  • ToolDefinition
  • ContentPart
  • Message
  • MessageContent
  • ToolChoice
  • response
  • AnnotatedLlmResponse
  • CostEstimate
  • ResponseToolCall
  • Usage
  • ApiSpecificResponse
  • CostSource
  • FinishReason
  • plugin
  • ConfigDiagnostic
  • DiagnosticLevel
  • Json
  • nemo-relay-plugin
  • nemo_relay_plugin
  • AnnotatedLlmRequest
  • AnnotatedLlmResponse
  • CategoryProfile
  • ConfigDiagnostic
  • EventCategory
  • LlmAttributes
  • LlmNext
  • LlmRequest
  • LlmRequestInterceptOutcome
  • LlmStream
  • LlmStreamNext
  • NemoRelayNativeHostApiV1
  • NemoRelayNativeLlmStreamV1
  • NemoRelayNativePluginContext
  • NemoRelayNativePluginV1
  • NemoRelayNativeScopeHandle
  • NemoRelayNativeScopeStack
  • NemoRelayNativeScopeStackBinding
  • NemoRelayNativeString
  • PendingMarkSpec
  • PluginContext
  • PluginRuntime
  • ScopeAttributes
  • ScopeGuard
  • ScopeHandle
  • ScopeStack
  • ScopeStackBinding
  • ThreadScopeStackGuard
  • ToolAttributes
  • ToolExecutionInterceptOutcome
  • ToolNext
  • DiagnosticLevel
  • Event
  • HandleAttributes
  • NemoRelayNativeScopeType
  • NemoRelayStatus
  • ScopeCategory
  • ScopeType
  • NEMO_RELAY_NATIVE_ABI_VERSION
  • NativePlugin
  • capture_scope_stack_thread
  • create_scope_stack
  • current_scope
  • emit_mark
  • export_plugin
  • pop_scope
  • push_scope
  • Json
  • LlmJsonStream
  • NemoRelayNativeEventSubscriberCb
  • NemoRelayNativeFreeFn
  • NemoRelayNativeJsonCb
  • NemoRelayNativeLlmConditionalCb
  • NemoRelayNativeLlmExecutionCb
  • NemoRelayNativeLlmNextFn
  • NemoRelayNativeLlmRequestCb
  • NemoRelayNativeLlmRequestInterceptCb
  • NemoRelayNativeLlmStreamCancelFn
  • NemoRelayNativeLlmStreamDropFn
  • NemoRelayNativeLlmStreamExecutionCb
  • NemoRelayNativeLlmStreamNextFn
  • NemoRelayNativeLlmStreamPollFn
  • NemoRelayNativePluginDropFn
  • NemoRelayNativePluginEntry
  • NemoRelayNativePluginRegisterFn
  • NemoRelayNativePluginValidateFn
  • NemoRelayNativeToolConditionalCb
  • NemoRelayNativeToolExecutionCb
  • NemoRelayNativeToolJsonCb
  • NemoRelayNativeToolNextFn
  • NemoRelayNativeWithScopeStackCb
  • Result
  • nemo-relay-worker-proto
  • v1
  • invoke_request
  • Payload
  • invoke_response
  • Result
  • plugin_worker_client
  • PluginWorkerClient
  • plugin_worker_server
  • PluginWorkerServer
  • SERVICE_NAME
  • PluginWorker
  • relay_host_runtime_client
  • RelayHostRuntimeClient
  • relay_host_runtime_server
  • RelayHostRuntimeServer
  • SERVICE_NAME
  • RelayHostRuntime
  • stream_chunk
  • Item
  • CancelInvocationRequest
  • CreateScopeStackRequest
  • CreateScopeStackResponse
  • DropScopeStackRequest
  • EmitMarkRequest
  • EmptyResult
  • GuardrailResult
  • HandshakeRequest
  • HandshakeResponse
  • HealthRequest
  • HealthResponse
  • HostAck
  • InvokeRequest
  • InvokeResponse
  • JsonEnvelope
  • JsonResult
  • LlmInvocation
  • LlmNextRequest
  • LlmRequestInterceptResult
  • LlmStreamNextRequest
  • PopScopeRequest
  • PushScopeRequest
  • PushScopeResponse
  • RegisterRequest
  • RegisterResponse
  • Registration
  • ScopeContext
  • ShutdownRequest
  • StreamChunk
  • ToolExecutionInterceptResult
  • ToolInvocation
  • ToolNextRequest
  • ValidateRequest
  • ValidateResponse
  • WorkerAck
  • WorkerError
  • RegistrationSurface
  • ScopeType
  • WORKER_PROTOCOL_GRPC_V1
  • decode_json_envelope
  • json_envelope
  • nemo-relay-worker
  • ConfigDiagnostic
  • LlmNext
  • LlmRequest
  • LlmRequestInterceptOutcome
  • LlmStreamNext
  • PendingMarkSpec
  • PluginContext
  • PluginRuntime
  • ToolExecutionInterceptOutcome
  • ToolNext
  • WorkerServerConfig
  • DiagnosticLevel
  • Event
  • ScopeType
  • WorkerSdkError
  • WorkerPlugin
  • serve_plugin
  • serve_plugin_arc
  • serve_plugin_arc_with_config
  • BoxFutureResult
  • Json
  • JsonStream
  • Result
  • Performance
  • LLM Request Intercept Outcomes
  • Tool Execution Intercept Outcomes
  • Support and FAQs
  • Glossary
  • Troubleshooting Guide
  • Trace Incident Runbook
  • Community
  • Legal
  • Open Source Software (OSS)
  • License Agreement
On this page
  • Packages
ReferenceAPIs

Python Library Reference

||View as Markdown|

Packages

  • nemo-relay: Python runtime SDK for NeMo Relay.
  • nemo-relay-plugin: Python SDK for NeMo Relay dynamic worker plugins.
Previous

APIs

Next

nemo_relay

NVIDIANVIDIA
Developer-friendly docs for your API
Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.