nemoguardrails.llm.call | NVIDIA NeMo Guardrails Library Developer Guide

The shared entry point for invoking an LLM model.

llm_call wraps an :class:~nemoguardrails.types.LLMModel with the project’s prompt logging, token accounting, reasoning-trace extraction, tool-call capture, and error wrapping, so every caller gets the same observability and the same :class:~nemoguardrails.exceptions.LLMCallException contract.

This module deliberately carries no Colang dependency. Built-in rail actions in nemoguardrails.library call llm_call, and those actions must stay runnable by engines that have no Colang runtime; tests/llm/test_call_import_graph.py enforces it.

Neighbouring modules: text helpers for interpreting a completion live in :mod:nemoguardrails.llm.completion_parsing, and the Colang-specific history and event helpers in :mod:nemoguardrails.actions.llm.utils.

Module Contents

Functions

Name	Description
`_ensure_chat_messages`	-
`_extract_and_remove_think_tags`	Extract reasoning from <think> tags and remove them from `response.content`.
`_extract_chunk_metadata`	-
`_extract_content`	Extract text content from response.
`_extract_http_status`	Extract an HTTP status code from a provider exception, if present.
`_log_completion`	-
`_log_prompt`	Log the prompt to LLM call info.
`_prepend_think_tags`	Re-attach `reasoning_content` to `content` as a leading <think> block, the inverse of `_extract_and_remove_think_tags`.
`_prompt_message_content`	Return a message’s textual content, or "" when it is absent or non-textual.
`_prompt_message_role`	Return a message’s role, accepting either a raw dict or a `ChatMessage`.
`_raise_llm_call_exception`	-
`_setup_llm_call_info`	Initialize or update LLM call info in context.
`_store_reasoning_traces`	-
`_store_request_id`	Record the provider’s response id on the current call, when it returned one.
`_store_response_metadata`	-
`_store_tool_calls`	-
`_stream_llm_call`	-
`_update_token_stats`	-
`_update_token_stats_from_chunk`	-
`llm_call`	-
`warn_if_truncated`	Return True and emit a warning if the LLM produced no visible content because it hit the max_tokens budget.

Data

logger

API

nemoguardrails.llm.call._ensure_chat_messages(
    prompt: typing.Union[str, list]
) -> typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]]

nemoguardrails.llm.call._extract_and_remove_think_tags(
    response: nemoguardrails.types.LLMResponse
) -> typing.Optional[str]

Extract reasoning from <think> tags and remove them from response.content.

This function looks for <think>…</think> tags in the response content, and if found, extracts the reasoning content inside the tags. It has a side-effect: it removes the full reasoning trace and tags from response.content.

Parameters:

response

LLMResponse

The LLM response object

Returns: Optional[str]

The extracted reasoning content, or None if no <think> tags found

nemoguardrails.llm.call._extract_chunk_metadata(
    chunk: nemoguardrails.types.LLMResponseChunk
) -> typing.Optional[typing.Dict[str, typing.Any]]

nemoguardrails.llm.call._extract_content(
    response: nemoguardrails.types.LLMResponse
) -> str

Extract text content from response.

nemoguardrails.llm.call._extract_http_status(
    exception: BaseException
) -> typing.Optional[int]

Extract an HTTP status code from a provider exception, if present.

Checks, in order:

LLMClientError.status_code (NeMo Guardrails client layer).
exception.status_code (OpenAI SDK, httpx).
exception.response.status_code (requests-style wrappers).

Returns None when no status can be determined or when the status is 0 (used by LLMTimeoutError / LLMConnectionError for client-side failures where no HTTP response was received).

nemoguardrails.llm.call._log_completion(
    response: nemoguardrails.types.LLMResponse
) -> None

nemoguardrails.llm.call._log_prompt(
    prompt: typing.Union[str, typing.List[dict], typing.List[nemoguardrails.types.ChatMessage]]
) -> None

Log the prompt to LLM call info.

Accepts the normalized ChatMessage form as well as raw dicts, because llm_call logs the prompt before _ensure_chat_messages runs and callers may already supply ChatMessage objects.

nemoguardrails.llm.call._prepend_think_tags(
    content: str,
    reasoning_content: typing.Optional[str]
) -> str

Re-attach reasoning_content to content as a leading <think> block, the inverse of _extract_and_remove_think_tags.

nemoguardrails.llm.call._prompt_message_content(
    message: typing.Union[dict, nemoguardrails.types.ChatMessage]
) -> str

Return a message’s textual content, or "" when it is absent or non-textual.

Multimodal payloads carry a list of content parts rather than a string; those are omitted from the logged prompt, matching the previous dict-only behavior.

nemoguardrails.llm.call._prompt_message_role(
    message: typing.Union[dict, nemoguardrails.types.ChatMessage]
) -> str

Return a message’s role, accepting either a raw dict or a ChatMessage.

Role is a str subclass, so it is returned as-is: it maps correctly through type_map and supports str methods. Passing it through str() would yield "Role.USER" because Enum.__str__ wins over the str mixin.

nemoguardrails.llm.call._raise_llm_call_exception(
    exception: Exception,
    model: nemoguardrails.types.LLMModel
) -> typing.NoReturn

nemoguardrails.llm.call._setup_llm_call_info(
    model: nemoguardrails.types.LLMModel,
    model_name: typing.Optional[str],
    model_provider: typing.Optional[str]
) -> None

Initialize or update LLM call info in context.

nemoguardrails.llm.call._store_reasoning_traces(
    response: nemoguardrails.types.LLMResponse
) -> None

nemoguardrails.llm.call._store_request_id(
    response: nemoguardrails.types.LLMResponse
) -> None

Record the provider’s response id on the current call, when it returned one.

Kept separate from LLMCallInfo.id, which track_llm_call generates client-side: only this value can be quoted to a provider, and only this value matches gen_ai.response.id on the OTEL span for the same call, so overloading one field with both meanings would make log-to-trace correlation unreliable.

nemoguardrails.llm.call._store_response_metadata(
    response: nemoguardrails.types.LLMResponse
) -> None

nemoguardrails.llm.call._store_tool_calls(
    response: nemoguardrails.types.LLMResponse
) -> None

nemoguardrails.llm.call._stream_llm_call(
    model: nemoguardrails.types.LLMModel,
    prompt: typing.Union[str, typing.List[nemoguardrails.types.ChatMessage]],
    handler: nemoguardrails.streaming.StreamingHandler,
    stop: typing.Optional[typing.List[str]],
    llm_params: typing.Optional[dict] = None
) -> nemoguardrails.types.LLMResponse

async

nemoguardrails.llm.call._update_token_stats(
    response: nemoguardrails.types.LLMResponse
) -> None

nemoguardrails.llm.call._update_token_stats_from_chunk(
    chunk: nemoguardrails.types.LLMResponseChunk
) -> None

nemoguardrails.llm.call.llm_call(
    llm: typing.Optional[typing.Any],
    prompt: typing.Union[str, typing.List[dict]],
    model_name: typing.Optional[str] = None,
    model_provider: typing.Optional[str] = None,
    stop: typing.Optional[typing.List[str]] = None,
    llm_params: typing.Optional[dict] = None,
    streaming_handler: typing.Optional[nemoguardrails.streaming.StreamingHandler] = None
) -> nemoguardrails.types.LLMResponse

async

nemoguardrails.llm.call.warn_if_truncated(
    response: nemoguardrails.types.LLMResponse,
    task: str
) -> bool

Return True and emit a warning if the LLM produced no visible content because it hit the max_tokens budget.

Reasoning models (OpenAI o-series, gpt-5, DeepSeek-R1, Gemini 2.5, Qwen QwQ, etc.) spend output tokens on internal reasoning before emitting visible text. A small max_tokens budget can be fully consumed by the reasoning phase, leaving empty content and finish_reason=“length”. The call succeeds silently and callers that only inspect response.content see nothing. Callers whose downstream parser does not fail safely on empty input (e.g. self_check_facts, whose parser inverts the result) should use the return value to take an explicit fail-safe branch.

nemoguardrails.llm.call.logger = logging.getLogger(__name__)