`core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions`#

Module Contents#

Functions#

`_is_int_list_like`	Return True for integer lists, including nested integer lists.
`_is_numeric_list_like`	Return True for numeric lists, including nested numeric lists.
`_redact_token_id_lists_for_logging`	Redact verbose token-id arrays from logs.
`_get_field`	Read a field from dict-like or object-like values.
`_try_parse_jsonish`
`_extract_declared_types`	Recursively extract declared JSON-schema type names.
`_get_tool_argument_schemas`	Build function-name to argument-schema mapping from request tools.
`_normalize_structured_tool_arguments`	Coerce structured (array/object) args from JSON strings to native types.
`_normalize_tool_calls`	Normalize tool calls to OpenAI-compatible JSON primitives.
`_apply_tool_call_guardrails`	Apply conservative post-parse guardrails to tool call lists.
`_normalize_assistant_content`	Normalize assistant content for policy-sensitive tool transitions.
`_coerce_arguments_mapping`	Coerce function.arguments to a mapping for HF/Jinja chat templates.
`_sanitize_messages_for_template`	Prepare messages so tokenizer chat templates can safely consume them.
`_sanitize_tools_for_template`	Ensure tools payload is template-safe and has mapping parameters.
`_reconstruct_reasoning_content`	Reconstruct tags from reasoning_content fields on assistant messages.
`_replace_prefix_tokens`	Replace the token ids that are associated with the previous turn with the actual tokens from the previous generation (rather than the ones from the chat template application).

Data#

`logger`
`_TOKEN_ID_FIELDS_TO_REDACT`
`_INDEX_FIELDS_TO_REDACT`
`_HASH_FIELDS_TO_REDACT`
`_NUMERIC_SERIES_FIELDS_TO_REDACT`
`_TRANSFER_TOOL_NAME`
`_TRANSFER_HOLD_MESSAGE`
`_RESERVATION_UPDATE_TOOLS`
`_RESERVATION_DESTRUCTIVE_TOOLS`

API#

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions.logger#: ‘getLogger(…)’

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._TOKEN_ID_FIELDS_TO_REDACT#: None

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._INDEX_FIELDS_TO_REDACT#: None

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._HASH_FIELDS_TO_REDACT#: None

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._NUMERIC_SERIES_FIELDS_TO_REDACT#: None

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._is_int_list_like(value)#: Return True for integer lists, including nested integer lists.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._is_numeric_list_like(value)#: Return True for numeric lists, including nested numeric lists.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._redact_token_id_lists_for_logging(value)#: Redact verbose token-id arrays from logs.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._get_field(obj, key, default=None)#: Read a field from dict-like or object-like values.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._TRANSFER_TOOL_NAME#: ‘transfer_to_human_agents’

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._TRANSFER_HOLD_MESSAGE#: ‘YOU ARE BEING TRANSFERRED TO A HUMAN AGENT. PLEASE HOLD ON.’

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._RESERVATION_UPDATE_TOOLS#: None

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._RESERVATION_DESTRUCTIVE_TOOLS#: None

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._try_parse_jsonish(value)#

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._extract_declared_types(schema)#: Recursively extract declared JSON-schema type names.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._get_tool_argument_schemas(tools)#: Build function-name to argument-schema mapping from request tools.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._normalize_structured_tool_arguments( arguments, function_name, tool_argument_schemas, )#: Coerce structured (array/object) args from JSON strings to native types.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._normalize_tool_calls(tool_calls, tools=None)#: Normalize tool calls to OpenAI-compatible JSON primitives.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._apply_tool_call_guardrails(tool_calls)#

Apply conservative post-parse guardrails to tool call lists.

If update-style reservation tools are already present in the same response, suppress cancel+book style calls to avoid destructive replanning patterns.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._normalize_assistant_content(message_text, tool_calls)#: Normalize assistant content for policy-sensitive tool transitions.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._coerce_arguments_mapping(arguments)#

Coerce function.arguments to a mapping for HF/Jinja chat templates.

Examples:

{“x”: 1} -> {“x”: 1}
‘{“x”: 1}’ -> {“x”: 1}
“[1, 2]” -> {} # JSON parses, but not a mapping
“not-json” -> {}
None -> {}

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._sanitize_messages_for_template(messages)#

Prepare messages so tokenizer chat templates can safely consume them.

This only normalizes tool-call argument payloads inside each message:

messages[].tool_calls[].function.arguments is coerced to a dict.

Example transformation: Input: [{“role”: “assistant”, “tool_calls”: [{“function”: {“name”: “f”, “arguments”: “{“x”: 1}”}}]}] Output: [{“role”: “assistant”, “tool_calls”: [{“function”: {“name”: “f”, “arguments”: {“x”: 1}}}]}]

Another example:

arguments: “[1,2,3]” -> arguments: {}

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._sanitize_tools_for_template(tools)#

Ensure tools payload is template-safe and has mapping parameters.

Example transformations:

{“function”: {“name”: “f”, “parameters”: “not-a-dict”}} -> {“function”: {“name”: “f”, “parameters”: {“type”: “object”, “properties”: {}}}}
non-dict tool entries are dropped.
non-list input returns None.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._reconstruct_reasoning_content(messages: list[dict]) → list[dict]#

Reconstruct tags from reasoning_content fields on assistant messages.

For parity with vLLM, assistant messages may carry reasoning in the reasoning_content field. Before applying the chat template, we must inline those tags back into content.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._replace_prefix_tokens( eos_token_id, previous_turn_token_ids, retokeenized_previous_turn_token_ids, current_turn_token_ids, )#: Replace the token ids that are associated with the previous turn with the actual tokens from the previous generation (rather than the ones from the chat template application).

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions#

Module Contents#

Functions#

Data#

API#

`core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions`#