Is this page helpful?

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions#

Module Contents#

Functions#

_get_field

Read a field from dict-like or object-like values.

_normalize_tool_calls

Normalize tool calls to OpenAI-compatible JSON primitives.

_coerce_arguments_mapping

Coerce function.arguments to a mapping for HF/Jinja chat templates.

_sanitize_messages_for_template

Prepare messages so tokenizer chat templates can safely consume them.

_sanitize_tools_for_template

Ensure tools payload is template-safe and has mapping parameters.

_replace_prefix_tokens

Replace the token ids that are associated with the previous turn with the actual tokens from the previous generation (rather than the ones from the chat template application).

Data#

logger

API#

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions.logger#

‘getLogger(…)’

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._get_field(obj, key, default=None)#

Read a field from dict-like or object-like values.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._normalize_tool_calls(tool_calls)#

Normalize tool calls to OpenAI-compatible JSON primitives.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._coerce_arguments_mapping(arguments)#

Coerce function.arguments to a mapping for HF/Jinja chat templates.

Examples:

  • {“x”: 1} -> {“x”: 1}

  • ‘{“x”: 1}’ -> {“x”: 1}

  • “[1, 2]” -> {} # JSON parses, but not a mapping

  • “not-json” -> {}

  • None -> {}

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._sanitize_messages_for_template(messages)#

Prepare messages so tokenizer chat templates can safely consume them.

This only normalizes tool-call argument payloads inside each message:

  • messages[].tool_calls[].function.arguments is coerced to a dict.

Example transformation: Input: [{“role”: “assistant”, “tool_calls”: [{“function”: {“name”: “f”, “arguments”: “{“x”: 1}”}}]}] Output: [{“role”: “assistant”, “tool_calls”: [{“function”: {“name”: “f”, “arguments”: {“x”: 1}}}]}]

Another example:

  • arguments: “[1,2,3]” -> arguments: {}

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._sanitize_tools_for_template(tools)#

Ensure tools payload is template-safe and has mapping parameters.

Example transformations:

  • {“function”: {“name”: “f”, “parameters”: “not-a-dict”}} -> {“function”: {“name”: “f”, “parameters”: {“type”: “object”, “properties”: {}}}}

  • non-dict tool entries are dropped.

  • non-list input returns None.

core.inference.text_generation_server.dynamic_text_gen_server.endpoints.chat_completions._replace_prefix_tokens(
eos_token_id,
previous_turn_token_ids,
retokeenized_previous_turn_token_ids,
current_turn_token_ids,
)#

Replace the token ids that are associated with the previous turn with the actual tokens from the previous generation (rather than the ones from the chat template application).