LlmCallInfo#

class nemo_microservices.types.LlmCallInfo(*args: Any, **kwargs: Any)

Bases: BaseModel

id: str | None = None: The unique prompt identifier.

completion: str | None = None: The completion generated by the LLM.

completion_tokens: int | None = None: The number of output tokens.

duration: float | None = None: The duration in seconds.

finished_at: float | None = None: The timestamp for when the LLM call finished.

llm_model_name: str | None = None: The name of the model use for the LLM call.

prompt: str | None = None: The prompt that was used for the LLM call.

prompt_tokens: int | None = None: The number of input tokens.

raw_response: object | None = None

The raw response received from the LLM.

May contain additional information, e.g. logprobs.

started_at: float | None = None: The timestamp for when the LLM call started.

task: str | None = None: The internal task that made the call.

total_tokens: int | None = None: The total number of used tokens.