Skip to navigation
API Reference

Llama Stack-compatible chat endpoint

View as Markdown

Request

This endpoint expects an object.
modelstringRequired
messageslist of objectsRequired
sampling_paramsobject or nullOptionalDefaults to {"max_tokens":0,"repetition_penalty":1,"strategy":"greedy","temperature":0,"top_k":0,"top_p":0.95}
toolslist of objects or nullOptional
tool_choice"auto" or "required" or nullOptional
streamboolean or nullOptionalDefaults to false
logprobsobject or nullOptional

Response

Successful Response
ChatCompletionResponseobject
OR
ChatCompletionResponseStreamChunkobject

SSE-stream of these events.

Errors

400
Bad Request Error
404
Not Found Error
415
Unsupported Media Type Error
422
Unprocessable Entity Error
500
Internal Server Error