Create Chat Completion

POST

http://localhost:8000/v1/chat/completions

POST

/v1/chat/completions

$ curl -X POST http://localhost:8000/v1/chat/completions \
>      -H "Content-Type: application/json" \
>      -d '{
>   "model": "meta/llama-3.1-8b-instruct",
>   "messages": [
>     {
>       "role": "user",
>       "content": "What is the capital of France?"
>     }
>   ],
>   "guardrails": {
>     "config_id": "content_safety"
>   }
> }'

1 {
2   "id": "chatcmpl-abc123",
3   "object": "chat.completion",
4   "created": 1709424000,
5   "model": "meta/llama-3.1-8b-instruct",
6   "choices": [
7     {
8       "index": 0,
9       "message": {
10         "role": "assistant",
11         "content": "Paris is the capital of France."
12       },
13       "finish_reason": "stop"
14     }
15   ],
16   "guardrails": {
17     "config_id": "content_safety",
18     "state": null,
19     "llm_output": null,
20     "output_data": null,
21     "log": null
22   }
23 }

Generate a chat completion with guardrails applied.

The request shape is compatible with the OpenAI Chat Completions API and accepts Guardrails-specific options in the guardrails object.

Request

This endpoint expects an object.

modelstringRequired

LLM model to use for the completion.

messageslist of objectsOptional

Chat messages in the current conversation.

streambooleanOptionalDefaults to false

Return partial message deltas as server-sent events.

max_tokensintegerOptional

Maximum number of tokens to generate.

temperaturedoubleOptional

Sampling temperature.

top_pdoubleOptional

Top-p sampling parameter.

stopstring or list of stringsOptional

Stop sequence or sequences.

presence_penaltydoubleOptional

Presence penalty parameter.

frequency_penaltydoubleOptional

Frequency penalty parameter.

function_callmap from strings to anyOptional

Function call parameter.

logit_biasmap from strings to anyOptional

Logit bias parameter.

logprobsbooleanOptional

Log probabilities parameter.

guardrailsobjectOptional

Guardrails-specific request options.

Response

Chat completion response or server-sent event stream.

idstring

objectstring

createdinteger

modelstring

choiceslist of objects

guardrailsobject

Errors

422

Unprocessable Entity Error