> This page is for version 1.0.0.
> For other versions, use one of these documentation indexes:
> - 1.10.1 (default): https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-topiccontrol/1.10.1/llms.txt
> - 1.0.0: https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-topiccontrol/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-topiccontrol/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-topiccontrol/_mcp/server.

# OpenAI-compatible completions endpoint

POST /v1/completions
Content-Type: application/json

Reference: https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-topiccontrol/api-reference/create-completion-v-1-completions-post

## Request

### Body (application/json)

This endpoint expects a NIMLLMCompletionRequest.

- `model` (string, required) — The model to use.
- `prompt` (NimllmCompletionRequestPrompt, required)
- `echo` (boolean, optional, nullable, default: false) — If true, echo back the prompt in addition to the completion. 'echo' and 'stream' fields cannot be true simultaneously.
- `frequency_penalty` (double, optional, nullable, default: 0) — Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
- `logit_bias` (map from string to double, optional, nullable) — Modify the likelihood of specified tokens appearing in the completion. Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.
- `logprobs` (integer, optional, nullable) — Include the log probabilities on the `logprobs` most likely output tokens, as well the chosen tokens. For example, if `logprobs` is 5, the API will return a list of the 5 most likely tokens. The API will always return the `logprob` of the sampled token, so there may be up to `logprobs+1` elements in the response.
- `max_tokens` (integer, optional, nullable, default: 16) — The maximum number of tokens that can be generated.
- `n` (integer, optional, nullable, default: 1) — How many completions to generate for each prompt.
- `presence_penalty` (double, optional, nullable, default: 0) — Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
- `seed` (integer, optional, nullable) — Changing the seed will produce a different response with similar characteristics. Fixing the seed will reproduce the same results if all other parameters are also kept constant.
- `stop` (NimllmCompletionRequestStop, optional, nullable) — Sequences where the API will stop generating further tokens.
- `stream` (boolean, optional, nullable, default: false) — If set, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only [server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#Event_stream_format) as they become available, with the stream terminated by a `data: [DONE]`
- `stream_options` (StreamOptions, optional, nullable)
- `suffix` (string, optional, nullable) — The suffix that comes after a completion of inserted text.
- `temperature` (double, optional, nullable, default: 1) — What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
- `top_p` (double, optional, nullable, default: 1) — An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or `temperature` but not both.
- `user` (string, optional, nullable)
- `min_p` (double, optional, default: 0)
- `prompt_logprobs` (integer, optional, nullable)
- `add_special_tokens` (boolean, optional, default: true) — If true (the default), special tokens (e.g. BOS) will be added to the prompt.
- `response_format` (ResponseFormat, optional, nullable) — An object specifying the format that the model must output. Setting to `{ "type": "json_object" }` enables JSON mode, which guarantees the message the model generates is valid JSON.
- `guided_whitespace_pattern` (string, optional, nullable) — If specified, will override the default whitespace pattern for guided json decoding.
- `nvext` (NVExt, optional, nullable) — Extension dictionary for NIM API.

## Response

### 200

Successful Response

- `create_completion_v1_completions_post_Response_200`
  - CompletionResponse: vllm__entrypoints__openai__protocol__CompletionResponse
  - CompletionStreamResponse: CompletionStreamResponse

## Errors

### 400 Bad Request Error

Received an invalid request possibly containing unsupported or out-of-range parameter values.

- `message` (string, required)
- `type` (string, required)
- `code` (integer, required)
- `object` (string, optional, default: error)
- `param` (string, optional, nullable)

### 404 Not Found Error

The requested model does not exist.

- `message` (string, required)
- `type` (string, required)
- `code` (integer, required)
- `object` (string, optional, default: error)
- `param` (string, optional, nullable)

### 415 Unsupported Media Type Error

Unsupported Media Type

- `message` (string, required)
- `type` (string, required)
- `code` (integer, required)
- `object` (string, optional, default: error)
- `param` (string, optional, nullable)

### 422 Unprocessable Entity Error

Validation Error

- `detail` (list of ValidationError, optional)

### 500 Internal Server Error

Internal Server Error

- `message` (string, required)
- `type` (string, required)
- `code` (integer, required)
- `object` (string, optional, default: error)
- `param` (string, optional, nullable)

## Types

### NimllmCompletionRequestPrompt

- list of integer
- list of list of integer
- string
- list of string

### NimllmCompletionRequestStop

Sequences where the API will stop generating further tokens.

- string
- list of string

### StreamOptions

- `include_usage` (boolean, optional, nullable, default: true)
- `continuous_usage_stats` (boolean, optional, nullable, default: true)

### ResponseFormat

- `type` (enum, required) — Must be one of `text` or `json_object`.
  - Allowed values: `text`, `json_object`

### NVExt

- `ignore_eos` (boolean, optional, nullable) — Whether to ignore End of Sequence (EOS) tokens. It is advised to set max_tokens when this is set to avoid overly long response times.
- `repetition_penalty` (double, optional, nullable, default: 1) — How much to penalize tokens based on how frequently they occur in the text. A value of 1 means no penalty, while values larger than 1 discourage and values smaller encourage.
- `top_k` (integer, optional, default: -1) — How many of the top tokens to sample from. Must be -1 or greater than or equal to 1, and cannot be null. If not set, then the default is -1 which disables top_k sampling (greedy).
- `guided_choice` (list of string, optional, nullable) — If specified, the output will be exactly one of the choices.
- `guided_json` (NvExtGuidedJson, optional, nullable) — If specified, the output will follow the JSON schema. Can be a string, an object, a reference to a BaseModel, or null.
- `guided_regex` (string, optional, nullable) — If specified, the output will follow the regex pattern. Can be a string or null.
- `guided_grammar` (string, optional, nullable) — If specified, the output will follow the context-free grammar. Can be a string or null.

### vllm__entrypoints__openai__protocol__CompletionResponse

- `model` (string, required)
- `choices` (list of CompletionResponseChoice, required)
- `usage` (UsageInfo, required)
- `id` (string, optional)
- `object` (string, optional, default: text_completion)
- `created` (integer, optional)

### CompletionStreamResponse

- `model` (string, required)
- `choices` (list of CompletionResponseStreamChoice, required)
- `id` (string, optional)
- `object` (string, optional, default: text_completion)
- `created` (integer, optional)
- `usage` (UsageInfo, optional, nullable)

### ValidationError

- `loc` (list of ValidationErrorLocItems, required)
- `msg` (string, required)
- `type` (string, required)

### NvExtGuidedJson

If specified, the output will follow the JSON schema. Can be a string, an object, a reference to a BaseModel, or null.

- string
- map from string to any
- BaseModel: BaseModel

### CompletionResponseChoice

- `index` (integer, required)
- `text` (string, required)
- `logprobs` (CompletionLogProbs, optional, nullable)
- `finish_reason` (string, optional, nullable)
- `stop_reason` (CompletionResponseChoiceStopReason, optional, nullable) — The stop string or token id that caused the completion to stop, None if the completion finished for some other reason including encountering the EOS token
- `prompt_logprobs` (list of map from string to Logprob, optional, nullable)

### UsageInfo

- `prompt_tokens` (integer, optional, default: 0)
- `total_tokens` (integer, optional, default: 0)
- `completion_tokens` (integer, optional, nullable, default: 0)

### CompletionResponseStreamChoice

- `index` (integer, required)
- `text` (string, required)
- `logprobs` (CompletionLogProbs, optional, nullable)
- `finish_reason` (string, optional, nullable)
- `stop_reason` (CompletionResponseStreamChoiceStopReason, optional, nullable) — The stop string or token id that caused the completion to stop, None if the completion finished for some other reason including encountering the EOS token

### ValidationErrorLocItems

- string
- integer

### BaseModel

### CompletionLogProbs

- `text_offset` (list of integer, optional)
- `token_logprobs` (list of double, optional)
- `tokens` (list of string, optional)
- `top_logprobs` (list of map from string to double, optional)

### CompletionResponseChoiceStopReason

The stop string or token id that caused the completion to stop, None if the completion finished for some other reason including encountering the EOS token

- integer
- string

### Logprob

- `logprob` (double, required)
- `rank` (integer, optional, nullable)
- `decoded_token` (string, optional, nullable)

### CompletionResponseStreamChoiceStopReason

The stop string or token id that caused the completion to stop, None if the completion finished for some other reason including encountering the EOS token

- integer
- string

## Examples

**Request**

```json
{
  "model": "string",
  "prompt": [
    1
  ]
}
```

**Response**

```json
{
  "choices": [
    {
      "finish_reason": "string",
      "index": 1,
      "logprobs": {
        "text_offset": [
          1
        ],
        "token_logprobs": [
          1.1
        ],
        "tokens": [
          "string"
        ],
        "top_logprobs": [
          {}
        ]
      },
      "prompt_logprobs": [
        {}
      ],
      "stop_reason": 1,
      "text": "string"
    }
  ],
  "created": 1,
  "id": "string",
  "model": "string",
  "object": "text_completion",
  "usage": {
    "completion_tokens": 1,
    "prompt_tokens": 0,
    "total_tokens": 0
  }
}
```

**SDK Code**

```python
import requests

url = "https://api.example.com/v1/completions"

payload = {
    "model": "string",
    "prompt": [1]
}
headers = {"Content-Type": "application/json"}

response = requests.post(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.example.com/v1/completions';
const options = {
  method: 'POST',
  headers: {'Content-Type': 'application/json'},
  body: '{"model":"string","prompt":[1]}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.example.com/v1/completions"

	payload := strings.NewReader("{\n  \"model\": \"string\",\n  \"prompt\": [\n    1\n  ]\n}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.example.com/v1/completions")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Content-Type"] = 'application/json'
request.body = "{\n  \"model\": \"string\",\n  \"prompt\": [\n    1\n  ]\n}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.post("https://api.example.com/v1/completions")
  .header("Content-Type", "application/json")
  .body("{\n  \"model\": \"string\",\n  \"prompt\": [\n    1\n  ]\n}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('POST', 'https://api.example.com/v1/completions', [
  'body' => '{
  "model": "string",
  "prompt": [
    1
  ]
}',
  'headers' => [
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.example.com/v1/completions");
var request = new RestRequest(Method.POST);
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{\n  \"model\": \"string\",\n  \"prompt\": [\n    1\n  ]\n}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = ["Content-Type": "application/json"]
let parameters = [
  "model": "string",
  "prompt": [1]
] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.example.com/v1/completions")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "POST"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```