Get Started with MiniMax M3.1 Flash

This page shows you how to deploy MiniMax M3.1 Flash using the SGLang Model-Free NIM Experimental Collection. The model-free container downloads model assets from a remote source at startup. For more information, refer to Model-Free NIM. For supported hardware, refer to the Support Matrix.

This model supports text, image, and video input.

For the generic NIM LLM and VLM onboarding path, refer to Quickstart. For full endpoint examples, refer to API Reference. For tool calling and MCP integration, refer to Tool Calling and MCP Integration.

Prerequisites

Before deploying a NIM LLM and VLM container, ensure your environment meets the following requirements:

Hardware Requirements

The following are the minimum required specifications for supported hardware components:

Requirement	Specification
CPU	AMD64, ARM64
GPU	Refer to the Support Matrix

Software Requirements

Minimum required versions for supported software components.

Requirement	Specification
Operating System	Ubuntu 22.04 LTS or later recommended
Container Toolkit	1.14.0 or later
CUDA SDK	12.9 or later
GPU Driver	580 or later
Docker	24.0 or later

Operating System

While other Linux distributions can be compatible with NIM, they have not been officially validated.

We recommend using Ubuntu 22.04 LTS or later for the best experience.

CUDA SDK

Install CUDA SDK by following the CUDA installation guide for Linux.

GPU Drivers

Install the NVIDIA GPU drivers by following the NVIDIA Driver Installation Guide.

Docker

Docker is required to run the containerized NIM services.

1. Install Docker Engine for your Linux distribution by following the Docker Engine installation guide.

2. Verify that the Docker daemon is running and that your user can execute docker commands without sudo. Add your user to the docker group if needed:

   sudo groupadd docker
   sudo usermod -aG docker $USER
   

3. Log out and back in for the group change to take effect.

Container Toolkit

The NVIDIA Container Toolkit enables Docker containers to access the host GPU.

1. Install the toolkit by following the NVIDIA Container Toolkit installation guide.

2. Configure Docker to use the NVIDIA runtime by following the Docker configuration steps.

3. Restart the Docker daemon after configuration:

   sudo systemctl restart docker
   

NIM Container Access

To download and deploy NIM containers, you need one of the following:

• A free NVIDIA Developer Program membership.

• An NVIDIA AI Enterprise license. To request a free 90-day evaluation license, refer to Ways to Get Started With NVIDIA AI Enterprise and Activate Your NVIDIA AI Enterprise License.

Generate Access Credentials

NGC

An NGC Personal API key is not required to access most NVIDIA NIM containers and models hosted on NGC. An NGC API key is only required to download Production Branch (PB) models. This Get Started documentation uses a feature branch model as an example, so creating an NGC Personal API key is optional.

If you want to create an NGC Personal API key, do the following:

1. Generate the Personal API Key on the Setup API Keys page.

2. When creating the Personal API key, select at least NGC Catalog from the Services Included list. You can also include additional services if you want to use the same key for other purposes.

Warning

Legacy API keys are not supported by NIM. Always use a Personal API Key.

Hugging Face

Create a Hugging Face access token with read permissions (or higher, only if your org or private repos require it). This token allows model-free NIM to fetch models directly from Hugging Face.

Verify NVIDIA Runtime Access

To ensure that your setup is correct, run the following command:

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

This command should produce output similar to one of the following, where you can confirm CUDA driver version, and available GPUs.

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 12.9     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA H100 80GB HBM3          On  |   00000000:1B:00.0 Off |                    0 |
| N/A   36C    P0            112W /  700W |   78489MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Configuration

Use environment variables to control authentication and model caching. Accept any model access terms on Hugging Face before downloading the weights.

Export a Hugging Face Access Token

Export a Hugging Face token to download models from Hugging Face.

1. Export the variable in your shell (temporary), replacing <token-value> with your actual token:

   export HF_TOKEN="<token-value>"
   

2. Optional: Persist the variable in your shell profile for future terminals.

   If using bash:

   echo 'export HF_TOKEN="<token-value>"' >> ~/.bashrc
   

   If using zsh:

   echo 'export HF_TOKEN="<token-value>"' >> ~/.zshrc
   

3. Verify the variable is set:

   echo "$HF_TOKEN"
   

Export an NGC API Key

Note

An NGC API key is only required to download Production Branch (PB) models. You can skip this section if you did not create an NGC API key.

1. Export the variable in your shell (temporary), replacing <VALUE> with your actual API key:

   export NGC_API_KEY=<VALUE>
   

2. Optional: Persist the variable in your shell profile for future terminals.

   If using bash:

   echo "export NGC_API_KEY=$NGC_API_KEY" >> ~/.bashrc
   

   If using zsh:

   echo "export NGC_API_KEY=$NGC_API_KEY" >> ~/.zshrc
   

3. Verify the variable is set:

   echo "$NGC_API_KEY"
   

Model Cache

NIM downloads model weights to a cache on the host that you mount into the container. Artifacts persist across restarts, so you do not pull the full model on every run.

Local Cache

An essential variable to configure on your host system is the cache path directory. This directory is mapped from the host machine to container; assets (for example, model weights) are downloaded to this host directory and persist across container restarts. Configuring a local cache is highly recommended, as it avoids re-downloading large model files upon subsequent container restarts. You can name the environment variable containing the path to the local cache whatever you want.

Create the cache directory and export an environment variable:

export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p $LOCAL_NIM_CACHE
# Optionally add sticky bit to avoid issues writing to the cache if the container is running as a different user
chmod -R a+rwxt $LOCAL_NIM_CACHE

When you start the NIM container, you must map your host machine’s local cache directory ($LOCAL_NIM_CACHE) to the container’s internal cache path (/opt/nim/.cache) using a Docker volume mount, such as -v "$LOCAL_NIM_CACHE:/opt/nim/.cache". This mapping ensures that the large model weights downloaded by the container are saved to your host machine. Because containers are ephemeral, any data stored only inside the container is lost when it stops. By using a volume mount, subsequent container runs detect the existing model files in your local cache and skip the lengthy download process, allowing the NIM to start up faster.

Provide enough disk space for the checkpoint and runtime cache files.

Authenticate

If you created an NGC API key, authenticate with the NVIDIA Container Registry using the following command:

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.

Choose the Model Source

The collection supplies the container and serving recipes. Model weights are distributed separately through Hugging Face. Set NIM_MODEL_PATH to the publisher’s repository using the hf:// prefix:

export NIM_MODEL_PATH="hf://MiniMaxAI/MiniMax-M3.1-Flash"

NIM downloads the weights into the mounted cache at startup. For other sources, refer to Supported Model Sources.

Pull the Container Image

Use the docker pull command to fetch the model-free NIM container from the NGC registry:

docker pull nvcr.io/nim/nvidia/sglang-model-free-nim-experimental:2.1.7-variant

Download the Serving Recipe

Open the SGLang Model-Free NIM Experimental Collection and select the recipe for your model, GPU, and topology. Use the container and recipe version 2.1.7-variant together. Recipes carry model-specific settings that are not included in the model-free container.

Install the NGC CLI, then download the recipe resource:

ngc registry resource download-version "nim/nvidia/minimax-m3.1-flash-recipe:2.1.7-variant"

Select one of the following configurations from the Support Matrix. Run the corresponding commands to set the recipe path and GPU count:

B200, 4 GPUs:

export RECIPE="$PWD/minimax-m3.1-flash-recipe_v2.1.7-variant/b200_throughput_fp8_tp4.yaml"
export GPU_COUNT=4

B200, 8 GPUs:

export RECIPE="$PWD/minimax-m3.1-flash-recipe_v2.1.7-variant/b200_throughput_fp8_tp8.yaml"
export GPU_COUNT=8

GB200, 4 GPUs:

export RECIPE="$PWD/minimax-m3.1-flash-recipe_v2.1.7-variant/gb200_throughput_fp8_tp4.yaml"
export GPU_COUNT=4

H200, 8 GPUs:

export RECIPE="$PWD/minimax-m3.1-flash-recipe_v2.1.7-variant/h200_throughput_fp8_tp8.yaml"
export GPU_COUNT=8

H20-3e, 8 GPUs:

export RECIPE="$PWD/minimax-m3.1-flash-recipe_v2.1.7-variant/h20-3e_throughput_fp8_tp8.yaml"
export GPU_COUNT=8

Make the selected recipe readable:

chmod 0644 "$RECIPE"

The recipe must be readable by the container user. The chmod command makes it readable when the NGC CLI saves it with mode 0600 under another user.

Run NIM

Use RECIPE and GPU_COUNT from the configuration selected above to expose the required GPUs, mount the recipe read-only, and load it with NIM_CONFIG_FILE:

docker run --rm --gpus "$GPU_COUNT" --ipc=host --shm-size=32g \
  -e NIM_MODEL_PATH="$NIM_MODEL_PATH" \
  -e NIM_SERVED_MODEL_NAME=MiniMaxAI/MiniMax-M3.1-Flash \
  -e NIM_TRUST_CUSTOM_CODE=1 \
  -e HF_TOKEN \
  -e NIM_CONFIG_FILE=/opt/nim/recipe.yaml \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  -v "$RECIPE:/opt/nim/recipe.yaml:ro" \
  -p 8000:8000 \
  nvcr.io/nim/nvidia/sglang-model-free-nim-experimental:2.1.7-variant \
  nim-serve

Use the GPU count listed with the recipe. Loading the matching recipe is required to reproduce its validated settings. A launch without NIM_CONFIG_FILE uses generic defaults.

NIM_TRUST_CUSTOM_CODE=1 allows the model’s custom code to run. Use model assets from a publisher you trust.

Override a Recipe Setting

When customizing a published recipe, follow these rules:

• Use one source for your customized settings: specific NIM_* environment variables, NIM_PASSTHROUGH_ARGS, or nim-serve command-line options. Setting the same parameter to different values in more than one of these sources can cause startup to fail.

• Settings from any of these three sources override the matching settings in NIM_CONFIG_FILE, which loads the published recipe.

Overriding recipe settings changes the validated configuration.

Multimodal Input

For more information on including image and video input, refer to Image, Audio, and Video Input. Inline base64 data is accepted up to 256 MiB by default. Set NIM_MAX_REQUEST_BODY_MB to change the request-body limit.

Video input requires an external FFmpeg installation to be mounted when the container starts. FFmpeg is not included in the image. Use a recent FFmpeg 8.x build.

Add the following options to the docker run command:

-v /path/to/external-ffmpeg:/opt/external-ffmpeg:ro \
-e PATH=/opt/external-ffmpeg/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/cuda/nvvm/bin \
-e LD_LIBRARY_PATH=/opt/external-ffmpeg/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/nvidia/lib

The mounted directory must provide bin/ffmpeg, bin/ffprobe, and the required FFmpeg/libav shared libraries under lib/.

Include a software decoder in the FFmpeg bundle for every codec you serve. For AV1, use libdav1d or libaom; FFmpeg’s built-in av1 decoder is hardware-only. If a stream cannot be decoded, NIM returns HTTP 400 and names the unsupported codec in the error response.

When setting PATH and LD_LIBRARY_PATH, preserve the image’s existing CUDA and NVIDIA paths. Do not keep only the standard system paths, and do not rely on a trailing colon in LD_LIBRARY_PATH=/opt/external-ffmpeg/lib: to inherit the original value. Otherwise, CUDA components such as FlashInfer and DeepGemm can fail to find required tools or libraries during service startup.

Reasoning

You can control reasoning in chat completion requests. To disable thinking, add one of the following to the request payload:

• "chat_template_kwargs": {"thinking_mode": "disabled"}

• "reasoning_effort": "none"

• "thinking": {"type": "disabled"}

To require reasoning, set "chat_template_kwargs": {"thinking_mode": "enabled"}. Set reasoning_effort to low, high, or max to select the effort when reasoning is enabled. In adaptive mode, the model can skip reasoning on some prompts.

Model Identifier

Every request must send the served model identifier in the model field. The launch command above sets it explicitly with NIM_SERVED_MODEL_NAME, so the identifier is MiniMaxAI/MiniMax-M3.1-Flash, independently of the model repository path.

Interact with the API

The NIM exposes OpenAI-compatible inference endpoints:

• Chat Completions: /v1/chat/completions

• Text Completions: /v1/completions

• Responses: /v1/responses

Tip

Chat Completions, Text Completions, and Responses support streaming.

Verify Health Endpoints

Verify that the NIM container is running by checking its health endpoints. By default, these endpoints are served on port 8000. If you set NIM_HEALTH_PORT, use that port instead.

Live Endpoint

Check whether the server is running:

curl -v http://localhost:8000/v1/health/live

Example response:

GET /v1/health/live HTTP/1.1
Host: localhost:8000
User-Agent: curl/7.81.0
Accept: */*

HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Content-Type: application/json
Content-Length: 61
Connection: keep-alive
Cache-Control: no-store, no-cache, must-revalidate

{
  "object": "health.response",
  "message": "live",
  "status": "live"
}

Ready Endpoint

Check whether the model is fully loaded and ready for inference:

curl -v http://localhost:8000/v1/health/ready

Example response:

GET /v1/health/ready HTTP/1.1
Host: localhost:8000
User-Agent: curl/7.81.0
Accept: */*

HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Content-Type: application/json
Content-Length: 63
Connection: keep-alive
Cache-Control: no-store, no-cache, must-revalidate

{
  "object": "health.response",
  "message": "ready",
  "status": "ready"
}

Send a Chat Completion Request

After the server is running, you can send a request to the chat completion endpoint:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-M3.1-Flash",
    "messages": [
      {
        "role": "user",
        "content": "Hello! How are you?"
      }
    ],
    "max_tokens": 4096,
    "reasoning_effort": "low"
  }'

You can set reasoning_effort to max, high, or low. Allow enough output tokens for reasoning and the answer. If the response ends with finish_reason: "length", increase max_tokens and retry.

Send an Image with a Text Prompt

Include both text and image_url content parts in the request. The image URL must be reachable from the container:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-M3.1-Flash",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Describe the image."},
        {"type": "image_url", "image_url": {"url": "https://assets.ngc.nvidia.com/products/api-catalog/phi-3-5-vision/example1b.jpg"}}
      ]
    }],
    "max_tokens": 4096,
    "reasoning_effort": "low"
  }'

Streaming

To receive responses incrementally as they are generated, enable streaming by adding "stream": true to your request payload. Streaming is supported across the /v1/chat/completions, /v1/completions, and /v1/responses endpoints.

When streaming is enabled, the API returns a sequence of Server-Sent Events (SSE).

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-M3.1-Flash",
    "messages": [
      {
        "role": "user",
        "content": "Write a short poem about a robot."
      }
    ],
    "max_tokens": 4096,
    "reasoning_effort": "low",
    "stream": true
  }'

For Chat Completions and Text Completions, the response streams back in chunks. Each chunk contains a data JSON object. These streams terminate with a data: [DONE] message:

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"MiniMaxAI/MiniMax-M3.1-Flash","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":null},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"MiniMaxAI/MiniMax-M3.1-Flash","choices":[{"index":0,"delta":{"reasoning_content":"Hmm, the user asked"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"MiniMaxAI/MiniMax-M3.1-Flash","choices":[{"index":0,"delta":{"content":"Steel"},"logprobs":null,"finish_reason":null}]}

...

data: [DONE]

For the Responses API, stream events use typed SSE events such as response.output_text.delta and terminate with response.completed.