Get Started with Beam-1.0-501B-A23B

This page shows you how to deploy Beam-1.0-501B-A23B using the SGLang Model-Free NIM Experimental collection. The model-free container downloads model weights at startup. For an overview, refer to About Model-Free NIM.

Beam supports text input and text output, reasoning, tool calling, and guided decoding. Image, video, and audio input and LoRA adapters are not supported in this release.

For the generic onboarding path, refer to Quickstart. For endpoint schemas, refer to API Reference. For tool execution and MCP integration, refer to Tool Calling and MCP Integration.

Prerequisites

Use the GPU count in the Beam support matrix and provide enough host memory and disk space for the checkpoint, download staging, and runtime cache.

Software Requirements

Minimum required versions for supported software components.

Requirement	Specification
Operating System	Ubuntu 22.04 LTS or later recommended
Container Toolkit	1.14.0 or later
CUDA SDK	12.9 or later
GPU Driver	580 or later
Docker	24.0 or later

Operating System

While other Linux distributions can be compatible with NIM, they have not been officially validated.

We recommend using Ubuntu 22.04 LTS or later for the best experience.

CUDA SDK

Install CUDA SDK by following the CUDA installation guide for Linux.

GPU Drivers

Install the NVIDIA GPU drivers by following the NVIDIA Driver Installation Guide.

Docker

Docker is required to run the containerized NIM services.

1. Install Docker Engine for your Linux distribution by following the Docker Engine installation guide.

2. Verify that the Docker daemon is running and that your user can execute docker commands without sudo. Add your user to the docker group if needed:

   sudo groupadd docker
   sudo usermod -aG docker $USER
   

3. Log out and back in for the group change to take effect.

Container Toolkit

The NVIDIA Container Toolkit enables Docker containers to access the host GPU.

1. Install the toolkit by following the NVIDIA Container Toolkit installation guide.

2. Configure Docker to use the NVIDIA runtime by following the Docker configuration steps.

3. Restart the Docker daemon after configuration:

   sudo systemctl restart docker
   

NIM Container Access

To download and deploy NIM containers, you need one of the following:

• A free NVIDIA Developer Program membership.

• An NVIDIA AI Enterprise license. To request a free 90-day evaluation license, refer to Ways to Get Started With NVIDIA AI Enterprise and Activate Your NVIDIA AI Enterprise License.

Generate Access Credentials

NGC

An NGC Personal API key is not required to access most NVIDIA NIM containers and models hosted on NGC. An NGC API key is only required to download Production Branch (PB) models. This Get Started documentation uses a feature branch model as an example, so creating an NGC Personal API key is optional.

If you want to create an NGC Personal API key, do the following:

1. Generate the Personal API Key on the Setup API Keys page.

2. When creating the Personal API key, select at least NGC Catalog from the Services Included list. You can also include additional services if you want to use the same key for other purposes.

Warning

Legacy API keys are not supported by NIM. Always use a Personal API Key.

Hugging Face

Create a Hugging Face access token with read permissions (or higher, only if your org or private repos require it). This token allows model-free NIM to fetch models directly from Hugging Face.

Verify NVIDIA Runtime Access

To ensure that your setup is correct, run the following command:

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

This command should produce output similar to one of the following, where you can confirm CUDA driver version, and available GPUs.

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 12.9     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA H100 80GB HBM3          On  |   00000000:1B:00.0 Off |                    0 |
| N/A   36C    P0            112W /  700W |   78489MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Configuration

Review the model access terms on Hugging Face before downloading weights. Configure your token and a persistent model cache:

Export a Hugging Face Access Token

Export a Hugging Face token to download models from Hugging Face.

1. Export the variable in your shell (temporary), replacing <token-value> with your actual token:

   export HF_TOKEN="<token-value>"
   

2. Optional: Persist the variable in your shell profile for future terminals.

   If using bash:

   echo 'export HF_TOKEN="<token-value>"' >> ~/.bashrc
   

   If using zsh:

   echo 'export HF_TOKEN="<token-value>"' >> ~/.zshrc
   

3. Verify the variable is set:

   echo "$HF_TOKEN"
   

Export an NGC API Key

Note

An NGC API key is only required to download Production Branch (PB) models. You can skip this section if you did not create an NGC API key.

1. Export the variable in your shell (temporary), replacing <VALUE> with your actual API key:

   export NGC_API_KEY=<VALUE>
   

2. Optional: Persist the variable in your shell profile for future terminals.

   If using bash:

   echo "export NGC_API_KEY=$NGC_API_KEY" >> ~/.bashrc
   

   If using zsh:

   echo "export NGC_API_KEY=$NGC_API_KEY" >> ~/.zshrc
   

3. Verify the variable is set:

   echo "$NGC_API_KEY"
   

Model Cache

NIM downloads model weights to a cache on the host that you mount into the container. Artifacts persist across restarts, so you do not pull the full model on every run.

Local Cache

An essential variable to configure on your host system is the cache path directory. This directory is mapped from the host machine to container; assets (for example, model weights) are downloaded to this host directory and persist across container restarts. Configuring a local cache is highly recommended, as it avoids re-downloading large model files upon subsequent container restarts. You can name the environment variable containing the path to the local cache whatever you want.

Create the cache directory and export an environment variable:

export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p $LOCAL_NIM_CACHE
# Optionally add sticky bit to avoid issues writing to the cache if the container is running as a different user
chmod -R a+rwxt $LOCAL_NIM_CACHE

When you start the NIM container, you must map your host machine’s local cache directory ($LOCAL_NIM_CACHE) to the container’s internal cache path (/opt/nim/.cache) using a Docker volume mount, such as -v "$LOCAL_NIM_CACHE:/opt/nim/.cache". This mapping ensures that the large model weights downloaded by the container are saved to your host machine. Because containers are ephemeral, any data stored only inside the container is lost when it stops. By using a volume mount, subsequent container runs detect the existing model files in your local cache and skip the lengthy download process, allowing the NIM to start up faster.

Choose the Model Source

Choose the checkpoint precision and GPU count from the Beam support matrix.

For H100, H20-3e, or H200, use the FP8 checkpoint:

export NIM_MODEL_PATH="hf://reflection/Beam-1.0-501B-A23B-FP8"
export GPU_COUNT=8

For B200 or GB200, use the NVFP4 checkpoint:

export NIM_MODEL_PATH="hf://reflection/Beam-1.0-501B-A23B-NVFP4"
export GPU_COUNT=4

Run only the configuration for your hardware. NIM downloads weights to the mounted cache. For local checkpoints and other sources, refer to Model Download.

Pull the Container Image

If you created an NGC API key, authenticate with the NVIDIA Container Registry using the following command:

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

Use $oauthtoken as the username and NGC_API_KEY as the password. The $oauthtoken username is a special name that indicates that you will authenticate with an API key and not a user name and password.

Pull the experimental SGLang model-free container:

docker pull nvcr.io/nim/nvidia/sglang-model-free-nim-experimental:2.1.7-variant-beam-1.0-501b

Download the Serving Recipe

Download the Beam recipe resource for the same version as your container. The experimental collection lists the available artifacts. Use the NGC CLI to download the recipes:

ngc registry resource download-version nim/nvidia/beam-1.0-recipe:2.1.7-variant-beam-1.0-501b

Select the YAML file that matches your GPU and model precision:

GPU	Precision	Tensor parallelism	Recipe filename
H100	FP8	8	beam-1.0_h100_latency_fp8_tp8.yaml
H20-3e	FP8	8	beam-1.0_h20-3e_latency_fp8_tp8.yaml
H200	FP8	8	beam-1.0_h200_latency_fp8_tp8.yaml
B200	NVFP4	4	beam-1.0_b200_latency_nvfp4_tp4.yaml
GB200	NVFP4	4	beam-1.0_gb200_latency_nvfp4_tp4.yaml

Keep the container and recipe versions together.

Set RECIPE to the absolute path of the downloaded YAML file for your configuration. Replace the example path before running this command:

export RECIPE="/absolute/path/to/your/beam-recipe.yaml"
test -f "$RECIPE"
chmod 0644 "$RECIPE"

Run NIM

Mount the selected recipe read-only and load it with NIM_CONFIG_FILE:

docker run --rm --name beam-nim --gpus "$GPU_COUNT" --ipc=host --shm-size=32g \
  -e NIM_MODEL_PATH="$NIM_MODEL_PATH" \
  -e NIM_SERVED_MODEL_NAME=reflection/beam-1.0-501b \
  -e NIM_CONFIG_FILE=/opt/nim/recipe.yaml \
  -e HF_TOKEN \
  -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
  -v "$RECIPE:/opt/nim/recipe.yaml:ro" \
  -p 8000:8000 \
  nvcr.io/nim/nvidia/sglang-model-free-nim-experimental:2.1.7-variant-beam-1.0-501b \
  nim-serve

Use a second terminal for API requests. The first startup downloads the checkpoint; wait until the readiness endpoint returns HTTP 200 before sending inference requests. Preserve the recipe’s model-specific settings when reproducing a validated configuration.

Verify Health and Runtime Identity

curl --fail http://localhost:8000/v1/health/live
curl --fail http://localhost:8000/v1/health/ready
curl --fail http://localhost:8000/v1/models
curl --fail http://localhost:8000/v1/version
curl --fail http://localhost:8000/v1/metadata

The launch command explicitly sets NIM_SERVED_MODEL_NAME, so /v1/models should report reflection/beam-1.0-501b. Send that identifier in every request’s model field, for both FP8 and NVFP4. The downloaded checkpoint repository name can differ from the served model name.

Interact with the API

The container provides /v1/chat/completions, /v1/completions, /v1/responses, and the Anthropic-compatible /v1/messages endpoint. Use the running container’s /docs explorer for request and response schemas.

Send a Chat Completion Request

curl --fail http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "reflection/beam-1.0-501b",
    "messages": [{"role": "user", "content": "Explain tensor parallelism."}],
    "max_tokens": 4096
  }'

Beam supports reasoning. Allow enough output tokens for both reasoning and the final answer. Check finish_reason: length means generation reached the token limit. Read reasoning from choices[0].message.reasoning and keep it separate from the assistant’s final answer in choices[0].message.content.

Hold a Multi-Turn Conversation

Chat Completions is stateless. Resend prior user and assistant messages in order with each request. Append the assistant’s final content to the history; do not copy reasoning into the conversation.

Stream a Response

Add "stream": true to the Chat Completions request and use curl -N to receive Server-Sent Events as they arrive. Assemble reasoning from choices[0].delta.reasoning and the final answer from choices[0].delta.content separately. Chat Completions streams end with data: [DONE].

Send a Responses Request

curl --fail http://localhost:8000/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "reflection/beam-1.0-501b",
    "input": "Explain why a persistent model cache is useful.",
    "max_output_tokens": 4096
  }'

For streaming Responses requests, treat both response.completed and response.incomplete as terminal events. An output token limit can end a response with response.incomplete.

Send an Anthropic-Compatible Messages Request

curl --fail http://localhost:8000/v1/messages \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "reflection/beam-1.0-501b",
    "max_tokens": 4096,
    "messages": [{"role": "user", "content": "What is tensor parallelism?"}]
  }'

Tool Calling

Supply function definitions in tools. The model can return a tool call; your application executes it and sends the result in a subsequent turn. Refer to Tool Calling and MCP Integration for a complete execution loop.

curl --fail http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "reflection/beam-1.0-501b",
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{"type": "function", "function": {
      "name": "get_weather",
      "description": "Get the current weather for a city.",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    }}],
    "max_tokens": 4096
  }'

Guided Decoding

Use response_format to request an answer constrained by a JSON schema:

curl --fail http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "reflection/beam-1.0-501b",
    "messages": [{"role": "user", "content": "Return a JSON object naming Paris as a city."}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "city",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"],
          "additionalProperties": false
        }
      }
    },
    "max_tokens": 4096
  }'

Troubleshooting

• Container or recipe download fails: Check the collection’s artifact version and your NGC access. Use the container and recipe from the same release.

• Checkpoint download fails: Check HF_TOKEN, model access terms, network access, cache permissions, and available disk space.

• Startup fails or runs out of GPU memory: Verify the checkpoint precision, GPU count, and selected recipe against the support matrix. Inspect the output in the launch terminal. The launch uses --rm, so Docker removes the container when it exits. To use docker logs beam-nim after a failure, rerun without --rm. Refer to Troubleshooting GPU Memory Out-of-Memory Errors.

• Requests fail with a model-name error: Query /v1/models and use the reported ID. The example launch sets it to reflection/beam-1.0-501b.

• An answer is empty: Inspect reasoning output and the finish reason. Increase the output token allowance if generation reached the limit.

For additional diagnostics, refer to Troubleshooting.