GPT-OSS Models | NVIDIA NeMo Platform

This page provides detailed technical specifications for the OpenAI GPT-OSS model family supported by NeMo Customizer. For supported features and capabilities, refer to Tested Models.

Before You Start

These models require a Hugging Face token to download. Create a secret with your Hugging Face API key, then create a FileSet and Model Entity referencing the model. See index for setup instructions.

GPT-OSS 20B

Property	Value
Creator	OpenAI
Architecture	Mixture of Experts (MoE) Transformer
Description	GPT-OSS 20B provides lower latency for local or specialized use cases, featuring full chain-of-thought reasoning and agentic capabilities.
Context Length	131,072 tokens
Parameters	21B parameters (3.6B active parameters)
Training Data	Trained on harmony response format
Checkpoint Quantization	MXFP4 quantization of the MoE weights
Inference Memory	The official checkpoint can run within 16 GB of memory; Customizer training requires the GPU configurations below
Default Name	openai/gpt-oss-20b
Hugging Face	openai/gpt-oss-20b

Training Options (20B)

The Automodel contract matrix validates these configurations:

Fine-Tuning	Dataset Format	GPUs	Sequence Packing
LoRA	Prompt-completion	1	Supported
Full SFT	Prompt-completion or chat	8	Not supported in the tested configurations

For the tested Full SFT configuration, tensor and pipeline parallel sizes are 1 and expert parallel size is 8. The tested LoRA configuration uses one GPU without expert parallelism.

Default training max sequence length: 4096.

Deployment Configuration

LoRA:
NIM Image: nvcr.io/nim/openai/gpt-oss-20b:1.6.1-variant
GPU Count: 2x 80GB
Additional Environment Variables:
NIM_DISABLE_MODEL_DOWNLOAD: 1
NIM_WORKSPACE: /model-store
Full SFT:
NIM Image: nvcr.io/nim/nvidia/llm-nim:1.15.5
GPU Count: 2x 80GB
Additional Environment Variables:
NIM_MODEL_PROFILE: vllm

Usage Recommendations

Reasoning Levels

GPT-OSS models support configurable reasoning levels that you can set in system prompts:

Low: Fast responses for general dialogue
Medium: Balanced speed and detail
High: Deep and detailed analysis

Example: Set reasoning level using “Reasoning: high” in the system prompt.

MoE Model Parallelization

GPT-OSS models use a Mixture of Experts (MoE) architecture and benefit from specialized parallelization across expert layers for optimal performance.

MoE Parallelism Constraints

MoE models only support expert parallelism for distributing experts across GPUs. When expert_parallel_size > 1, tensor_parallel_size must be set to 1. Additionally, expert_parallel_size must evenly divide the number of GPUs. These constraints apply to training parallelism only and NIM deployment may use different GPU counts optimized for inference.

Model Selection Guidelines

GPT-OSS 20B: Ideal for lower latency requirements, local deployment, specialized use cases, and consumer hardware (with Ollama support)

Important Usage Note

GPT-OSS models use the harmony response format and require this format for proper functionality.

For GPT-OSS 20B, sequence packing is supported by the tested Automodel LoRA configuration with prompt-completion data. It is not supported by the tested Full SFT or chat configurations.