Model CoverageVision Language Models

Kimi-K3

View as Markdown

Kimi-K3 is a Moonshot AI vision-language model with a MoonViT3d vision tower and a Kimi Linear Mixture-of-Experts language backbone.

TaskImage-Text-to-Text / Text Generation
ArchitecturesKimiK3ForConditionalGeneration, KimiK3ForCausalLM
HF Orgmoonshotai

Architecture

  • KimiK3ForConditionalGeneration combines the MoonViT3d vision tower and multimodal projector with the Kimi Linear language backbone.
  • KimiK3ForCausalLM exposes the text-only Kimi Linear backbone.

Example HF Models

ModelHF ID
Kimi-K3moonshotai/Kimi-K3

Example Recipes

RecipeDatasetDescription
k3_hellaswag.yamlHellaSwagSFT — text-only Kimi-K3 with EP32 + PP8

Try with NeMo AutoModel

1. Clone and install from source (full instructions):

$git clone https://github.com/NVIDIA-NeMo/Automodel.git
$cd Automodel
$uv sync --locked --all-groups --all-extras --extra vlm-media

2. Run the text-only recipe from inside the repo:

$uv run automodel --nproc-per-node=8 examples/llm_finetune/kimi/k3_hellaswag.yaml

1. Pull the container and mount a checkpoint directory:

$docker run --gpus all -it --rm \
>  --shm-size=8g \
>  -v $(pwd)/checkpoints:/opt/Automodel/checkpoints \
>  nvcr.io/nvidia/nemo-automodel:26.06.00

2. Navigate to the AutoModel directory:

$cd /opt/Automodel

3. Run the recipe:

$automodel --nproc-per-node=8 examples/llm_finetune/kimi/k3_hellaswag.yaml

See the Installation Guide and LLM Fine-Tuning Guide.

Hugging Face Model Cards