Model CoverageLarge Language ModelsBaiduERNIE-4.5-0.3B-PT

ERNIE-4.5-0.3B-PT

View as Markdown

ERNIE 4.5 is Baidu’s dense and Mixture-of-Experts language model family with long-context text checkpoints on Hugging Face.

Set up NeMo AutoModel with the latest container or follow the installation instructions.

Fine-Tune ERNIE-4.5-0.3B-PT

From the repository root, run:

uv run automodel --nproc-per-node=8 examples/llm_finetune/ernie4_5/ernie4_5_0p3b_hellaswag.yaml

Choose a Workflow

GoalStart Here
Supervised fine-tuning (SFT) - ERNIE 4.5 0.3B on HellaSwag with the Hugging Face implementationUse ernie4_5_0p3b_hellaswag.yaml.

Model Reference

Model Architecture

PropertyValue
TaskText Generation
ArchitecturesErnie4_5ForCausalLM, Ernie4_5_MoeForCausalLM
Parameters0.36B
Context Length131,072 tokens
Hugging Face Organizationbaidu
  • Ernie4_5ForCausalLM: dense Hugging Face implementation path.
  • Ernie4_5_MoeForCausalLM: custom NeMo AutoModel implementation with expert parallelism support.

Available Models

ModelHF ID
ERNIE 4.5 0.3B PTbaidu/ERNIE-4.5-0.3B-PT