StarCoder2

Released in 2024, BigCode’s StarCoder2 is a code-generation large language mode based on the transformer decoder framework. The model was trained on a massive code dataset called The Stack v2 which contains over 900B tokens. StarCoder2 is offered in 3B, 7B, and 15B sizes and competes with models well above its size on code-generation tasks. More information can be found in the companion paper “StarCoder 2 and The Stack v2: The Next Generation”.

• StarCoder2
• Data Preparation for SFT and PEFT
  
  • Step 1: Download dataset
  • Step 2: Data Preprocessing
  • Step 3: Split the data into train, validation and test.
  • Step 4: Prepare Packed Dataset
• Checkpoint Conversion
• Model Evaluation
  
  • Run Evaluation
    
    • Configuration
• Supervised Fine-tuning (SFT)
  
  • Run SFT inside NeMo Container
    
    • Step 1: Start NeMo Container
    • Step 2: Run SFT
    • Step 3: Run evaluation
  • Run SFT with NeMo Launcher
    
    • Configuration
• Parameter Efficient Fine-Tuning (PEFT)
  
  • Run PEFT inside NeMo Container
    
    • Step 1: Start NeMo Container
    • Step 2: Run PEFT
    • Step 3: Run evaluation
    • Step 4 (Optional): Merge LORA weights
  • Run PEFT with NeMo Launcher
    
    • Configuration
• Model Export to TensorRT-LLM
  
  • Run Export
• Model Deployment
  
  • Serve in-framework or TensorRT-LLM model on Triton
    
    • Sending a Query using NeMo APIs
    • Sending a Query using PyTriton
  • Deploy a LLM model with NeMo APIs