Llama and CodeLlama

Released in February 2023, Meta’s Llama builds on the general transformer decoder framework with some key additions such as pre-normalization, SwiGLU activations, and Rotary Positional Embeddings (RoPE). More information is available in the companion paper “LLaMA: Open and Efficient Foundation Language Models”. With 7B, 13B, 33B, and 65B offerings - Llama has options for every inference budget.

• Data Preparation
• Training with Predefined Configurations
• Checkpoint Conversion
• Model Evaluation
• Parameter Efficient Fine-Tuning (PEFT)
• Model Export to TensorRT-LLM
• Model Deployment
• Llama-2 Results