Tutorials for NVIDIA NIM for LLMs

Use the following GitHub projects to learn about NIM.

Playbooks

The NVIDIA NIM for Large Language Models (LLMs) playbooks demonstrate how to use NVIDIA NIM for LLMs to self-host RAG, deploy on Hugging Face, and fine-tune with LoRA.

• The NVIDIA AI Blueprint: Bring Your LLM to NIM playbooks show three deployment approaches using the multi-LLM compatible NIM container:

  • Deploy HuggingFace Safetensors

  • Deploy TensorRT-LLM Checkpoints and Engines

  • Deploy GGUF Checkpoints.

• The Build a RAG using a locally hosted NIM playbook demonstrates how to build a RAG using NVIDIA NIM for LLMs with a locally hosted Llama 3 8B Instruct NIM and deploy it using NVIDIA AI Endpoints for LangChain.

• The Llama 3 LoRA Fine-Tuning and Deployment with the NeMo framework and NVIDIA NIM playbook demonstrates how to perform LoRA PEFT on Llama 3 8B Instruct using a dataset for biomedical domain question answering and deploy multiple LoRA adapters with NVIDIA NIM for LLMs.

• The Llama 3.1 Law-Domain LoRA Fine-Tuning and Deployment with the NeMo framework and NVIDIA NIM playbook demonstrates how to perform LoRA PEFT on Llama 3.1 8B Instruct using a synthetically augmented version of Law StackExchange with the NeMo framework, followed by deployment with NVIDIA NIM for LLMs. As a prerequisite, follow the tutorial for data curation using NeMo Curator.

Platform Deployment Guides

• The NIM on Azure Kubernetes Service (AKS) deployment guide provides step-by-step instructions for deploying AKS.

• The NIM on Azure Machine Learning (AzureML) deployment guide provides step-by-step instructions for deploying AzureML using Azure CLI and Jupyter Notebook.

• The End to End LLM App development with Azure AI Studio, Prompt Flow and NIMs deployment guide provides end-to-end LLM App development with Azure AI Studio, Prompt Flow, and NIMs.

• The NIM on AWS Elastic Kubernetes Service (EKS) deployment guide provides step-by-step instructions for deploying on AWS EKS.

• The NIM on AWS SageMaker deployment guide provides step-by-step instructions for deploying on AWS SageMaker using Jupyter Notebooks, Python CLI, and the shell.

• The NIM on KServe deployment guide provides step-by-step instructions on how to deploy on KServe.