Tutorials for NVIDIA NIM for LLMs

Use the following GitHub projects to learn about NIM.

Playbooks

The NVIDIA NIM for Large Language Models (LLMs) playbooks demonstrate how to use NVIDIA NIM for LLMs to self-host RAG, deploy on Hugging Face, and fine-tune with LoRA.

• The Build a RAG using a locally hosted NIM playbook demonstrates how to build an RAG using NVIDIA NIM for LLMs with a locally hosted Llama3-8b-instruct NIM and deploy it using NVIDIA AI Endpoints for LangChain.

• The Llama 3 LoRA Fine-Tuning and Deployment with NeMo Framework and NVIDIA NIM playbook demonstrates how to perform LoRA PEFT on a Llama 3 8B Instruct using a dataset for bio-medical domain question answering and deploy multiple LoRA adapters with NVIDIA NIM for LLMs.

• The Llama 3.1 Law-Domain LoRA Fine-Tuning and Deployment with NeMo Framework and NVIDIA NIM playbook demonstrates how to perform LoRA PEFT on Llama 3.1 8B Instruct using a synthetically augmented version of Law StackExchange with NeMo Framework, followed by deployment with NVIDIA NIM for LLMs. As a pre-requisite, follow the tutorial for data curation using NeMo Curator.

Platform Deployment Guides

• The NIM on Azure Kubernetes Service (AKS) deployment guide provides step-by-step instructions for deploying AKS.

• The NIM on Azure Machine Learning (AzureML) deployment guide provides step-by-step instructions for deploying AzureML using Azure CLI and Jupyter Notebook.

• The End to End LLM App development with Azure AI Studio, Prompt Flow and NIMs deployment guide provides end-to-end LLM App development with Azure AI Studio, Prompt Flow, and NIMs.

• The NIM on AWS Elastic Kubernetes Service (EKS) deployment guide provides step-by-step instructions for deploying on AWS EKS.

• The NIM on AWS SageMaker deployment guide provides step-by-step instructions for deploying on AWS SageMaker using Jupyter Notebooks, Python CLI, and the shell.

• The NIM on KServe deployment guide provides step-by-step on how to deploy on KServe.