Summary
This is the first general release of NIM.
Language Models
Llama 3 8B Instruct
Llama 3 70B Instruct
Known Issues
P-Tuning is not supported.
Empty metrics values on multi-GPU TensorRT-LLM model Metrics items gpu_cache_usage_perc, num_requests_running, and num_requests_waiting will not be reported for multi-GPU TensorRT-LLM model, because TensorRT-LLM currently doesn’t expose iteration statistics in orchestrator mode.
No tokenizer found error when running PEFT This warning can be safely ignored.