nemo_automodel.components.quantization.qlora#

Module Contents#

Functions#

create_bnb_config

Create BitsAndBytes config for quantization.

verify_qlora_quantization

Verify that the model has been properly quantized.

API#

nemo_automodel.components.quantization.qlora.create_bnb_config(config: dict)#

Create BitsAndBytes config for quantization.

nemo_automodel.components.quantization.qlora.verify_qlora_quantization(model) bool#

Verify that the model has been properly quantized.