nemo_automodel.components.quantization.qlora

Module Contents

Name	Description
`create_bnb_config`	Create BitsAndBytes config for quantization.
`verify_qlora_quantization`	Verify that the model has been properly quantized.

nemo_automodel.components.quantization.qlora.create_bnb_config(
    config: dict
)

Create BitsAndBytes config for quantization.

nemo_automodel.components.quantization.qlora.verify_qlora_quantization(
    model
) -> bool

Verify that the model has been properly quantized.