Evaluation
For the Vision Transformer, our evaluation script processes the ImageNet 1K validation folder and computes the final validation accuracy.
To enable the evaluation stage with a ViT model, configure the configuration files:
In the
defaults
section ofconf/config.yaml
, update theevaluation
field to point to the desired ViT configuration file. For example, if you want to use thevit/imagenet_val
configuration, change theevaluation
field tovit/imagenet_val
.defaults: - evaluation: vit/imagenet_val ...
In the
stages
field ofconf/config.yaml
, make sure theevaluation
stage is included. For example,stages: - evaluation ...
Configure
imagenet_val
field ofconf/evaluation/vit/imagenet_val.yaml
to be the ImageNet 1K validation folder.Execute launcher pipeline:
python3 main.py
Remarks:
To load a pretrained checkpoint for inference, set the
restore_from_path
field in themodel
section to the path of the pretrained checkpoint in.nemo
format inconf/evaluation/vit/imagenet_val.yaml
. By default, this field links to the.nemo
format checkpoint located in the ImageNet 1K fine-tuning checkpoints folder.We highly recommend users to use the same precision (i.e.,
trainer.precision
) for evaluation as was used during training.