NovoGrad is an optimizer developed by a team in NVIDIA. It seems to be able to make models converge more quickly and achieve a slightly better accuracy for some models.
To use NovoGrad, just set the optimizer in config_train.json to NovoGrad:
"optimizer": {
"name": "NovoGrad"
},
For more details, see the API doc