On the Fly Model Update

 
One of the desirable requirements for IVA applications is to change the Model on the fly. Current release supports this feature with the assumption that the Model being updated should have the same network parameters. This is an alpha feature and only supported in deepstream-test5-app. DeepStream subsequent releases will address mentioned assumptions for on the fly model update.
Currently on the fly model update helps to deploy newly trained models with better accuracy without the necessity of stopping and re-launching the running DeepStream application / container.
This feature helps to apply model-update with no/zero DeepStream application downtime.
The figure below shows how on the fly models works currently:
A screenshot of a cell phone Description automatically generated
Refer to deepstream-test5-app/README section 7 for instructions on how to test model update feature.

Assumptions

1. New model must have same network parameter configuration as of previous model (e.g. network resolution, network architecture, number of classes)
2. Engine file or cache file of new model to be provided by developer
3. Other primary gie configuration parameters like group-threshold, bbox color, gpu-id, nvbuf-memory-type etc. updated parameters if provided in the override file, will not have any effect after model switch.