On the Fly Model Update

One of the desirable requirements for DeepStream applications is to change the Model on the fly. The current release supports this feature with the assumption that the model being updated should have the same network parameters. This is an alpha feature and only supported in deepstream-test5-app. Subsequent releases will address mentioned assumptions for on the fly model update. Currently on the fly model update helps to deploy more accurate, newly trained models without the necessity of stopping and re-launching the DeepStream application or container. This feature helps to apply model-update with zero DeepStream application downtime. The image below shows how on the fly models works currently:

On the Fly Model Update

Refer to “section 7” in the deepstream-test5-app/README for instructions on how to test model update feature.

Assumptions

  1. New model must have same network parameter configuration as of previous model (e.g. network resolution, network architecture, number of classes).

  2. Engine file or cache file of new model to be provided by developer.

  3. Other primary gie configuration parameters like group-threshold, bbox color, gpu-id, nvbuf-memory-type etc., will not have any effect after model switch even if updated parameters are provided in the override file.