Smart Video Record
This module can be used for event (local or cloud) based recording of original data feed. This way data feed having only events of importance is recorded and hence avoiding the need to save the whole feed all the time. This recording happens in parallel to the inference pipeline running over the feed.
A video cache is maintained so that recorded video not only has frames after the event is generated but it can also have frames just before the event. This size of video cache can be configured as per use case.
In smart record, encoded frames are itself cached to save on CPU memory and to avoid transcoding and based on the event these cached frames are encapsulated under the container of choice to generate the recorded video. There is one downside to this approach. We can’t start recording until we have an Iframe and hence some of the frames from the cache are dropped in case the first frame in the cache is not an Iframe. Therefore, duration of the generated video might be less than the specified value.
Below diagram shows the smart record architecture:
This module provides the following APIs. Please refer gst-nvdssr.h for more details.
• NvDsSRStatus NvDsSRCreate (NvDsSRContext **ctx, NvDsSRInitParams *params);
• This function creates the instance of smart record and returns the pointer to an allocated NvDsSRContext. The params structure must be filled with initialization parameters required to create the instance.
• recordbin of NvDsSRContext is the gstbin which must be added to the pipeline. It expects encoded frames which will be muxed and saved to the file. Add this bin after parser element in the pipeline.
• Call NvDsSRDestroy() to free resources allocated by this function.
• NvDsSRStatus NvDsSRStart (NvDsSRContext *ctx, NvDsSRSessionId *sessionId, guint startTime, guint duration, gpointer userData);
• This function starts writing the cached video data to a file. It returns the session id which later can be used in NvDsSRStop() to stop the corresponding recording.
• Here startTime specifies the seconds before the current time and duration specifies the seconds after the start of recording.
• If current time is t1, content from t1 - startTime to t1 + duration will be saved to file. Therefore, a total of startTime + duration seconds of data will be recorded. In case duration is set to zero, recording will be stopped after defaultDuration seconds set in NvDsSRCreate().
• NvDsSRStatus NvDsSRStop (NvDsSRContext *ctx, NvDsSRSessionId sessionId);
• This function stops the previously started recording.
• NvDsSRStatus NvDsSRDestroy (NvDsSRContext *ctx);
• This function releases the resources previously allocated by NvDsSRCreate()
Please refer deepstream_source_bin.c for more details on usage of this module.
In existing deepstream-test5-app only RTSP sources are enabled for smart record. To enable smart record in deepstream-test5-app set the following under [sourceX] group:
• smart-record=1
This will enable smart record with default configuration. By default, smart record Start / Stop events will be generated every 10 seconds. Following are the default values of configuration parameters:
video cache size = 30 seconds,
container = MP4,
default duration = 10 seconds,
interval = 10 seconds,
file prefix = Smart_Record etc.
Following fields can be used under [sourceX] groups to configure these parameters.
• smart-rec-video-cache=<val in seconds>
Size of video cache in seconds. This parameter will increase the overall memory usages of the application.
• smart-rec-duration=<val in seconds>
Duration of recording.
• smart-rec-start-time=<val in seconds>
Here, start time of recording is the number of seconds earlier to the current time to start the recording.
E.g. if t0 is the current time and N is the start time in seconds that means recording will start from t0 – N. For it to work, the video cache size must be greater than the N.
• smart-rec-default-duration=<val in seconds>
In case a Stop event is not generated. This parameter will ensure the recording is stopped after a predefined default duration.
• smart-rec-container=<0/1>
MP4 and MKV containers are supported.
• smart-rec-interval=<val in seconds>
This is the time interval in seconds for SR start / stop events generation.
In the deepstream-test5-app, to demonstrate the use case smart record Start / Stop events are generated every interval second.
• smart-rec-file-prefix=<file name prefix>
Prefix of file name for generated video. By default, “Smart_Record” is the prefix in case this field is not set. For unique names every source must be provided with a unique prefix.
• smart-rec-dir-path=<path of directory to save the file>
Path of directory to save the recorded file. By default, the current directory is used.
Recording also can be triggered by JSON messages received from the cloud. The message format is as follows:
{
command: string // <start-recording / stop-recording>
start: string // "2020-05-18T20:02:00.051Z"
end: string // "2020-05-18T20:02:02.851Z",
sensor: {
id: string
}
}
Receiving and processing such messages from the cloud is demonstrated in the deepstream-test5 sample application. This is currently supported for Kafka. To activate this functionality, populate and enable the following block in the application configuration file:
# Configure this group to enable cloud message consumer.
[message-consumer0]
enable=1
proto-lib=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_kafka_proto.so
conn-str=<host>;<port>
config-file=<broker config file e.g. cfg_kafka.txt>
subscribe-topic-list=<topic1>;<topic2>;<topicN>
# Use this option if message has sensor name as id instead of index (0,1,2 etc.).
#sensor-list-file=dstest5_msgconv_sample_config.txt
While the application is running, use a Kafka broker to publish the above JSON messages on topics in the subscribe-topic-list to start and stop recording.