NVIDIA TAO Toolkit v4.0.1

Gesture Recognition

The model described in this card is a classification network that aims to classify hand-crop images into five gesture types:

  • thumbs up

  • fist

  • stop

  • ok

  • two

  • random

GestureNet is generally cascaded with Bodypose for gesture-based applications.

GestureNet use case

This is a classification model with a Resnet18 backbone.

The training algorithm optimizes the network to minimize the categorical cross entropy loss for the classes. This model was trained using the Gesture Recognition training app in TAO Toolkit v3.0.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).

GestureNet is cascaded with a hand detect or a bodypose network. For example, BodyPoseNet detects human body joints, which are used to create hand crops, and GestureNet acts as a classifier that determines the gesture of the hand.

Input

  • RGB Images of 160 X 160 X 3

Output

  • Gesture category labels

The datasheet for the model is captured in its model card hosted on NGC.

© Copyright 2023, NVIDIA.. Last updated on Aug 2, 2023.