Data Analytics

The Data Analytics service analyzes object-detection annotation files and image files, calculates insights, and generates graphs and a summary. This service supports the following tasks:

• analyze - This task analyzes the input files and generates graphs for calculated statistics. It can also generate the images with bounding boxes.

• validate - This task validates the input files by calculating the invalid coordinates and suggesting whether data needs to be revised.

• kpi_analyze - This task calculates the accuracy and average precision (AP) for a given test set.

These tasks can be invoked from the TAO Toolkit Launcher using the following convention on the command-line:

            

            
tao dataset analytics <sub_task> <args_per_subtask>
        

Where args_per_subtask are the command-line arguments required for a given subtask. Each subtask is explained in detail in the following sections.

Data Analytics expects a directory of images and a directory of annotated KITTI text files or a COCO JSON file.

Refer to the Data Annotation Format KITTI and COCO sections for more information about the data formats.

Here is an example spec file for analyzing KITTI input data.

            

            
data:
    input_format: "KITTI"
    output_dir: /path/to/results_dir/
    image_dir: /path/to/images_dir/
    ann_path: /path/to/annotation_dir/
workers: 36
image:
  generate_image_with_bounding_box: False
  image_sample_size: 100
graph:
    generate_summary_and_graph: True
    height: 15
    width: 15
    show_all: False
wandb:
    visualize: False
    project: "tao data analytics"
        

data

image

graph

wandb

Here is an example spec file for validating COCO input data.

            

            
apply_correction: True
data:
  output_dir: /path/to/result_dir/
  input_format: "COCO"
  image_dir: /path/to/images_dir/
  ann_path: /path/to/annotation_dir/
workers: 36
        

data

Use the following command to analyze the data:

            

            
tao dataset analytics analyze -e <experiment_spec>
        

Required Arguments

• -e, --experiment_spec_file: The experiment spec file to set up the analyze experiment

Here’s an example of using the data analyze command:

            

            
tao dataset analytics analyze -e $DEFAULT_SPEC
        

Result

The result directory or wandb contains the generated images with bounding boxes and graph PDFs for the below attributes.

• Bounding box area

• Object count

• Occlusion(only for kitti input)

• Truncation(only for kitti input)

• Image size

• Invalid bounding box coordinates (contains information about inverted and out-of-bounds coordinates)

Use the following command to validate the data:

            

            
tao dataset analytics validate -e <experiment_spec>
        

Required Arguments

• -e, --experiment_spec_file: The experiment spec file to set up the validate experiment

Here’s an example of using the data validate command:

            

            
tao dataset analytics validate -e $DEFAULT_SPEC
        

Result

The console output contains the validation summary. The results directory contains the corrected input files if apply_correction=True is specified. Below are the correction conditions for bounding box coordinates.

• Set negative coordinates to 0.

• Swap the inverted coordinates.

• If xmax is greater than image_width, then set xmax = image_width.

• If ymax is greater than image_height, then set ymax = image_height.

Here is an example spec file for calculating the KPI accuracy and average precision (AP) using KITTI data.

            

            
data:
  input_format: KITTI
  output_dir: /path/to/result_dir/
  kpi_sources:
    - image_dir: /path/to/raw_images_dir/
      ground_truth_ann_path: /path/to/gt_annotation_dir/
      inference_ann_path: /path/to/infer_annotation_dir/
  mapping: /path/to/mapping_json/
visualize:
  platform: wandb
kpi:
  iou_threshold: 0.5
  filter: False
  num_recall_points: 11
  conf_threshold: 0.3
  ignore_sqwidth: 40
wandb:
  visualize: True
  project: kpi_calculation
        

data

visualize

kpi

wandb

Use the following command to calculate KPI on the data:

            

            
tao dataset analytics kpi_analyze -e <experiment_spec>
        

Required Arguments

• -e, --experiment_spec_file: The experiment spec file to configure the kpi_analyze experiment

Here’s an example of using the data kpi_analyze command:

            

            
tao dataset analytics kpi_analyze -e $DEFAULT_SPEC
        

Result

The precision recall curve will be saved as an image in the output results directory (output_dir) or displayed in WandB.