kitti_config

The dataset_convert tool provides several configurable parameters. The parameters are encapsulated in a spec file to convert data from the original annotation format to the TFRecords format which the trainer can ingest. KITTI and COCO formats can be configured by using either kitti_config or coco_config respectively. You may use only one of the two in a single spec file. The spec file is a prototxt format file with following global parameters:

kitti_config: A nested prototxt configuration with multiple input parameters
coco_config: A nested prototxt configuration with multiple input parameters
image_directory_path: The path to the dataset root. The image_dir_name is appended to this path to get the input images and must be the same path specified in the experiment spec file.
target_class_mapping: The prototxt dictionary that maps the class names in the tfrecords to the target class to be trained in the network.

Here are descriptions of the configurable parameters for the kitti_config field:

Parameter	Datatype	Default	Description	Supported Values
root_directory_path	string	–	The path to the dataset root directory	–
image_dir_name	string	–	The relative path to the directory containing images from the path in root_directory_path.	–
label_dir_name	string	–	The relative path to the directory containing labels from the path in root_directory_path.	–
partition_mode	string	–	The method employed when partitioning the data to multiple folds. Two methods are supported: Random partitioning: The data is divided in to 2 folds, train and val. This mode requires that the val_split parameter be set. Sequence-wise partitioning: The data is divided into n partitions (defined by the num_partitions parameter) based on the number of sequences available.	random sequence
num_partitions	int	2 (if partition_mode is random)	The number of partitions to use to split the data (N folds). This field is ignored when the partition model is set to random, as by default only two partitions are generated: val and train. In sequence mode, the data is split into n-folds. The number of partitions is ideally fewer than the total number of sequences in the kitti_sequence_to_frames file.	n=2 for random partition n< number of sequences in the kitti_sequence_to_frames_file
image_extension	str	.png	The extension of the images in the image_dir_name parameter.	.png .jpg .jpeg
val_split	float	20	The percentage of data to be separated for validation. This only works under “random” partition mode. This partition is available in fold 0 of the TFrecords generated. Set the validation fold to 0 in the dataset_config.	0-100
kitti_sequence_to_frames_file	str		The name of the KITTI sequence to frame mapping file. This file must be present within the dataset root as mentioned in the root_directory_path.
num_shards	int	10	The number of shards per fold.	1-20

The sample configuration file shown below converts the 100% KITTI dataset to the training set.

Copy
Copied!

            
            kitti_config {
  root_directory_path: "/workspace/tao-experiments/data/"
  image_dir_name: "training/image_2"
  label_dir_name: "training/label_2"
  image_extension: ".png"
  partition_mode: "random"
  num_partitions: 2
  val_split: 0
  num_shards: 10
}
image_directory_path: "/workspace/tao-experiments/data/"
  target_class_mapping {
      key: "car"
      value: "car"
  }
  target_class_mapping {
      key: "pedestrian"
      value: "pedestrian"
  }
  target_class_mapping {
      key: "cyclist"
      value: "cyclist"
  }
  target_class_mapping {
      key: "van"
      value: "car"
  }
  target_class_mapping {
      key: "person_sitting"
      value: "pedestrian"
  }
  target_class_mapping {
      key: "truck"
      value: "car"
  }

coco_config

Here are descriptions of the configurable parameters for the coco_config field:

Parameter	Datatype	Default	Description	Supported Values
root_directory_path	string	–	The path to the dataset root directory	–
image_dir_names	string (repated)	–	The relative path to the directory containing images from the path in root_directory_path for each partition.	–
annotation_files	string (repated)	–	The relative path to the directory containing JSON file from the path in root_directory_path for each partition.	–
num_partitions	int	2	The number of partitions in the data. The number of partition must match the length of the list for image_dir_names and annotation_files. By default, two partitions are generated: val and train.	n==len(annotation_files)
num_shards	int (repeated)	[10]	The number of shards per partitions. If only one value is provided, same number of shards is applied in all partitions

The sample configuration file shown below converts the COCO dataset with training and validation data where number of shard is 32 for validation and 256 for training.

Copy
Copied!

            
            coco_config {
  root_directory_path: "/workspace/tao-experiments/data/coco"
  image_dir_names: ["val2017", "train2017"]
  annotation_files: ["annotations/instances_val2017.json", "annotations/instances_train2017.json"]
  num_partitions: 2
  num_shards: [32, 256]
}
image_directory_path: "/workspace/tao-experiments/data/coco"