Performance | NVIDIA Medical NIM Documentation

Accuracy Performance

The accuracy performance is measured by calculating the dice score on ground truth and inference results on 7 images. The following table shows the dice score for each class and the mean dice score for each case.

Case	Image	Class	Dice Score	Mean Dice
1	spleen_2_image.nii.gz	spleen	0.9652	0.9652
2	s0996.nii	aorta	0.9682	0.9682
3	liver_129.nii.gz	liver	0.9177	0.8008
		hepatic tumor	0.6840
4	lung_034.nii.gz	lung tumor	0.8779	0.8779
5	colon_203.nii	colon cancer primaries	0.8421	0.8421
6	s0459.nii	stomach	0.9360	0.9004
		inferior vena cava	0.9087
		pancreas	0.7810
		vertebrae L1	0.9793
		vertebrae T8	0.9769
		brain	0.8203
7	s0675.nii	left rib 8	0.8731	0.9165
		right rib 3	0.9463
		right rib 12	0.9498
		right iliopsoas	0.8696
		heart	0.9439

Speed Performance

The speed performance is measured by calculating the time cost on 6 images with different GPUs. The following tables show the results for both PyTorch and TensorRT models for each case.

Case 1

Image: 256cubic.nii.gz
Classes: []
Size: 256 x 256 x 256

GPU	Model	Mean Speed (s)	Median Speed (s)
H100	PyTorch	3.028	2.968
	TensorRT	2.702	2.710
A100	PyTorch	7.126	7.147
	TensorRT	5.856	5.674
RTX 6000 Ada	PyTorch	6.757	6.763
	TensorRT	6.539	6.237
L40s	PyTorch	8.767	8.823
	TensorRT	8.543	8.720

Case 2

Image: 256cubic.nii.gz
Classes: [spleen]
Size: 256 x 256 x 256

GPU	Model	Mean Speed (s)	Median Speed (s)
H100	PyTorch	2.511	2.521
	TensorRT	2.287	2.298
A100	PyTorch	6.117	6.125
	TensorRT	4.0935	4.086
RTX 6000 Ada	PyTorch	5.514	5.593
	TensorRT	5.169	5.168
L40s	PyTorch	6.903	6.944
	TensorRT	6.597	6.594

Case 3

Image: 512cubic.nii.gz
Classes: []
Size: 512 x 512 x 512

GPU	Model	Mean Speed (s)	Median Speed (s)
H100	PyTorch	16.308	16.294
	TensorRT	15.235	15.226
A100	PyTorch	91.567	92.848
	TensorRT	87.108	84.881
RTX 6000 Ada	PyTorch	109.338	109.405
	TensorRT	103.284	103.386
L40s	PyTorch	96.068	97.135
	TensorRT	95.064	91.982

Case 4

Image: 512cubic.nii.gz
Classes: [liver]
Size: 512 x 512 x 512

GPU	Model	Mean Speed (s)	Median Speed (s)
H100	PyTorch	14.156	14.147
	TensorRT	13.079	13.066
A100	PyTorch	37.947	37.963
	TensorRT	34.156	33.852
RTX 6000 Ada	PyTorch	32.242	32.256
	TensorRT	31.434	31.149
L40s	PyTorch	33.164	34.715
	TensorRT	33.244	33.151

Case 5

Image: 512-768.nii.gz
Classes: []
Size: 512 x 512 x 768

GPU	Model	Mean Speed (s)	Median Speed (s)
H100	PyTorch	89.312	88.864
	TensorRT	87.692	87.501
A100	PyTorch	144.568	143.292
	TensorRT	140.919	140.490
RTX 6000 Ada	PyTorch	160.502	160.697
	TensorRT	151.359	144.844
L40s	PyTorch	146.684	150.386
	TensorRT	147.603	144.717

Case 6

Image: 512-768.nii.gz
Classes: [heart]
Size: 512 x 512 x 768

GPU	Model	Mean Speed (s)	Median Speed (s)
H100	PyTorch	22.957	22.991
	TensorRT	21.499	21.515
A100	PyTorch	58.676	58.831
	TensorRT	55.636	54.910
RTX 6000 Ada	PyTorch	49.822	50.126
	TensorRT	33.956	33.831
L40s	PyTorch	49.225	51.239
	TensorRT	48.805	49.690