TensorFlow For Jetson Platform

This document describes the key features, software enhancements and improvements, and known issues regarding Tensorflow 1.13.1 on Jetson Platform.

Key Features and Enhancements

This release includes the following key features and enhancements.


Using TensorFlow With Jetson Platform

If you observe any out-of-memory problems, use: config.gpu_options.allow_growth = True and/or decrease the fraction of the memory TensorFlow will use.

For example:

config.gpu_options.per_process_gpu_memory_fraction = 0.4.

If you need more storage (Jetson AGX Xavier has 32GB), you can connect an external SSD using a cable.
If you want to see which operators of your graph are placed on Jetson AGX Xavier, use tf.ConfigProto(log_device_placement=True) to see all the device placements.

Known Issues

  • Certain tasks with large memory requirements may fail due to the memory limitations of embedded systems. Creating a swap partition will effectively enable your device to make use of more memory at the cost of speed and storage space, and may solve the problem.
  • When accelerating the inference in TensorFlow with TensorRT (TF-TRT), you may experience problems with tf.estimator and standard allocator(BFC allocator). It may cause crashes and nondeterministic accuracy. If you run into issues, use cuda_malloc as an allocator (type exportTF_GPU_ALLOCATOR=”cuda_malloc”).
  • Convolutions with a large kernel size using native TensorFlow are slow. It’s because the SwapDimension0And2InTensor3Simple kernel has no implementation using tiling. This issue will be resolved in a future release. Use TF-TRT to speed up the process.
  • The TF-TRT conversion sometimes crashes without any error message. It usually happens when the OS runs out of swap memory. In this case, it can kill processes to reclaim space. If you encounter this issue, add more swap memory.
  • Some NVIDIA tools, for example nvidia-smi, are not supported on the Jetson platform. For more information, see the JetPack documentation.
  • Jetson AGX Xavier has 16GB of memory which is shared between the CPU and GPU. As a result, some applications that work on a GPU with 16GB of memory may not work on Jetson AGX Xavier because not all of the 16GB is available to the GPU.
  • The following TensorFlow unit tests will fail on Jetson AGX Xavier. Some of the TensorFlow failures are due to bugs in the test scripts; which affect only the AArch64 platform.
    • //tensorflow/core/kernels:sparse_matmul_op_test_gpu
    • //tensorflow/core/kernels:requantize_op_test
    • //tensorflow/core/kernels:quantized_bias_add_op_test
    • //tensorflow/core:util_tensor_slice_set_test
    • //tensorflow/core:platform_stacktrace_handler_test
    • //tensorflow/core:platform_profile_utils_cpu_utils_test
    • //tensorflow/core:common_runtime_gpu_gpu_device_test
    • //tensorflow/core:gpu_device_unified_memory_test_gpu
    • //tensorflow/core:common_runtime_direct_session_with_debug_test
    • //tensorflow/core:common_runtime_direct_session_with_debug_test
    • //tensorflow/core:device_tracer_test
    • //tensorflow/core:framework_unique_tensor_references_test
    • //tensorflow/python:basic_session_run_hooks_test