Profile model or callable

The Triton Model Navigator enhances models and pipelines and provides a uniform method for profiling any Python function, callable, or model. At present, our support is limited strictly to static batch profiling scenarios. Profiling is conducted for each sample provided in the dataloader. This allows for the use of various batch sizes in the dataloader, enabling testing of different batch size distributions. As an example, we will use a simple function that simply sleeps for 50ms:

import time


def custom_fn(input_):
    # wait 50ms
    time.sleep(0.05)
    return input_

Let’s provide a dataloader we will use for profiling:

# Tuple of batch size and data sample
dataloader = [(1, ["This is example input"])]

Finally, run the profiling of the function with prepared dataloader:

nav.profile(custom_fn, dataloader)

Review all possible options in the profile API.