Advanced Usage of Evo 2 NIM

This page describes how to use the advanced and non-standard features of the Evo 2 NIM, including benchmarking the performance of the NIM locally and controlling the log levels of the NIM for monitoring and debugging.

Logging

Controlling Logging Level

The logging level for the NIM can be controlled using the environment variable NIM_LOG_LEVEL. This variable allows you to specify the level of logging detail you want to see in the container’s logs.

Available Logging Levels

The following logging levels are available:

• DEBUG: Logs all inputs and outputs for each endpoint of the server, which can be used for debugging. It also produces very large logs and should only be used when necessary.

• INFO: Logs important events and information about the server’s operation.

• WARNING: Logs warnings about potential issues or errors.

• ERROR: Logs errors that occur during the server’s operation.

• CRITICAL: Logs critical errors that prevent the server from functioning properly.

Setting the Logging Level

To set the logging level, pass the NIM_LOG_LEVEL environment variable when starting the NIM. For example:

docker run -e NIM_LOG_LEVEL=DEBUG

This sets the logging level to DEBUG, which logs all inputs and outputs for each endpoint of the server.

Best Practices

When setting the logging level, consider the trade-off between logging detail and log size. If you set the logging level to DEBUG, you might generate very large logs that can be difficult to manage. However, if you set the logging level to a higher level, such as INFO or WARNING, you might miss important debugging information.

It’s recommended to set the logging level to INFO or WARNING unless you have a specific need for more detailed logging.

Default Logging Level

If no value is provided for NIM_LOG_LEVEL, the default logging level will be INFO. This means that only critical errors will be logged, and other levels of logging will be suppressed.