Advanced Usage

This page describes advanced and non-standard usage of the MolMIM NIM, including benchmarking the performance of the NIM locally and controlling the log levels of the NIM for monitoring and debugging.

Benchmarking the accuracy and performance of autoencoder-style models, such as MolMIM, is crucial to evaluate their effectiveness. Accuracy can be measured by evaluating the reconstruction accuracy, which is the ability of the model to accurately generate the original input SMILES string from its encoded representation. Performance can be measured by the time it takes for the model to generate the output starting from the input, which is known as the decoding time. The script provided below demonstrates how to benchmark the accuracy and performance of such a model for a single SMILES string example. It sends a POST request to the model’s /hidden endpoint to encode the input SMILES string, and then sends another POST request to the /decode endpoint to decode the encoded representation back into a SMILES string. The script measures the time it takes for each of these steps and checks if the generated SMILES string matches the original input. This script can be easily extended to cover multiple SMILES strings by modifying the smiles variable and sending multiple requests to the model’s endpoints.

            

            
import requests
import time

# You can change this line to match the URL and/or port of the MolMIM NIM
url = "http://localhost:8000"

# Input SMILES
smiles = "CC[C@@H](OC)[C@H](Cl)C(=O)N1CCCCC1"

# Send POST request to /hidden endpoint
data = {"sequences": [smiles]}
start_time = time.time()
response = requests.post(f"{url}/hidden", json=data)
hidden_time = time.time() - start_time
hidden_output = response.json()


# Send POST request to /decode endpoint
response = requests.post(f"{url}/decode", json=hidden_output)
decode_time = time.time() - (start_time + hidden_time)
output = response.json()["generated"][0]

if output == smiles:
    matches = True
else:
    matches = False

print(f"Matches:{matches}")
print(f"Hidden time:{hidden_time:.03f}")
print(f"Decode time:{decode_time:.03f}")
        

The script should generate output similar to the following:

            

            
Matches: True
Hidden time: 0.029
Decode time: 1.660
        

Controlling Logging Level

The logging level for the NIM can be controlled using the environment variable NIM_LOG_LEVEL. This variable allows you to specify the level of logging detail you want to see in the container’s logs.

Available Logging Levels

The following logging levels are available:

• DEBUG: This level logs all inputs and outputs for each endpoint of the server. This can be useful for debugging purposes, but it can also produce very large logs and should only be used when necessary.

• INFO: This level logs important events and information about the server’s operation.

• WARNING: This level logs warnings about potential issues or errors.

• ERROR: This level logs errors that occur during the server’s operation.

• CRITICAL: This level logs critical errors that prevent the server from functioning properly.

Default Logging Level

If no value is provided for NIM_LOG_LEVEL, the default logging level will be CRITICAL. This means only critical errors are logged and other levels of logging are suppressed.

Setting the Logging Level

To set the logging level, you can pass the NIM_LOG_LEVEL environment variable when starting the NIM. For example:

            

            
docker run -e NIM_LOG_LEVEL=DEBUG
        

This sets the logging level to DEBUG, which logs all inputs and outputs for each endpoint of the server.

Best Practices

When setting the logging level, you should consider the trade-off between logging detail and log size. If you set the logging level to DEBUG, you can generate very large logs that can be difficult to manage. However, if you set the logging level to a higher level (such as INFO or WARNING), you can miss important debugging information.

It is generally recommended to set the logging level to INFO or WARNING unless you have a specific need for more detailed logging.