Quickstart

To quickly get the inference server up and running follow these steps. After you’ve seen the inference server in action you can revisit the rest of the User Guide to learn more about all the inference server features.

First, follow the instructions in Installing Prebuilt Containers to install the inference server container.

Next, use the Example Model Repository section to create an example model repository containing a couple of models that you can serve with the inference server.

Now that you have a model repository, follow the instructions in Running The Inference Server to start the inference server. Use the server’s Status endpoint to make sure the server and the models are ready for inferencing.

Finally, build and run the example image-client application to perform image classification using the models on the inference server.