===================================
NVIDIA Self-Hosted Server Overview
===================================

NVIDIA cuOpt is also an example containerized server built upon the core functionality of cuOpt and can be easily deployed.

.. image:: images/genms_cuopt.png
  :width: 600
  :align: center

The example server uses HTTP POST requests on port 5000 to accept optimization input data. cuOpt works on the given data with options/constraints provided, and returns optimized route plans in response.

.. image:: images/sh-cuOpt-stack.png
  :width: 500
  :align: center

=================
Quickstart Guide
=================

Step 1: Get an NVIDIA AI Enterprise Account
---------------------------------------------

#. Get a subscription for `NVIDIA AI Enterprise (NVAIE) <https://www.nvidia.com/en-us/data-center/products/ai-enterprise/>`__ to get the cuOpt container to host in your cloud.

Step 2: Access NGC
-------------------

#. Log into NGC using the invite and choose the appropriate NGC org.

#. Generate an NGC API key from settings. If you have not generated an API Key, you can generate it by going to the Setup option in your profile and choose Get API Key. Store this or generate a new one next time. More information can be found `here <https://docs.nvidia.com/ngc/ngc-private-registry-user-guide/index.html#generating-api-key>`__.


Step 3: Pull a cuOpt Container
-------------------------------

#. Prerequisites for running the container:

    * `nvidia-docker2 <https://docs.nvidia.com/deeplearning/frameworks/user-guide/index.html#nvcontainers>`_ needs to be installed

    * A single NVIDIA GPU of `Ampere <https://en.wikipedia.org/wiki/Ampere_(microarchitecture)>`_ or `Hopper <https://en.wikipedia.org/wiki/Hopper_(microarchitecture)>`_ architecture.

    * Multi GPU is not supported. cuOpt uses only one GPU.
   
    * CPU - x86-64 >= 8 core (Recommended)

    * Memory >= 16 GB (Recommended)

    * Minimum Storage: 20 GB (8 GB container size)

    * CUDA 11.8+

    * Compute Capability >= 8.x

    * Minimum NVIDIA Driver Version: 450.36.06


#. Go to the container section for cuOpt and copy the pull tag for the latest image.

   #. Within the **Select a tag** dropdown, locate the container image release that you want to run.

   #. Click the **Copy Image Path** button to copy the container image path.


#. Log into the nvcr.io container registry in your cluster setup, using the NGC API key as shown below.

.. code-block:: bash

    docker login nvcr.io
    Username: $oauthtoken
    Password: <my-api-key>

.. note::
    The username is ``$oauthtoken`` and the password is your API key.

#. Pull the cuOpt container.

	The container for cuOpt can be found in the ``Containers`` tab in NGC. Please copy the tag from the cuOpt container page and use it as follows,

.. code-block:: bash

    # Save the image tag in a variable for use below
    export CUOPT_IMAGE="CONTAINER_TAG_COPIED_FROM_NGC"

    docker pull $CUOPT_IMAGE


Step 4: Running cuOpt
---------------------

.. note::
   The commands below run the cuOpt container in detached mode. To stop a detached container, use the ``docker stop`` command. To instead run the container in interactive mode, remove the ``-d`` option.


#. If you have Docker 19.03 or later:

    - A typical command to launch the container is:

    .. code-block:: bash

        docker run -it -d --gpus=1 --rm -p 5000:5000 $CUOPT_IMAGE


#. If you have Docker 19.02 or earlier:

    - A typical command to launch the container is:

    .. code-block:: bash

        nvidia-docker run -it -d --gpus=1 --rm -p 5000:5000 $CUOPT_IMAGE


#. By default the container runs on port ``5000``, but this can be changed using environment variable ``CUOPT_SERVER_PORT``,

    .. code-block:: bash

        docker run -it -d --gpus=1 -e CUOPT_SERVER_PORT=8080 --rm -p 8080:8080 $CUOPT_IMAGE

#. If you have multiple GPUs and would like to choose particular gpu, please use ``--gpus device=<GPU_ID>``, ``GPU_ID`` can be found using command ``nvidia-smi``.

#. Server can be configured for log levels and other options using environment variables, options are listed as follows,

    - CUOPT_SERVER_PORT      : For server port (default 5000).
    - CUOPT_SERVER_IP        : For server IP (default 0.0.0.0).
    - CUOPT_SERVER_LOG_LEVEL : Options are ``critical``, ``error``, ``warning``, ``info``, ``debug`` (default info). 
    - CUOPT_DATA_DIR         : A shared mount path used to optionally pass cuopt problem data files to routes endpoint, instead of sending data over network (default None).
    - CUOPT_RESULT_DIR       : A shared mount path used to optionally pass cuopt result files from routes endpoint, instead of sending data over network (default None).
    - CUOPT_MAX_RESULT       : Maximum size (kilobytes) of a result returned over http from routes endpoint when ``CUOPT_RESULT_DIR`` is set. Set to 0 to have all results written to ``CUOPT_RESULT_DIR`` (default 250).
    - CUOPT_SSL_CERTFILE     : Filepath in container for ssl certificate, may need to mount a directory for this file with read and write access.
    - CUOPT_SSL_KEYFILE      : Filepath in container for key file, may need to mount a directory for this file with read and write access.

#. This command would launch the container and :doc:`cuOpt API endpoints <open-api>` should be available for testing.


Step 5: Testing container
----------------------------------------------
#. ``curl`` can be used instead of thin client to communicate with server, for example,

    .. code-block:: bash

        curl --location 'http://<SERVER_IP>:<SERVER:PORT>/cuopt/routes' \
        --header 'Content-Type: application/json' \
        --header "CLIENT-VERSION: custom" \
        -d '{
            "cost_matrix_data": {"data": {"0": [[0, 1], [1, 0]]}},
            "task_data": {"task_locations": [1], "demand": [[1]], "task_time_windows": [[0, 10]], "service_times": [1]},
            "fleet_data": {"vehicle_locations":[[0, 0]], "capacities": [[2]], "vehicle_time_windows":[[0, 20]] },
            "solver_config": {"time_limit": 2}
            }' 


Step 6: Installing Thin Client Using Pip Index 
----------------------------------------------

.. note::
    The self-hosted thin client requires Python 3.9+

#. The thin client enables users to test quickly, but users can design their own clients using :doc:`this<sh-cli-build>`.

#. Whenever thin clients need to be updated or installed, you can directly install it using NVIDIA pip index.

    .. code-block:: bash

        pip install --upgrade --extra-index-url https://pypi.nvidia.com cuopt-sh-client
#. For more information navigate to :doc:`Self-Hosted Thin Client<sh-cli-overview>`.


Step 7: Installing cuOpt Using a Helm Chart
--------------------------------------------

#. Create a namespace in Helm.

    .. code-block:: bash
        :linenos:
    
        kubectl create namespace <some name>
        export NAMESPACE="<some name>"


#. Configure the NGC API Key as secret.

    .. code-block:: bash

        kubectl create secret docker-registry ngc-docker-reg-secret \
        -n $NAMESPACE --docker-server=nvcr.io --docker-username='$oauthtoken' \
        --docker-password=$NGC_CLI_API_KEY

#. Fetch the Helm Chart.

	Helm chart for cuOpt can be found in ``Helm Charts`` tab in NGC. Please copy the fetch tag from cuOpt helm page and use it as follows,

    .. code-block:: bash

        helm fetch <FETCH-TAG-COPIED-FROM-NGC> --username='$oauthtoken' --password=<YOUR API KEY> --untar


#. Run cuOpt server.

    .. code-block:: bash

        helm install --namespace $NAMESPACE nvidia-cuopt-chart cuopt --values cuopt/values.yaml

#. It might take some time to download the container, which is shown in the status as **PodInitializing**; otherwise it would be **Running**. Use the following commands to verify:

    .. code-block:: bash

        kubectl -n $NAMESPACE get all
    

    .. code-block:: bash
    
        NAME                                          READY   STATUS    RESTARTS   AGE
        pod/cuopt-cuopt-deployment-595656b9d6-dbqcb   1/1     Running   0          21s

        NAME                                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
        service/cuopt-cuopt-deployment-cuopt-service   ClusterIP   X.X.X.X          <none>        5000/TCP,8888/TCP   21s

        NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
        deployment.apps/cuopt-cuopt-deployment   1/1     1            1           21s

        NAME                                                DESIRED   CURRENT   READY   AGE
        replicaset.apps/cuopt-cuopt-deployment-595656b9d6   1         1         1       21s

#. Uninstalling NVIDIA cuOpt Server:

    .. code-block:: bash

        helm uninstall -n $NAMESPACE nvidia-cuopt-chart