Advance Framework Configuration

NVIDIA AI Enterprise 2.0 or later

Securing a notebook server

The Jupyter notebook web application is based on a server-client structure. This document describes how you can secure a notebook server .

Important

The following scripts don’t take jupyter notebook security into consideration. To properly secure your Jupyter notebook use the guide listed above.

Startup Scripts for Individual Containers

Startup Scripts for Jupyter

Create a dataset directory to store all your datasets using Jupyter notebooks.
Copy

Copied!
```
            
            mkdir ~/dataset
        
```

Create a startup script and place it in the home directory.

Copy
Copied!

            
            vim /home/nvidia/startup.sh

RAPIDS Container

Add the following contents to the startup.sh script created in Startup Scripts for Jupyter section.

Copy
Copied!

            
            #!/bin/bash
podman rm -f $(podman ps -a -q)
podman run --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8888:8888 --name rapids_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/nvidia-rapids-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root  --ip='0.0.0.0'

Note

Replace /home/nvidia with your home path. Do not use $HOME, this script requires the absolute path.

TensorFlow1 Container

Add the following contents to the startup.sh script created in Startup Scripts for Jupyter section.

Copy
Copied!

            
            #!/bin/bash
podman rm -f $(podman ps -a -q)
podman run --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8888:8888 -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/tensorflow-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root  --ip='0.0.0.0'

Note

Replace /home/nvidia with your home path. Do not use $HOME, this script requires the absolute path.

TensorFlow2 Container

Add the following contents to the startup.sh script created in Startup Scripts for Jupyter section.

Copy
Copied!

            
            #!/bin/bash
podman rm -f $(podman ps -a -q)
podman run --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8888:8888 -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/tensorflow-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root  --ip='0.0.0.0'

Note

Replace /home/nvidia with your home path. Do not use $HOME, this script requires the absolute path.

PyTorch Container

Add the following contents to the startup.sh script created in Startup Scripts for Jupyter section.

Copy
Copied!

            
            #!/bin/bash
podman rm -f $(podman ps -a -q)
podman run --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8888:8888 -v /home/nvidia/dataset:/workspace/dataset nnvcr.io/nvaie/pytorch-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root  --ip='0.0.0.0'

Note

Replace /home/nvidia with your home path. Do not use $HOME, this script requires the absolute path.

Combined Startup Script

The script below autostarts Jupyter notebook for all the NVIDIA AI Enterprise containers together on a single system. In this example, Jupyter notebook for PyTorch, TensorFlow1, TensorFlow2 and RAPIDS are started on port 8888, 8889, 8890 and 8891 respectively.

Copy
Copied!

            
            #!/bin/bash
podman rm -f $(podman ps -a -q)
podman run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8888:8888  -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/pytorch-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root  --NotebookApp.token='' --ip='0.0.0.0' --port 8888
podman run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8889:8889 -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/tensorflow-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token=''  --ip='0.0.0.0' --port 8889
podman run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8890:8890 -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/tensorflow-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root  --NotebookApp.token='' --ip='0.0.0.0' --port 8890
podman run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p 8891:8891 --name rapids_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/nvidia-rapids-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token='' --ip='0.0.0.0' --port 8891

Enabling Startup Script

Give execution privileges to the script.
Copy

Copied!
```
            
            chmod +x /home/nvidia/startup.sh
        
```
Note

Replace /home/nvidia with your home path. Do not use $HOME, this script requires the absolute path.

Create a systemd process for the startup script.

Copy
Copied!

            
            sudo vim /etc/systemd/system/jupyter.service

Add the following content to the jupyter.service file.

Copy
Copied!

            
            [Unit]
Description=Starts Jupyter server

[Service]
ExecStart=/home/nvidia/startup.sh #Use your home path

[Install]
WantedBy=multi-user.target

Start and enable the startup service on reboot.

Copy
Copied!

            
            sudo systemctl start jupyter.service

Copy
Copied!

            
            sudo systemctl enable jupyter.service

Reboot the system.

Note

For the :ref: combined-startup-scripts section you can skip the next step and directly access PyTorch container, TensorFlow-v1, TensorFlow-v2 and RAPIDS Jupyter notebooks at: http://system_IP:8888, http://system_IP:8889, http://system_IP:8890, http://system_IP:8891 respectively.
To open the Jupyter Notebook you will need the token/password. This is needed to prevent unauthorized access to a Jupyter Notebook. To access the token, look at the Jupyter service logs using the command below.
Copy

Copied!
```
            
            journalctl -f -u jupyter.service
        
```

The logs will display the full URL of the Jupyter Notebook including the token.

Copy
Copied!

            
            Sep 15 16:33:58 triton-inference-server startup.sh[6315]: To access the notebook,
http://341eed905e2a:8888/?token=0a13f9068c4ea9bb2f1ca5d8ad212a26accc085da896a368

As an IT Administrator, you need to provide the data scientist with the IP of the system and the token below.

Copy
Copied!

            
            http://system_IP:8888/?token=

Example:

Copy
Copied!

            
            http://192.168.100.10:8888/?token=0a13f9068c4ea9bb2f1ca5d8ad212a26accc085da896a368

Startup Scripts for Triton Inference Server

Create a triton directory inside the system for the AI Practitioner to host the model.
Copy

Copied!
```
            
            mkdir ~/triton
        
```

Pull the latest Triton Inference Server container.

Copy
Copied!

            
            sudo podman pull nvcr.io/nvaie/tritonserver-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>

Create a startup script to run Triton Inference Server automatically on system.
Copy

Copied!
```
            
            vim ~/startup.sh
        
```

Add the following content to the startup.sh file.

Copy
Copied!

            
            #!/bin/bash
podman rm -f $(podman ps -a -q)
podman run --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8000:8000 -p8001:8001 -p8002:8002 --name triton_server_cont -v $HOME/triton_models:/models nvcr.io/nvaie/tritonserver-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> tritonserver --model-store=/models --strict-model-config=false --log-verbose=1

Make the startup script executable.

Copy
Copied!

            
            chmod +x ~/startup.sh

Create a systemd process for startup script.

Copy
Copied!

            
            sudo vim /etc/systemd/system/triton.service

Add the following content to the triton.service file.

Copy
Copied!

            
            [Unit]
Description=Starts Triton server

[Service]
ExecStart=/home/nvidia/startup.sh

[Install]
WantedBy=multi-user.target

Start and enable the startup service on reboot.

Copy
Copied!

            
            sudo systemctl start triton.service

Copy
Copied!

            
            sudo systemctl enable triton.service

Reboot the system.