Launching a VM Instance from the AWS Console

Logging In and Selecting the AWS Zone

  1. Log into the AWS Console (, then under the Compute section, click EC2 .
  2. Select the AWS Zone from the upper right of the top menu.

    In order to use NVIDIA Volta and Turing GPUs in AWS, you must select a region that has Amazon EC2 P3 or G4 instances available. The examples in this guide use instances in US West (Oregon) - us-west-2. Check with AWS for Amazom EC2 P3 or G4 instance availability in other regions.

Selecting the NVIDIA Deep Learning AMI

NVIDIA publishes and maintains an AMI with all the software needed to pull and run the NGC deep learning containers. This AMI should be used as the basis for your Volta instance types.

  1. Click Launch Instance.

  2. Select the NVIDIA Deep Learning AMI.
    1. Select AWS Marketplace on the left, and search for "NVIDIA Deep Learning AMI", and select the NVIDIA AMI you want to use - either the basic, the PyTorch AMI, or the TensorFlow AMI.
    2. Click Continue on the details page.

Selecting an Amazon EC2 P3/G4 Instance Type and Configuring Instance Settings

  1. Select one of the Amazon EC2 P3 or G4 instance types according to your GPU, CPU, and memory requirements.
  2. Click Review and Launch to review the default configuration settings, or continue with the instructions in the next section to configure each setting step-by-step
  3. After choosing an instance type, click Next: Configure Instance Details.

    There are no instance details that need to be configured, so you can proceed to the next step.

  4. Add storage.

    Click Next: Add Storage.

    While the default 32 GiB for the root volume can be changed, users should not use the root volume for storing datasets since the root volume is destroyed when the instance is terminated. NVIDIA provides directions in this guide for using EFS volumes for datasets and deep learning job results. See the chapter Using the Amazon Elastic File System (EFS) for Persistent Data Storage.

  5. Add tags.

    Naming your instances helps to keep multiple instances organized.

    1. Click Next: Add Tag.
    2. Click Add Tag and then fill in the following information:

      Key: "Name"

      Value: <instance name, such as "My GPU">

  6. Configure a Security Group
    1. Click Next: Configure Security Group.
    2. Click Select an existing security group and select the Security Group you created during Preliminary Setup.

Launching Your VM Instance

  1. Click Review and Launch.

    A window pops up and asks which key pair to use.

  2. Select Choose an existing key pair, select your key pair, then check the acknowledgement checkbox.
  3. Click Launch Instances.

Connecting to Your VM Instance

  1. After launching your instance, click View Instances, locate your instance from the list, then wait until it is in the ‘running’ state.
  2. When it is in the running state, select it from the list and then click Connect.
  3. Follow the instructions in the pop-up window to establish an SSH connection to the instance.

    Be sure to use 'ubuntu' for the username.

    If you plan to access locked NGC containers, you will need to log in to the NGC container registry. See Logging in to the NGC Container Registry) for instructions.

    If the instructions for SSH login do not work, see the AWS Connect to Your Linux Instance documentation for additional information.

Starting, Stopping, and Terminating Your VM Instance

Once you are done with your instance you can stop (to be started again later) or terminate (delete) it. Refer to the Instance Lifecycle in the AWS documentation for more information.

Instances can be controlled from the Instances page, using the "Actions”->"Instance State" menu to stop, start, or terminate Instances.