User Guide (Latest)
User Guide (Latest Version)

spark-rapids/user-guide/latest/partials/tools-setup-dataproc.html

  • Install gcloud CLI. Follow the instructions on gcloud-sdk-install

  • Set the configuration settings and credentials of the gcloud CLI:

    • Initialize the gcloud CLI by following these instructions

    • Grant authorization to the gcloud CLI with a user account

    • Set up “application default credentials” to the gcloud CLI by logging in

    • Manage gcloud CLI configurations. For more details, visit gcloud-sdk-configurations

    • Verify that the following gcloud CLI properties are properly defined:

      • dataproc/region

      • compute/zone

      • compute/region

      • core/project

    • If the configuration is not set to default values, then make sure to explicitly set some environment variables to be picked up by the tools cmd such as: CLOUDSDK_DATAPROC_REGION, and CLOUDSDK_COMPUTE_REGION.

    • The tools CLI follows the process described in this doc to resolve the credentials. If not running on (GCP), the environment variable GOOGLE_APPLICATION_CREDENTIALS is required to point to a JSON file containing credentials.

© Copyright 2024, NVIDIA. Last updated on Jul 26, 2024.