tools-setup-db-azure.html

The tool currently only supports event logs stored on ABFS. The remote output storage is also expected to be ABFS (no DBFS paths). In order to get complete eventlogs for a given run-id : ` databricks clusters list | grep <run-id> databricks fs cp -r <databricks log location/<cluster id got from the above command> <destination_location> ` are a couple of commands that can be used to download all the logs associated with a given run. Please refer to the latest Databricks documentation on up-to-date information. Due to some platform limitations, it is likely that the logs may be incomplete. Thq qualification tool attempts to process them as best as possible. If the results come back empty, the rapids_4_spark_qualification_output_status.csv file can call out the failed run due to incomplete logs.

Install Databricks CLI
- Install the Databricks CLI version 0.200+. Follow the instructions on Install the CLI.
- Set the configuration settings and credentials of the Databricks CLI:
- Set up authentication by following these instructions
- Verify that the access credentials are stored in the file ~/.databrickscfg on Unix, Linux, or macOS, or in another file defined by environment variable DATABRICKS_CONFIG_FILE.
- If the configuration isn’t set to default values, then make sure to explicitly set some environment variables to be picked up by the tools cmd such as: DATABRICKS_CONFIG_FILE, DATABRICKS_HOST and DATABRICKS_TOKEN. Refer to the description of the variables in environment variables docs.
Install Azure CLI
- Install the Azure CLI. Follow the instructions on How to install the Azure CLI.
- Set the configuration settings and credentials of the Azure CLI:
  - Set up the authentication by following these instructions.
  - Configure the Azure CLI by following these instructions.
    - location is used for retreving instance type description (default is westus).
    - output should use default of json in core section.
    - Verify that the configurations are stored in the file $AZURE_CONFIG_DIR/config where the default value of AZURE_CONFIG_DIR is $HOME/.azure on Linux or macOS.
- If the configuration isn’t set to default values, then make sure to explicitly set some environment variables to be picked up by the tools cmd such as: AZURE_CONFIG_DIR and AZURE_DEFAULTS_LOCATION.