spark-rapids/user-guide/23.12.2/partials/tools-install-package-from-source.html
Checkout the code repository
git clone git@github.com:NVIDIA/spark-rapids-tools.git cd spark-rapids-tools/user_tools
Optional: Run the project in a virtual environment
python -m venv .venv source .venv/bin/activate
Build wheel file using one of the following modes:
- Fat mode
Similar to fat jar in Java, this mode solves the problem when web access is not available to download resources having Url-paths (http/https). The command builds the tools jar file and downloads the necessary dependencies and packages them with the source code into a single wheel file. You may consider this mode if the development environment has no access to download dependencies (i.e., Spark jars) during runtime.
./build.sh fat
- Default mode
This mode builds a wheel package without any jar dependencies
./build.sh
Finally, install the package using the wheel file
pip install <wheel-file>