# Getting started with nvForest

## Installation

You can install nvForest using Pip or Conda.

```console
# Using Pip: need a suffix corresponding to your CUDA version, e.g. for CUDA 13:
$ pip install nvforest-cu13
```

```console
# Using Conda: need to specify the rapidsai channel
$ conda install -c rapidsai -c conda-forge nvforest
```

Please see the [Installation Guide](https://docs.nvidia.com/datascience/install/#system-requirements)
for NVIDIA CUDA-X libraries for data science for information about supported operating systems,
GPU drivers, and CUDA versions.

## nvForest with Python

<!-- import numpy as np
from sklearn.ensemble import RandomForestClassifier

X = np.array([[1, 2], [-1, 2]], dtype="float32")
y = np.array([0, 1], dtype="int32")

skl_model = RandomForestClassifier(n_estimators=1, max_depth=1)
skl_model.fit(X, y) -->

### First example

To run inference for decision tree models, it takes only two lines of code:

```python
import nvforest

# model_dir: pathlib.Path object, pointing to a directory containing model files.
fm = nvforest.load_model(model_dir / "xgboost_model.json")
y = fm.predict(X)
```

Let us now look into all available options for nvForest.

### Model import and device selection

With nvForest, you can run tree models using CPUs and NVIDIA GPUs.
You may explicitly select which device to use by specifying the `device` parameter
in [`load_model()`](../python_api/index.html.md#nvforest.load_model). If no `device` is given, it is set to `"auto"`.
(See the note below for the behavior of `"auto"`.)

```python
# Load XGBoost JSON model, to run inference on CPU
fm = nvforest.load_model(model_dir / "xgboost_model.json", device="cpu")

# Load LightGBM model, to run inference on GPU
fm = nvforest.load_model(model_dir / "lightgbm_model.txt", device="gpu")

# Load scikit-learn random forest, to run inference on GPU
fm = nvforest.load_from_sklearn(skl_model, device="gpu")
```

#### NOTE
Automatically detecting `device`

Setting `device="auto"` in [`load_model()`](../python_api/index.html.md#nvforest.load_model) will load the tree model
onto GPU memory, if a GPU is available. If no GPU is available, the tree model will be
loaded to the main memory instead. If no `device` parameter is specified, `device="auto"`
will be used.

This feature allows you to use a single script to deploy tree models to heterogeneous array
of machines, some with NVIDIA GPUs and some without.

#### NOTE
Automatically detecting `model_type`

By default, nvForest will attempt to detect the type of the model file using the file
extension:

* `.json`: XGBoost JSON
* `.ubj`: XGBoost UBJSON
* `.txt`: LightGBM
* Other: Treelite checkpoint (produced with [`treelite.Model.serialize()`](https://treelite.readthedocs.io/en/latest/treelite-api.html#treelite.Model.serialize))

In cases where nvForest fails to detect the right model type, you may want to
specify the `model_type` explicitly:

```python
fm = nvforest.load_model(model_dir / "lightgbm_model.txt", device="gpu",
                         model_type="lightgbm")
```

The `fm` object will be one of the following types:

* [`GPUForestInferenceClassifier`](../python_api/index.html.md#nvforest.GPUForestInferenceClassifier): a classification model, to run on GPU.
  The model will reside in the GPU memory.
* [`GPUForestInferenceRegressor`](../python_api/index.html.md#nvforest.GPUForestInferenceRegressor): a regression model, to run on GPU.
  The model will reside in the GPU memory.
* [`CPUForestInferenceClassifier`](../python_api/index.html.md#nvforest.CPUForestInferenceClassifier): a classification model, to run on CPU.
  The model object will reside in the main memory.
* [`CPUForestInferenceRegressor`](../python_api/index.html.md#nvforest.CPUForestInferenceRegressor): a regression model, to run on CPU.
  The model object will reside in the main memory.

You can inspect the type of the model by printing its type:

```pycon
>>> print(type(fm).__name__)
GPUForestInferenceClassifier
```

#### NOTE
Selecting among multiple GPUs

If your system has more than one NVIDIA GPU, you can select one of them to run tree inference
by passing `device_id` parameter.

```python
# Load model to GPU device 1
fm = nvforest.load_model(model_dir / "xgboost_model.json",
                         device="gpu", device_id=1)
fm = nvforest.load_from_sklearn(skl_model, device="gpu", device_id=1)
```

Each model object is associated with a single device. Use the `device_id` property to look up
which device the model object is located on.

### CUDA stream selection

By default, nvForest creates a new
[CUDA stream](https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/asynchronous-execution.html#cuda-streams)
for a GPU model. To use a custom stream, pass in a [`cuda.core.Stream`](https://nvidia.github.io/cuda-python/cuda-core/generated/cuda.core.Stream.html#cuda.core.Stream) (or
any stream-like object <sup>[1](#id2)</sup>) via the `stream` parameter. A supplied stream must be associated with the same device as `device_id`.

```python
from cuda.core import Device

device = Device(0)
device.set_current()
stream = device.create_stream()

fm = nvforest.load_from_sklearn(
    skl_model, device="gpu", device_id=0, stream=stream
)
predictions = fm.predict(X)
```

nvForest retains a reference to a caller-supplied stream. Python prediction methods synchronize
the model’s stream before returning, so their results are ready for immediate use.

* <a id='id2'>**[1]**</a> A stream-like object is an object that exposes a method named `__cuda_stream__`. See [Stream Protocol](https://nvidia.github.io/cuda-python/cuda-core/latest/interoperability.html#cuda-stream-protocol) for more details.

### Running inference

After importing the model, run inference using [`predict()`](../python_api/index.html.md#nvforest.GPUForestInferenceRegressor.predict)
or its variants.

```python
# Run inference
pred = fm.predict(X)

# Run inference and output class probabilities
# Only applicable for classification models
class_probs = fm.predict_proba(X)

# Run inference and obtain leaf indices in each decision tree
leaf_ids = fm.apply(X)

# Run inference and obtain prediction per individual tree
pred_per_tree = fm.predict_per_tree(X)
```

<!-- assert pred.shape == (X.shape[0], 1)
assert class_probs.shape == (X.shape[0], fm.num_outputs)
assert leaf_ids.shape == (X.shape[0], fm.num_trees)
assert pred_per_tree.shape == (X.shape[0], fm.num_trees) -->

### Auto-optimization

Forest inference capabilities in nvForest allow users to fine-tune performance with a variety of hyperparameters.
It is impossible to predict what the optimal values will be for any given model and batch size, so it is
necessary to determine them empirically. nvForest significantly simplifies this process with a built-in method for
auto-optimization at any given batch size:

```python
fm_optimized = fm.optimize(batch_size=1000)
```

The [`optimize()`](../python_api/index.html.md#nvforest.GPUForestInferenceRegressor.optimize) returns a new instance of `ForestInference`, and
subsequent prediction calls will use the optimal performance hyperparameters found for the indicated batch size.
You can also check what hyperparameters were selected by looking at the attributes.

```python
# Optimal layout
fm_optimized.layout
# Optimal chunk size
fm_optimized.default_chunk_size
```

<a id="nvforest-with-c-advanced"></a>

## nvForest with C++ (Advanced)

### Integrating your C++ application with nvForest

nvForest provides a CMake config file so that other C++ projects can find and use it.

```cmake
find_package(nvforest CONFIG REQUIRED)

target_link_libraries(my_target PRIVATE nvforest::nvforest treelite::treelite)
```

To ensure that CMake can locate nvForest and Treelite, we recommend the use of Conda to install nvForest.

### How to load tree models and run inference

To import tree models into nvForest, first load the tree models as a Treelite model object.

```cpp
#include <treelite/model_loader.h>
#include <treelite/tree.h>
#include <memory>

std::unique_ptr<treelite::Model> treelite_model
    = treelite::model_loader::LoadXGBoostModelUBJSON(
        "/path/to/xgboost_model.ubj", "{}");
```

Refer to [the Treelite documentation](https://treelite.readthedocs.io/en/latest/) for
the full list of model loader utilities.

Once the tree model is available as a Treelite object, pass it to the
[`import_from_treelite_model()`](../cpp_api/index.html.md#_CPPv4N8nvforest26import_from_treelite_modelERKN8treelite5ModelE11tree_layout10index_typeNSt8optionalIbEEN8nvforest11device_typeEiN8nvforest11cuda_streamE) to load it into nvForest.

```cpp
#include <nvforest/constants.hpp>
#include <nvforest/device_type.hpp>
#include <nvforest/treelite_importer.hpp>
#include <nvforest/detail/index_type.hpp>
#include <cuda_runtime_api.h>
#include <optional>

constexpr int device_id = 0;
cudaSetDevice(device_id);
cudaStream_t stream{};
cudaStreamCreate(&stream);

auto fm = nvforest::import_from_treelite_model(
    *treelite_model,
    nvforest::preferred_tree_layout,
    nvforest::index_type{},
    std::nullopt,
    nvforest::device_type::gpu,
    device_id,
    stream);
```

Now that the tree model is fully imported into nvForest, let’s run inference:

```cpp
// Assumption:
// * Both output and input are in the GPU memory.
// * The input buffer should be of dimension (num_rows, num_features)
// * The output buffer should be of dimension (num_rows, fm.num_outputs())
fm.predict(stream, output, input, num_rows,
           nvforest::device_type::gpu, nvforest::device_type::gpu,
           nvforest::infer_kind::default_kind);

// Wait before reading or freeing output.
cudaStreamSynchronize(stream);

// Use output
// ...

// Clean up by destroying stream
// Also free input, output buffers
```

Use a stream associated with the same device as the model. Re-use the stream for import and
inference to preserve the ordering between model initialization and prediction without a global
synchronization. Keep the stream alive until the model is destroyed and all queued
work has completed.