# Python API ## Model loading functions ### nvforest.load_model(model_file: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [Path](https://docs.python.org/3/library/pathlib.html#pathlib.Path), , model_type: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, device: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'auto', layout: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'depth_first', default_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, align_bytes: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, precision: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, device_id: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, stream: StreamLike | [None](https://docs.python.org/3/builtins/constants.html#None) = None, handle: Handle | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → ForestInference[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_factory.py#L67) Load a model into nvForest from a serialized model file. * **Parameters:** **model_file** : The path to the serialized model file. This can be an XGBoost binary or JSON file, a LightGBM text file, or a Treelite checkpoint file. If the model_type parameter is not passed, an attempt will be made to load the file based on its extension. **model_type** : “treelite_checkpoint”, None}, default=None The serialization format for the model file. If None, a best-effort guess will be made based on the file extension. **device: {“auto”, “gpu”, “cpu”}, default=”auto”** : Whether to use GPU or CPU for inferencing. If set to “auto”, GPU will be selected if it is available. **layout** : The in-memory layout to be used during inference for nodes of the forest model. This parameter is available purely for runtime optimization. For performance-critical applications, it is recommended that available layouts be tested with realistic batch sizes to determine the optimal value. **default_chunk_size** : If set, predict calls without a specified chunk size will use this default value. **align_bytes** : Pad each tree with empty nodes until its in-memory size is a multiple of the given value. If None, use 0 for GPU and 64 for CPU. **precision** : Use the given floating point precision for evaluating the model. If None, use the native precision of the model. Note that single-precision execution is substantially faster than double-precision execution, so double-precision is recommended only for models trained in double precision and when exact conformance between results from nvForest and the original training framework is of paramount importance. **device_id** : For GPU execution, the device on which to load and execute this model. For CPU execution, this value is currently ignored. **stream** : For GPU execution, the CUDA stream to use during model loading and inference. If not given, a new stream will be created. For CPU execution, this value is ignored. **handle** : **Deprecated**. This argument will be removed in 27.02. Please use the `stream` argument instead. For GPU execution, the nvForest handle containing the stream or stream pool to use during loading and inference. ### nvforest.load_from_sklearn(skl_model: [Any](https://docs.python.org/3/library/typing.html#typing.Any), , device: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'auto', layout: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'depth_first', default_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, align_bytes: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, precision: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, device_id: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, stream: StreamLike | [None](https://docs.python.org/3/builtins/constants.html#None) = None, handle: Handle | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → ForestInference[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_factory.py#L177) Load a Scikit-Learn forest model to nvForest * **Parameters:** **skl_model** : The Scikit-Learn forest model to load. **device: {“auto”, “gpu”, “cpu”}, default=”auto”** : Whether to use GPU or CPU for inferencing. If set to “auto”, GPU will be selected if it is available. **layout** : The in-memory layout to be used during inference for nodes of the forest model. This parameter is available purely for runtime optimization. For performance-critical applications, it is recommended that available layouts be tested with realistic batch sizes to determine the optimal value. **default_chunk_size** : If set, predict calls without a specified chunk size will use this default value. **align_bytes** : Pad each tree with empty nodes until its in-memory size is a multiple of the given value. If None, use 0 for GPU and 64 for CPU. **precision** : Use the given floating point precision for evaluating the model. If None, use the native precision of the model. Note that single-precision execution is substantially faster than double-precision execution, so double-precision is recommended only for models trained in double precision and when exact conformance between results from nvForest and the original training framework is of paramount importance. **device_id** : For GPU execution, the device on which to load and execute this model. For CPU execution, this value is currently ignored. **stream** : For GPU execution, the CUDA stream to use during model loading and inference. If not given, a new stream will be created. For CPU execution, this value is ignored. **handle** : **Deprecated**. This argument will be removed in 27.02. Please use the `stream` argument instead. For GPU execution, the nvForest handle containing the stream or stream pool to use during loading and inference. ### nvforest.load_from_treelite_model(tl_model: [Model](https://treelite.readthedocs.io/en/latest/treelite-api.html#treelite.Model), , device: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'auto', layout: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'depth_first', default_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, align_bytes: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, precision: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, device_id: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, stream: StreamLike | [None](https://docs.python.org/3/builtins/constants.html#None) = None, handle: Handle | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → ForestInference[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_factory.py#L246) Load a Treelite forest model to nvForest * **Parameters:** **tl_model** : The Treelite model to load. **device: {“auto”, “gpu”, “cpu”}, default=”auto”** : Whether to use GPU or CPU for inferencing. If set to “auto”, GPU will be selected if it is available. **layout** : The in-memory layout to be used during inference for nodes of the forest model. This parameter is available purely for runtime optimization. For performance-critical applications, it is recommended that available layouts be tested with realistic batch sizes to determine the optimal value. **default_chunk_size** : If set, predict calls without a specified chunk size will use this default value. **align_bytes** : Pad each tree with empty nodes until its in-memory size is a multiple of the given value. If None, use 0 for GPU and 64 for CPU. **precision** : Use the given floating point precision for evaluating the model. If None, use the native precision of the model. Note that single-precision execution is substantially faster than double-precision execution, so double-precision is recommended only for models trained in double precision and when exact conformance between results from nvForest and the original training framework is of paramount importance. **device_id** : For GPU execution, the device on which to load and execute this model. For CPU execution, this value is currently ignored. **stream** : For GPU execution, the CUDA stream to use during model loading and inference. If not given, a new stream will be created. For CPU execution, this value is ignored. **handle** : **Deprecated**. This argument will be removed in 27.02. Please use the `stream` argument instead. For GPU execution, the nvForest handle containing the stream or stream pool to use during loading and inference. ### *class* nvforest.CPUForestInferenceClassifier(, treelite_model: [Model](https://treelite.readthedocs.io/en/latest/treelite-api.html#treelite.Model), stream: StreamLike | [None](https://docs.python.org/3/builtins/constants.html#None) = None, layout: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'depth_first', default_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, align_bytes: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, precision: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, handle: Handle | [None](https://docs.python.org/3/builtins/constants.html#None) = None)[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L258) * **Attributes:** **align_bytes** **default_chunk_size** **device_id** **layout** **num_features** **num_outputs** **num_trees** **precision** ### Methods | [`apply`](#nvforest.CPUForestInferenceClassifier.apply)(X, \*[, chunk_size]) | Output the ID of the leaf node for each tree. | |------------------------------------------------------------------------------------------|--------------------------------------------------------| | [`optimize`](#nvforest.CPUForestInferenceClassifier.optimize)(\*[, data, batch_size, ...]) | Find the optimal layout and chunk size for this model. | | [`predict`](#nvforest.CPUForestInferenceClassifier.predict)(X, \*[, chunk_size, threshold]) | Predict the class for each row. | | [`predict_per_tree`](#nvforest.CPUForestInferenceClassifier.predict_per_tree)(X, \*[, chunk_size]) | Output prediction of each tree. | | [`predict_proba`](#nvforest.CPUForestInferenceClassifier.predict_proba)(X, \*[, chunk_size]) | Predict the class probabilities for each row in X. | #### apply(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L336) Output the ID of the leaf node for each tree. * **Parameters:** **X** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. #### optimize(, data=None, batch_size: [int](https://docs.python.org/3/builtins/functions.html#int) = 1024, unique_batches: [int](https://docs.python.org/3/builtins/functions.html#int) = 10, timeout: [float](https://docs.python.org/3/builtins/functions.html#float) = 0.2, predict_method: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'predict', max_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, seed: [int](https://docs.python.org/3/builtins/functions.html#int) = 0) → [Self](https://docs.python.org/3/library/typing.html#typing.Self)[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L344) Find the optimal layout and chunk size for this model. Returns a new model instance with the optimal layout and chunk size. The optimal values depend on the model, batch size, and available hardware. In order to get the most realistic performance distribution, example data can be provided. If it is not, random data will be generated based on the indicated batch size. * **Parameters:** **data** : Example data either of shape unique_batches x batch_size x features or batch_size x features or None. If None, random data will be generated instead. **batch_size** : If example data is not provided, random data with this many rows per batch will be used. **unique_batches** : The number of unique batches to generate if random data are used. Increasing this number decreases the chance that the optimal configuration will be skewed by a single batch with unusual performance characteristics. **timeout** : Time in seconds to target for optimization. The optimization loop will be repeatedly run a number of times increasing in the sequence 1, 2, 5, 10, 20, 50, … until the time taken is at least the given value. Note that for very large batch sizes and large models, the total elapsed time may exceed this timeout; it is a soft target for elapsed time. Setting the timeout to zero will run through the indicated number of unique batches exactly once. Defaults to 0.2s. **predict_method** : If desired, optimization can occur over one of the prediction method variants (e.g. “predict_per_tree”) rather than the default [`predict`](#nvforest.CPUForestInferenceClassifier.predict) method. To do so, pass the name of the method here. **max_chunk_size** : The maximum chunk size to explore during optimization. If not set, a value will be picked based on the current device type. Setting this to a lower value will reduce the optimization search time but may not result in optimal performance. **seed** : The random seed used for generating example data if none is provided. * **Returns:** Self : A new model instance with optimal layout and default_chunk_size. #### predict(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, threshold: [float](https://docs.python.org/3/builtins/functions.html#float) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L310) Predict the class for each row. * **Parameters:** **X** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. **threshold** : For binary classifiers, output probabilities above this threshold will be considered positive detections. If None, a threshold of 0.5 will be used for binary classifiers. For multiclass classifiers, the highest probability class is chosen regardless of threshold. #### predict_per_tree(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L328) Output prediction of each tree. This function computes one or more margin scores per tree. * **Parameters:** **X:** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. #### predict_proba(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L320) Predict the class probabilities for each row in X. * **Parameters:** **X** : The input data of shape Rows \* Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. ### *class* nvforest.CPUForestInferenceRegressor(, treelite_model: [Model](https://treelite.readthedocs.io/en/latest/treelite-api.html#treelite.Model), stream: StreamLike | [None](https://docs.python.org/3/builtins/constants.html#None) = None, layout: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'depth_first', default_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, align_bytes: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, precision: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, handle: Handle | [None](https://docs.python.org/3/builtins/constants.html#None) = None)[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L399) * **Attributes:** **align_bytes** **default_chunk_size** **device_id** **layout** **num_features** **num_outputs** **num_trees** **precision** ### Methods | [`apply`](#nvforest.CPUForestInferenceRegressor.apply)(X, \*[, chunk_size]) | Output the ID of the leaf node for each tree. | |----------------------------------------------------------------------------------------|--------------------------------------------------------| | [`optimize`](#nvforest.CPUForestInferenceRegressor.optimize)(\*[, data, batch_size, ...]) | Find the optimal layout and chunk size for this model. | | [`predict`](#nvforest.CPUForestInferenceRegressor.predict)(X, \*[, chunk_size]) | Predict the output for each row. | | [`predict_per_tree`](#nvforest.CPUForestInferenceRegressor.predict_per_tree)(X, \*[, chunk_size]) | Output prediction of each tree. | #### apply(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L465) Output the ID of the leaf node for each tree. * **Parameters:** **X** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. #### optimize(, data=None, batch_size: [int](https://docs.python.org/3/builtins/functions.html#int) = 1024, unique_batches: [int](https://docs.python.org/3/builtins/functions.html#int) = 10, timeout: [float](https://docs.python.org/3/builtins/functions.html#float) = 0.2, predict_method: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'predict', max_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, seed: [int](https://docs.python.org/3/builtins/functions.html#int) = 0) → [Self](https://docs.python.org/3/library/typing.html#typing.Self)[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L473) Find the optimal layout and chunk size for this model. Returns a new model instance with the optimal layout and chunk size. The optimal values depend on the model, batch size, and available hardware. In order to get the most realistic performance distribution, example data can be provided. If it is not, random data will be generated based on the indicated batch size. * **Parameters:** **data** : Example data either of shape unique_batches x batch_size x features or batch_size x features or None. If None, random data will be generated instead. **batch_size** : If example data is not provided, random data with this many rows per batch will be used. **unique_batches** : The number of unique batches to generate if random data are used. Increasing this number decreases the chance that the optimal configuration will be skewed by a single batch with unusual performance characteristics. **timeout** : Time in seconds to target for optimization. The optimization loop will be repeatedly run a number of times increasing in the sequence 1, 2, 5, 10, 20, 50, … until the time taken is at least the given value. Note that for very large batch sizes and large models, the total elapsed time may exceed this timeout; it is a soft target for elapsed time. Setting the timeout to zero will run through the indicated number of unique batches exactly once. Defaults to 0.2s. **predict_method** : If desired, optimization can occur over one of the prediction method variants (e.g. “predict_per_tree”) rather than the default [`predict`](#nvforest.CPUForestInferenceRegressor.predict) method. To do so, pass the name of the method here. **max_chunk_size** : The maximum chunk size to explore during optimization. If not set, a value will be picked based on the current device type. Setting this to a lower value will reduce the optimization search time but may not result in optimal performance. **seed** : The random seed used for generating example data if none is provided. * **Returns:** Self : A new model instance with optimal layout and default_chunk_size. #### predict(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L449) Predict the output for each row. * **Parameters:** **X** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. #### predict_per_tree(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L457) Output prediction of each tree. This function computes one or more margin scores per tree. * **Parameters:** **X:** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. ### *class* nvforest.GPUForestInferenceClassifier(, treelite_model: [Model](https://treelite.readthedocs.io/en/latest/treelite-api.html#treelite.Model), stream: StreamLike | [None](https://docs.python.org/3/builtins/constants.html#None) = None, layout: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'depth_first', default_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, align_bytes: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, precision: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, handle: Handle | [None](https://docs.python.org/3/builtins/constants.html#None) = None, device_id: [int](https://docs.python.org/3/builtins/functions.html#int))[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L528) * **Attributes:** **align_bytes** **default_chunk_size** **device_id** **layout** **num_features** **num_outputs** **num_trees** **precision** ### Methods | [`apply`](#nvforest.GPUForestInferenceClassifier.apply)(X, \*[, chunk_size]) | Output the ID of the leaf node for each tree. | |------------------------------------------------------------------------------------------|--------------------------------------------------------| | [`optimize`](#nvforest.GPUForestInferenceClassifier.optimize)(\*[, data, batch_size, ...]) | Find the optimal layout and chunk size for this model. | | [`predict`](#nvforest.GPUForestInferenceClassifier.predict)(X, \*[, chunk_size, threshold]) | Predict the class for each row. | | [`predict_per_tree`](#nvforest.GPUForestInferenceClassifier.predict_per_tree)(X, \*[, chunk_size]) | Output prediction of each tree. | | [`predict_proba`](#nvforest.GPUForestInferenceClassifier.predict_proba)(X, \*[, chunk_size]) | Predict the class probabilities for each row in X. | #### apply(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L608) Output the ID of the leaf node for each tree. * **Parameters:** **X** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. #### optimize(, data=None, batch_size: [int](https://docs.python.org/3/builtins/functions.html#int) = 1024, unique_batches: [int](https://docs.python.org/3/builtins/functions.html#int) = 10, timeout: [float](https://docs.python.org/3/builtins/functions.html#float) = 0.2, predict_method: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'predict', max_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, seed: [int](https://docs.python.org/3/builtins/functions.html#int) = 0) → [Self](https://docs.python.org/3/library/typing.html#typing.Self)[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L616) Find the optimal layout and chunk size for this model. Returns a new model instance with the optimal layout and chunk size. The optimal values depend on the model, batch size, and available hardware. In order to get the most realistic performance distribution, example data can be provided. If it is not, random data will be generated based on the indicated batch size. * **Parameters:** **data** : Example data either of shape unique_batches x batch_size x features or batch_size x features or None. If None, random data will be generated instead. **batch_size** : If example data is not provided, random data with this many rows per batch will be used. **unique_batches** : The number of unique batches to generate if random data are used. Increasing this number decreases the chance that the optimal configuration will be skewed by a single batch with unusual performance characteristics. **timeout** : Time in seconds to target for optimization. The optimization loop will be repeatedly run a number of times increasing in the sequence 1, 2, 5, 10, 20, 50, … until the time taken is at least the given value. Note that for very large batch sizes and large models, the total elapsed time may exceed this timeout; it is a soft target for elapsed time. Setting the timeout to zero will run through the indicated number of unique batches exactly once. Defaults to 0.2s. **predict_method** : If desired, optimization can occur over one of the prediction method variants (e.g. “predict_per_tree”) rather than the default [`predict`](#nvforest.GPUForestInferenceClassifier.predict) method. To do so, pass the name of the method here. **max_chunk_size** : The maximum chunk size to explore during optimization. If not set, a value will be picked based on the current device type. Setting this to a lower value will reduce the optimization search time but may not result in optimal performance. **seed** : The random seed used for generating example data if none is provided. * **Returns:** Self : A new model instance with optimal layout and default_chunk_size. #### predict(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, threshold: [float](https://docs.python.org/3/builtins/functions.html#float) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L582) Predict the class for each row. * **Parameters:** **X** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. **threshold** : For binary classifiers, output probabilities above this threshold will be considered positive detections. If None, a threshold of 0.5 will be used for binary classifiers. For multiclass classifiers, the highest probability class is chosen regardless of threshold. #### predict_per_tree(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L600) Output prediction of each tree. This function computes one or more margin scores per tree. * **Parameters:** **X:** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. #### predict_proba(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L592) Predict the class probabilities for each row in X. * **Parameters:** **X** : The input data of shape Rows \* Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. ### *class* nvforest.GPUForestInferenceRegressor(, treelite_model: [Model](https://treelite.readthedocs.io/en/latest/treelite-api.html#treelite.Model), stream: StreamLike | [None](https://docs.python.org/3/builtins/constants.html#None) = None, layout: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'depth_first', default_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, align_bytes: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, precision: [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, handle: Handle | [None](https://docs.python.org/3/builtins/constants.html#None) = None, device_id: [int](https://docs.python.org/3/builtins/functions.html#int))[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L671) * **Attributes:** **align_bytes** **default_chunk_size** **device_id** **layout** **num_features** **num_outputs** **num_trees** **precision** ### Methods | [`apply`](#nvforest.GPUForestInferenceRegressor.apply)(X, \*[, chunk_size]) | Output the ID of the leaf node for each tree. | |----------------------------------------------------------------------------------------|--------------------------------------------------------| | [`optimize`](#nvforest.GPUForestInferenceRegressor.optimize)(\*[, data, batch_size, ...]) | Find the optimal layout and chunk size for this model. | | [`predict`](#nvforest.GPUForestInferenceRegressor.predict)(X, \*[, chunk_size]) | Predict the output for each row. | | [`predict_per_tree`](#nvforest.GPUForestInferenceRegressor.predict_per_tree)(X, \*[, chunk_size]) | Output prediction of each tree. | #### apply(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L739) Output the ID of the leaf node for each tree. * **Parameters:** **X** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. #### optimize(, data=None, batch_size: [int](https://docs.python.org/3/builtins/functions.html#int) = 1024, unique_batches: [int](https://docs.python.org/3/builtins/functions.html#int) = 10, timeout: [float](https://docs.python.org/3/builtins/functions.html#float) = 0.2, predict_method: [str](https://docs.python.org/3/builtins/stdtypes.html#str) = 'predict', max_chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None, seed: [int](https://docs.python.org/3/builtins/functions.html#int) = 0) → [Self](https://docs.python.org/3/library/typing.html#typing.Self)[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L747) Find the optimal layout and chunk size for this model. Returns a new model instance with the optimal layout and chunk size. The optimal values depend on the model, batch size, and available hardware. In order to get the most realistic performance distribution, example data can be provided. If it is not, random data will be generated based on the indicated batch size. * **Parameters:** **data** : Example data either of shape unique_batches x batch_size x features or batch_size x features or None. If None, random data will be generated instead. **batch_size** : If example data is not provided, random data with this many rows per batch will be used. **unique_batches** : The number of unique batches to generate if random data are used. Increasing this number decreases the chance that the optimal configuration will be skewed by a single batch with unusual performance characteristics. **timeout** : Time in seconds to target for optimization. The optimization loop will be repeatedly run a number of times increasing in the sequence 1, 2, 5, 10, 20, 50, … until the time taken is at least the given value. Note that for very large batch sizes and large models, the total elapsed time may exceed this timeout; it is a soft target for elapsed time. Setting the timeout to zero will run through the indicated number of unique batches exactly once. Defaults to 0.2s. **predict_method** : If desired, optimization can occur over one of the prediction method variants (e.g. “predict_per_tree”) rather than the default [`predict`](#nvforest.GPUForestInferenceRegressor.predict) method. To do so, pass the name of the method here. **max_chunk_size** : The maximum chunk size to explore during optimization. If not set, a value will be picked based on the current device type. Setting this to a lower value will reduce the optimization search time but may not result in optimal performance. **seed** : The random seed used for generating example data if none is provided. * **Returns:** Self : A new model instance with optimal layout and default_chunk_size. #### predict(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L723) Predict the output for each row. * **Parameters:** **X** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512. #### predict_per_tree(X: [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray, , chunk_size: [int](https://docs.python.org/3/builtins/functions.html#int) | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [ndarray](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray) | cupy.ndarray[[source]](https://github.com/rapidsai/nvforest/blob/1a1b92b/python/nvforest/nvforest/_forest_inference.py#L731) Output prediction of each tree. This function computes one or more margin scores per tree. * **Parameters:** **X:** : The input data of shape Rows X Features. This can be a numpy array or cupy array. nvForest is optimized for C-major arrays (e.g. numpy/cupy arrays). Inputs whose datatype does not match the precision of the loaded model (float/double) will be converted to the correct datatype before inference. If this input is in a memory location that is inaccessible to the current device type (as set with the ‘device’ parameter in the constructor), it will be copied to the correct location on the model’s CUDA stream. **chunk_size** : The number of rows to simultaneously process in one iteration of the inference algorithm. Batches are further broken down into “chunks” of this size when assigning available threads to tasks. The choice of chunk size can have a substantial impact on performance, but the optimal choice depends on model and hardware and is difficult to predict a priori. In general, larger batch sizes benefit from larger chunk sizes, and smaller batch sizes benefit from small chunk sizes. On GPU, valid values are powers of 2 from 1 to 32. On CPU, valid values are any power of 2, but little benefit is expected above a chunk size of 512.