# C++ API

## Model importer utilities

### inline auto nvforest::import_from_treelite_model(treelite::Model const &tl_model, [tree_layout](#_CPPv4N8nvforest11tree_layoutE) layout = [preferred_tree_layout](#_CPPv4N8nvforest21preferred_tree_layoutE), index_type align_bytes = index_type{}, std::optional<bool> use_double_precision = std::nullopt, nvforest::device_type dev_type = nvforest::device_type::cpu, int device = 0, nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE) stream = nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE){})

Import a treelite model to nvForest

Load a model from Treelite to a nvForest [forest_model](#structnvforest_1_1forest__model). The model will be inspected to determine the correct underlying [decision_forest](#structnvforest_1_1decision__forest) variant to use within the [forest_model](#structnvforest_1_1forest__model) object.

* **Parameters:**
  * **tl_model** – The Treelite Model to load 
  * **layout** – The in-memory layout of nodes in the loaded forest 
  * **align_bytes** – If non-zero, ensure that each tree is stored in a multiple of this value of bytes by padding with empty nodes. This can be useful for increasing the likelihood that successive reads will take place within a single cache line. On GPU, a value of 128 can be used for this purpose. On CPU, a value of either 0 or 64 typically produces optimal performance. 
  * **use_double_precision** – Whether or not to use 64 bit floats for model evaluation and 64 bit ints for applicable indexing 
  * **dev_type** – Which device type to use for inference (CPU or GPU) 
  * **device** – For GPU execution, the device id for the device on which this model is to be loaded 
  * **stream** – The CUDA stream to use for loading this model (can be omitted for CPU). 

### inline auto nvforest::import_from_treelite_handle([TreeliteModelHandle](https://treelite.readthedocs.io/en/latest/treelite-c-api.html#_CPPv419TreeliteModelHandle) tl_handle, [tree_layout](#_CPPv4N8nvforest11tree_layoutE) layout = [preferred_tree_layout](#_CPPv4N8nvforest21preferred_tree_layoutE), index_type align_bytes = index_type{}, std::optional<bool> use_double_precision = std::nullopt, nvforest::device_type dev_type = nvforest::device_type::cpu, int device = 0, nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE) stream = nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE){})

Import a treelite model handle to nvForest

Load a model from a Treelite model handle (type-erased treelite::Model object) to a nvForest [forest_model](#structnvforest_1_1forest__model). The model will be inspected to determine the correct underlying [decision_forest](#structnvforest_1_1decision__forest) variant to use within the [forest_model](#structnvforest_1_1forest__model) object.

* **Parameters:**
  * **tl_handle** – The Treelite ModelHandle to load 
  * **layout** – The in-memory layout of nodes in the loaded forest 
  * **align_bytes** – If non-zero, ensure that each tree is stored in a multiple of this value of bytes by padding with empty nodes. This can be useful for increasing the likelihood that successive reads will take place within a single cache line. On GPU, a value of 128 can be used for this purpose. On CPU, a value of either 0 or 64 typically produces optimal performance. 
  * **use_double_precision** – Whether or not to use 64 bit floats for model evaluation and 64 bit ints for applicable indexing 
  * **dev_type** – Which device type to use for inference (CPU or GPU) 
  * **device** – For GPU execution, the device id for the device on which this model is to be loaded 
  * **stream** – The CUDA stream to use for loading this model (can be omitted for CPU). 

### template<[tree_layout](#_CPPv4N8nvforest11tree_layoutE) layout>struct treelite_importer

Struct used to import a model from Treelite to nvForest

* **Template Parameters:**
  **layout** – The in-memory layout for nodes to be loaded into nvForest 

### Public Functions

### template<index_type variant_index>inline auto import_to_specific_variant(index_type target_variant_index, treelite::Model const &tl_model, index_type num_class, index_type num_feature, index_type max_num_categories, std::vector<index_type> const &offsets, index_type align_bytes = index_type{}, nvforest::device_type mem_type = nvforest::device_type::cpu, int device = 0, nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE) stream = nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE){})

Assuming that the correct [decision_forest](#structnvforest_1_1decision__forest) variant has been identified, import to that variant 

### inline [forest_model](#_CPPv4N8nvforest12forest_modelE) import(treelite::Model const &tl_model, index_type align_bytes = index_type{}, std::optional<bool> use_double_precision = std::nullopt, nvforest::device_type dev_type = nvforest::device_type::cpu, int device = 0, nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE) stream = nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE){})

Import a treelite model to nvForest

Load a model from Treelite to a nvForest [forest_model](#structnvforest_1_1forest__model). The model will be inspected to determine the correct underlying [decision_forest](#structnvforest_1_1decision__forest) variant to use within the [forest_model](#structnvforest_1_1forest__model) object.

* **Parameters:**
  * **tl_model** – The Treelite Model to load 
  * **align_bytes** – If non-zero, ensure that each tree is stored in a multiple of this value of bytes by padding with empty nodes. This can be useful for increasing the likelihood that successive reads will take place within a single cache line. On GPU, a value of 128 can be used for this purpose. On CPU, a value of either 0 or 64 typically produces optimal performance. 
  * **use_double_precision** – Whether or not to use 64 bit floats for model evaluation and 64 bit ints for applicable indexing 
  * **dev_type** – Which device type to use for inference (CPU or GPU) 
  * **device** – For GPU execution, the device id for the device on which this model is to be loaded 
  * **stream** – The CUDA stream to use for loading this model (can be omitted for CPU). 

## Forest classes

### struct forest_model

A model used for performing inference with nvForest

This struct is a wrapper for all variants of [decision_forest](#structnvforest_1_1decision__forest) supported by a standard nvForest build. 

### Public Functions

### inline forest_model([decision_forest_variant](#_CPPv4N8nvforest23decision_forest_variantE) &&forest = [decision_forest_variant](#_CPPv4N8nvforest23decision_forest_variantE){})

Wrap a [decision_forest](#structnvforest_1_1decision__forest) in a full [forest_model](#structnvforest_1_1forest__model) object 

### inline auto num_features()

The number of features per row expected by the model 

### inline auto num_outputs([infer_kind](#_CPPv4N8nvforest10infer_kindE) inference_kind = [infer_kind](#_CPPv4N8nvforest10infer_kindE)::[default_kind](#_CPPv4N8nvforest10infer_kind12default_kindE))

The number of outputs per row generated by the model for the given type of inference 

### inline auto num_trees()

The number of trees in the model 

### inline auto has_vector_leaves()

Whether or not leaf nodes use vector outputs 

### inline auto row_postprocessing()

The operation used for postprocessing all outputs for a single row 

### inline void set_row_postprocessing([row_op](#_CPPv4N8nvforest6row_opE) val)

Setter for [row_postprocessing()](#structnvforest_1_1forest__model_1aac4f8d740181bfcee2ba563c875df4a7)

### inline auto elem_postprocessing()

The operation used for postprocessing each element of the output for a single row 

### inline auto memory_type()

The type of memory (device/host) where the model is stored 

### inline auto device_index()

The ID of the device on which this model is loaded 

### inline auto is_double_precision()

Whether or not model is loaded at double precision 

### template<typename io_t>inline void predict([cuda_stream](#_CPPv4N8nvforest11cuda_streamE) stream, buffer<[io_t](#_CPPv4I0EN8nvforest12forest_model7predictEv11cuda_streamR6bufferI4io_tERK6bufferI4io_tE10infer_kindNSt8optionalI10index_typeEE)> &output, buffer<[io_t](#_CPPv4I0EN8nvforest12forest_model7predictEv11cuda_streamR6bufferI4io_tERK6bufferI4io_tE10infer_kindNSt8optionalI10index_typeEE)> const &input, [infer_kind](#_CPPv4N8nvforest10infer_kindE) predict_type = [infer_kind](#_CPPv4N8nvforest10infer_kindE)::[default_kind](#_CPPv4N8nvforest10infer_kind12default_kindE), std::optional<index_type> specified_chunk_size = std::nullopt)

Perform inference on given input

* **Parameters:**
  * **stream** – **[in]** A [nvforest::cuda_stream](#namespacenvforest_1a66c2cc734f8818b2ee515c61a440c98b), which (on GPU-enabled builds) is a transparent wrapper for the cudaStream_t or (on CPU-only builds) a CUDA-free placeholder object. 
  * **output** – **[out]** The buffer where model output should be stored. This must be of size at least ROWS x num_outputs(predict_type). 
  * **input** – **[in]** The buffer containing input data. 
  * **predict_type** – **[in]** Type of inference to perform. Defaults to summing the outputs of all trees and produce an output per row. If set to “per_tree”, we will instead output all outputs of individual trees. If set to “leaf_id”, we will output the integer ID of the leaf node for each tree. 
  * **specified_chunk_size** – **[in]** Specifies the mini-batch size for processing. This has different meanings on CPU and GPU, but on GPU it corresponds to the number of rows evaluated per inference iteration on a single block. It can take on any power of 2 from 1 to 32, and runtime performance is quite sensitive to the value chosen. In general, larger batches benefit from higher values, but it is hard to predict the optimal value a priori. If omitted, a heuristic will be used to select a reasonable value. On CPU, this argument can generally just be omitted. 

### template<typename io_t>inline void predict(std::function<[cuda_stream](#_CPPv4N8nvforest11cuda_streamE)()> const &stream_pool_next, std::size_t stream_pool_size, buffer<[io_t](#_CPPv4I0EN8nvforest12forest_model7predictEvRKNSt8functionIF11cuda_streamvEEENSt6size_tER6bufferI4io_tERK6bufferI4io_tE10infer_kindNSt8optionalI10index_typeEE)> &output, buffer<[io_t](#_CPPv4I0EN8nvforest12forest_model7predictEvRKNSt8functionIF11cuda_streamvEEENSt6size_tER6bufferI4io_tERK6bufferI4io_tE10infer_kindNSt8optionalI10index_typeEE)> const &input, [infer_kind](#_CPPv4N8nvforest10infer_kindE) predict_type = [infer_kind](#_CPPv4N8nvforest10infer_kindE)::[default_kind](#_CPPv4N8nvforest10infer_kind12default_kindE), std::optional<index_type> specified_chunk_size = std::nullopt)

Perform inference on given input, using multiple CUDA streams. A stream pool lets you overlap memory transfers with partitioned inference.

* **Parameters:**
  * **stream_pool_next** – **[in]** A callback function that returns a CUDA stream from the pool. 
  * **stream_pool_size** – **[in]** The number of CUDA streams in the pool. This must be greater than zero. 
  * **output** – **[out]** The buffer where model output should be stored. If this buffer is on host while the model is on device or vice versa, work will be distributed across available streams to copy the data back to this output location. This must be of size at least ROWS x num_outputs(predict_type). 
  * **input** – **[in]** The buffer containing input data. If this buffer is on host while the model is on device or vice versa, work will be distributed across available streams to copy the input data to the appropriate location and perform inference. 
  * **predict_type** – **[in]** Type of inference to perform. Defaults to summing the outputs of all trees and produce an output per row. If set to “per_tree”, we will instead output all outputs of individual trees. If set to “leaf_id”, we will output the integer ID of the leaf node for each tree. 
  * **specified_chunk_size** – **[in]** Specifies the mini-batch size for processing. This has different meanings on CPU and GPU, but on GPU it corresponds to the number of rows evaluated per inference iteration on a single block. It can take on any power of 2 from 1 to 32, and runtime performance is quite sensitive to the value chosen. In general, larger batches benefit from higher values, but it is hard to predict the optimal value a priori. If omitted, a heuristic will be used to select a reasonable value. On CPU, this argument can generally just be omitted. 

### template<typename io_t>inline void predict([cuda_stream](#_CPPv4N8nvforest11cuda_streamE) stream, [io_t](#_CPPv4I0EN8nvforest12forest_model7predictEv11cuda_streamP4io_tP4io_tNSt6size_tE11device_type11device_type10infer_kindNSt8optionalI10index_typeEE) \*output, [io_t](#_CPPv4I0EN8nvforest12forest_model7predictEv11cuda_streamP4io_tP4io_tNSt6size_tE11device_type11device_type10infer_kindNSt8optionalI10index_typeEE) \*input, std::size_t num_rows, device_type out_mem_type, device_type in_mem_type, [infer_kind](#_CPPv4N8nvforest10infer_kindE) predict_type = [infer_kind](#_CPPv4N8nvforest10infer_kindE)::[default_kind](#_CPPv4N8nvforest10infer_kind12default_kindE), std::optional<index_type> specified_chunk_size = std::nullopt)

Perform inference on given input

* **Parameters:**
  * **stream** – **[in]** A [nvforest::cuda_stream](#namespacenvforest_1a66c2cc734f8818b2ee515c61a440c98b), which (on GPU-enabled builds) is a transparent wrapper for the cudaStream_t or (on CPU-only builds) a CUDA-free placeholder object. If this argument is cudaStream_t, ensure that this stream is associated with the same GPU device as the model object. 
  * **output** – **[out]** Pointer to the memory location where output should end up 
  * **input** – **[in]** Pointer to the input data 
  * **num_rows** – **[in]** Number of rows in input 
  * **out_mem_type** – **[in]** The memory type (device/host) of the output buffer 
  * **in_mem_type** – **[in]** The memory type (device/host) of the input buffer 
  * **predict_type** – **[in]** Type of inference to perform. Defaults to summing the outputs of all trees and produce an output per row. If set to “per_tree”, we will instead output all outputs of individual trees. If set to “leaf_id”, we will output the integer ID of the leaf node for each tree. 
  * **specified_chunk_size** – **[in]** Specifies the mini-batch size for processing. This has different meanings on CPU and GPU, but on GPU it corresponds to the number of rows evaluated per inference iteration on a single block. It can take on any power of 2 from 1 to 32, and runtime performance is quite sensitive to the value chosen. In general, larger batches benefit from higher values, but it is hard to predict the optimal value a priori. If omitted, a heuristic will be used to select a reasonable value. On CPU, this argument can generally just be omitted. 

### template<typename io_t>inline void predict(std::function<[cuda_stream](#_CPPv4N8nvforest11cuda_streamE)()> const &stream_pool_next, std::size_t stream_pool_size, [io_t](#_CPPv4I0EN8nvforest12forest_model7predictEvRKNSt8functionIF11cuda_streamvEEENSt6size_tEP4io_tP4io_tNSt6size_tE11device_type11device_type10infer_kindNSt8optionalI10index_typeEE) \*output, [io_t](#_CPPv4I0EN8nvforest12forest_model7predictEvRKNSt8functionIF11cuda_streamvEEENSt6size_tEP4io_tP4io_tNSt6size_tE11device_type11device_type10infer_kindNSt8optionalI10index_typeEE) \*input, std::size_t num_rows, device_type out_mem_type, device_type in_mem_type, [infer_kind](#_CPPv4N8nvforest10infer_kindE) predict_type = [infer_kind](#_CPPv4N8nvforest10infer_kindE)::[default_kind](#_CPPv4N8nvforest10infer_kind12default_kindE), std::optional<index_type> specified_chunk_size = std::nullopt)

Perform inference on given input, using multiple CUDA streams. A stream pool lets you overlap memory transfers with partitioned inference.

* **Parameters:**
  * **stream_pool_next** – **[in]** A callback function that returns a CUDA stream from the pool. 
  * **stream_pool_size** – **[in]** The number of CUDA streams in the pool. This must be greater than zero. 
  * **output** – **[out]** Pointer to the memory location where output should end up 
  * **input** – **[in]** Pointer to the input data 
  * **num_rows** – **[in]** Number of rows in input 
  * **out_mem_type** – **[in]** The memory type (device/host) of the output buffer 
  * **in_mem_type** – **[in]** The memory type (device/host) of the input buffer 
  * **predict_type** – **[in]** Type of inference to perform. Defaults to summing the outputs of all trees and produce an output per row. If set to “per_tree”, we will instead output all outputs of individual trees. If set to “leaf_id”, we will output the integer ID of the leaf node for each tree. 
  * **specified_chunk_size** – **[in]** Specifies the mini-batch size for processing. This has different meanings on CPU and GPU, but on GPU it corresponds to the number of rows evaluated per inference iteration on a single block. It can take on any power of 2 from 1 to 32, and runtime performance is quite sensitive to the value chosen. In general, larger batches benefit from higher values, but it is hard to predict the optimal value a priori. If omitted, a heuristic will be used to select a reasonable value. On CPU, this argument can generally just be omitted. 

### template<[tree_layout](#_CPPv4N8nvforest11tree_layoutE) layout_v, typename threshold_t, typename index_t, typename metadata_storage_t, typename offset_t>struct decision_forest

A general-purpose decision forest implementation

This template provides an optimized but generic implementation of a decision forest. Template parameters are used to specialize the implementation based on the size and characteristics of the forest. For instance, the smallest integer that can express the offset between a parent and child node within a tree is used in order to minimize the size of a node, increasing the number that can fit within the L2 or L1 cache.

* **Template Parameters:**
  * **layout_v** – The in-memory layout of nodes in this forest 
  * **threshold_t** – The floating-point type used for quantitative splits 
  * **index_t** – The integer type used for storing many things within a forest, including the category value of categorical nodes and the index at which vector output for a leaf node is stored. 
  * **metadata_storage_t** – The type used for storing node metadata. The first several bits will be used to store flags indicating various characteristics of the node, and the remaining bits provide the integer index of the feature for this node’s split 
  * **offset_t** – An integer used to indicate the offset between a node and its most distant child. This type must be large enough to store the largest such offset in the entire forest. 

### Public Types

### using forest_type = forest<[layout](#_CPPv4N8nvforest15decision_forest6layoutE), [threshold_t](#_CPPv4I_11tree_layout0000EN8nvforest15decision_forestE), [index_t](#_CPPv4I_11tree_layout0000EN8nvforest15decision_forestE), [metadata_storage_t](#_CPPv4I_11tree_layout0000EN8nvforest15decision_forestE), [offset_t](#_CPPv4I_11tree_layout0000EN8nvforest15decision_forestE)>

The type of the forest object which is actually passed to the CPU/GPU for inference 

### using node_type = typename [forest_type](#_CPPv4N8nvforest15decision_forest11forest_typeE)::node_type

The type of nodes within the forest 

### using io_type = typename [forest_type](#_CPPv4N8nvforest15decision_forest11forest_typeE)::io_type

The type used for input and output to the model 

### using threshold_type = [threshold_t](#_CPPv4I_11tree_layout0000EN8nvforest15decision_forestE)

The type used for quantitative splits within the model 

### using postprocessor_type = postprocessor<[io_type](#_CPPv4N8nvforest15decision_forest7io_typeE)>

The type used to indicate how leaf output should be post-processed 

### using categorical_storage_type = typename [node_type](#_CPPv4N8nvforest15decision_forest9node_typeE)::index_type

The type used for storing data on categorical nodes 

### Public Functions

### inline decision_forest()

Construct an empty decision forest 

### inline decision_forest(buffer<[node_type](#_CPPv4N8nvforest15decision_forest9node_typeE)> &&nodes, buffer<index_type> &&root_node_indexes, buffer<index_type> &&node_id_mapping, buffer<[io_type](#_CPPv4N8nvforest15decision_forest7io_typeE)> &&bias, index_type num_features, index_type num_outputs = index_type{2}, bool has_categorical_nodes = false, std::optional<buffer<[io_type](#_CPPv4N8nvforest15decision_forest7io_typeE)>> &&vector_output = std::nullopt, std::optional<buffer<typename [node_type](#_CPPv4N8nvforest15decision_forest9node_typeE)::index_type>> &&categorical_storage = std::nullopt, index_type leaf_size = index_type{1}, [row_op](#_CPPv4N8nvforest6row_opE) row_postproc = [row_op](#_CPPv4N8nvforest6row_opE)::[disable](#_CPPv4N8nvforest6row_op7disableE), [element_op](#_CPPv4N8nvforest10element_opE) elem_postproc = [element_op](#_CPPv4N8nvforest10element_opE)::[disable](#_CPPv4N8nvforest10element_op7disableE), [io_type](#_CPPv4N8nvforest15decision_forest7io_typeE) average_factor = [io_type](#_CPPv4N8nvforest15decision_forest7io_typeE){1}, [io_type](#_CPPv4N8nvforest15decision_forest7io_typeE) postproc_constant = [io_type](#_CPPv4N8nvforest15decision_forest7io_typeE){1})

Construct a decision forest with the indicated data

* **Parameters:**
  * **nodes** – A buffer containing all nodes within the forest 
  * **root_node_indexes** – A buffer containing the index of the root node of every tree in the forest 
  * **node_id_mapping** – Mapping to use to convert nvForest’s internal node ID into Treelite’s node ID. Only relevant when predict_type == [infer_kind::leaf_id](#namespacenvforest_1af0554e6b9407e80881d128a3fb3317c2ae7ccf35b51e7551e8f07fec0e4172297)
  * **bias** – The bias term that is added to the output as part of the postprocessing step. The bias term should have same length as num_outputs. 
  * **num_features** – The number of features per input sample for this model 
  * **num_outputs** – The number of outputs per row from this model 
  * **has_categorical_nodes** – Whether this forest contains any categorical nodes 
  * **vector_output** – A buffer containing the output from all vector leaves for this model. Each leaf node will specify the offset within this buffer at which its vector output begins, and leaf_size will be used to determine how many subsequent entries from the buffer should be used to construct the vector output. A value of std::nullopt indicates that this is not a vector leaf model. 
  * **categorical_storage** – For models with inputs on too many categories to be stored in the bits of an `index_t`, it may be necessary to store categorical information external to the node itself. This buffer contains the necessary storage for this information. 
  * **leaf_size** – The number of output values per leaf (1 for non-vector leaves; >1 for vector leaves) 
  * **row_postproc** – The post-processing operation to be applied to an entire row of the model output 
  * **elem_postproc** – The per-element post-processing operation to be applied to the model output 
  * **average_factor** – A factor which is used for output normalization 
  * **postproc_constant** – A constant used by some post-processing operations, including sigmoid, exponential, and logarithm_one_plus_exp 

### inline auto num_features() const

The number of features per row expected by the model 

### inline auto num_trees() const

The number of trees in the model 

### inline auto has_vector_leaves() const

Whether or not leaf nodes have vector outputs 

### inline auto num_outputs([infer_kind](#_CPPv4N8nvforest10infer_kindE) inference_kind = [infer_kind](#_CPPv4N8nvforest10infer_kindE)::[default_kind](#_CPPv4N8nvforest10infer_kind12default_kindE)) const

The number of outputs per row generated by the model for the given type of inference. Note: This will differ from num_outputs argument passed to the constructor, if inference_kind is not default_kind. 

### inline auto row_postprocessing() const

The operation used for postprocessing all outputs for a single row 

### inline auto elem_postprocessing() const

The operation used for postprocessing each element of the output for a single row 

### inline auto memory_type()

The type of memory (device/host) where the model is stored 

### inline auto device_index()

The ID of the device on which this model is loaded 

### inline void predict(buffer<typename [forest_type](#_CPPv4N8nvforest15decision_forest11forest_typeE)::io_type> &output, buffer<typename [forest_type](#_CPPv4N8nvforest15decision_forest11forest_typeE)::io_type> const &input, nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE) stream = nvforest::[cuda_stream](#_CPPv4N8nvforest11cuda_streamE){}, [infer_kind](#_CPPv4N8nvforest10infer_kindE) predict_type = [infer_kind](#_CPPv4N8nvforest10infer_kindE)::[default_kind](#_CPPv4N8nvforest10infer_kind12default_kindE), std::optional<index_type> specified_rows_per_block_iter = std::nullopt)

Perform inference with this model

* **Parameters:**
  * **output** – **[out]** The buffer where the model output should be stored. This must be of size ROWS x [num_outputs()](#structnvforest_1_1decision__forest_1a588df0640e5d39ddd9c1dae58ad0ad13). 
  * **input** – **[in]** The buffer containing the input data 
  * **stream** – **[in]** For GPU execution, the CUDA stream. For CPU execution, this optional parameter can be safely omitted. 
  * **predict_type** – **[in]** Type of inference to perform. Defaults to summing the outputs of all trees and produce an output per row. If set to “per_tree”, we will instead output all outputs of individual trees. If set to “leaf_id”, we will output the integer ID of the leaf node for each tree. 
  * **specified_rows_per_block_iter** – **[in]** If non-nullopt, this value is used to determine how many rows are evaluated for each inference iteration within a CUDA block. Runtime performance is quite sensitive to this value, but it is difficult to predict a priori, so it is recommended to perform a search over possible values with realistic batch sizes in order to determine the optimal value. Any power of 2 from 1 to 32 is a valid value, and in general larger batches benefit from larger values. 

### Public Static Attributes

### static constexpr auto const layout = [layout_v](#_CPPv4I_11tree_layout0000EN8nvforest15decision_forestE)

The in-memory layout of nodes in this forest 

## Enums and constants

### static constexpr auto const nvforest::preferred_tree_layout = [tree_layout](#_CPPv4N8nvforest11tree_layoutE)::[breadth_first](#_CPPv4N8nvforest11tree_layout13breadth_firstE)

The default memory layout for nvForest trees if not otherwise specified 

### enum class nvforest::tree_layout : unsigned char

Enum representing possible memory layout for nvForest trees 

*Values:*

### enumerator depth_first

### enumerator breadth_first

### enumerator layered_children_together

### enum class nvforest::infer_kind : unsigned char

Enum representing distinct prediction tasks 

*Values:*

### enumerator default_kind

### enumerator per_tree

### enumerator leaf_id

### enum class nvforest::row_op : unsigned char

Enum representing possible row-wise operations on output 

*Values:*

### enumerator disable

### enumerator softmax

### enumerator max_index

### enum class nvforest::element_op : unsigned char

Enum representing possible element-wise operations on output 

*Values:*

### enumerator disable

### enumerator signed_square

### enumerator hinge

### enumerator sigmoid

### enumerator exponential

### enumerator logarithm_one_plus_exp

## Type aliases

### using nvforest::cuda_stream = int

### using nvforest::decision_forest_variant = std::variant<detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<0, detail::specialization_variant>::layout, std::variant_alternative_t<0, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<0, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<1, detail::specialization_variant>::layout, std::variant_alternative_t<1, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<1, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<2, detail::specialization_variant>::layout, std::variant_alternative_t<2, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<2, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<3, detail::specialization_variant>::layout, std::variant_alternative_t<3, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<3, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<4, detail::specialization_variant>::layout, std::variant_alternative_t<4, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<4, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<5, detail::specialization_variant>::layout, std::variant_alternative_t<5, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<5, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<6, detail::specialization_variant>::layout, std::variant_alternative_t<6, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<6, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<7, detail::specialization_variant>::layout, std::variant_alternative_t<7, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<7, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<8, detail::specialization_variant>::layout, std::variant_alternative_t<8, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<8, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<9, detail::specialization_variant>::layout, std::variant_alternative_t<9, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<9, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<10, detail::specialization_variant>::layout, std::variant_alternative_t<10, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<10, detail::specialization_variant>::has_large_trees>, detail::[preset_decision_forest](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)<std::variant_alternative_t<11, detail::specialization_variant>::layout, std::variant_alternative_t<11, detail::specialization_variant>::is_double_precision, std::variant_alternative_t<11, detail::specialization_variant>::has_large_trees>>

A variant containing all standard [decision_forest](#structnvforest_1_1decision__forest) instantiations 

### template<[tree_layout](#_CPPv4N8nvforest11tree_layoutE) layout, bool double_precision, bool large_trees>using nvforest::detail::preset_decision_forest = [decision_forest](#_CPPv4I_11tree_layout0000EN8nvforest15decision_forestE)<[layout](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE), typename specialization_types<[layout](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE), [double_precision](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE), [large_trees](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)>::threshold_type, typename specialization_types<[layout](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE), [double_precision](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE), [large_trees](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)>::index_type, typename specialization_types<[layout](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE), [double_precision](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE), [large_trees](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)>::metadata_type, typename specialization_types<[layout](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE), [double_precision](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE), [large_trees](#_CPPv4I_11tree_layout_b_bEN8nvforest6detail22preset_decision_forestE)>::offset_type>

A convenience wrapper to simplify template instantiation of [decision_forest](#structnvforest_1_1decision__forest)

This template takes the large range of available template parameters and reduces them to just three standard choices.

* **Template Parameters:**
  * **layout** – The in-memory layout of nodes in this forest 
  * **double_precision** – Whether this model should use double-precision for floating-point evaluation and 64-bit integers for indexes 
  * **large_trees** – Whether this forest expects more than 2\*\*(16 -3) - 1 = 8191 features or contains nodes whose child is offset more than 2\*\*16 - 1 = 65535 nodes away.