cuml::genetic Namespace

Warning

Primarily internal API: it may change or disappear without notice and has no stability, deprecation, backward-compatibility, or input-validation guarantees. Callers must validate inputs and satisfy all memory, stream, and lifetime preconditions. Prefer the supported Python API.

namespace genetic

Typedefs

typedef program *program_t

program_t is a shorthand for device programs

Enums

enum class metric_t : uint32_t

fitness metric types

Values:

enumerator mae

mean absolute error (regression-only)

enumerator mse

mean squared error (regression-only)

enumerator rmse

root mean squared error (regression-only)

enumerator pearson

pearson product-moment coefficient (regression and transformation)

enumerator spearman

spearman’s rank-order coefficient (regression and transformation)

enumerator logloss

binary cross-entropy loss (classification-only)

enum class init_method_t : uint32_t

Type of initialization of the member programs in the population

Values:

enumerator grow

random nodes chosen, allowing shorter or asymmetrical trees

enumerator full

growing till a randomly chosen depth

enumerator half_and_half

50% of the population on grow and the rest with full

enum class transformer_t : uint32_t

Values:

enumerator sigmoid

sigmoid function

enum class mutation_t : uint32_t

Mutation types for a program

Values:

enumerator none

Placeholder for first generation programs

enumerator crossover

Crossover mutations

enumerator subtree

Subtree mutations

enumerator hoist

Hoise mutations

enumerator point

Point mutations

enumerator reproduce

Program reproduction

Functions

std::string stringify(const program &prog)

Visualize an AST.

Parameters:

prog – host object containing the AST

Returns:

String representation of the AST

void symFit(

const raft::handle_t &handle,

const float *input,

const float *labels,

const float *sample_weights,

const int n_rows,

const int n_cols,

param &params,

program_t &final_progs,

std::vector<std::vector<program>> &history

)

Fit either a regressor, classifier or a transformer to the given dataset.

Note

This module allocates extra device memory for the nodes of the last generation that is pointed by final_progs[i].nodes for each program i in final_progs. The amount of memory allocated is found at runtime, and is final_progs[i].len * sizeof(node) for each program i. The reason this isn’t deallocated within the function is because the resulting memory is needed for executing predictions in symRegPredict, symClfPredict, symClfPredictProbs and symTransform functions. The above device memory is expected to be explicitly deallocated by the caller AFTER calling the predict function.

Parameters:

• handle – cuML handle

• input – device pointer to the feature matrix

• labels – device pointer to the label vector of length n_rows

• sample_weights – device pointer to the sample weights of length n_rows

• n_rows – number of rows of the feature matrix

• n_cols – number of columns of the feature matrix

• params – host struct containing hyperparameters needed for training

• final_progs – device pointer to the final generation of programs(sorted by decreasing fitness)

• history – host vector containing the list of all programs in every generation (sorted by decreasing fitness)

void symRegPredict(

const raft::handle_t &handle,

const float *input,

const int n_rows,

const program_t &best_prog,

float *output

)

Make predictions for a symbolic regressor.

Parameters:

• handle – cuML handle

• input – device pointer to feature matrix

• n_rows – number of rows of the feature matrix

• best_prog – device pointer to best AST fit during training

• output – device pointer to output values

void symClfPredictProbs(

const raft::handle_t &handle,

const float *input,

const int n_rows,

const param &params,

const program_t &best_prog,

float *output

)

Probability prediction for a symbolic classifier. If a transformer(like sigmoid) is specified, then it is applied on the output before returning it.

Parameters:

• handle – cuML handle

• input – device pointer to feature matrix

• n_rows – number of rows of the feature matrix

• params – host struct containing training hyperparameters

• best_prog – The best program obtained during training. Inferences are made using this

• output – device pointer to output probability(in col major format)

void symClfPredict(

const raft::handle_t &handle,

const float *input,

const int n_rows,

const param &params,

const program_t &best_prog,

float *output

)

Return predictions for a binary classification program defining the decision boundary.

Parameters:

• handle – cuML handle

• input – device pointer to feature matrix

• n_rows – number of rows of the feature matrix

• params – host struct containing training hyperparameters

• best_prog – Best program obtained after training

• output – Device pointer to output predictions

void symTransform(

const raft::handle_t &handle,

const float *input,

const param &params,

const program_t &final_progs,

const int n_rows,

const int n_cols,

float *output

)

Transform the values in the input feature matrix according to the supplied programs.

Parameters:

• handle – cuML handle

• input – device pointer to feature matrix

• params – Hyperparameters used during training

• final_progs – List of ASTs used for generating new features

• n_rows – number of rows of the feature matrix

• n_cols – number of columns of the feature matrix

• output – device pointer to transformed input

void execute(

const raft::handle_t &h,

const program_t &d_progs,

const int n_rows,

const int n_progs,

const float *data,

float *y_pred

)

Calls the execution kernel to evaluate all programs on the given dataset.

Parameters:

• h – cuML handle

• d_progs – Device pointer to programs

• n_rows – Number of rows in the input dataset

• n_progs – Total number of programs being evaluated

• data – Device pointer to input dataset (in col-major format)

• y_pred – Device pointer to output of program evaluation

void compute_metric(

const raft::handle_t &h,

int n_rows,

int n_progs,

const float *y,

const float *y_pred,

const float *w,

float *score,

const param &params

)

Compute the loss based on the metric specified in the training hyperparameters. It performs a batched computation for all programs in one shot.

Parameters:

• h – cuML handle

• n_rows – The number of labels/rows in the expected output

• n_progs – The number of programs being batched

• y – Device pointer to the expected output (SIZE = n_samples)

• y_pred – Device pointer to the predicted output (SIZE = n_samples * n_progs)

• w – Device pointer to sample weights (SIZE = n_samples)

• score – Device pointer to final score (SIZE = n_progs)

• params – Training hyperparameters

void find_fitness(

const raft::handle_t &h,

program_t &d_prog,

float *score,

const param &params,

const int n_rows,

const float *data,

const float *y,

const float *sample_weights

)

Computes the fitness scores for a sngle program on the given dataset.

Parameters:

• h – cuML handle

• d_prog – Device pointer to program

• score – Device pointer to fitness vals

• params – Training hyperparameters

• n_rows – Number of rows in the input dataset

• data – Device pointer to input dataset

• y – Device pointer to input labels

• sample_weights – Device pointer to sample weights

void find_batched_fitness(

const raft::handle_t &h,

int n_progs,

program_t &d_progs,

float *score,

const param &params,

const int n_rows,

const float *data,

const float *y,

const float *sample_weights

)

Computes the fitness scores for all programs on the given dataset.

Parameters:

• h – cuML handle

• n_progs – Batch size(Number of programs)

• d_progs – Device pointer to list of programs

• score – Device pointer to fitness vals computed for all programs

• params – Training hyperparameters

• n_rows – Number of rows in the input dataset

• data – Device pointer to input dataset

• y – Device pointer to input labels

• sample_weights – Device pointer to sample weights

void set_fitness(

const raft::handle_t &h,

program_t &d_prog,

program &h_prog,

const param &params,

const int n_rows,

const float *data,

const float *y,

const float *sample_weights

)

Computes and sets the fitness scores for a single program on the given dataset.

Parameters:

• h – cuML handle

• d_prog – Device pointer to program

• h_prog – Host program object

• params – Training hyperparameters

• n_rows – Number of rows in the input dataset

• data – Device pointer to input dataset

• y – Device pointer to input labels

• sample_weights – Device pointer to sample weights

void set_batched_fitness(

const raft::handle_t &h,

int n_progs,

program_t &d_progs,

std::vector<program> &h_progs,

const param &params,

const int n_rows,

const float *data,

const float *y,

const float *sample_weights

)

Computes and sets the fitness scores for all programs on the given dataset.

Parameters:

• h – cuML handle

• n_progs – Batch size

• d_progs – Device pointer to list of programs

• h_progs – Host vector of programs corresponding to d_progs

• params – Training hyperparameters

• n_rows – Number of rows in the input dataset

• data – Device pointer to input dataset

• y – Device pointer to input labels

• sample_weights – Device pointer to sample weights

float get_fitness(const program &prog, const param &params)

Returns precomputed fitness score of program on the host, after accounting for parsimony.

Parameters:

• prog – The host program

• params – Training hyperparameters

Returns:

Fitness score corresponding to trained program

int get_depth(const program &p_out)

Evaluates and returns the depth of the current program.

Parameters:

p_out – The given program

Returns:

The depth of the current program

void build_program(

program &p_out,

const param &params,

std::mt19937 &rng

)

Build a random program with depth at most 10.

Parameters:

• p_out – The output program

• params – Training hyperparameters

• rng – RNG to decide nodes to add

void point_mutation(

const program &prog,

program &p_out,

const param &params,

std::mt19937 &rng

)

Perform a point mutation on the given program(AST)

Parameters:

• prog – The input program

• p_out – The result program

• params – Training hyperparameters

• rng – RNG to decide nodes to mutate

void crossover(

const program &prog,

const program &donor,

program &p_out,

const param &params,

std::mt19937 &rng

)

Perform a ‘hoisted’ crossover mutation using the parent and donor programs. The donor subtree selected is hoisted to ensure our constrains on total depth.

Parameters:

• prog – The input program

• donor – The donor program

• p_out – The result program

• params – Training hyperparameters

• rng – RNG for subtree selection

void subtree_mutation(

const program &prog,

program &p_out,

const param &params,

std::mt19937 &rng

)

Performs a crossover mutation with a randomly built new program. Since crossover is ‘hoisted’, this will ensure that depth constrains are not violated.

Parameters:

• prog – The input program

• p_out – The result mutated program

• params – Training hyperparameters

• rng – RNG to control subtree selection and temporary program addition

void hoist_mutation(

const program &prog,

program &p_out,

const param &params,

std::mt19937 &rng

)

Perform a hoist mutation on a random subtree of the given program (replace a subtree with a subtree of a subtree)

Parameters:

• prog – The input program

• p_out – The output program

• params – Training hyperparameters

• rng – RNG to control subtree selection

Variables

const int GENE_TPB = 256

const int MAX_STACK_SIZE = 20

struct param

#include <common.h>

contains all the hyper-parameters for training

Note

Unless otherwise mentioned, all the parameters below are applicable to all of classification, regression and transformation.

Public Functions

float p_reproduce() const

Computes the probability of ‘reproduction’

int max_programs() const

maximum possible number of programs

int criterion() const

criterion for scoring based on metric used

Public Members

int population_size = 1000

number of programs in each generation

int hall_of_fame = 100

number of fittest programs to compare during correlation (transformation-only)

int n_components = 10

number of fittest programs to return from hall_of_fame top programs (transformation-only)

int generations = 20

number of generations to evolve

int tournament_size = 20

number of programs that compete in the tournament to become part of next generation

float stopping_criteria = 0.0f

metric threshold used for early stopping

float const_range[2] = {-1.0f, 1.0f}

minimum/maximum value for constant nodes

int init_depth[2] = {2, 6}

minimum/maximum depth of programs after initialization

init_method_t init_method = init_method_t::half_and_half

initialization method

std::vector<node::type> function_set = {node::type::add, node::type::mul, node::type::div, node::type::sub}

list of functions to choose from

std::map<int, std::vector<node::type>> arity_set{{2, {node::type::add, node::type::mul, node::type::div, node::type::sub}}}

map of functions ordered by their arity

transformer_t transformer = transformer_t::sigmoid

transformation function to class probabilities (classification-only)

metric_t metric = metric_t::mae

fitness metric

float parsimony_coefficient = 0.001f

penalization factor for large programs

float p_crossover = 0.9f

crossover mutation probability of the tournament winner

float p_subtree_mutation = 0.01f

subtree mutation probability of the tournament winner

float p_hoist_mutation = 0.01f

hoist mutation probability of the tournament winner

float p_point_mutation = 0.01f

point mutation probabiilty of the tournament winner

float p_point_replace = 0.05f

point replace probabiility for point mutations

float max_samples = 1.0f

subsampling factor

float terminalRatio = 0.0f

Terminal ratio for node selection during grow initialization. 0 -> auto-selection

std::vector<std::string> feature_names

list of feature names for generating syntax trees from the programs

int num_features

number of features in current dataset

uint64_t random_state = 0UL

Todo:

: feature_names

: verbose

random seed used for RNG

int num_epochs = 0

Number of epochs for which the algorithm ran

bool low_memory = false

Low memory flag for program history

struct node

#include <node.h>

Represents a node in the syntax tree.

// A non-terminal (aka function) node
node func_node{node::type::sub};
// A constant node
float const_value = 2.f;
node const_node{const_value};
// A variable (aka feature) node
node var_node{20};

Public Types

enum class type : uint32_t

All possible types of nodes. For simplicity, all the terminal and non-terminal types are clubbed together.

Values:

enumerator variable

enumerator constant

enumerator functions_begin

enumerator binary_begin

enumerator add

enumerator atan2

enumerator div

enumerator fdim

enumerator max

enumerator min

enumerator mul

enumerator pow

enumerator sub

enumerator binary_end

enumerator unary_begin

enumerator abs

enumerator acos

enumerator acosh

enumerator asin

enumerator asinh

enumerator atan

enumerator atanh

enumerator cbrt

enumerator cos

enumerator cosh

enumerator cube

enumerator exp

enumerator inv

enumerator log

enumerator neg

enumerator rcbrt

enumerator rsqrt

enumerator sin

enumerator sinh

enumerator sq

enumerator sqrt

enumerator tan

enumerator tanh

enumerator unary_end

enumerator functions_end

Public Functions

explicit node()

Default constructor for node.

explicit node(type ft)

Construct a function node.

Parameters:

ft – [in] function type

explicit node(int fid)

Construct a variable node.

Parameters:

fid – [in] feature id that represents the variable

explicit node(float val)

Construct a constant node.

Parameters:

val – [in] constant value

explicit node(const node &src)

Parameters:

src – [in] source node to be copied

node &operator=(const node &src)

assignment operator

Parameters:

src – [in] source node to be copied

Returns:

current node reference

bool is_terminal() const

whether the current is either a variable or a constant

bool is_nonterminal() const

whether the current node is a function

int arity() const

Get the arity of the node. If it is a terminal, then a 0 is returned

Public Members

type t

node type

int fid

if the node is variable type, then this is the column id to be used to fetch its value, from the input dataset

float val

if the node is constant type, then this is the value of the node

Public Static Functions

static type from_str(const std::string &ntype)

Helper method to get node type from input string.

Parameters:

ntype – [in] node type in string. Possible strings correlate one-to-one with the enum values for type

Returns:

type

Public Static Attributes

static const int kInvalidFeatureId

constant used to represent invalid feature id

struct program

#include <program.h>

The main data structure to store the AST that represents a program in the current generation.

Public Functions

explicit program()

the AST. It is stored in the reverse of DFS-right-child-first order. In other words, construct a regular AST in the form of depth-first, but instead of storing the left child first, store the right child and so on. Now take the resulting 1D array and reverse it.

Note

The pointed memory buffer is NOT owned by this class and further it is assumed to be a zero-copy (aka pinned memory) buffer, at least in this initial version Default constructor

~program()

Destroy the program object.

explicit program(const program &src)

Copy constructor for a new program object.

Parameters:

src –

program &operator=(const program &src)

assignment operator

Parameters:

src – [in] source program to be copied

Returns:

current program reference

Public Members

int len

total number of nodes in this AST

int depth

maximum depth of this AST

float raw_fitness_

fitness score of current AST

metric_t metric

fitness metric used for current AST

mutation_t mut_type

mutation type responsible for production