Is this page helpful?

cuml::genetic Namespace#

Warning

Primarily internal API: it may change or disappear without notice and has no stability, deprecation, backward-compatibility, or input-validation guarantees. Callers must validate inputs and satisfy all memory, stream, and lifetime preconditions. Prefer the supported Python API.

namespace genetic#

Typedefs

typedef program *program_t#

program_t is a shorthand for device programs

Enums

enum class metric_t : uint32_t#

fitness metric types

Values:

enumerator mae#

mean absolute error (regression-only)

enumerator mse#

mean squared error (regression-only)

enumerator rmse#

root mean squared error (regression-only)

enumerator pearson#

pearson product-moment coefficient (regression and transformation)

enumerator spearman#

spearman’s rank-order coefficient (regression and transformation)

enumerator logloss#

binary cross-entropy loss (classification-only)

enum class init_method_t : uint32_t#

Type of initialization of the member programs in the population

Values:

enumerator grow#

random nodes chosen, allowing shorter or asymmetrical trees

enumerator full#

growing till a randomly chosen depth

enumerator half_and_half#

50% of the population on grow and the rest with full

enum class transformer_t : uint32_t#

Values:

enumerator sigmoid#

sigmoid function

enum class mutation_t : uint32_t#

Mutation types for a program

Values:

enumerator none#

Placeholder for first generation programs

enumerator crossover#

Crossover mutations

enumerator subtree#

Subtree mutations

enumerator hoist#

Hoise mutations

enumerator point#

Point mutations

enumerator reproduce#

Program reproduction

Functions

std::string stringify(const program &prog)#

Visualize an AST.

Parameters:

prog – host object containing the AST

Returns:

String representation of the AST

void symFit(
const raft::handle_t &handle,
const float *input,
const float *labels,
const float *sample_weights,
const int n_rows,
const int n_cols,
param &params,
program_t &final_progs,
std::vector<std::vector<program>> &history
)#

Fit either a regressor, classifier or a transformer to the given dataset.

Note

This module allocates extra device memory for the nodes of the last generation that is pointed by final_progs[i].nodes for each program i in final_progs. The amount of memory allocated is found at runtime, and is final_progs[i].len * sizeof(node) for each program i. The reason this isn’t deallocated within the function is because the resulting memory is needed for executing predictions in symRegPredict, symClfPredict, symClfPredictProbs and symTransform functions. The above device memory is expected to be explicitly deallocated by the caller AFTER calling the predict function.

Parameters:

  • handle – cuML handle

  • input – device pointer to the feature matrix

  • labels – device pointer to the label vector of length n_rows

  • sample_weights – device pointer to the sample weights of length n_rows

  • n_rows – number of rows of the feature matrix

  • n_cols – number of columns of the feature matrix

  • params – host struct containing hyperparameters needed for training

  • final_progs –

    device pointer to the final generation of programs(sorted by decreasing

    fitness)

  • history – host vector containing the list of all programs in every generation (sorted by decreasing fitness)

void symRegPredict(
const raft::handle_t &handle,
const float *input,
const int n_rows,
const program_t &best_prog,
float *output
)#

Make predictions for a symbolic regressor.

Parameters:

  • handle – cuML handle

  • input – device pointer to feature matrix

  • n_rows – number of rows of the feature matrix

  • best_prog – device pointer to best AST fit during training

  • output – device pointer to output values

void symClfPredictProbs(
const raft::handle_t &handle,
const float *input,
const int n_rows,
const param &params,
const program_t &best_prog,
float *output
)#

Probability prediction for a symbolic classifier. If a transformer(like sigmoid) is specified, then it is applied on the output before returning it.

Parameters:

  • handle – cuML handle

  • input – device pointer to feature matrix

  • n_rows – number of rows of the feature matrix

  • params – host struct containing training hyperparameters

  • best_prog – The best program obtained during training. Inferences are made using this

  • output – device pointer to output probability(in col major format)

void symClfPredict(
const raft::handle_t &handle,
const float *input,
const int n_rows,
const param &params,
const program_t &best_prog,
float *output
)#

Return predictions for a binary classification program defining the decision boundary.

Parameters:

  • handle – cuML handle

  • input – device pointer to feature matrix

  • n_rows – number of rows of the feature matrix

  • params – host struct containing training hyperparameters

  • best_prog – Best program obtained after training

  • output – Device pointer to output predictions

void symTransform(
const raft::handle_t &handle,
const float *input,
const param &params,
const program_t &final_progs,
const int n_rows,
const int n_cols,
float *output
)#

Transform the values in the input feature matrix according to the supplied programs.

Parameters:

  • handle – cuML handle

  • input – device pointer to feature matrix

  • params – Hyperparameters used during training

  • final_progs – List of ASTs used for generating new features

  • n_rows – number of rows of the feature matrix

  • n_cols – number of columns of the feature matrix

  • output – device pointer to transformed input

void execute(
const raft::handle_t &h,
const program_t &d_progs,
const int n_rows,
const int n_progs,
const float *data,
float *y_pred
)#

Calls the execution kernel to evaluate all programs on the given dataset.

Parameters:

  • h – cuML handle

  • d_progs – Device pointer to programs

  • n_rows – Number of rows in the input dataset

  • n_progs – Total number of programs being evaluated

  • data – Device pointer to input dataset (in col-major format)

  • y_pred – Device pointer to output of program evaluation

void compute_metric(
const raft::handle_t &h,
int n_rows,
int n_progs,
const float *y,
const float *y_pred,
const float *w,
float *score,
const param &params
)#

Compute the loss based on the metric specified in the training hyperparameters. It performs a batched computation for all programs in one shot.

Parameters:

  • h – cuML handle

  • n_rows – The number of labels/rows in the expected output

  • n_progs – The number of programs being batched

  • y – Device pointer to the expected output (SIZE = n_samples)

  • y_pred – Device pointer to the predicted output (SIZE = n_samples * n_progs)

  • w – Device pointer to sample weights (SIZE = n_samples)

  • score – Device pointer to final score (SIZE = n_progs)

  • params – Training hyperparameters

void find_fitness(
const raft::handle_t &h,
program_t &d_prog,
float *score,
const param &params,
const int n_rows,
const float *data,
const float *y,
const float *sample_weights
)#

Computes the fitness scores for a sngle program on the given dataset.

Parameters:

  • h – cuML handle

  • d_prog – Device pointer to program

  • score – Device pointer to fitness vals

  • params – Training hyperparameters

  • n_rows – Number of rows in the input dataset

  • data – Device pointer to input dataset

  • y – Device pointer to input labels

  • sample_weights – Device pointer to sample weights

void find_batched_fitness(
const raft::handle_t &h,
int n_progs,
program_t &d_progs,
float *score,
const param &params,
const int n_rows,
const float *data,
const float *y,
const float *sample_weights
)#

Computes the fitness scores for all programs on the given dataset.

Parameters:

  • h – cuML handle

  • n_progs – Batch size(Number of programs)

  • d_progs – Device pointer to list of programs

  • score – Device pointer to fitness vals computed for all programs

  • params – Training hyperparameters

  • n_rows – Number of rows in the input dataset

  • data – Device pointer to input dataset

  • y – Device pointer to input labels

  • sample_weights – Device pointer to sample weights

void set_fitness(
const raft::handle_t &h,
program_t &d_prog,
program &h_prog,
const param &params,
const int n_rows,
const float *data,
const float *y,
const float *sample_weights
)#

Computes and sets the fitness scores for a single program on the given dataset.

Parameters:

  • h – cuML handle

  • d_prog – Device pointer to program

  • h_prog – Host program object

  • params – Training hyperparameters

  • n_rows – Number of rows in the input dataset

  • data – Device pointer to input dataset

  • y – Device pointer to input labels

  • sample_weights – Device pointer to sample weights

void set_batched_fitness(
const raft::handle_t &h,
int n_progs,
program_t &d_progs,
std::vector<program> &h_progs,
const param &params,
const int n_rows,
const float *data,
const float *y,
const float *sample_weights
)#

Computes and sets the fitness scores for all programs on the given dataset.

Parameters:

  • h – cuML handle

  • n_progs – Batch size

  • d_progs – Device pointer to list of programs

  • h_progs – Host vector of programs corresponding to d_progs

  • params – Training hyperparameters

  • n_rows – Number of rows in the input dataset

  • data – Device pointer to input dataset

  • y – Device pointer to input labels

  • sample_weights – Device pointer to sample weights

float get_fitness(const program &prog, const param &params)#

Returns precomputed fitness score of program on the host, after accounting for parsimony.

Parameters:

  • prog – The host program

  • params – Training hyperparameters

Returns:

Fitness score corresponding to trained program

int get_depth(const program &p_out)#

Evaluates and returns the depth of the current program.

Parameters:

p_out – The given program

Returns:

The depth of the current program

void build_program(
program &p_out,
const param &params,
std::mt19937 &rng
)#

Build a random program with depth at most 10.

Parameters:

  • p_out – The output program

  • params – Training hyperparameters

  • rng – RNG to decide nodes to add

void point_mutation(
const program &prog,
program &p_out,
const param &params,
std::mt19937 &rng
)#

Perform a point mutation on the given program(AST).

Parameters:

  • prog – The input program

  • p_out – The result program

  • params – Training hyperparameters

  • rng – RNG to decide nodes to mutate

void crossover(
const program &prog,
const program &donor,
program &p_out,
const param &params,
std::mt19937 &rng
)#

Perform a ‘hoisted’ crossover mutation using the parent and donor programs. The donor subtree selected is hoisted to ensure our constrains on total depth.

Parameters:

  • prog – The input program

  • donor – The donor program

  • p_out – The result program

  • params – Training hyperparameters

  • rng – RNG for subtree selection

void subtree_mutation(
const program &prog,
program &p_out,
const param &params,
std::mt19937 &rng
)#

Performs a crossover mutation with a randomly built new program. Since crossover is ‘hoisted’, this will ensure that depth constrains are not violated.

Parameters:

  • prog – The input program

  • p_out – The result mutated program

  • params – Training hyperparameters

  • rng – RNG to control subtree selection and temporary program addition

void hoist_mutation(
const program &prog,
program &p_out,
const param &params,
std::mt19937 &rng
)#

Perform a hoist mutation on a random subtree of the given program (replace a subtree with a subtree of a subtree).

Parameters:

  • prog – The input program

  • p_out – The output program

  • params – Training hyperparameters

  • rng – RNG to control subtree selection

Variables

const int GENE_TPB = 256#
const int MAX_STACK_SIZE = 20#
struct node#
#include <node.h>

Represents a node in the syntax tree. 

// A non-terminal (aka function) node
node func_node{node::type::sub};
// A constant node
float const_value = 2.f;
node const_node{const_value};
// A variable (aka feature) node
node var_node{20};

Public Types

enum class type : uint32_t#

All possible types of nodes. For simplicity, all the terminal and non-terminal types are clubbed together.

Values:

enumerator variable#
enumerator constant#
enumerator functions_begin#
enumerator binary_begin#
enumerator add#
enumerator atan2#
enumerator div#
enumerator fdim#
enumerator max#
enumerator min#
enumerator mul#
enumerator pow#
enumerator sub#
enumerator binary_end#
enumerator unary_begin#
enumerator abs#
enumerator acos#
enumerator acosh#
enumerator asin#
enumerator asinh#
enumerator atan#
enumerator atanh#
enumerator cbrt#
enumerator cos#
enumerator cosh#
enumerator cube#
enumerator exp#
enumerator inv#
enumerator log#
enumerator neg#
enumerator rcbrt#
enumerator rsqrt#
enumerator sin#
enumerator sinh#
enumerator sq#
enumerator sqrt#
enumerator tan#
enumerator tanh#
enumerator unary_end#
enumerator functions_end#

Public Functions

explicit node()#

Default constructor for node.

explicit node(type ft)#

Construct a function node.

Parameters:

ft – [in] function type

explicit node(int fid)#

Construct a variable node.

Parameters:

fid – [in] feature id that represents the variable

explicit node(float val)#

Construct a constant node.

Parameters:

val – [in] constant value

explicit node(const node &src)#
Parameters:

src – [in] source node to be copied

node &operator=(const node &src)#

assignment operator

Parameters:

src – [in] source node to be copied

Returns:

current node reference

bool is_terminal() const#

whether the current is either a variable or a constant

bool is_nonterminal() const#

whether the current node is a function

int arity() const#

Get the arity of the node. If it is a terminal, then a 0 is returned

Public Members

type t#

node type

Public Static Functions

static type from_str(const std::string &ntype)#

Helper method to get node type from input string.

Parameters:

ntype – [in] node type in string. Possible strings correlate one-to-one with the enum values for type

Returns:

type

Public Static Attributes

static const int kInvalidFeatureId#

constant used to represent invalid feature id

union data#
#include <node.h>

Public Members

int fid#

if the node is variable type, then this is the column id to be used to fetch its value, from the input dataset

float val#

if the node is constant type, then this is the value of the node

struct param#
#include <common.h>

contains all the hyper-parameters for training

Note

Unless otherwise mentioned, all the parameters below are applicable to all of classification, regression and transformation.

Public Functions

float p_reproduce() const#

Computes the probability of ‘reproduction’

int max_programs() const#

maximum possible number of programs

int criterion() const#

criterion for scoring based on metric used

Public Members

int population_size = 1000#

number of programs in each generation

int hall_of_fame = 100#

number of fittest programs to compare during correlation (transformation-only)

int n_components = 10#

number of fittest programs to return from hall_of_fame top programs (transformation-only)

int generations = 20#

number of generations to evolve

int tournament_size = 20#

number of programs that compete in the tournament to become part of next generation

float stopping_criteria = 0.0f#

metric threshold used for early stopping

float const_range[2] = {-1.0f, 1.0f}#

minimum/maximum value for constant nodes

int init_depth[2] = {2, 6}#

minimum/maximum depth of programs after initialization

init_method_t init_method = init_method_t::half_and_half#

initialization method

std::vector<node::type> function_set = {node::type::add, node::type::mul, node::type::div, node::type::sub}#

list of functions to choose from

std::map<int, std::vector<node::type>> arity_set{{2, {node::type::add, node::type::mul, node::type::div, node::type::sub}}}#

map of functions ordered by their arity

transformer_t transformer = transformer_t::sigmoid#

transformation function to class probabilities (classification-only)

metric_t metric = metric_t::mae#

fitness metric

float parsimony_coefficient = 0.001f#

penalization factor for large programs

float p_crossover = 0.9f#

crossover mutation probability of the tournament winner

float p_subtree_mutation = 0.01f#

subtree mutation probability of the tournament winner

float p_hoist_mutation = 0.01f#

hoist mutation probability of the tournament winner

float p_point_mutation = 0.01f#

point mutation probabiilty of the tournament winner

float p_point_replace = 0.05f#

point replace probabiility for point mutations

float max_samples = 1.0f#

subsampling factor

float terminalRatio = 0.0f#

Terminal ratio for node selection during grow initialization. 0 -> auto-selection

std::vector<std::string> feature_names#

list of feature names for generating syntax trees from the programs

int num_features#

number of features in current dataset

uint64_t random_state = 0UL#

Todo:

: feature_names

Todo:

: verbose

random seed used for RNG

int num_epochs = 0#

Number of epochs for which the algorithm ran

bool low_memory = false#

Low memory flag for program history

struct program#
#include <program.h>

The main data structure to store the AST that represents a program in the current generation.

Public Functions

explicit program()#

the AST. It is stored in the reverse of DFS-right-child-first order. In other words, construct a regular AST in the form of depth-first, but instead of storing the left child first, store the right child and so on. Now take the resulting 1D array and reverse it.

Note

The pointed memory buffer is NOT owned by this class and further it is assumed to be a zero-copy (aka pinned memory) buffer, at least in this initial version Default constructor

~program()#

Destroy the program object.

explicit program(const program &src)#

Copy constructor for a new program object.

Parameters:

src –

program &operator=(const program &src)#

assignment operator

Parameters:

src – [in] source program to be copied

Returns:

current program reference

Public Members

int len#

total number of nodes in this AST

int depth#

maximum depth of this AST

float raw_fitness_#

fitness score of current AST

metric_t metric#

fitness metric used for current AST

mutation_t mut_type#

mutation type responsible for production