Skip to navigation

Model Plan Guide

View as Markdown

Use a model plan to control how Kumo Engine searches for and trains a model. Start with a suggested plan, change the settings you need, and pass the plan to Trainer. This guide covers the public model controls shared by Linux, Snowflake notebooks, and Databricks notebooks. For the column types and encoder choices introduced during graph setup, see Encoder Selection. Apply your encoder overrides in this plan’s Column Processing section before training.

Create and Use a Plan

Start with the query, snapshot, and training_table from Predictive Queries:

from kumo.engine import Trainer
from kumoapi.model_plan import RunMode
plan = query.suggest_model_plan(snapshot, run_mode=RunMode.FAST)
print(plan.as_json())
trainer = Trainer(plan)
fit_result = trainer.fit(training_table, snapshot=snapshot)
print(fit_result.metrics)

Suggestion uses graph and query metadata and does not start training. Choose RunMode.FAST, RunMode.NORMAL, or RunMode.BEST for progressively larger search budgets. RunMode.DEBUG is for diagnostic runs, not model quality evaluation. For a supplied training table with sample weights, pass its TrainingTableSpec as train_table_spec= to the suggestion call.

Keep suggested values for the first run; change a setting when its behavior matches a specific requirement. Compare validation results on the same data and splits when evaluating a change; reserve holdout data for the final assessment. See Training Diagnostics for experiment and epoch history.

Read Values and Search Spaces

The plan is a kumoapi.model_plan.ModelPlan with five sections:

SectionPurpose
training_jobExperiment count, evaluation, trial pruning, and refits.
column_processingColumn encoders and missing values.
neighbor_samplingGraph neighborhood sampling.
optimizationTraining budget, losses, optimizers, and batch sizes.
model_architectureMessage passing, prediction heads, and embeddings.

Most tunable settings take a nonempty list of candidates. For example, base_lr=[0.001, 0.003] offers two learning rates, while base_lr=[0.001] fixes that choice. num_experiments limits the trials; Kumo does not necessarily exhaust every combination. Nested lists group one candidate: num_neighbors=[[20, 10]] means one two-hop sampling configuration.

Scalar settings such as max_epochs and num_experiments take a single value. None is an explicit option only where the tables below allow it; it does not universally mean “infer a default.” Unspecified settings are filled from the task, graph, and run mode. Inspect plan.as_json() instead of assuming the same suggested values for every dataset. The bounds below describe valid configuration values, not recommended capacity settings.

Training Job Plan

Set these fields on plan.training_job. Keep the suggested search budget unless you need to cap trials. Choose tune_metric for the outcome you want to select on. Disable refits when you only need trial results; use refit_full only with separate unseen evaluation data.

Training job parameters and pruning
ParameterSupported valuesMeaning
num_experimentsPositive integerMaximum number of AutoML experiments. Suggested budget depends on run mode.
manual_seedInteger or None; default NoneSeed model training and the trial search to help repeat an experiment.
metricsList of metric names valid for the taskMetrics computed during evaluation. See Choose Metrics.
tune_metricOne metric nameMetric used to compare experiments and select a model. Include it in metrics; "loss" is also available for tasks other than link prediction.
pruningPruningConfig or NoneStop unpromising trials using previous trials as a reference. None disables trial pruning.
refit_trainvalTrue or False; suggested by run modeRefit the selected configuration on training plus validation rows.
refit_fullTrue or False; default FalseAlso refit on training, validation, and test rows. Reserve separate unseen data to evaluate this model.

PruningConfig takes min_epochs (integer at least 1), k (integer at least 1), min_delta (number at least 0), and patience (positive integer). It compares the trial with the best k previous trials after min_epochs, using the improvement threshold and patience to decide when to stop it. Trial pruning differs from early stopping, which monitors one trial’s own validation progress.

Choose Metrics

Use metric names for the query’s task. Common choices are:

TaskMetric choices
Binary classification"auroc", "ap", "auprc", "acc", "f1", "precision", "recall".
Multiclass classification"acc", "f1", "precision", "recall".
Multilabel classification"ap_macro", "ap_micro", "auroc_macro", "auroc_micro", "auprc_macro", "auprc_micro".
Regression and forecasting"mae", "mse", "rmse", "mape", "smape".
Link prediction and multilabel ranking"map@10", "precision@10", "recall@10", "ndcg@10", "mrr@10". Replace 10 with the desired positive cutoff.

For example, for a regression query:

plan.training_job.metrics = ["mae", "mse", "rmse"]
plan.training_job.tune_metric = "mae"

These fields select built-in metrics, not Python callbacks. To evaluate a custom metric, score held-out predictions against separately retained ground truth.

Column Processing

Set these fields on plan.column_processing.

ParameterSupported valuesMeaning
encoder_overridesDictionary from "table.column" to an encoder, or NoneReplace the inferred encoder for specific columns. Names are case sensitive and use graph table names.
na_strategyDictionary from Stype to NAStrategy, or NoneSet missing-value defaults by semantic type for compatible inferred feature encoders.

Both fields default to None. See Missing Values for the type-wide defaults recipe, column-specific overrides, precedence, and validation behavior. Encoder Selection lists encoder arguments and examples.

Neighbor Sampling

Set these fields on plan.neighbor_sampling. Keep suggested fanouts unless a relationship needs a different context limit. Larger neighborhoods usually require more memory and time.

ParameterSupported valuesMeaning
num_neighborsList of per-hop integer lists, or NumNeighborsConfig candidatesUp to six hops per candidate. Each count is -1 (all neighbors), 0 (none), or 1 to 128.
sample_from_entity_tableTrue or FalseInclude the entity table when sampling. Applies to static node prediction tasks.
adaptive_samplingTrue or False; default FalseCompensate for undersampling at an earlier hop by sampling more at later hops.

For one candidate with at most 20 neighbors at the first hop and 10 at the second:

plan.neighbor_sampling.num_neighbors = [[20, 10]]

Override a Relationship’s Fanout

Use NumNeighborsConfig when different relationships need different amounts of context.

Configure fanouts by relationship and adaptive sampling

The following example uses the SalesOrderHeader.CustomerID relationship to Customer.CustomerID in Create a Graph:

from kumoapi.model_plan import HopConfig, NumNeighborsConfig
hop1 = HopConfig(default=20)
hop1["Customer.CustomerID->SalesOrderHeader.CustomerID"] = 40
hop1["SalesOrderHeader.CustomerID->Customer.CustomerID"] = 1
plan.neighbor_sampling.num_neighbors = [
NumNeighborsConfig(hop1=hop1, hop2=HopConfig(default=10)),
]
plan.neighbor_sampling.adaptive_sampling = False

Use graph table names, including aliases, and the actual primary-key and foreign-key columns. The two directions have separate overrides. Unlisted relationships use that hop’s default; at hop2, both directions above use 10. Each default accepts integers from -1 to 128; a named edge accepts integers from -1 to 512. 0 samples none and -1 requests all neighbors, which can exhaust memory on a large relationship. Define consecutive hops beginning with hop1, up to six hops. An unknown relationship or a gap in hop numbers is invalid; correct the graph names or configuration before fitting.

Some suggested plans use "inferred" for a target-to-entity connection in hop1. This is a task-specific fanout estimated from the data, not a general substitute for an integer on any edge. It is invalid on other connections or later hops. Use explicit integer overrides when setting your own edge counts.

To compensate for sparse earlier hops, set plan.neighbor_sampling.adaptive_sampling = True. Later hops can then sample more neighbors than their nominal count. All counts must be nonnegative: -1 is incompatible with adaptive sampling. Inspect memory use and runtime before increasing the graph context; enabling adaptation does not guarantee better validation quality.

Optimization

Set these fields on plan.optimization. Retain suggested losses and optimizer settings unless your task requires a different objective or a bounded comparison. Cap epochs or steps to limit work; a shorter run is not evidence of equivalent model quality.

Optimization parameters and learning-rate search example
ParameterSupported valuesMeaning
max_epochsPositive integerMaximum epochs per experiment.
min_steps_per_epochInteger 1 to 4000Minimum training batches per epoch.
max_steps_per_epochPositive integerMaximum training batches per epoch. Keep it at least as large as min_steps_per_epoch.
max_val_stepsInteger 1 to 8000Maximum batches in a validation pass.
max_test_stepsInteger 1 to 8000Maximum batches in a test pass.
lossList of loss names or loss configurationsTraining objective. Must match the task; see Loss Functions.
optimizerList containing "Adam", "AdamW", or "Adagrad"Optimizer candidates.
base_lrList of numbers greater than 0Initial learning rates.
weight_decayList of numbers at least 0Weight regularization coefficients.
batch_sizeList of integers 1 to 2048Training examples per batch.
early_stoppingList containing EarlyStoppingConfig instances or NoneStop a trial when validation improvement stalls. [None] disables it.
lr_schedulerList containing LRSchedulerConfig instances or NoneLearning rate schedules. [None] disables scheduling.
majority_sampling_ratioList containing positive numbers or NoneMajority-class sampling ratios for binary classification. [None] disables class rebalancing. A numeric candidate requires every weight_mode candidate to be non-None.
weight_modeList containing "sample", "weighted_loss", or NoneSample rows according to weights, weight their loss, or ignore weights during training. Supported for binary classification or weighted training tables, except forecasting; weighted link prediction supports "weighted_loss" or None.
lookup_optimizationList containing OptimizerConfig instances or "inferred"Optimizer and learning rate for link prediction lookup embeddings. OptimizerConfig takes optimizer and positive base_lr.
target_fractionList of numbers from 0 to 1, inclusiveFraction of ground-truth links used for supervision in static link prediction; at least one link per entity is retained.
overlap_ratioList of numbers from 0 to 1, inclusiveOverlap between message passing and supervision links in static link prediction.

EarlyStoppingConfig(min_delta=0.0, patience=3) requires a nonnegative improvement threshold and positive integer patience. LRSchedulerConfig takes name, interval ("step" or "epoch"), and an optional kwargs dictionary for that scheduler. Supported warmup schedule names include "constant_with_warmup", "linear_with_warmup", "cosine_with_warmup", and "cosine_with_warmup_restarts". Additional scheduler names and arguments must resolve in the installed training runtime.

For a bounded learning-rate search:

from kumoapi.model_plan import EarlyStoppingConfig
plan.training_job.num_experiments = 2
plan.training_job.manual_seed = 42
plan.optimization.base_lr = [0.001, 0.003]
plan.optimization.batch_size = [256]
plan.optimization.early_stopping = [
EarlyStoppingConfig(min_delta=0.0, patience=3),
]

A fixed seed helps repeat a run with the same data, plan, package versions, and execution environment. It does not guarantee identical scores across devices, library versions, or parallel execution. Retain plan.as_json() and the data snapshot identity with your results. The search selects a model; it does not ensemble its trials.

Use Row Weights and Balance Binary Classes

Use row weights when examples should contribute differently, or a class ratio when binary class influence needs rebalancing.

Configure row weights and class balancing

Start with the scalar weighted custom training table. Reuse its custom_query, custom_snapshot, custom_training_table, and spec. Pass the same specification when suggesting the plan so Kumo knows the table contains weights:

plan = custom_query.suggest_model_plan(
custom_snapshot, run_mode=RunMode.FAST, train_table_spec=spec
)
plan.optimization.weight_mode = ["weighted_loss"]
plan.optimization.majority_sampling_ratio = [None]

Choose one mode before creating Trainer(plan) and calling fit(custom_training_table, snapshot=custom_snapshot):

weight_modeTraining behavior
["weighted_loss"]Multiply each example’s loss contribution by its weight.
["sample"]Use weights to choose training rows more or less often.
[None]Ignore row weights for training; evaluation can still use them.

Use finite, nonnegative scalar weights, with positive weight remaining in the training population. A zero weight removes that row’s training contribution in weighted loss or its sampling probability in weighted sampling. This scalar example does not define weights for list targets or forecast sequences. Weighted link prediction permits "weighted_loss" or None, not "sample"; forecasting does not support these training weight modes.

For binary classification, optionally add a class-balancing ratio:

plan.optimization.weight_mode = ["sample"]
plan.optimization.majority_sampling_ratio = [1.0]

The ratio is the desired majority-to-minority balance. 1.0 requests equal class influence when the original classes are imbalanced. Kumo increases the minority class weight only when the observed imbalance exceeds the requested ratio. When row weights are also present, Kumo multiplies their effect by the class weights, so arbitrary row weights can change the final class balance. With "sample", these weights affect row selection; with "weighted_loss", they affect the loss. Neither mode promises an exact class count in each batch. A numeric ratio requires every weight_mode candidate to be non-None. To ignore weights, set both weight_mode=[None] and majority_sampling_ratio=[None].

Fit using the bounded procedure under Training, then compare validation metrics and class-specific behavior before retaining a configuration. Different sampling frequencies or a lower training loss do not by themselves establish better prediction quality.

Loss Functions

Choose distribution losses only when their assumptions match your target values.

Loss names and configuration arguments
TaskSupported loss names
Binary and multilabel classification"binary_cross_entropy", "focal".
Multiclass classification and multilabel ranking"cross_entropy", "ordinal_log".
Static and temporal link prediction"cross_entropy", "binary_cross_entropy".
Regression and forecasting"mae", "mse", "huber", "wape", "normal_distribution", "negative_binomial_distribution", "log_normal_distribution", "multi_quantile".

For custom loss parameters, import the corresponding configuration from kumoapi.model_plan:

ConfigurationArguments
FocalLossConfigname="focal"; alpha greater than 0 and less than 1 (default 0.25); gamma at least 1 (default 2.0). Controls class weighting and emphasis on difficult examples.
HuberLossConfigname="huber"; delta greater than 0 (default 1.0). Controls the transition between squared and absolute error.
OrdinalLogLossConfigname="ordinal_log"; alpha at least 0 (default 1.0). Controls the ordinal loss coefficient.
QuantileLossConfigname="quantile"; q from 0 to 1, inclusive. Selects one quantile for regression or forecasting.

For a regression query, plan.optimization.loss = [HuberLossConfig(name="huber", delta=2.0)] selects a Huber loss with a custom threshold.

Model Architecture

Set plan.model_architecture.model to a list of architecture configurations. Keep the suggested architecture initially. Set explicit candidates to compare architectures on the same task and data. The following illustrates candidate syntax, not recommended channel counts:

from kumoapi.model_plan import GNNModelPlan
plan.model_architecture.model = [
GNNModelPlan(
channels=[64, 128],
aggregation=[["sum", "mean", "max"]],
dropout=[0.0],
),
]

GNN and Graph Transformer

GNNModelPlan supports all task types. GraphTransformerModelPlan supports node prediction tasks, including forecasting, but not link prediction. Import both from kumoapi.model_plan.

GNN and Graph Transformer parameters
ConfigurationParameterSupported values and meaning
BothchannelsList of even integers from 2 to 512. Hidden representation size.
GNNaggregationList of lists containing "sum", "mean", "min", "max", "std", or "var". Each inner list combines neighbor aggregation operations.
BothdropoutList of rates at least 0 and less than 1. Fraction of features dropped during training.
Graph Transformernum_layersList of integers 1 to 12. Transformer depth.
Graph Transformernum_headsList of integers 1 to 32. Attention heads; every candidate channel count must be divisible by every candidate head count.
Graph Transformerpositional_encodingsList of lists containing "hop" or "random_gnn". Use [[]] for no positional encoding.

Shared Architecture Settings

These fields belong directly to plan.model_architecture.

Shared architecture parameter reference
ParameterSupported valuesMeaning
num_pre_message_passing_layersList of integers 0 to 4MLP layers before message passing.
num_post_message_passing_layersList of integers 1 to 8MLP layers after message passing.
activationList containing "relu", "leaky_relu", "elu", or "gelu"Activation function candidates.
normalizationList containing "layer_norm", "batch_norm", or NoneNormalization layer candidates.
use_seq_idList of booleansEncode the order of fact records as a model feature.
prediction_time_encodingsList of booleansEncode absolute prediction time for temporal node prediction.
past_encoderList containing "decomposed", "normalized", "mlp", "transformer", or NoneEncode historical labels for forecasting with positive lag_timesteps. Does not apply to temporal regression.
handle_new_entitiesBooleanForecasting only. True uses a shared prediction head; False adds learned entity-specific parameters to that head.

Forecasting Architecture and Historical Inputs

For a forecasting query, the suggested architecture uses a GNN with a forecasting head. You can choose GNN or Graph Transformer candidates using the configuration classes above. The forecasting head combines graph context with the historical target inputs available in the training table; these controls do not turn a regression query into a forecasting query.

Configure forecasting history and entity handling

Historical inputs are created by TrainingTableGenerationPlan, separately from the model plan. Generated forecasts require a positive integer lag_timesteps; leaving it unspecified uses one previous target window, while 0 is rejected. year_over_year=True requests the corresponding previous-year sequence. The encoder is activated only when those values are present in the materialized table. Changing past_encoder after generation does not change the table’s lag windows or create missing historical values. Follow Generate Forecast Inputs to generate the intended horizon and historical inputs before fitting.

For a forecasting table with positive lag windows, select a historical encoder before fitting:

plan.model_architecture.past_encoder = ["decomposed"]
plan.model_architecture.handle_new_entities = True

"decomposed" is the suggested historical encoder. "normalized", "mlp", and "transformer" are alternative candidates. handle_new_entities=True uses a shared head. False adds learned parameters for individual training entities; it does not merely replace the shared head. Keep it True when you need forecasts for new entities, subject to the graph and feature requirements of your model. These settings apply to forecasting, not ordinary temporal regression.

To disable only the historical target encoder, set this before creating the trainer and fitting:

plan.model_architecture.past_encoder = [None]

Keep the generated training table’s positive lag setting unchanged. The forecasting head still receives the raw lag values; disabling their encoder does not remove historical inputs or stop generation of the lag columns. It also does not disable the separate year-over-year encoder when the table contains those inputs. This choice is distinct from lag_timesteps=0, which is rejected when constructing generated or supplied forecast training tables.

After fitting or loading the model, reuse trainer.training_table_generation_plan when generating prediction rows so the horizon, lags, year-over-year inputs, and step size match training. See Select the Prediction Population and the forecast output example. The default output has ENTITY, TIMESTAMP, and TARGET_PRED; select optional ANCHOR_TIMESTAMP explicitly when you need the common forecast anchor.

These fields also belong directly to plan.model_architecture, and apply to link prediction.

Link prediction and embedding parameters
ParameterSupported valuesMeaning
module"shallow_rhs", "two_tower", "context", "pairwise"Prediction head. "context" and "pairwise" require temporal link prediction.
handle_new_target_entitiesBooleanAllow predictions involving target entities unseen during training. Keep target feature and embedding choices compatible with this requirement.
target_embedding_modeList containing "lookup", "feature", "shallow_feature", or "fusion"Use learned target lookup embeddings, target features, shallow target features, or a combination of lookup and features.
output_embedding_dimList of integers 1 to 256Output embedding dimension.
ranking_embedding_loss_coeffList of numbers at least 0Auxiliary embedding loss coefficient for temporal link prediction with module="context".
embedding_norm_reg_coeffList of numbers at least 0L2 regularization coefficient for final embeddings.
distance_measureList containing "dot_product" or "cosine"Similarity function for link scores.

target_embedding_mode controls the target representation for "shallow_rhs" and "context". For these modules, handle_new_target_entities=True requires "feature" or "shallow_feature"; "lookup" and "fusion" are incompatible. The "two_tower" and "pairwise" modules do not use target lookup embeddings.

Use "shallow_rhs" or "two_tower" when exported embeddings need to represent the model’s ranking. "context" produces approximate embeddings, and "pairwise" does not support embedding output. Legacy "ranking" and "embedding" module names remain compatibility aliases; use the explicit module names in new plans. See Making Predictions for embedding export.

Training Table Generation

Labels, time windows, and train/validation/test splits use a separate TrainingTableGenerationPlan passed to query.generate_training_table(). Changing a model plan does not regenerate existing training rows or splits. See Generate a Training Table for generation settings and supplied training tables.

Return to Training to fit your customized plan.