Refresh a Model

Refresh your model on a fixed cadence, in response to observed performance, or both. Keep retraining separate from recurring prediction: refreshing a graph snapshot supplies new data without fitting new model weights.

Choose a Refresh Cadence

Base the interval on how quickly the data changes, when labels become available, and training cost. For example, score customers daily and evaluate a new candidate monthly.

For temporal tasks, allow the outcome window to complete before evaluating a prediction batch. A prediction of purchases over the next 30 days needs those outcomes before its accuracy can be measured.

Schedule a separate training workflow. Keep prediction jobs on the accepted model while the candidate trains and is evaluated.

Retrain Based on Observed Performance

Retain the accepted batch’s model version and predictions with the identifiers needed to join them to ground truth. Use entity and class or candidate identifiers where applicable. For temporal tasks, select anchor metadata before scoring; forecast outputs also identify their forecast window with TIMESTAMP. Scoring time is not a substitute for either. See prediction result columns for each task’s keys.

Track a task-appropriate metric and business outcome, including important population segments. Define the threshold or sustained decline that prompts review, and assign an owner.

Score distributions and population changes can flag issues before labels arrive, but do not measure accuracy. Investigate source freshness, schema, scoring filters, and label definitions before retraining. Fix data pipeline problems at the source.

Use these observations to trigger retraining or adjust the regular refresh cadence.

Evaluate and Replace the Model

1. Prepare updated data and regenerate training rows and labels with appropriate evaluation splits. Existing training tables retain their original labels.
2. Train a candidate and retain its bundle under a new path or registry version.
3. Compare it with the current model using the same evaluation population and outcome definition. Keep evaluation outcomes out of candidate training. Check your quality criteria and test prediction in the scheduled runtime with the candidate’s required packages and embedding models.
4. If accepted, update the prediction job to load the new version and apply the same batch acceptance checks to its first production output.

Keep the previous model, its required embedding assets, and compatible configuration for rollback. If the candidate fails acceptance checks, continue using the current version. Record which model produced each accepted batch so later performance measurements can distinguish the versions.