> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server.

# Publish and Retrieve Models

> Retain a trained model beyond local scratch and load it in a later Python or notebook session

Publish a trained model when you need to reuse it after the current Python process or notebook session ends.
Publication copies the complete model bundle to durable storage or a supported model registry.
Retrieval validates that bundle and restores a fitted `Trainer`.
Connector retrieval first fetches a local copy.

This page continues from [Training](/kumo-engine/1.0.0/user-code-guide/train-a-model/training).
Choose the retention method by destination. Linux deployments commonly use a retained filesystem, S3, or GCS. Snowflake notebooks can use Snowflake Model Registry. The demonstrated Databricks notebook copies the complete bundle to a Unity Catalog volume and loads it from that path in a later process.

## Understand What a Model Bundle Contains

Every successful fit creates a reloadable bundle at `fit_result.bundle_dir`.
The bundle includes the trained checkpoint, model plan, query, effective training-table generation settings, metrics, and manifests required by `Trainer.load()`.

The bundle does not include:

* Source tables or credentials
* The materialized graph snapshot or index
* Generated training or prediction inputs
* Prediction or embedding outputs, including the local holdout prediction file
* Complete epoch-by-epoch training history
* Post-training explanation files

Retain these artifacts separately before deleting scratch if you need to reuse them.
Keep the entire bundle unchanged.
A checkpoint by itself is not sufficient, and a model bundle cannot recreate missing source data or graph storage.
Use [Training Diagnostics](/kumo-engine/1.0.0/user-code-guide/train-a-model/training-diagnostics#read-holdout-predictions) to retain holdout predictions and detailed training history separately, and [Model Explainability](/kumo-engine/1.0.0/user-code-guide/train-a-model/model-explainability) for explanation files produced for supported tasks.

## Publish During Training

Supply `ModelOutputConfig` to the fit call when you want a durable model copy.
This replaces the ordinary fit call; it does not require an additional training run.

`ModelOutputConfig(connector, path=None, location=None)` requires exactly one destination:

* Use `path` for filesystem or object-storage publication.
* Use `location` with `ModelRegistryLocation` for a supported model registry.

#### Filesystem and Object Storage

Publish a model to a separate filesystem destination with a `FileConnector`:

```python
from kumo.engine import FileConnector, ModelOutputConfig

destination = FileConnector(root_dir="/path/to/retained-models")
fit_result = trainer.fit(
    training_table,
    snapshot=snapshot,
    export_config=ModelOutputConfig(destination, path="model-v1"),
)
print(fit_result.published_model)
```

Choose a new or empty destination that does not overlap the working bundle.
Filesystem publication creates an independent model copy; its durability depends on the destination filesystem.
For S3 or GCS, use the corresponding output connector and a path within that connector.
`fit_result.bundle_dir` remains the working copy; `fit_result.published_model` describes the published copy.

#### Snowflake Model Registry

Publish to Snowflake Model Registry with the connector configured for the notebook session:

```python
from kumo.engine import ModelOutputConfig, ModelRegistryLocation

fit_result = trainer.fit(
    training_table,
    snapshot=snapshot,
    export_config=ModelOutputConfig(
        connector,
        location=ModelRegistryLocation("CUSTOMER_ORDER_MODEL"),
    ),
)
published = fit_result.published_model
print("Model:", published.name)
print("Version:", published.version)
```

The registry uses the connector's database and schema.
The notebook identity needs permission to create the model or add a version.
Record the returned name and assigned version; do not substitute an assumed version later.

### Databricks Notebook

`DatabricksConnector` does not support model publication through `ModelOutputConfig` in this local Python path. After a successful fit, copy the complete bundle to a Unity Catalog volume. Use `ARTIFACT_ROOT` from [Databricks Notebook Setup](/kumo-engine/1.0.0/get-started-with-kumo-engine/installation/databricks-notebook#configure-working-storage) and give each run a separate directory:

```python
from pathlib import Path
import shutil
from uuid import uuid4

run_root = ARTIFACT_ROOT / uuid4().hex
run_root.mkdir(parents=True, exist_ok=False)
model_dir = run_root / "model"
shutil.copytree(Path(fit_result.bundle_dir), model_dir)
print("Record this model path:", model_dir)
```

Keep the complete copied directory, including its manifests. Record the path and verify that you can load it before deleting local scratch. The demonstrated Databricks run used this copy procedure; it did not publish a model registry entry.

The volume copy survives a cluster restart while the volume and its grants are retained. The graph snapshot, generated training table, working model files, and detailed training history remain in driver local scratch unless retained separately. A model bundle does not contain source data, credentials, or the graph index. See [Training Diagnostics](/kumo-engine/1.0.0/user-code-guide/train-a-model/training-diagnostics) to retain history files.

## Retrieve a Published Model

Load only model bundles from publishers you trust.
Retrieval validates the bundle and restores a fitted trainer for local execution; it does not perform registry-hosted inference.

For models with text embeddings, keep the matching word vectors available in the prediction environment. Retain the text package for its installed models, or recreate the registered embedding store and pass `embedding_models=store` when loading. Keep any custom embedding module importable after restart. Follow [Reload a Trained Text Model](/kumo-engine/1.0.0/user-code-guide/set-up-a-graph/encoder-selection#reload-a-trained-text-model) and [Custom Word Vectors](/kumo-engine/1.0.0/user-code-guide/set-up-a-graph/encoder-selection#custom-word-vectors) before using the retrieval examples below.

`Trainer.load()` and `Trainer.load_from_connector()` restore a completed model for inference.
To initialize a new training run from the saved weights, pass the loaded trainer explicitly as [`warm_start`](/kumo-engine/1.0.0/user-code-guide/train-a-model/training#start-a-new-run-from-a-trained-model).
Loading alone does not resume interrupted training.
Calling `fit()` without `warm_start` starts fresh.

#### Filesystem and Object Storage

Load a retained local bundle directly:

```python
from kumo.engine import Trainer, configure

configure(scratch_dir="/path/to/local-storage/kumo-scratch")
trainer = Trainer.load("/path/to/retained-models/model-v1")
```

Alternatively, fetch the filesystem publication through a `FileConnector`:

```python
from kumo.engine import FileConnector

file_connector = FileConnector(root_dir="/path/to/retained-models")
trainer = Trainer.load_from_connector(file_connector, path="model-v1")
```

Use the same root as publication and the recorded path relative to that root.
Here, `path="model-v1"` identifies `/path/to/retained-models/model-v1`.
Retrieval copies the bundle into configured scratch and validates it before loading.
Keep scratch separate from the retained publication, allow room for the copy, and retain read access to the source.

For S3 or GCS, set `storage_connector` to the configured connector used for publication, then retrieve the bundle with its recorded path:

```python
trainer = Trainer.load_from_connector(
    storage_connector,
    path="models/model-v1",
)
```

The connector must point to the publication storage and have read access.
Use the address recorded at publication; filesystem paths are relative to the configured root.

#### Snowflake Model Registry

Reconnect to Snowflake, then load the exact version returned by publication:

```python
from kumo.engine import ModelRegistryLocation, Trainer

trainer = Trainer.load_from_connector(
    connector,
    location=ModelRegistryLocation(
        "CUSTOMER_ORDER_MODEL",
        version="RECORDED_VERSION",
    ),
)
```

Replace `RECORDED_VERSION` with `published.version` from the completed fit.
The connector must use the database and schema containing the registered model.
Omitting `version` selects the registry's default version, which is not necessarily the most recently created version.
Use an explicit recorded version for repeatable retrieval.

#### Databricks Notebook

Install `kumo-engine[databricks]`, configure new local scratch, and load the complete bundle from its recorded Unity Catalog volume path:

```python
from kumo.engine import Trainer

trainer = Trainer.load(
    "/Volumes/<catalog>/<schema>/<volume>/runs/<recorded-run-id>/model"
)
```

Recreate the source connection, table and graph definitions, and a compatible graph snapshot before scoring. Use the restored trainer's `training_table_generation_plan` to generate a new prediction table. The demonstrated Databricks workflow loaded a saved bundle in a fresh process; it did not retrieve a Databricks model registry entry. Keep the original run directory until the new process completes a scoring check.

FileConnector, S3, GCS, and Snowflake support connector-based model retrieval.
The local Databricks connector does not publish or retrieve model bundles; use a retained Unity Catalog volume and `Trainer.load()` as shown above.
BigQuery does not support model publication or retrieval through this API.
It does support [prediction export](/kumo-engine/1.0.0/user-code-guide/making-predictions#bigquery-output).

### Find a Published Snowflake Model

Use a Snowflake connector whose identity can read the registry in the intended database and schema.
List the visible Kumo model versions before choosing one to retrieve:

```python
models = connector.sink_backend.list_models(connector)
for model in models[:10]:
    print(model.name, model.version, model.uri)
    print("Metadata:", dict(list(model.metadata.items())[:5]))
    print("Metrics:", dict(list(model.metrics.items())[:5]))
```

The listing returns published-model records ordered by name and version, with recorded training metadata and metrics for each version.
The slice limits displayed records, not the registry listing itself.
Discovery does not download model bundles.
Only versions carrying Kumo's metadata marker are included; an empty list can mean no visible Kumo versions even when other models exist in the registry.
Check the connector's database, schema, and registry permissions if an expected version is missing.

Copy the chosen record's `name` and `version` into the explicit `ModelRegistryLocation` retrieval example above.
After retrieval, inspect the retained [configuration and lineage](/kumo-engine/1.0.0/user-code-guide/train-a-model/training-diagnostics#inspect-the-trained-configuration) to confirm the query and training snapshot associated with the model.

## Recreate the Prediction Context

Loading a model restores its fitted state, but not its data.
Before making predictions in a new process:

1. Reconnect to the source data.
2. Recreate compatible table, graph, and predictive-query definitions.
3. Materialize or restore a compatible graph snapshot.
4. Generate the population to score using the restored trainer's generation settings.

The snapshot can contain updated data, provided the trained target, entity, relationships, and feature requirements remain compatible.
Continue to [Making Predictions](/kumo-engine/1.0.0/user-code-guide/making-predictions#select-the-population) to score entities, generate embeddings, and save the outputs. To repeat this workflow in a scheduled process, follow [Production Workflow](/kumo-engine/1.0.0/user-code-guide/deploy-a-model/overview#prepare-the-production-job).