Kubernetes DeploymentCloud Provider GuidesAzure

Storage for Model Caching on AKS

View as Markdown

For implementing tiered storage on AKS, you can take advantage of the different storage options available in Azure. This guide covers choosing the right storage for each Dynamo cache type and configuring PVCs.

Available Storage Options

Storage OptionPerformanceBest For
Local CSI (Ephemeral Disk)Very highFast model caching, warm restarts
Azure Managed LustreExtremely highLarge multi-node models, shared cache
Azure Disk (Managed Disk)HighPersistent single-writer model cache
Azure FilesMediumShared small/medium models
Azure Blob (via Fuse or init)Low-MediumCold model storage, bootstrap downloads

Azure Managed Lustre and Local CSI (ephemeral disk) are not installed by default in AKS and require additional setup before use. Azure Disk, Azure Files, and Azure Blob CSI drivers are available out of the box. See the Azure Lustre CSI Driver guide for Lustre setup, or the AKS CSI storage options documentation for a full overview of built-in drivers.

For Azure Managed Lustre setup, see the Azure Lustre CSI Driver guide.

Recommendations by Cache Type

  • Model Cache — raw model artifacts, configuration files, tokenizers, etc.

    • Persistence: Required to avoid repeated downloads and reduce cold-start latency.
    • Recommended storage: Azure Managed Lustre (shared, high throughput) or Azure Disk (single-replica, persistent).

  • Compilation Cache — backend-specific compiled artifacts (e.g., TensorRT engines).

    • Persistence: Optional.
    • Recommended storage: Local CSI (fast, node-local) or Azure Disk (persistent when GPU configuration is fixed).

  • Performance Cache — runtime tuning and profiling data.

    • Persistence: Not required.
    • Recommended storage: Local CSI (or other ephemeral storage).

Check Available Storage Classes

List the storage classes available in your AKS cluster:

kubectl get storageclass
NAME                           PROVISIONER                 RECLAIMPOLICY
azureblob-csi                  blob.csi.azure.com          Delete
azurefile                      file.csi.azure.com          Delete
azurefile-csi                  file.csi.azure.com          Delete
azurefile-csi-premium          file.csi.azure.com          Delete
azurefile-premium              file.csi.azure.com          Delete
default                        disk.csi.azure.com          Delete
managed                        disk.csi.azure.com          Delete
managed-csi                    disk.csi.azure.com          Delete
managed-csi-premium            disk.csi.azure.com          Delete
managed-premium                disk.csi.azure.com          Delete
sc.azurelustre.csi.azure.com   azurelustre.csi.azure.com   Retain

Example PVC Configuration

In the cache.yaml in the different recipes, you can set the storageClassName to a storage option available in your AKS cluster:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: model-cache
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: "sc.azurelustre.csi.azure.com"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: compilation-cache
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Gi
  storageClassName: "azurefile-csi"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: perf-cache
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Gi
  storageClassName: "local-ephemeral"

See Also