Storage for Model Caching on AKS

For implementing tiered storage on AKS, you can take advantage of the different storage options available in Azure. This guide covers choosing the right storage for each Dynamo cache type and configuring PVCs.

Available Storage Options

For Azure Managed Lustre setup, see the Azure Lustre CSI Driver guide.

Recommendations by Cache Type

• Model Cache — raw model artifacts, configuration files, tokenizers, etc.

  • Persistence: Required to avoid repeated downloads and reduce cold-start latency.
  • Recommended storage: Azure Managed Lustre (shared, high throughput) or Azure Disk (single-replica, persistent).

• Compilation Cache — backend-specific compiled artifacts (e.g., TensorRT engines).

  • Persistence: Optional.
  • Recommended storage: Local CSI (fast, node-local) or Azure Disk (persistent when GPU configuration is fixed).

• Performance Cache — runtime tuning and profiling data.

  • Persistence: Not required.
  • Recommended storage: Local CSI (or other ephemeral storage).

Check Available Storage Classes

List the storage classes available in your AKS cluster:

$
kubectl get storageclass
$
$
NAME                           PROVISIONER                 RECLAIMPOLICY
$
azureblob-csi                  blob.csi.azure.com          Delete
$
azurefile                      file.csi.azure.com          Delete
$
azurefile-csi                  file.csi.azure.com          Delete
$
azurefile-csi-premium          file.csi.azure.com          Delete
$
azurefile-premium              file.csi.azure.com          Delete
$
default                        disk.csi.azure.com          Delete
$
managed                        disk.csi.azure.com          Delete
$
managed-csi                    disk.csi.azure.com          Delete
$
managed-csi-premium            disk.csi.azure.com          Delete
$
managed-premium                disk.csi.azure.com          Delete
$
sc.azurelustre.csi.azure.com   azurelustre.csi.azure.com   Retain

Example PVC Configuration

In the cache.yaml in the different recipes, you can set the storageClassName to a storage option available in your AKS cluster:

1
apiVersion: v1
2
kind: PersistentVolumeClaim
3
metadata:
4
  name: model-cache
5
spec:
6
  accessModes:
7
    - ReadWriteMany
8
  resources:
9
    requests:
10
      storage: 100Gi
11
  storageClassName: "sc.azurelustre.csi.azure.com"
12
---
13
apiVersion: v1
14
kind: PersistentVolumeClaim
15
metadata:
16
  name: compilation-cache
17
spec:
18
  accessModes:
19
    - ReadWriteMany
20
  resources:
21
    requests:
22
      storage: 50Gi
23
  storageClassName: "azurefile-csi"
24
---
25
apiVersion: v1
26
kind: PersistentVolumeClaim
27
metadata:
28
  name: perf-cache
29
spec:
30
  accessModes:
31
    - ReadWriteMany
32
  resources:
33
    requests:
34
      storage: 50Gi
35
  storageClassName: "local-ephemeral"

See Also

• Azure Lustre CSI Driver — Full setup guide for Azure Managed Lustre
• Model Caching — Full walkthrough for setting up model caching with Dynamo, including download Jobs and mount configuration
• AKS CSI Storage Drivers — Microsoft documentation for all built-in CSI drivers