> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nvcf/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nvcf/_mcp/server.

# Distributed Shader Cache (GXCache)

This guide provides detailed information on the Distributed Shader Cache (GXCache) component and its use in NVCF GPU clusters.

GXCache is a cloud-native shader cache for NVIDIA that provides distributed shader caching capabilities. It consists of the GXCache server and mutating webhook to optimize shader compilation and caching across your cluster.

GXCache improves rendering performance by caching compiled shaders, reducing compilation time for frequently used shaders and enabling faster scene loading and rendering.

## Installation

### Prerequisites

- A running Kubernetes cluster with `kubectl` access
- **Helm** >= 3.12
- Credentials for the registry where your NVCF charts and images are stored

### Step 1. Authenticate Helm to your chart registry

<details>
<summary>BYOC (NGC Helm Repo)</summary>

</details>

```bash
 helm repo add nvcf-byoc \
   https://helm.ngc.nvidia.com/nvidia/nvcf-byoc \
   --username '$oauthtoken' --password "${NGC_API_KEY}" --force-update
 helm repo update
```

<details>
<summary>Self-Hosted (NGC OCI)</summary>

</details>

```bash
 echo "${NGC_API_KEY}" | helm registry login nvcr.io \
   --username '$oauthtoken' --password-stdin
```

<details>
<summary>Self-Hosted (Amazon ECR)</summary>

</details>

```bash
 aws ecr get-login-password --region <region> | \
   helm registry login <account-id>.dkr.ecr.<region>.amazonaws.com \
   --username AWS --password-stdin
```

### Step 2. Create the namespace and image pull secret

```bash
 kubectl create namespace gxcache
```

Create an image pull secret so that pods can pull container images from your registry.

<details>
<summary>BYOC / NGC</summary>

</details>

```bash
 kubectl create secret docker-registry nvcr-creds \
   --docker-server=nvcr.io \
   --docker-username='$oauthtoken' \
   --docker-password="${NGC_API_KEY}" \
   --namespace=gxcache
```

<details>
<summary>Amazon ECR</summary>

</details>

```bash
 kubectl create secret docker-registry nvcr-creds \
   --docker-server=<account-id>.dkr.ecr.<region>.amazonaws.com \
   --docker-username=AWS \
   --docker-password="$(aws ecr get-login-password --region <region>)" \
   --namespace=gxcache
```

<details>
<summary>Other Registry</summary>

</details>

```bash
 kubectl create secret docker-registry nvcr-creds \
   --docker-server=<your-registry> \
   --docker-username=<your-username> \
   --docker-password=<your-password> \
   --namespace=gxcache
```

<Note>
The secret name ``nvcr-creds`` is referenced in the values file under
``webhook.deployment.secret`` and ``service.deployment.imageSecret``.
If you use a different secret name, update the values file to match.
</Note>

### Step 3. Create a values file

Create a `values.yaml` using the complete example in Base Configuration below.

- **BYOC users** pulling directly from NGC can use the `nvcr.io/nvidia/nvcf-byoc/` image
  paths without mirroring.
- **Self-hosted users** should replace the `<your-registry>/<your-repo>` placeholders with
  their mirrored registry path (see the Artifact Manifest in the self-hosted installation guide).

At minimum, configure the storage class for your environment (e.g., `gp3` for AWS EKS)
and the node selector for your GPU nodes.

### Step 4. Install the chart

<details>
<summary>BYOC (NGC Helm Repo)</summary>

</details>

```bash
 helm upgrade --install gxcache \
   nvcf-byoc/gxcache \
   --namespace gxcache \
   --values values.yaml
```

<details>
<summary>Self-Hosted (OCI)</summary>

Replace `<your-registry>/<your-repo>` with your mirrored registry path.

</details>

```bash
 helm upgrade --install gxcache \
   oci://<your-registry>/<your-repo>/gxcache \
   --version 0.8.2 \
   --namespace gxcache \
   --values values.yaml
```

### Step 5. Verify the installation

```bash
 # All pods should reach Running status
 kubectl get pods -n gxcache

 # Check the service and webhook are available
 kubectl get svc -n gxcache

 # Check persistent volume claims are bound
 kubectl get pvc -n gxcache
```

## Base Configuration

The following is a complete example `values.yaml` for deploying GXCache. Copy this file and
adjust values for your environment. Each section is explained in detail below.

- **BYOC users** can use the `nvcr.io/nvidia/nvcf-byoc/` paths directly.
- **Self-hosted users** should replace `<your-registry>/<your-repo>` with their mirrored
  registry path (see the Artifact Manifest in the self-hosted installation guide for source image paths).

```yaml
 webhook:
   nodeSelector:
     nvcf.nvidia.com/workload: gpu  # Adjust based on your node labels, or set to null
   deployment:
     image: <your-registry>/<your-repo>/gxcache-webhook
     version: 59bd8ec5
     secret: nvcr-creds  # Image pull secret created in Step 2
   client:
     version:
       image: <your-registry>/<your-repo>/gxcache-init
       version: 1e47f722
     config:
       tls:
         enabled: false
   metrics:
     enabled: false  # Requires monitoring.coreos.com/v1 CRD

 service:
   nodeSelector:
     nvcf.nvidia.com/workload: gpu  # Adjust based on your node labels, or set to null
   deployment:
     image: <your-registry>/<your-repo>/gxcache-service:b206ce39  # Tag is part of the image field
     imageSecret: nvcr-creds  # Image pull secret created in Step 2
   vault:
     enabled: false
   kns:
     enabled: true
     keyset:
       endpoint: https://notary-service.nvcf.nvidia.com
       api: /.well-known/jwks.json
   persistence:
     enabled: true
     storageClass: gp3  # Use gp3 for AWS EKS, adjust for other platforms
     accessMode: ReadWriteOnce
     size: 20Gi
   metrics:
     enabled: false  # Requires monitoring.coreos.com/v1 CRD
```

Replace `<your-registry>/<your-repo>` with your registry path. BYOC users pulling
directly from NGC can use `nvcr.io/nvidia/nvcf-byoc` as the registry/repo path.

<Tip>
If your cluster does not use node labels, set ``nodeSelector`` to ``null``.
</Tip>

## Configuration Sections

### Webhook Configuration

The GXCache webhook is responsible for intercepting and modifying pod specifications to enable shader caching:

```yaml
 # values.yaml

 webhook:
   nodeSelector:
     nvcf.nvidia.com/workload: gpu  # Adjust based on your node labels
   deployment:
     image: <your-registry>/<your-repo>/gxcache-webhook
     version: 59bd8ec5
     secret: nvcr-creds  # Registry credentials secret
   client:
     version:
       image: <your-registry>/<your-repo>/gxcache-init
       version: 1e47f722
     config:
       tls:
         enabled: false
   metrics:
     enabled: false  # Requires monitoring.coreos.com/v1 CRD
```

### Service Configuration

The GXCache service provides the shader cache functionality:

```yaml
 # values.yaml

 service:
   nodeSelector:
     nvcf.nvidia.com/workload: gpu  # Adjust based on your node labels
   deployment:
     image: <your-registry>/<your-repo>/gxcache-service:b206ce39
     imageSecret: nvcr-creds  # Registry credentials secret
   vault:
     enabled: false
```

### Node Selection

GXCache components are scheduled on nodes with appropriate labels to ensure they run on GPU compute nodes. Adjust the node selector based on your cluster's node labeling scheme.

```yaml
 # values.yaml

 webhook:
   nodeSelector:
     nvcf.nvidia.com/workload: gpu
 service:
   nodeSelector:
     nvcf.nvidia.com/workload: gpu
```

### Key Management

GXCache uses NVIDIA's Key Notary Service (KNS) for secure key management:

```yaml
 # values.yaml

 service:
   kns:
     enabled: true
     keyset:
       # Production endpoint
       endpoint: https://notary-service.nvcf.nvidia.com
       api: /.well-known/jwks.json
```

### Storage Configuration

GXCache requires persistent storage for caching shader data:

```yaml
 # values.yaml

 service:
   persistence:
     enabled: true
     storageClass: gp3  # Use gp3 for AWS EKS, adjust for other platforms
     accessMode: ReadWriteOnce
     size: 20Gi
```

### Metrics Configuration

GXCache supports Prometheus metrics for monitoring:

```yaml
 # values.yaml

 webhook:
   metrics:
     enabled: false  # Requires monitoring.coreos.com/v1 CRD
 service:
   metrics:
     enabled: false  # Requires monitoring.coreos.com/v1 CRD
```

## Architecture

### GXCache Architecture

GXCache consists of two main components:

1. **GXCache Service**: Provides the shader cache storage and retrieval functionality
2. **GXCache Webhook**: Intercepts pod creation to inject shader cache configuration

### Data Flow

1. **Pod Creation**: Application pod is created with GPU requirements
2. **Webhook Interception**: GXCache webhook intercepts pod creation
3. **Configuration Injection**: Webhook injects shader cache configuration
4. **Shader Compilation**: Application compiles shaders during runtime
5. **Cache Storage**: Compiled shaders are stored in GXCache
6. **Cache Retrieval**: Subsequent requests retrieve cached shaders

## Performance Considerations

### Storage Performance

For optimal shader cache performance:

- **Storage Type**: Use high-performance SSD storage (gp3, io1, io2)
- **Storage Size**: Size based on expected shader cache usage (20GB minimum)
- **Access Mode**: ReadWriteOnce is sufficient for most deployments

### Network Configuration

GXCache should be deployed to minimize network latency:

- Deploy in the same availability zone as GPU nodes
- Use high-bandwidth network connections
- Consider dedicated network interfaces for cache traffic

### Cache Size Planning

Plan cache size based on your workload:

- **Small workloads**: 20-50GB
- **Medium workloads**: 50-200GB
- **Large workloads**: 200GB+

## Monitoring and Observability

### Metrics

GXCache provides several key metrics:

- **Cache hit ratio**: Percentage of shader requests served from cache
- **Cache size**: Current size of cached shader data
- **Request latency**: Time to serve cached shaders
- **Storage utilization**: Disk space usage

### Prometheus Integration

To enable Prometheus metrics:

1. **Install Prometheus Operator** (if not already installed):

```bash
 helm install --wait --timeout 15m \
   --namespace monitoring --create-namespace \
   --repo https://prometheus-community.github.io/helm-charts \
   prometheus-agent kube-prometheus-stack
```

1. **Enable Metrics in GXCache**:

```bash
 helm upgrade gxcache -n gxcache . \
   --values values.yaml \
   --set webhook.metrics.enabled=true \
   --set service.metrics.enabled=true
```

1. **Access Prometheus**:

```bash
 kubectl port-forward -n monitoring svc/prometheus-operated 9090:9090
```

### Logging

GXCache logs include:

- Cache hit/miss events
- Shader compilation requests
- Error conditions and troubleshooting information
- Performance metrics

## Troubleshooting

### Common Issues

**Webhook Not Working**

- Verify webhook is running and healthy
- Check webhook configuration and secrets
- Ensure proper RBAC permissions

**Cache Not Storing Shaders**

- Verify GXCache service is running
- Check storage configuration and persistent volumes
- Review application logs for shader compilation errors

**Low Cache Hit Ratio**

- Review cache size configuration
- Check cache eviction policies
- Monitor storage performance

**Storage Issues**

- Verify storage class availability
- Check persistent volume claims
- Monitor disk space usage

## Best Practices

### Deployment

- Deploy GXCache before deploying GPU workloads
- Use high-performance storage for better cache performance
- Monitor cache performance and adjust configuration as needed

### Configuration

- Size cache appropriately for your shader workload
- Use high-performance storage for better performance
- Enable metrics for monitoring and optimization

### Security

- Use proper RBAC permissions for webhook and service
- Secure communication between components
- Regularly update GXCache images for security patches

### Maintenance

- Monitor cache hit ratios and adjust cache size as needed
- Regularly review and clean up unused cached shaders
- Update GXCache images regularly for security patches
- Monitor storage usage and plan for growth