Title: Sharing Storage between Pod Replicas#

URL Source: https://docs.nvidia.com/ucf/2.10.0/text/tutorials/04_Sharing_Storage_Replicas.html

Published Time: Thu, 30 Oct 2025 07:23:04 GMT

Markdown Content:
Housekeeping[#](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/04_Sharing_Storage_Replicas.html#housekeeping "Link to this heading")
---------------------------------------------------------------------------------------------------------------------------------------

See the Getting Started documentation for installing and setting up:

*   microk8s

*   UCS Tools

    *   Accessing NGC

    *   Setting up repositories

Introduction[#](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/04_Sharing_Storage_Replicas.html#introduction "Link to this heading")
---------------------------------------------------------------------------------------------------------------------------------------

In this tutorial, you will learn how to share storage between Pod Replicas of a microservice.

This tutorial continues from [Adding Storage](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/03_Adding_Storage.html#tutorial-3).

Updating the microservices and application[#](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/04_Sharing_Storage_Replicas.html#updating-the-microservices-and-application "Link to this heading")
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

No special changes are required in the manifest for sharing storage to work. However, you will make some changes in the manifest to demonstrate that the storage is accessible by each of the pod replicas.

Update the microservice manifest of the HTTP server microservice `http-server` to the following:

type: msapplication
specVersion: 2.5.0
name: ucf.svc.http-server
chartName: http-server
description: http server
version: 0.0.3
tags: []
keywords: []
publish: false
ingress-endpoints:
 - name: http
 description: REST API endpoint
 protocol: TCP
 scheme: http
 mandatory: False
 data-flow: in-out
---
spec:
 - name: http-server-deployment
 type: ucf.k8s.app.deployment
 parameters:
 apptype: stateless

 - name: http-server-container
 type: ucf.k8s.container
 parameters:
 image:
 repository: nvcr.io/nvidia/pytorch
 tag: 22.04-py3
 command: [sh, -c]
 args: [
 "cd /localvol && echo $PWD && touch $POD_NAME.txt && ls && python -m http.server 8080
 "]
 env:
 - name: POD_NAME
 valueFrom:
 fieldRef:
 fieldPath: metadata.name
 ports:
 - containerPort: 8080
 name: http
 volumeMounts:
 - name: localvol
 mountPath: /localvol

 - name: svc
 type: ucf.k8s.service
 parameters:
 ports:
 - port: 8080
 protocol: TCP
 name: http

 - name: localvol
 type: ucf.k8s.volume
 parameters:
 persistentVolumeClaim:
 claimName: local-path-pvc

You have two changes to the manifest file:

*   Current pod name is exposed as an environment variable `POD_NAME` inside the container.

*   The container args have been updated to create an empty file `<pod-name>.txt`.

Update the microservice manifest of the Curl Client microservice `curl-client` to the following:

type: msapplication
specVersion: 2.5.0
name: ucf.svc.curl-client
chartName: curl-client
description: Curl Client
version: 0.0.2
tags: []
keywords: []
publish: false

egress-endpoints:
 - name: http
 description: REST API endpoint
 protocol: TCP
 scheme: http
 mandatory: False
 data-flow: in-out

---
spec:
 - name: curl-client-deployment
 type: ucf.k8s.app.deployment
 parameters:
 apptype: stateless

 - name: curl-client-container
 type: ucf.k8s.container
 parameters:
 image:
 repository: nvcr.io/nvidia/pytorch
 tag: 22.04-py3
 command: [sh, -c]
 args: [
 "while true; do sleep 10 && curl $egress.http.address:$egress.http.port; done
 "]

You have a single change to the manifest file:

*   The container arguments have been updated to periodically call the HTTP API every 10 seconds.

Update the `app.yaml` file to the following:

specVersion: 2.5.0
version: 0.0.3
doc: README.md
name: server-client-app
description: Server Client Application

dependencies:
- ucf.svc.curl-client:0.0.2
- ucf.svc.http-server:0.0.3

components:
- name: client
 type: ucf.svc.curl-client
- name: http-server
 type: ucf.svc.http-server

connections:
 client/http: http-server/http

The version of the http-server, curl-client and the app has been updated.

Building Microservices and Applications and Deploying them[#](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/04_Sharing_Storage_Replicas.html#building-microservices-and-applications-and-deploying-them "Link to this heading")
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Follow the steps mentioned below but remember to update the version of the `http-server` and `curl-client` under the `dependencies` section in app.yaml:

*   [Connecting Microservices - Building Microservices and Applications](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/02_Connecting_Microservices.html#build-ms-app-cli-02) to build the services and app using CLI

*   [Connecting Microservices - Creating and Building Application using Studio](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/02_Connecting_Microservices.html#create-app-studio-02) to create and build the app visually using Studio

*   [Connecting Microservices - Deploying and Running Microservices and Application](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/02_Connecting_Microservices.html#deploy-app-02) - to deploy the app

Inspecting and Debugging Microservices and Application[#](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/04_Sharing_Storage_Replicas.html#inspecting-and-debugging-microservices-and-application "Link to this heading")
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Verify that the application was deployed successfully:

$ microk8s kubectl get all
NAME READY STATUS RESTARTS AGE
pod/curl-client-curl-client-deployment-77d5f5465d-jfw47 1/1 Running 0 11s
pod/http-server-http-server-deployment-6bb7754dfc-kw2dh 1/1 Running 0 11s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 49d
service/http-server-http-server-deployment-svc ClusterIP 10.152.183.80 <none> 8080/TCP 11s

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/curl-client-curl-client-deployment 1/1 1 1 11s
deployment.apps/http-server-http-server-deployment 1/1 1 1 11s

NAME DESIRED CURRENT READY AGE
replicaset.apps/curl-client-curl-client-deployment-77d5f5465d 1 1 1 11s
replicaset.apps/http-server-http-server-deployment-6bb7754dfc 1 1 1 11s

You can check the `http-server` container to see that a file named `<pod-name>.txt` was created in the current working directory - in this case the `/localvol` folder.

$ microk8s kubectl logs --tail -1 -l "app=http-server-http-server-deployment"
/localvol
http-server-http-server-deployment-6bb7754dfc-kw2dh.txt
somefile.txt

You can also get logs to verify if the file was created in the mounted volume:

$ microk8s kubectl logs --tail -1 -l "app=curl-client-curl-client-deployment"
...
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 481 100 481 0 0 156k 0 --:--:-- --:--:-- --:--:-- 156k
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Directory listing for /</title>
</head>
<body>
<h1>Directory listing for /</h1>
<hr>
<ul>
<li><a href="http-server-http-server-deployment-6bb7754dfc-kw2dh.txt">http-server-http-server-deployment-6bb7754dfc-kw2dh.txt</a></li>
<li><a href="somefile.txt">somefile.txt</a></li>
</ul>
<hr>
</body>
</html>

As you can see, `http-server-http-server-deployment-6bb7754dfc-kw2dh.txt` shows up in the client, meaning that the file was indeed created in the mounted file path.

Now manually scale up the pod replicas to 3:

$ microk8s kubectl scale deploy/http-server-http-server-deployment --replicas=3

Confirm that there are 2 new pod replicas:

$ microk8s kubectl get all
NAME READY STATUS RESTARTS AGE
pod/curl-client-curl-client-deployment-77d5f5465d-jfw47 1/1 Running 0 11m
pod/http-server-http-server-deployment-6bb7754dfc-kw2dh 1/1 Running 0 11m
pod/http-server-http-server-deployment-6bb7754dfc-q68wq 1/1 Running 0 19s
pod/http-server-http-server-deployment-6bb7754dfc-xdb6p 1/1 Running 0 19s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 49d
service/http-server-http-server-deployment-svc ClusterIP 10.152.183.80 <none> 8080/TCP 11m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/curl-client-curl-client-deployment 1/1 1 1 11m
deployment.apps/http-server-http-server-deployment 3/3 3 3 11m

NAME DESIRED CURRENT READY AGE
replicaset.apps/curl-client-curl-client-deployment-77d5f5465d 1 1 1 11m
replicaset.apps/http-server-http-server-deployment-6bb7754dfc 3 3 3 11m

Check the logs of the curl-client pod. You must wait for all the http-server pods to go into the RUNNING state plus an additional 10 seconds for the curl client pod to execute the curl command.

$ microk8s kubectl logs --tail -1 -l "app=curl-client-curl-client-deployment"
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 751 100 751 0 0 183k 0 --:--:-- --:--:-- --:--:-- 183k
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Directory listing for /</title>
</head>
<body>
<h1>Directory listing for /</h1>
<hr>
<ul>
<li><a href="http-server-http-server-deployment-6bb7754dfc-kw2dh.txt">http-server-http-server-deployment-6bb7754dfc-kw2dh.txt</a></li>
<li><a href="http-server-http-server-deployment-6bb7754dfc-q68wq.txt">http-server-http-server-deployment-6bb7754dfc-q68wq.txt</a></li>
<li><a href="http-server-http-server-deployment-6bb7754dfc-xdb6p.txt">http-server-http-server-deployment-6bb7754dfc-xdb6p.txt</a></li>
<li><a href="somefile.txt">somefile.txt</a></li>
</ul>
<hr>
</body>
</html>

As you can see, there is a file corresponding to each pod replica of http-server. This confirms that the same storage (PVC) was mounted in all the pod replicas.

Stopping and Cleaning up Microservices and Applications[#](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/04_Sharing_Storage_Replicas.html#stopping-and-cleaning-up-microservices-and-applications "Link to this heading")
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Finally, to stop and clean up the application run:

$ microk8s helm3 uninstall server-client

Some Notes[#](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/04_Sharing_Storage_Replicas.html#some-notes "Link to this heading")
-----------------------------------------------------------------------------------------------------------------------------------

*   The microservice must handle read/write access contention to the shared storage itself. Kubernetes does not provide any in-built mechanism.

*   Persistent Volumes created by the **Local Path Provisioner** can only be shared by pods running on the same node.

> *   Pod replicas mounting PVC with `mdx-local-path` storageClass will never get scheduled on different nodes because of nodeAffinity of the backing Persistent Volume.
> 
>     *   This can lead to pod replicas not getting scheduled at all if the node where the PV has been created has insufficient resources to launch more pods.

*   To share storage between pod replicas scheduled on different nodes, use a different PV provisioner that allows PVs to be mounted across nodes.

> *   One such example is [Kubernetes NFS Subdir External Provisioner](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner).

Links/Buttons:
- [#](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/04_Sharing_Storage_Replicas.html#some-notes)
- [Adding Storage](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/03_Adding_Storage.html#tutorial-3)
- [Connecting Microservices - Building Microservices and Applications](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/02_Connecting_Microservices.html#build-ms-app-cli-02)
- [Connecting Microservices - Creating and Building Application using Studio](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/02_Connecting_Microservices.html#create-app-studio-02)
- [Connecting Microservices - Deploying and Running Microservices and Application](https://docs.nvidia.com/ucf/2.10.0/text/tutorials/02_Connecting_Microservices.html#deploy-app-02)
- [Kubernetes NFS Subdir External Provisioner](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner)