NFS External Exporter Script | How to export Sharedv4 volume to an external node as NFS target

Portworx Volumes
1

This document describes how to export Portworx sharedv4 (ReadWriteMany) volumes in your Kubernetes cluster to an external node / host.

Sharedv4 volumes are useful when you want multiple PODs to access the same PVC (volume) at the same time. They can use the same volume even if they are running on different hosts. They provide a global namespace and the semantics are POSIX compliant.

The basic steps are to create a SharedV4 StorageClass, PVC and Pod that will attach and mount the PV to one of the portworx nodes and retrieve the information to export and mount it.

NOTE: Use with caution on production clusters. The volume and its data will be available outside the Kubernetes cluster.

This method may be followed for use-cases where end users want to Read or Monitor the logs, reports etc. generated by their application workloads using SharedV4 Volumes.

Prerequisites:-

  • A Kubernetes Master node or a Kubectl client node.
  • An external node with nfs-utils installed.

Step-1: Create a Storage Class Spec

nfs-sharedv4-sc.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
    name: export-sharedv4-sc
provisioner: kubernetes.io/portworx-volume
parameters:
   repl: "2"
   sharedv4: "true"
   allow_all_ips: "true"

Step-2: Create a sharedv4 Persistent Volume Claim Spec

nfs-sharedv4-pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
   name: export-sharedv4-pvc
   annotations:
     volume.beta.kubernetes.io/storage-class: export-sharedv4-sc
spec:
   accessModes:
     - ReadWriteMany
   resources:
     requests:
       storage: 10Gi

Step-3:Create a Pod which uses Persistent Volume Claim

nfs-sharedv4-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: export-sharedv4-pod
  label:
     app: nfs
spec:
  containers:
  - name: nfsmountexporter
    image: busybox:1.28
    volumeMounts:
    - name: sharedv4-volume
      mountPath: /sharedv4-portworx-volume
    command: ["/bin/sh", "-ec", "sleep 3600"]
  volumes:
  - name: sharedv4-volume
    persistentVolumeClaim:
      claimName: export-sharedv4-pvc

Step-4: Run the NFS exporter script from Master/Kubectl Client node.

nfs-mounter.sh

#!/bin/bash
# Declaring the modified Specs
podspec=/root/nfs-sharedv4-pod.yaml
storageclassspec=/root/nfs-sharedv4-sc.yaml
pvcspec=/root/nfs-sharedv4-pvc.yaml

# Check if Storage Class is already present and create PVC, else create it and then create PVC.
if kubectl get sc export-sharedv4-sc
then
    echo ""
   if kubectl get pvc export-sharedv4-pvc
    then
            pvc_id=$(kubectl get pvc export-sharedv4-pvc | grep -v STATUS | awk '{print $3}')               
    else 
                 {
            pvc_id=$(kubectl apply -f $pvcspec | awk '{print $1}'| cut -d "/" -f 2)
                 }
     fi
else
        kubectl apply -f $storageclassspec
        pvc_id=$(kubectl apply -f $pvcspec | awk '{print $1}'| cut -d "/" -f 2)
fi        
        while [ $(kubectl get pvc export-sharedv4-pvc | grep -v NAME | awk '{print $2}') != 'Bound'  ]
        do 
                    sleep 3
                    echo "Waiting for PVC $pvc_id to get Bound !"
            done
            echo ""
            echo " The export-sharedv4-pvc is now in Bound State ... "

    pods=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath='{.items[0].metadata.name}')
    vol_id=$(kubectl exec $pods -n kube-system -- /opt/pwx/bin/pxctl v i $pvc_id| grep -i "Device" |awk '{print $4}'|cut -b 13-)
    kubectl apply -f $podspec
    echo ""
    while [ $(kubectl get pods | grep export-sharedv4-pod | awk '{print $3}') != 'Running'  ]
    do
            sleep 3
            echo " The pod export-sharedv4-pod is NOT Running yet ..."
    done
    echo ""
    # Capture the NFS endpoint ..
    target_nfs_ep=$(kubectl get nodes $(kubectl get pods -o wide | grep export-sharedv4-pod | awk '{print $7}') -o wide | awk '{print $6}' | grep -v INTERNAL-IP)
    echo " TARGET NFS END-POINT: $target_nfs_ep"
    # Capture exported NFS Sharedv4 Volume device PATH
    nfs_path=$(showmount -e $target_nfs_ep |grep -i $vol_id | awk '{print $1}')
    echo " TARGET NFS DEVICE PATH: $nfs_path"
    # Mount the nfs endpoint
    echo ""
    echo "Use the following information and commands to mount the sharedv4 Portworx volume:- "
    echo ""
    echo "  1. Create a mount point directory for example, mkdir /mnt/Portworx_NFS_Mount"
    echo "  2. mount -t nfs" $target_nfs_ep:$nfs_path /mnt/Portworx_NFS_Mount
    echo "  3. Verify mount df -hP /mnt/Portworx_NFS_Mount"
    echo ""

Step-5: Note down the details for mounting Shared-V4 volume as NFS in your instance.

Script Output :-

[root@master-node ~]# ./nfs-mounter.sh 
NAME                 PROVISIONER                     RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
export-sharedv4-sc   kubernetes.io/portworx-volume   Delete          Immediate           false                  21m

NAME                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         AGE
export-sharedv4-pvc   Bound    pvc-78c391f0-46a6-439e-9533-37d3746e8b6a   10Gi       RWX            export-sharedv4-sc   7m8s

 The export-sharedv4-pvc is now in Bound State ... 
pod/export-sharedv4-pod configured

 TARGET NFS END-POINT: 70.0.71.13
 TARGET NFS DEVICE PATH: /var/lib/osd/pxns/780959580385042137

Use the following information and commands to mount the sharedv4 Portworx volume:- 

  1. Create a mount point directory for example, mkdir /mnt/Portworx_NFS_Mount
  2. mount -t nfs 70.0.71.13:/var/lib/osd/pxns/780959580385042137 /mnt/Portworx_NFS_Mount
  3. Verify mount df -hP /mnt/Portworx_NFS_Mount

Step-6: Mount and verify the Portworx volume onto an external node.

root@varun-virtual-machine:~# mkdir /mnt/Portworx_NFS_Mount
root@varun-virtual-machine:~# mount -t nfs 70.0.71.13:/var/lib/osd/pxns/780959580385042137 /mnt/Portworx_NFS_Mount
root@varun-virtual-machine:~# df -hP /mnt/Portworx_NFS_Mount
Filesystem                                       Size  Used Avail Use% Mounted on
70.0.71.13:/var/lib/osd/pxns/780959580385042137  9.8G   23M  9.2G   1% /mnt/Portworx_NFS_Mount
root@varun-virtual-machine:~#
1 Like