Skip to main content

Setup an NFS client provisioner in Kubernetes

Setup an NFS client provisioner in Kubernetes

One of the most common needs when deploying Kubernetes is the ability to use shared storage. While there are several options available, one of the most commons and easier to setup is to use an NFS server.
This post will explain how to setup a dynamic NFS client provisioner on Kubernetes, relying on an existing NFS server on your systems.

Step 1. Setup an NFS server (sample for CentOS)

First thing you will need, of course, is to have an NFS server. This can be easily achieved with some easy steps:

  • Install nfs package: yum install -y nfs-utils
  • Enable and start nfs service and rpcbind:
    systemctl enable rpcbind
    systemctl enable nfs-server
    systemctl start rpcbind
    systemctl start nfs-server

  • Create the directory that will be shared by NFS, and change the permissions:
    mkdir /var/nfsshare
    chmod -R 755 /var/nfsshare
    chown nfsnobody:nfsnobody /var/nfsshare

  •  Share the NFS directory over the network, creating the /etc/exports file:
    vi /etc/exports
    /var/nfsshare * (rw,sync,no_root_squash,no_all_squash)
  • Restart the nfs service to apply the content:
    systemctl restart nfs-server
  • Add NFS and rpcbind services to firewall:
    firewall-cmd --permanent --zone=public --add-service=nfs
    firewall-cmd --permanent --zone=public --add-service=rpcbind
    firewall-cmd --reload


    The NFS server is now ready to be used

Step 2. Install NFS client provisioner

To achieve that, we will rely on Kubernetes external storage provisioner (https://github.com/kubernetes-incubator/external-storage) . An external provisioner is a dynamic volume provisioner, whose code lives outside kubernetes code.
It relies on an StorageClass object, that defines the external provisioner instance. Then, that instance will wait for PersistentVolumeClaims asking for that specific StorageClass, and will automatically create PersistentVolumes.
In that case we are relying on NFS-client provisioner (https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client) , that will provide those volumes, relying on an existing NFS server.

In order to use that, several steps are needed:
  • Clone the external-storage repository and switch to the nfs-client folder:
    git clone https://github.com/kubernetes-incubator/external-storage
    cd external-storage/nfs-client
  • Customize the deploy/class.yaml file, to give a custom provisioner name to your instance:
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: managed-nfs-storage
    provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
    parameters:
      archiveOnDelete: "false"
  • Customize the deploy/deployment.yaml file, to specify the location and folder for your NFS server, and to give the right provisioner name:
    kind: ServiceAccount
    metadata:
      name: nfs-client-provisioner
    ---
    kind: Deployment
    apiVersion: extensions/v1beta1
    metadata:
      name: nfs-client-provisioner
    spec:
      replicas: 1
      strategy:
        type: Recreate
      template:
       metadata:
        labels:
          app: nfs-client-provisioner
       spec:
         serviceAccountName: nfs-client-provisioner
         containers:
           - name: nfs-client-provisioner
             image: quay.io/external_storage/nfs-client-provisioner:latest
             volumeMounts:
               - name: nfs-client-root
                 mountPath: /persistentvolumes
             env:
               - name: PROVISIONER_NAME
                 value: fuseim.pri/ifs
    # or choose another name, must match
               - name: NFS_SERVER
                 value: <<IP_OF_YOUR_NFS_SERVER>>
               - name: NFS_PATH
                 value: <<PATH_TO_NFS_SHARED_FOLDER>>
         volumes:
           - name: nfs-client-root
             nfs:
               server: <<IP_OF_YOUR_NFS_SERVER>>
               path: <<PATH_TO_NFS_SHARED_FOLDER>>
               apiVersion: v1
  • Create the objects into your kubernetes cluster:
    kubectl create -f deploy/rbac.yaml
    kubectl create -f deploy/class.yaml
    kubectl create -f deploy/deployment.yaml
  • To actually start testing the system, you will need to create a PersistentVolumeClaim using that StorageClass. Then you will need to create a pod that uses this PersistentVolumeClaim:
    claim.yaml
    ----------
    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: test-claim
      annotations:
        volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 1Mi

    pod.yaml
    --------
    kind: Pod

    apiVersion: v1
    metadata:
      name: test-pod
    spec:
      containers:
       - name: test-pod
         image: gcr.io/google_containers/busybox:1.24
         command:
           - "/bin/sh"
         args:
           - "-c"
           - "touch /mnt/SUCCESS && exit 0 || exit 1"
         volumeMounts:
           - name: nfs-pvc
             mountPath: "/mnt"
    restartPolicy: "Never"
    volumes:
       - name: nfs-pvc
         persistentVolumeClaim:
           claimName: test-claim


    kubectl create -f deploy/claim.yaml
    kubectl create -f deploy/pod.yaml
    You can see that the pods can be started now with NFS volumes that are dynamically provisioned, relying on your existing NFS server installation.

Comments

  1. I've got a great kubernetes interview guide here

    ReplyDelete
  2. Thank you so much for this nice information. Hope so many people will get aware of this and useful as well. And please keep update like this.

    Big Data Services

    Data Lake Services

    Advanced Analytics Solutions

    Full Stack Development Services

    ReplyDelete
  3. Few people are very much confused to make a clear picture of machine learning in their mind. This is the reason why they have different opinions for machine learning in both good and bad sense. Now it all depends upon machine learning development services and their output.

    ReplyDelete

Post a Comment

Popular posts from this blog

Create and restore external backups of virtual machines with libvirt

A common need for deployments in production, is to have the possibility of taking backups of your working virtual machines, and export them to some external storage.
Although libvirt offers the possibility of taking snapshots and restore them, those snapshots are intended to be managed locally, and are lost when you destroy your virtual machines.
There may be the need to just trash all your environment, and re-create the virtual machines from an external backup, so this article offers a procedure to achieve it.
First step, create an external snapshot So the first step will be taking an snapshot from your running vm. The best way to take an isolated backup is using blockcopy virsh command. So, how to proceed?

1. First you need to extract all the disks that your vm has. This can be achieved with domblklist command:
DISK_NAME=$(virsh domblklist {{domain}} --details | grep 'disk' | awk '{print $3}')

This will extract the name of the device that the vm is using (vda, hda, et…

Add RHEL8 nodes to OpenShift deployments

This blogpost is going to show how to automatically enroll RHEL 8 (realtime) to Openshift 4.1 deployments, using UPI method.
This assumes that you will setup an OpenShift cluster using UPI, following the according documentation: https://docs.openshift.com/container-platform/4.1/installing/installing_bare_metal
The procedure on how to spin up this cluster in a semi-automated way is also shown at https://github.com/redhat-nfvpe/upi-rt . This article will assume that you have this UPI cluster up and running.

Enroll RHEL 8 nodes as workers By default, all nodes added into an OpenShift cluster are based on RHCOS. But there are use caes where you may need RHEL nodes. This is the case of RT (real time) nodes, where you need an specific kernel.
This can be achieved with the help of kickstart, and some specific configuration of PXE kernel args (in this case achieved with matchbox).
We are going to use RHEL8 images, booted with PXE, but we are going to add some specific configuration to allow t…