OpenEBS: Create persistent storage in your Charmed Kubernetes cluster

I previously wrote a blog about using StorageOS as persistent storage solution for Kubernetes here. StorageOS is dependent on etcd. I was having difficulties getting etcd up again after a reboot. Since I wanted to get a storage solution working quickly and not focus too much on external dependencies so I decided to give OpenEBS a try. In this blog I'll describe a developer installation on Charmed Kubernetes (the environment described here). I used openebs-jiva-default as storage class. This is unsuitable for production scenario's. OpenEBS also provides cStor. Most of the development effort goes there. cStor however requires a mounted block device. I have not tried this yet in my environment.

Installing OpenEBS

First I created my environment as described here. I used 4 worker nodes.

For the creation of the environment I used the following yaml file as overlay

description: A highly-available, production-grade Kubernetes cluster.
series: bionic
applications:
  etcd:
    num_units: 2
  kubernetes-master:
    constraints: cores=1 mem=4G root-disk=16G
    num_units: 2
  kubernetes-worker:
    constraints: cores=1 mem=3G root-disk=20G
    num_units: 4

Next I enabled iscsi as per OpenEBS requirement in the worker nodes.

juju run "sudo systemctl enable iscsid && sudo systemctl start iscsid" --application kubernetes-worker

Allow creation of privileged containers. The containers need access to host devices to do their thing.

juju config kubernetes-master allow-privileged=true

Restart the environment

juju run "reboot" --application kubernetes-worker
juju run "reboot" --application kubernetes-master

Create a namespace

kubectl create namespace openebs

Add the OpenEBS Helm chart

helm repo add openebs https://openebs.github.io/charts
helm repo update

Add some configuration for OpenEBS. Parameters are described here.

cat << EOF > openebs-config.yaml
jiva:
   replicas: 2
EOF

Also it helps to indicate in the storage class only one replica is needed. This way you can make due with only 2 worker nodes instead of 4:

kubectl apply -n openebs -f - <<END  

kind: StorageClass  

apiVersion: storage.k8s.io/v1  

metadata:  

 name: openebs-jiva-default  

 annotations:  

  cas.openebs.io/config: |  

   - name: ReplicaCount  

    value: "1"  

  openebs.io/cas-type: jiva  

  storageclass.kubernetes.io/is-default-class: 'true'  

provisioner: openebs.io/provisioner-iscsi  

reclaimPolicy: Delete  

volumeBindingMode: Immediate  

END  

Install OpenEBS

helm install openebs stable/openebs --version 1.10.0 -f openebs-config.yaml --namespace openebs

Trying it out

Add the Jenkins repo

helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo update

Create a namespace

kubectl create namespace jenkins

Create a persistent volume claim

kubectl create -n jenkins -f - <<END
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pv-claim
spec:
  storageClassName: openebs-jiva-default
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
END

Create some Jenkins configuration to use the claim

 cat << EOF > jenkins-config.yaml
persistence:
    enabled: true
    size: 5Gi
    accessMode: ReadWriteOnce
    existingClaim: jenkins-pv-claim
    storageClass: "openebs-jiva-default"
EOF

Install Jenkins

helm install my-jenkins-release -f jenkins-config.yaml stable/jenkins --namespace jenkins

Get your 'admin' user password by running:

printf $(kubectl get secret --namespace jenkins my-jenkins-release -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode);echo

Get the Jenkins URL to visit by running these commands in the same shell and login!

export POD_NAME=$(kubectl get pods --namespace jenkins -l "app.kubernetes.io/component=jenkins-master" -l "app.kubernetes.io/instance=my-jenkins-release" -o jsonpath="{.items[0].metadata.name}")

kubectl --namespace jenkins port-forward $POD_NAME 8080:8080