Skip to main content
Skip table of contents

Tutorial - Clean up MicroK8s Cluster and Redeploy with Helm

In this tutorial, you will learn how to clean up the MicroK8s cluster removing all the containers, deployments, configmaps, secrets, etc from the previous tutorials. MariaDB and EJBCA are deployed using a Helm chart on the cleaned-up cluster. The database is backed up and restored that way all the configuration that has been done from the previous tutorials is not lost.

Over the last year, all the EJBCA tutorials have been demonstrated on a MicroK8s virtual machine. EJBCA was deployed with YAML files and it would be nice to redeploy EJBCA using Helm. It’s time to clean up and clear out the cluster starting with a fresh deployment of EJBCA using a backup of the MariaDB database.

If you want to clean house, start fresh, and use Helm then this tutorial is for you.

The tutorial covers these steps:

  • Review what is deployed in the MicroK8s cluster

  • Backup the MariaDB database

  • Reset the MicroK8s cluster

  • Install required services on the MicroK8s cluster

  • Deploy MariaDB using Helm and restore the MariaDB backup

  • Deploy EJBCA using Helm

Prerequisites

For this tutorial, EJBCA Community Docker container version 8.2.0.1 was used.

Before you begin, you need:

Step 1 - Review what is deployed in the cluster

If you have been trying out the previous tutorials, there are a lot of pieces and parts deployed and configured in MicroK8s from following the tutorials. Follow the steps below to review what is currently deployed in the cluster:

  1. Review all of the deployments and items deployed:

    BASH 
    $ kubectl get all,ingress,secret,pv,pvc,svc,no --all-namespaces

    • The output is similar to the following:

      BASH 
      NAMESPACE            NAME                                                  READY   STATUS        RESTARTS         AGE
cert-manager         pod/cert-manager-55cf8685cb-tztzk                     1/1     Running       15 (2d21h ago)   151d
cert-manager         pod/cert-manager-cainjector-fbd548cb8-nrr7l           1/1     Running       20 (2d21h ago)   151d
cert-manager         pod/cert-manager-webhook-655b4d58fb-46zzh             1/1     Running       12 (2d21h ago)   151d
ejbca-cert-manager   pod/ejbca-cert-manager-issuer-6c5549d575-qd52x        1/1     Running       30 (2d21h ago)   137d
ejbca-k8s            pod/ejbca-deployment-78fdbbb5c4-n5v2p                 1/1     Running       12 (2d21h ago)   167d
ejbca-k8s            pod/ejbca-httpd-7d646c85f9-2n68j                      1/1     Running       3 (2d21h ago)    48d
ejbca-k8s            pod/mariadb-deployment-5665d96759-sdg24               1/1     Running       3 (2d21h ago)    48d
ingress              pod/nginx-ingress-microk8s-controller-sqk8c           1/1     Running       2 (2d21h ago)    24d
istio-system         pod/istio-egressgateway-66fc798fb8-6cmc4              0/1     Running       22 (2d21h ago)   442d
istio-system         pod/istio-ingressgateway-668c8c965b-s7725             0/1     Running       22 (2d21h ago)   442d
istio-system         pod/istiod-7fc54f9bd-qhwxg                            1/1     Running       27 (2d21h ago)   442d
kube-system          pod/calico-kube-controllers-7c8c9d85c9-htp24          1/1     Running       3 (2d21h ago)    48d
kube-system          pod/calico-node-c4ffk                                 1/1     Running       0                2d21h
kube-system          pod/coredns-6f5f9b5d74-wxbmz                          1/1     Running       28 (2d21h ago)   449d
kube-system          pod/hostpath-provisioner-69cd9ff5b8-fx8ps             1/1     Running       45 (2d21h ago)   449d
metallb-system       pod/controller-9556c586f-gzbm6                        1/1     Running       28 (2d21h ago)   449d
metallb-system       pod/speaker-58lm5                                     1/1     Running       39 (2d21h ago)   449d
pkirules             pod/ejbca-cm-issuer-helloworld-one-6c886556f9-xth78   1/1     Terminating   8 (2d21h ago)    136d

NAMESPACE        NAME                             TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)                                                                      AGE
cert-manager     service/cert-manager             ClusterIP      10.152.183.223   <none>           9402/TCP                                                                     151d
cert-manager     service/cert-manager-webhook     ClusterIP      10.152.183.154   <none>           443/TCP                                                                      151d
default          service/kubernetes               ClusterIP      10.152.183.1     <none>           443/TCP                                                                      449d
ejbca-k8s        service/ejbca-database-service   ClusterIP      10.152.183.44    <none>           3306/TCP                                                                     449d
ejbca-k8s        service/ejbca-internal           ClusterIP      10.152.183.133   <none>           443/TCP,80/TCP                                                               449d
ejbca-k8s        service/ejbca-service            ClusterIP      10.152.183.46    <none>           8009/TCP                                                                     449d
ingress          service/ejbca-node1              LoadBalancer   10.152.183.171   172.16.170.240   80:31205/TCP,443:30622/TCP                                                   449d
istio-system     service/istio-egressgateway      ClusterIP      10.152.183.125   <none>           80/TCP,443/TCP                                                               442d
istio-system     service/istio-ingressgateway     LoadBalancer   10.152.183.92    172.16.170.241   15021:30350/TCP,80:31533/TCP,443:30430/TCP,31400:32547/TCP,15443:31322/TCP   442d
istio-system     service/istiod                   ClusterIP      10.152.183.76    <none>           15010/TCP,15012/TCP,443/TCP,15014/TCP                                        442d
kube-system      service/kube-dns                 ClusterIP      10.152.183.10    <none>           53/UDP,53/TCP,9153/TCP                                                       449d
metallb-system   service/webhook-service          ClusterIP      10.152.183.52    <none>           443/TCP                                                                      449d

NAMESPACE        NAME                                               DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
ingress          daemonset.apps/nginx-ingress-microk8s-controller   1         1         1       1            1           <none>                   449d
kube-system      daemonset.apps/calico-node                         1         1         1       1            1           kubernetes.io/os=linux   449d
metallb-system   daemonset.apps/speaker                             1         1         1       1            1           kubernetes.io/os=linux   449d

NAMESPACE            NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
cert-manager         deployment.apps/cert-manager                1/1     1            1           151d
cert-manager         deployment.apps/cert-manager-cainjector     1/1     1            1           151d
cert-manager         deployment.apps/cert-manager-webhook        1/1     1            1           151d
ejbca-cert-manager   deployment.apps/ejbca-cert-manager-issuer   1/1     1            1           137d
ejbca-k8s            deployment.apps/ejbca-deployment            1/1     1            1           167d
ejbca-k8s            deployment.apps/ejbca-httpd                 1/1     1            1           449d
ejbca-k8s            deployment.apps/mariadb-deployment          1/1     1            1           449d
istio-system         deployment.apps/istio-egressgateway         0/1     1            0           442d
istio-system         deployment.apps/istio-ingressgateway        0/1     1            0           442d
istio-system         deployment.apps/istiod                      1/1     1            1           442d
kube-system          deployment.apps/calico-kube-controllers     1/1     1            1           449d
kube-system          deployment.apps/coredns                     1/1     1            1           449d
kube-system          deployment.apps/hostpath-provisioner        1/1     1            1           449d
metallb-system       deployment.apps/controller                  1/1     1            1           449d

NAMESPACE            NAME                                                   DESIRED   CURRENT   READY   AGE
cert-manager         replicaset.apps/cert-manager-55cf8685cb                1         1         1       151d
cert-manager         replicaset.apps/cert-manager-cainjector-fbd548cb8      1         1         1       151d
cert-manager         replicaset.apps/cert-manager-webhook-655b4d58fb        1         1         1       151d
ejbca-cert-manager   replicaset.apps/ejbca-cert-manager-issuer-6c5549d575   1         1         1       137d
ejbca-k8s            replicaset.apps/ejbca-deployment-78fdbbb5c4            1         1         1       167d
ejbca-k8s            replicaset.apps/ejbca-httpd-7d646c85f9                 1         1         1       449d
ejbca-k8s            replicaset.apps/mariadb-deployment-5665d96759          1         1         1       449d
istio-system         replicaset.apps/istio-egressgateway-66fc798fb8         1         1         0       442d
istio-system         replicaset.apps/istio-egressgateway-7bf6457867         0         0         0       442d
istio-system         replicaset.apps/istio-ingressgateway-668c8c965b        1         1         0       442d
istio-system         replicaset.apps/istio-ingressgateway-8576d6c55f        0         0         0       442d
istio-system         replicaset.apps/istiod-77f9c8bf68                      0         0         0       442d
istio-system         replicaset.apps/istiod-7fc54f9bd                       1         1         1       442d
kube-system          replicaset.apps/calico-kube-controllers-5cdbddfc9      0         0         0       449d
kube-system          replicaset.apps/calico-kube-controllers-75b6ddb7b8     0         0         0       408d
kube-system          replicaset.apps/calico-kube-controllers-79568db7f8     0         0         0       449d
kube-system          replicaset.apps/calico-kube-controllers-7c8c9d85c9     1         1         1       48d
kube-system          replicaset.apps/calico-kube-controllers-7c9cdf4bfc     0         0         0       396d
kube-system          replicaset.apps/calico-kube-controllers-7fcc7d8496     0         0         0       167d
kube-system          replicaset.apps/calico-kube-controllers-b5fcf779f      0         0         0       60d
kube-system          replicaset.apps/calico-kube-controllers-df49bb54       0         0         0       364d
kube-system          replicaset.apps/coredns-6f5f9b5d74                     1         1         1       449d
kube-system          replicaset.apps/hostpath-provisioner-69cd9ff5b8        1         1         1       449d
metallb-system       replicaset.apps/controller-9556c586f                   1         1         1       449d

NAMESPACE   NAME                                    CLASS    HOSTS                   ADDRESS     PORTS     AGE
ejbca-k8s   ingress.networking.k8s.io/ejbca-node1   public   ejbca-node1.ejbca-k8s   127.0.0.1   80, 443   24d

NAMESPACE            NAME                                                     TYPE                 DATA   AGE
cert-manager         secret/cert-manager-webhook-ca                           Opaque               3      151d
cert-manager         secret/sh.helm.release.v1.cert-manager.v1                helm.sh/release.v1   1      151d
ejbca-cert-manager   secret/ejbca-ca-secret                                   Opaque               1      151d
ejbca-cert-manager   secret/ejbca-secret                                      kubernetes.io/tls    2      151d
ejbca-cert-manager   secret/sh.helm.release.v1.ejbca-cert-manager-issuer.v1   helm.sh/release.v1   1      137d
ejbca-csr-signer     secret/ejbca-client-cert                                 kubernetes.io/tls    2      442d
ejbca-csr-signer     secret/ejbca-credentials                                 Opaque               1      442d
ejbca-k8s            secret/ejbca-mariadb-secret                              Opaque               1      449d
ejbca-k8s            secret/mariadb-secret                                    Opaque               1      449d
ejbca-k8s            secret/mtls-trust-chain                                  Opaque               1      47d
ejbca-k8s            secret/tls-ejbca-node1                                   kubernetes.io/tls    2      60d
istio-system         secret/external-ca-cert                                  Opaque               1      442d
istio-system         secret/istio-ca-secret                                   istio.io/ca-root     5      442d
metallb-system       secret/memberlist                                        Opaque               1      449d
metallb-system       secret/webhook-server-cert                               Opaque               4      449d
vault                secret/tls-api-vault                                     kubernetes.io/tls    2      160d
vault                secret/vault-ha-tls                                      Opaque               3      160d

NAMESPACE   NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                   STORAGECLASS        VOLUMEATTRIBUTESCLASS   REASON   AGE
            persistentvolume/pvc-031c7c9d-63dd-42b7-8c20-83e9e57782d7   100M       RWO            Delete           Bound    ejbca-k8s/mariadb-pvc   microk8s-hostpath   <unset>                          449d
            persistentvolume/pvc-230714e3-ed56-498f-8386-c84b941e9b00   1Gi        RWO            Delete           Bound    vault/data-vault-1      microk8s-hostpath   <unset>                          160d
            persistentvolume/pvc-3d0cc22e-3c9a-4e8a-a867-9f99b9f97424   1Gi        RWO            Delete           Bound    vault/audit-vault-2     microk8s-hostpath   <unset>                          160d
            persistentvolume/pvc-4cacc08e-2802-4b54-9af5-b386750fa715   1Gi        RWO            Delete           Bound    vault/audit-vault-0     microk8s-hostpath   <unset>                          160d
            persistentvolume/pvc-5be3658f-abda-4512-873a-38b4e9e46556   1Gi        RWO            Delete           Bound    vault/audit-vault-1     microk8s-hostpath   <unset>                          160d
            persistentvolume/pvc-6d665f7a-c89e-4536-8b3a-f8f5a79a4b00   1Gi        RWO            Delete           Bound    vault/data-vault-0      microk8s-hostpath   <unset>                          160d
            persistentvolume/pvc-903289f8-db05-44c2-8ded-fdb93a34fb21   1Gi        RWO            Delete           Bound    vault/data-vault-2      microk8s-hostpath   <unset>                          160d
            persistentvolume/pvc-b1d34ec9-10db-4b8d-ab3d-d8ba7810be14   30M        RWO            Delete           Bound    ejbca-k8s/ejbca-pvc     microk8s-hostpath   <unset>                          449d

NAMESPACE   NAME                                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS        VOLUMEATTRIBUTESCLASS   AGE
ejbca-k8s   persistentvolumeclaim/ejbca-pvc       Bound    pvc-b1d34ec9-10db-4b8d-ab3d-d8ba7810be14   30M        RWO            microk8s-hostpath   <unset>                 449d
ejbca-k8s   persistentvolumeclaim/mariadb-pvc     Bound    pvc-031c7c9d-63dd-42b7-8c20-83e9e57782d7   100M       RWO            microk8s-hostpath   <unset>                 449d
vault       persistentvolumeclaim/audit-vault-0   Bound    pvc-4cacc08e-2802-4b54-9af5-b386750fa715   1Gi        RWO            microk8s-hostpath   <unset>                 160d
vault       persistentvolumeclaim/audit-vault-1   Bound    pvc-5be3658f-abda-4512-873a-38b4e9e46556   1Gi        RWO            microk8s-hostpath   <unset>                 160d
vault       persistentvolumeclaim/audit-vault-2   Bound    pvc-3d0cc22e-3c9a-4e8a-a867-9f99b9f97424   1Gi        RWO            microk8s-hostpath   <unset>                 160d
vault       persistentvolumeclaim/data-vault-0    Bound    pvc-6d665f7a-c89e-4536-8b3a-f8f5a79a4b00   1Gi        RWO            microk8s-hostpath   <unset>                 160d
vault       persistentvolumeclaim/data-vault-1    Bound    pvc-230714e3-ed56-498f-8386-c84b941e9b00   1Gi        RWO            microk8s-hostpath   <unset>                 160d
vault       persistentvolumeclaim/data-vault-2    Bound    pvc-903289f8-db05-44c2-8ded-fdb93a34fb21   1Gi        RWO            microk8s-hostpath   <unset>                 160d

NAMESPACE   NAME              STATUS   ROLES    AGE    VERSION
            node/microk8-01   Ready    <none>   449d   v1.29.4          microk8s-hostpath   <unset>                 159d

You have now reviewed what has been deployed in MicroK8s, continue to the next step to backup the EJBCA database.

Step 2 - Backup the EJBCA database

Time was invested to configure EJBCA following the tutorials and it would be great to not lose the EJBCA configuration. Follow these steps to backup the EJBCA database before the MicroK8s cluster is reset:

  1. Review what directories are present in the VM host:

    BASH 
    $ ls -l

    • The output is similar to the following:

      BASH 
      total 20
-rw-rw-r--. 1 user user     3612 Apr 24 07:56 ca.crt
drwxrwxr-x. 3 user user     4096 Jan 26 06:30 cert-manager
drwxrwxr-x. 3 user user       88 Mar 25  2023 csr-api
drwxrwxr-x. 2 user user     4096 May 17 11:58 k81
drwx------. 3 user microk8s   22 Mar 18  2023 snap
drwxrwxr-x. 2 user user     4096 Jun  9 15:34 sql
drwxrwxr-x. 2 user user     4096 Jan  2 12:01 vault

  2. Install the MariaDB client using the DNF package manager:

    BASH 
    $ sudo dnf install -y <https://dlm.mariadb.com/2690116/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-client-10.10.2-1.el8.x86_64.rpm> \
  <https://dlm.mariadb.com/2690045/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-backup-10.10.2-1.el8.x86_64.rpm> \
  <https://dlm.mariadb.com/2690100/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-common-10.10.2-1.el8.x86_64.rpm> \
  <https://dlm.mariadb.com/2690079/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-shared-10.10.2-1.el8.x86_64.rpm>

  3. Backup the database to the SQL directory:

    BASH 
    $ mariadb-dump -h 10.152.183.44 -u root -p --all-databases --quick --lock-tables=false --system=users --insert-ignore --force > sql/20240610_BackupToMigrate.sql

    • The output is similar to the following:

      BASH 
      # mariadb-dump: Couldn't execute 'SELECT CONCAT(QUOTE(u.user), '@', QUOTE(u.Host)) AS u FROM mysql.user u  /*!80001 LEFT JOIN mysql.role_edges e             ON u.user=e.from_user               AND u.host=e.from_host          WHERE e.from_user IS NULL */ /*M!100005 WHERE is_role='N' */': Illegal mix of collations (utf8mb4_general_ci,COERCIBLE) and (utf8mb4_uca1400_ai_ci,COERCIBLE) for operation '=' (1267)

You have now backed up the EJBCA MariaDB database using the mariadb-dump utility. Continue to the next step to reset the MicroK8s cluster.

Step 3 - Reset the MicroK8s cluster

When MicroK8s is reset, the cluster is cleaned up by removing the entire configuration and restoring the cluster to the default install from when MicroK8s was first installed. To reset the cluster follow these steps:

  1. Uninstall the ejbca cert-manager Issuer Helm deployment:

    BASH 
    $ helm uninstall -n ejbca-cert-manager ejbca-cert-manager-issuer

    • The output is similar to the following:

      BASH 
      release "ejbca-cert-manager-issuer" uninstalled

  2. Uninstall the cert-manager Helm deployment:

    BASH 
    $ helm uninstall -n cert-manager cert-manager

    • The output is similar to the following:

      BASH 
      release "cert-manager" uninstalled

  3. Uninstall istio using istioctl:

    BASH 
    $ csr-api/ejbca-csr-signer/istio-1.16.3/bin/istioctl uninstall --purge --force

    • The output is similar to the following:

      BASH 
      failed to get proxy infos: failure running port forward process: failure running port forward process: error upgrading connection: error dialing backend: dial tcp 172.16.170.187:10250: connect: connection refused
All Istio resources will be pruned from the cluster
Proceed? (y/N)

  4. Enter y to continue.

    • The output is similar to the following:

      BASH 
        Removed IstioOperator:istio-system:installed-state.
  Removed Deployment:istio-system:istio-egressgateway.
  Removed Deployment:istio-system:istio-ingressgateway.
  Removed Deployment:istio-system:istiod.
  Removed Service:istio-system:istio-egressgateway.
  Removed Service:istio-system:istio-ingressgateway.
  Removed Service:istio-system:istiod.
  Removed ConfigMap:istio-system:istio.
  Removed ConfigMap:istio-system:istio-sidecar-injector.
  Removed Pod:istio-system:istio-egressgateway-66fc798fb8-6cmc4.
  Removed Pod:istio-system:istio-ingressgateway-668c8c965b-s7725.
  Removed Pod:istio-system:istiod-7fc54f9bd-qhwxg.
  Removed ServiceAccount:istio-system:istio-egressgateway-service-account.
  Removed ServiceAccount:istio-system:istio-ingressgateway-service-account.
  Removed ServiceAccount:istio-system:istio-reader-service-account.
  Removed ServiceAccount:istio-system:istiod.
  Removed ServiceAccount:istio-system:istiod-service-account.
  Removed RoleBinding:istio-system:istio-egressgateway-sds.
  Removed RoleBinding:istio-system:istio-ingressgateway-sds.
  Removed RoleBinding:istio-system:istiod.
  Removed RoleBinding:istio-system:istiod-istio-system.
  Removed Role:istio-system:istio-egressgateway-sds.
  Removed Role:istio-system:istio-ingressgateway-sds.
  Removed Role:istio-system:istiod.
  Removed Role:istio-system:istiod-istio-system.
  Removed EnvoyFilter:istio-system:stats-filter-1.13.
  Removed EnvoyFilter:istio-system:stats-filter-1.14.
  Removed EnvoyFilter:istio-system:stats-filter-1.15.
  Removed EnvoyFilter:istio-system:stats-filter-1.16.
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.13.
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.14.
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.15.
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.16.
  Removed PodDisruptionBudget:istio-system:istio-egressgateway.
  Removed PodDisruptionBudget:istio-system:istio-ingressgateway.
  Removed PodDisruptionBudget:istio-system:istiod.
  Removed MutatingWebhookConfiguration::istio-revision-tag-default.
  Removed MutatingWebhookConfiguration::istio-sidecar-injector.
  Removed ValidatingWebhookConfiguration::istio-validator-istio-system.
  Removed ValidatingWebhookConfiguration::istiod-default-validator.
  Removed ClusterRole::istio-reader-clusterrole-istio-system.
  Removed ClusterRole::istio-reader-istio-system.
  Removed ClusterRole::istiod-clusterrole-istio-system.
  Removed ClusterRole::istiod-gateway-controller-istio-system.
  Removed ClusterRole::istiod-istio-system.
  Removed ClusterRoleBinding::istio-reader-clusterrole-istio-system.
  Removed ClusterRoleBinding::istio-reader-istio-system.
  Removed ClusterRoleBinding::istiod-clusterrole-istio-system.
  Removed ClusterRoleBinding::istiod-gateway-controller-istio-system.
  Removed ClusterRoleBinding::istiod-istio-system.
  Removed CustomResourceDefinition::authorizationpolicies.security.istio.io.
  Removed CustomResourceDefinition::destinationrules.networking.istio.io.
  Removed CustomResourceDefinition::envoyfilters.networking.istio.io.
  Removed CustomResourceDefinition::gateways.networking.istio.io.
  Removed CustomResourceDefinition::istiooperators.install.istio.io.
  Removed CustomResourceDefinition::peerauthentications.security.istio.io.
  Removed CustomResourceDefinition::proxyconfigs.networking.istio.io.
  Removed CustomResourceDefinition::requestauthentications.security.istio.io.
  Removed CustomResourceDefinition::serviceentries.networking.istio.io.
  Removed CustomResourceDefinition::sidecars.networking.istio.io.
  Removed CustomResourceDefinition::telemetries.telemetry.istio.io.

  5. Reset microk8s to restore MicroK8s to the default install:

    BASH 
    $ sudo /var/lib/snapd/snap/bin/microk8s reset

    • The output is similar to the following:

      BASH 
      Disabling all addons
Disabling addon : core/cert-manager
Disabling addon : core/dashboard
Disabling addon : core/dns
Disabling addon : core/gpu
Disabling addon : core/helm
Disabling addon : core/helm3
Disabling addon : core/host-access
Disabling addon : core/hostpath-storage
Disabling addon : core/ingress
Disabling addon : core/kube-ovn
Disabling addon : core/mayastor
Disabling addon : core/metallb
Disabling addon : core/metrics-server
Disabling addon : core/minio
Disabling addon : core/observability
Disabling addon : core/prometheus
Disabling addon : core/rbac
Disabling addon : core/registry
Disabling addon : core/storage
All addons are disabled.
Deleting the CNI
Cleaning resources in namespace bookinfo
Cleaning resources in namespace cert-manager
Cleaning resources in namespace default
Cleaning resources in namespace ejbca-cert-manager
Cleaning resources in namespace ejbca-csr-signer
Cleaning resources in namespace ejbca-k8s
Cleaning resources in namespace ingress
Cleaning resources in namespace istio-system
Cleaning resources in namespace kube-node-lease
Cleaning resources in namespace kube-public
Cleaning resources in namespace kube-system
Cleaning resources in namespace metallb-system
Cleaning resources in namespace pkirules
Cleaning resources in namespace vault
Removing CRDs
Removing PriorityClasses
Removing StorageClasses
Removing namespace/bookinfo
Removing namespace/cert-manager
Removing namespace/ejbca-cert-manager
Removing namespace/ejbca-csr-signer
Removing namespace/ejbca-k8s
Removing namespace/ingress
Removing namespace/istio-system
Removing namespace/metallb-system
Removing namespace/pkirules
Removing namespace/vault
Restarting cluster
Stopped.
Setting up the CNI

You have now reset MicroK8s back to the default. Continue to the next step to enable the required services in MicroK8s for deploying MariaDB and EJBCA.

Step 4 - Enable required services on MicroK8s

Enable DNS, Helm, and hostpath storage on the MicroK8s cluster which are used to deploy EJBCA and MariaDB and provide persistent storage for MariaDB. Follow the steps below to prepare the cluster:

  1. Enable the DNS, Helm, and hostpath-storage addons on the cluster:

    BASH 
    $ microk8s enable dns hostpath-storage helm

    • The output is similar to the following:

      BASH 
      Infer repository core for addon dns
Infer repository core for addon hostpath-storage
Infer repository core for addon helm
WARNING: Do not enable or disable multiple addons in one command.
         This form of chained operations on addons will be DEPRECATED in the future.
         Please, enable one addon at a time: 'microk8s enable <addon>'
Addon core/dns is already enabled
Enabling default storage class.
WARNING: Hostpath storage is not suitable for production environments.
         A hostpath volume can grow beyond the size limit set in the volume claim manifest.

deployment.apps/hostpath-provisioner created
storageclass.storage.k8s.io/microk8s-hostpath created
serviceaccount/microk8s-hostpath created
clusterrole.rbac.authorization.k8s.io/microk8s-hostpath created
clusterrolebinding.rbac.authorization.k8s.io/microk8s-hostpath created
Storage will be available soon.
Addon core/helm is already enabled

  2. Enable the Metal LB addon on the cluster using the IP range of the VM network the VM is on:

    BASH 
    $ microk8s enable metallb:172.16.170.240-172.16.170.245

    • The output is similar to the following:

      BASH 
      Infer repository core for addon metallb
Enabling MetalLB
Applying Metallb manifest
customresourcedefinition.apiextensions.k8s.io/addresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
namespace/metallb-system created
serviceaccount/controller created
serviceaccount/speaker created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
role.rbac.authorization.k8s.io/controller created
role.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/controller created
secret/webhook-server-cert created
service/webhook-service created
rolebinding.rbac.authorization.k8s.io/pod-lister created
daemonset.apps/speaker created
deployment.apps/controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/validating-webhook-configuration created
Waiting for Metallb controller to be ready.
deployment.apps/controller condition met
ipaddresspool.metallb.io/default-addresspool created
l2advertisement.metallb.io/default-advertise-all-pools created
MetalLB is enabled

  3. Create an alias for helm command:

    BASH 
    $ sudo snap alias microk8s.helm helm

    • The output is similar to the following:

      BASH 
      #Added:
#  - microk8s.helm as helm

  4. Review what is deployed in the cluster after the microk8s reset:

    BASH 
    $ kubectl get all,secret,pv,pvc,svc,no --all-namespaces

    • The output is similar to the following:

      BASH 
      NAMESPACE        NAME                                           READY   STATUS    RESTARTS   AGE
kube-system      pod/calico-kube-controllers-7cb4fd5784-xs4db   1/1     Running   0          3m39s
kube-system      pod/calico-node-s42fj                          1/1     Running   0          3m39s
kube-system      pod/coredns-85db789d58-2pfxz                   1/1     Running   0          2m3s
kube-system      pod/hostpath-provisioner-7df77bc496-xcdbm      1/1     Running   0          50s
metallb-system   pod/controller-5f7bb57799-wxcvx                1/1     Running   0          50s
metallb-system   pod/speaker-rdn59                              1/1     Running   0          50s

NAMESPACE        NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
default          service/kubernetes        ClusterIP   10.152.183.1     <none>        443/TCP                  6m31s
kube-system      service/kube-dns          ClusterIP   10.152.183.10    <none>        53/UDP,53/TCP,9153/TCP   2m3s
metallb-system   service/webhook-service   ClusterIP   10.152.183.225   <none>        443/TCP                  102s

NAMESPACE        NAME                         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system      daemonset.apps/calico-node   1         1         1       1            1           kubernetes.io/os=linux   4m44s
metallb-system   daemonset.apps/speaker       1         1         1       1            1           kubernetes.io/os=linux   102s

NAMESPACE        NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
kube-system      deployment.apps/calico-kube-controllers   1/1     1            1           4m44s
kube-system      deployment.apps/coredns                   1/1     1            1           2m3s
kube-system      deployment.apps/hostpath-provisioner      1/1     1            1           114s
metallb-system   deployment.apps/controller                1/1     1            1           102s

NAMESPACE        NAME                                                 DESIRED   CURRENT   READY   AGE
kube-system      replicaset.apps/calico-kube-controllers-7cb4fd5784   1         1         1       3m39s
kube-system      replicaset.apps/coredns-85db789d58                   1         1         1       2m3s
kube-system      replicaset.apps/hostpath-provisioner-7df77bc496      1         1         1       50s
metallb-system   replicaset.apps/controller-5f7bb57799                1         1         1       50s

NAMESPACE        NAME                         TYPE     DATA   AGE
metallb-system   secret/memberlist            Opaque   1      49s
metallb-system   secret/webhook-server-cert   Opaque   4      102s

NAMESPACE   NAME              STATUS   ROLES    AGE    VERSION
            node/microk8-01   Ready    <none>   449d   v1.29.4

You have now enabled the required services to deploy EJBCA and MariaDB using Helm. Continue to the next step to deploy MariaDB using Helm.

Step 5 - Deploy MariaDB with Helm & restore the EJBCA SQL backup

Before EJBCA is deployed you need to have the database (MariaDB) deployed and the SQL backup restored. That way when EJBCA is deployed you begin right where you left off from the last backup taken. Follow these steps to deploy MariaDB with Helm and restore the EJBCA database backup:

  1. Install Helm repository for MariaDB:

    BASH 
    $ helm repo add groundhog2k https://groundhog2k.github.io/helm-charts

    • The output is similar to the following:

      BASH 
      "groundhog2k" has been added to your repositories

  2. Create namespace for the EJBCA MariaDB deployment:

    BASH 
    $ kubectl create namespace ejbca-k8s

    • The output is similar to the following:

      BASH 
      namespace/ejbca-k8s created

  3. Create the MariaDB root password secret:

    BASH 
    $ kubectl -n ejbca-k8s create secret generic mariadb-secret --from-literal=MARIADB_ROOT_PASSWORD=ejbca

    • The output is similar to the following:

      BASH 
      secret/mariadb-secret created

  4. Change to the k81 directory:

    BASH 
    $ cd k81

  5. Create the mariadb-override.yaml file with settings to deploy MariaDB:

    BASH 
    $ cat > mariadb-override.yaml << EOF
## Default values for MariaDB deployment

## MariaDB docker image
image:
  registry: "docker.io"
  repository: "mariadb"
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "latest"

## Optional service account
serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

# Resource limits and requests
resources: 
   limits:
     cpu: 2
     memory: 2048Mi
   requests:
     cpu: 1000m
     memory: 1024Mi

## Use Kubernetes Deployment instead of StatefulSet
useDeployment: false

## Database configuration
settings:
  ## Arguments for the container entrypoint process
  arguments:
    - --character-set-server=utf8mb4
    - --collation-server=utf8mb4_unicode_ci
  ## Optional existing secret for the MariaDB root password
  existingSecret: mariadb-secret

  ## Set true to skip loading timezone data during init
  skipTZInfo: false

## Storage parameters
storage:
  ## Alternative set requestedSize to define a size for a dynamically created PVC
  requestedSize: 1Gi

EOF

  6. Deploy MariaDB using helm chart with override:

    BASH 
    $ helm install -n ejbca-k8s ejbca-mariadb groundhog2k/mariadb -f mariadb-override.yaml

    • The output is similar to the following:

      BASH 
      NAME: ejbca-mariadb
LAST DEPLOYED: Sun Jun  9 13:31:31 2024
NAMESPACE: ejbca-k8s
STATUS: deployed
REVISION: 1
TEST SUITE: None

  7. Check on the deployment after waiting for 1 min:

    BASH 
    $ kubectl -n ejbca-k8s get all

    • The output is similar to the following:

      BASH 
      NAME                  READY   STATUS    RESTARTS   AGE
pod/ejbca-mariadb-0   1/1     Running   0          85s

NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/ejbca-mariadb   ClusterIP   10.152.183.95   <none>        3306/TCP   85s

NAME                             READY   AGE
statefulset.apps/ejbca-mariadb   1/1     85s

  8. Connect to the database using the service/ejbca-mariadb to make sure the DB is accessible, enter the password for the root account when prompted:

    BASH 
    $ mariadb -h 10.152.183.234 -u root -p

  9. Enter the password for the root account.

    • The output is similar to the following:

      BASH 
      Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 51
Server version: 11.4.2-MariaDB-ubu2404 mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]>

  10. Enter exit to exit the database shell:

    BASH 
    MariaDB [(none)]> exit

    • The output is similar to the following:

      BASH 
      MariaDB [(none)]> exit
Bye

  11. Import the EJBCA SQL backup into MariaDB, enter the password for the root account when prompted:

    BASH 
    $ mariadb -h 10.152.183.234 -u root -p < ../sql/20240610_BackupToMigrate.sql

  12. Enter the password for the root account.

  13. Connect to the ejbca database as the ejbca user:

    BASH 
    $ mariadb -h 10.152.183.234 -u ejbca -p ejbca

  14. Enter the password for the ejbca account.

    • The output is similar to the following:

      BASH 
      Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 185
Server version: 11.4.2-MariaDB-ubu2404 mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [ejbca]>

  15. Verify the tables are present:

    BASH 
    MariaDB [ejbca]> show tables;

    • The output is similar to the following:

      BASH 
      MariaDB [ejbca]> show tables;

# Output similar to the following:
+-------------------------------+
| Tables_in_ejbca               |
+-------------------------------+
| AccessRulesData               |
| AcmeAccountData               |
| AcmeAuthorizationData         |
| AcmeChallengeData             |
| AcmeNonceData                 |
| AcmeOrderData                 |
| AdminEntityData               |
| AdminGroupData                |
| AdminPreferencesData          |
| ApprovalData                  |
| AuditRecordData               |
| AuthorizationTreeUpdateData   |
| Base64CertData                |
| BlacklistData                 |
| CAData                        |
| CRLData                       |
| CertReqHistoryData            |
| CertificateData               |
| CertificateProfileData        |
| CryptoTokenData               |
| EndEntityProfileData          |
| GlobalConfigurationData       |
| IncompleteIssuanceJournalData |
| InternalKeyBindingData        |
| KeyRecoveryData               |
| NoConflictCertificateData     |
| OcspResponseData              |
| PeerData                      |
| ProfileData                   |
| PublisherData                 |
| PublisherQueueData            |
| RoleData                      |
| RoleMemberData                |
| SctData                       |
| ServiceData                   |
| UserData                      |
| UserDataSourceData            |
+-------------------------------+
37 rows in set (0.001 sec)

  16. Enter exit to exit the database:

    BASH 
    MariaDB [ejbca]> exit

    • The output is similar to the following:

      BASH 
      MariaDB [ejbca]> exit
Bye

You have now deployed MariaDB using Helm and restored the EJBCA database backup. You then verified the tables created from restoring the backup. Continue to the next step to deploy EJBCA using Helm.

Step 6 - Deploy EJBCA with Helm

EJBCA is deployed after MariaDB is deployed and the database backup is restored. Follow the steps below to deploy EJBCA:

  1. Install the Helm repository for EJBCA:

    BASH 
    $ helm repo add ejbca-ce https://keyfactor.github.io/ejbca-community-helm

    • The output is similar to the following:

      BASH 
      "ejbca-ce" has been added to your repositories

  2. Create the ejbca-override.yaml file:

    BASH 
    $ cat > ejbca-overrides.yaml << EOF
ejbca:
  useEphemeralH2Database: false
  useH2Persistence: false
  importExternalCas: false
  importAppserverKeystore: false
  importAppserverTruststore: false
  importEjbcaConfFiles: false
  env:
    TLS_SETUP_ENABLED: "simple"
    DATABASE_JDBC_URL: "jdbc:mariadb://ejbca-mariadb.ejbca-k8s:3306/ejbca?characterEncoding=utf8"
    LOG_AUDIT_TO_DB: true
    LOG_LEVEL_APP: "INFO"
    LOG_LEVEL_SERVER: "INFO"
    OBSERVABLE_BIND: "127.0.0.1"
    METRICS_ENABLED: false
    OCSP_CHECK_SIGN_CERT_VALIDITY: false
    OCSP_NON_EXISTING_IS_GOOD: true

  envRaw:
    - name: DATABASE_USER
      valueFrom:
       secretKeyRef:
         name: ejbcadbcredentials
         key: database_user
    - name: DATABASE_PASSWORD
      valueFrom:
       secretKeyRef:
         name: ejbcadbcredentials
         key: database_password
  #  - name: PASSWORD_ENCRYPTION_KEY
  #    valueFrom:
  #     secretKeyRef:
  #       name: ejbca-confidentiality-secrets
  #       key: password_encrpytion_key
  envFrom: []
  initContainers: []
  sidecarContainers: []
  volumes: []
  volumeMounts: []
services:
  directHttp:
    enabled: false
  proxyAJP:
    enabled: false
  proxyHttp:
    enabled: true
    type: LoadBalancer
    bindIP: 0.0.0.0
    httpPort: 80
    httpsPort: 443
  sidecarPorts: []
nginx:
  enabled: true
  host: "ejbca-node1.ejbca-k8s"
  proxy_url_host: localhost
  mountInternalNginxCert: true
  secretInternalNginxCert: "internal-nginx-credential-secret-ca"
  service:
    enabled: false
    type: NodePort
    httpPort: 30080
    httpsPort: 30443 
ingress:
  enabled: false
replicaCount: 1
updateStrategy: {}
image:
  repository: keyfactor/ejbca-ce
  pullPolicy: IfNotPresent
  tag: latest
imagePullSecrets: []
nameOverride: "ejbca-ca"
fullnameOverride: "ejbca-ca"
serviceAccount:
  create: true
  annotations: {}
  name: ""
podLabels: {}
podAnnotations: {}
podSecurityContext: {}
securityContext: {}
resources:
  limits:
    cpu: 4
    memory: 4096Mi
  requests:
    cpu: 1000m
    memory: 2048Mi
autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 5
  targetCPUUtilizationPercentage: 80
  behavior: {}
podDisruptionBudget:
  create: false
  minAvailable: 1
  maxUnavailable: ""
nodeSelector: {}
affinity: {}
tolerations: []
topologySpreadConstraints: []
priorityClassName: ""
schedulerName: ""
terminationGracePeriodSeconds: ""

EOF

  3. Create the secret for the EJBCA Database username and password:

    BASH 
    $ kubectl -n ejbca-k8s create secret generic ejbcadbcredentials --from-literal=database_user=ejbca --from-literal=database_password=ejbca

    • The output is similar to the following:

      BASH 
      secret/ejbcadbcredentials created

  4. Create the TLS secret for the certificate, key, and CA cert used by NGINX that is deployed in the EJBCA pod:

    BASH 
    $ kubectl -n ejbca-k8s create secret generic internal-nginx-credential-secret-ca \
  --from-file=ejbca-node1.ejbca-k8s.pem=ejbca-node1.ejbca-k8s.crt \
  --from-file=ejbca-node1.ejbca-k8s-Key.pem=ejbca-node1.ejbca-k8s.key \
  --from-file=ejbca-node1.ejbca-k8s-CA.pem=../ca.crt

    • The output is similar to the following:

      BASH 
      secret/internal-nginx-credential-secret-ca created

  5. Deploy EJBCA using the Helm chart and the override values:

    BASH 
    $ helm install -n ejbca-k8s ejbca-ca ejbca-ce/ejbca-community-helm -f ejbca-overrides.yaml

    • The output is similar to the following:

      BASH 
      NAME: ejbca-ca
LAST DEPLOYED: Sun Jun  9 15:29:16 2024
NAMESPACE: ejbca-k8s
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing EJBCA CE!

Now that you have deployed EJBCA, you should look over the docs on using
EJBCA available here:

<https://doc.primekey.com/>


Your release is named ejbca-ca. To learn more about the release, try:

  $ helm status ejbca-ca
  $ helm get manifest ejbca-ca

  6. Review what has been deployed:

    BASH 
    kubectl -n ejbca-k8s get all

    • The output is similar to the following:

      BASH 
      NAME                            READY   STATUS    RESTARTS   AGE
pod/ejbca-ca-84649f7f7b-ck5bn   1/2     Running   0          15s
pod/ejbca-mariadb-0             1/1     Running   0          83m

NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                      AGE
service/ejbca-ca        LoadBalancer   10.152.183.78   172.16.170.240   80:31313/TCP,443:30725/TCP   15s
service/ejbca-mariadb   ClusterIP      10.152.183.95   <none>           3306/TCP                     83m

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ejbca-ca   0/1     1            0           15s

NAME                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/ejbca-ca-84649f7f7b   1         1         0       15s

NAME                             READY   AGE
statefulset.apps/ejbca-mariadb   1/1     83m

  7. Check the EJBCA container logs and verify the container has properly started:

    BASH 
    $ kubectl -n ejbca-k8s logs pod/ejbca-ca-84649f7f7b-spwh7 -c ejbca-ca

    • The output is similar to the following:

      BASH 
      2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #######################################################################################################
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #                                                                                                     #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) # Whenever you are ready for production:                                                              #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #                                                                                                     #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) # Try out the commercially supported EJBCA Enterprise Cloud on AWS or Azure, fully featured with:     #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) # - Additional enrollment APIs such as a complete REST API, EST, ACME and more.                       #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) # - Support for external Registration Authority and OCSP responders                                   #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) # - Hardware Security Module support                                                                  #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) # - Application updates                                                                               #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) # - ...and more!                                                                                      #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #                                                                                                     #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) # Feel free to contact us directly for a cloud, on-prem, SaaS, embedded or hybrid solution that fits  #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) # your specific needs.                                                                                #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #                                                                                                     #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #           <https://aws.amazon.com/marketplace/seller-profile?id=7edf9048-58e6-4086-9d98-b8e0c1d78fce> #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #     <https://azuremarketplace.microsoft.com/en-us/marketplace/apps/primekey.ejbca_enterprise_cloud_2> #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #                                                                                                     #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #                                                                           <https://www.keyfactor.com> #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #                                                                                 sales@keyfactor.com #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #                                                                                                     #
2024-06-09 19:29:50,009+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) #######################################################################################################
2024-06-09 19:29:50,015+0000 INFO  [/opt/keyfactor/bin/start.sh] (process:1) Health check now reports application status at /ejbca/publicweb/healthcheck/ejbcahealth

  8. Open a web browser and access EJBCA Administration.

You have now deployed EJBCA using Helm and logged into the AdminWeb to verify that all your configuration is present from the database restore. At this point, you can continue using EJBCA and try out some of the the other EJBCA integrations.

Next steps

In this tutorial, you learned how to clean up/reset the MicroK8s cluster and deploy MariaDB along with EJBCA using Helm.

Here are some next steps we recommend:

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close