.png){kind=logo}
Tutorial - Clean up MicroK8s Cluster and Redeploy with Helm
In this tutorial, you will learn how to clean up the MicroK8s cluster removing all the containers, deployments, configmaps, secrets, etc from the previous tutorials. MariaDB and EJBCA are deployed using a Helm chart on the cleaned-up cluster. The database is backed up and restored that way all the configuration that has been done from the previous tutorials is not lost.
Over the last year, all the EJBCA tutorials have been demonstrated on a MicroK8s virtual machine. EJBCA was deployed with YAML files and it would be nice to redeploy EJBCA using Helm. It’s time to clean up and clear out the cluster starting with a fresh deployment of EJBCA using a backup of the MariaDB database.
If you want to clean house, start fresh, and use Helm then this tutorial is for you.
The tutorial covers these steps:
Review what is deployed in the MicroK8s cluster
Backup the MariaDB database
Reset the MicroK8s cluster
Install required services on the MicroK8s cluster
Deploy MariaDB using Helm and restore the MariaDB backup
Deploy EJBCA using Helm
Prerequisites
For this tutorial, EJBCA Community Docker container version 8.2.0.1 was used.
Before you begin, you need:
A virtual machine running MicroK8s, see the tutorial Install Kubernetes MicroK8s runtime to run EJBCA container.
To have completed the tutorial deploy EJBCA and MariaDB containers in MicroK8s.
Step 1 - Review what is deployed in the cluster
If you have been trying out the previous tutorials, there are a lot of pieces and parts deployed and configured in MicroK8s from following the tutorials. Follow the steps below to review what is currently deployed in the cluster:
Review all of the deployments and items deployed:
BASH$ kubectl get all,ingress,secret,pv,pvc,svc,no --all-namespacesThe output is similar to the following:
BASHNAMESPACE NAME READY STATUS RESTARTS AGE cert-manager pod/cert-manager-55cf8685cb-tztzk 1/1 Running 15 (2d21h ago) 151d cert-manager pod/cert-manager-cainjector-fbd548cb8-nrr7l 1/1 Running 20 (2d21h ago) 151d cert-manager pod/cert-manager-webhook-655b4d58fb-46zzh 1/1 Running 12 (2d21h ago) 151d ejbca-cert-manager pod/ejbca-cert-manager-issuer-6c5549d575-qd52x 1/1 Running 30 (2d21h ago) 137d ejbca-k8s pod/ejbca-deployment-78fdbbb5c4-n5v2p 1/1 Running 12 (2d21h ago) 167d ejbca-k8s pod/ejbca-httpd-7d646c85f9-2n68j 1/1 Running 3 (2d21h ago) 48d ejbca-k8s pod/mariadb-deployment-5665d96759-sdg24 1/1 Running 3 (2d21h ago) 48d ingress pod/nginx-ingress-microk8s-controller-sqk8c 1/1 Running 2 (2d21h ago) 24d istio-system pod/istio-egressgateway-66fc798fb8-6cmc4 0/1 Running 22 (2d21h ago) 442d istio-system pod/istio-ingressgateway-668c8c965b-s7725 0/1 Running 22 (2d21h ago) 442d istio-system pod/istiod-7fc54f9bd-qhwxg 1/1 Running 27 (2d21h ago) 442d kube-system pod/calico-kube-controllers-7c8c9d85c9-htp24 1/1 Running 3 (2d21h ago) 48d kube-system pod/calico-node-c4ffk 1/1 Running 0 2d21h kube-system pod/coredns-6f5f9b5d74-wxbmz 1/1 Running 28 (2d21h ago) 449d kube-system pod/hostpath-provisioner-69cd9ff5b8-fx8ps 1/1 Running 45 (2d21h ago) 449d metallb-system pod/controller-9556c586f-gzbm6 1/1 Running 28 (2d21h ago) 449d metallb-system pod/speaker-58lm5 1/1 Running 39 (2d21h ago) 449d pkirules pod/ejbca-cm-issuer-helloworld-one-6c886556f9-xth78 1/1 Terminating 8 (2d21h ago) 136d NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE cert-manager service/cert-manager ClusterIP 10.152.183.223 <none> 9402/TCP 151d cert-manager service/cert-manager-webhook ClusterIP 10.152.183.154 <none> 443/TCP 151d default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 449d ejbca-k8s service/ejbca-database-service ClusterIP 10.152.183.44 <none> 3306/TCP 449d ejbca-k8s service/ejbca-internal ClusterIP 10.152.183.133 <none> 443/TCP,80/TCP 449d ejbca-k8s service/ejbca-service ClusterIP 10.152.183.46 <none> 8009/TCP 449d ingress service/ejbca-node1 LoadBalancer 10.152.183.171 172.16.170.240 80:31205/TCP,443:30622/TCP 449d istio-system service/istio-egressgateway ClusterIP 10.152.183.125 <none> 80/TCP,443/TCP 442d istio-system service/istio-ingressgateway LoadBalancer 10.152.183.92 172.16.170.241 15021:30350/TCP,80:31533/TCP,443:30430/TCP,31400:32547/TCP,15443:31322/TCP 442d istio-system service/istiod ClusterIP 10.152.183.76 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 442d kube-system service/kube-dns ClusterIP 10.152.183.10 <none> 53/UDP,53/TCP,9153/TCP 449d metallb-system service/webhook-service ClusterIP 10.152.183.52 <none> 443/TCP 449d NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE ingress daemonset.apps/nginx-ingress-microk8s-controller 1 1 1 1 1 <none> 449d kube-system daemonset.apps/calico-node 1 1 1 1 1 kubernetes.io/os=linux 449d metallb-system daemonset.apps/speaker 1 1 1 1 1 kubernetes.io/os=linux 449d NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE cert-manager deployment.apps/cert-manager 1/1 1 1 151d cert-manager deployment.apps/cert-manager-cainjector 1/1 1 1 151d cert-manager deployment.apps/cert-manager-webhook 1/1 1 1 151d ejbca-cert-manager deployment.apps/ejbca-cert-manager-issuer 1/1 1 1 137d ejbca-k8s deployment.apps/ejbca-deployment 1/1 1 1 167d ejbca-k8s deployment.apps/ejbca-httpd 1/1 1 1 449d ejbca-k8s deployment.apps/mariadb-deployment 1/1 1 1 449d istio-system deployment.apps/istio-egressgateway 0/1 1 0 442d istio-system deployment.apps/istio-ingressgateway 0/1 1 0 442d istio-system deployment.apps/istiod 1/1 1 1 442d kube-system deployment.apps/calico-kube-controllers 1/1 1 1 449d kube-system deployment.apps/coredns 1/1 1 1 449d kube-system deployment.apps/hostpath-provisioner 1/1 1 1 449d metallb-system deployment.apps/controller 1/1 1 1 449d NAMESPACE NAME DESIRED CURRENT READY AGE cert-manager replicaset.apps/cert-manager-55cf8685cb 1 1 1 151d cert-manager replicaset.apps/cert-manager-cainjector-fbd548cb8 1 1 1 151d cert-manager replicaset.apps/cert-manager-webhook-655b4d58fb 1 1 1 151d ejbca-cert-manager replicaset.apps/ejbca-cert-manager-issuer-6c5549d575 1 1 1 137d ejbca-k8s replicaset.apps/ejbca-deployment-78fdbbb5c4 1 1 1 167d ejbca-k8s replicaset.apps/ejbca-httpd-7d646c85f9 1 1 1 449d ejbca-k8s replicaset.apps/mariadb-deployment-5665d96759 1 1 1 449d istio-system replicaset.apps/istio-egressgateway-66fc798fb8 1 1 0 442d istio-system replicaset.apps/istio-egressgateway-7bf6457867 0 0 0 442d istio-system replicaset.apps/istio-ingressgateway-668c8c965b 1 1 0 442d istio-system replicaset.apps/istio-ingressgateway-8576d6c55f 0 0 0 442d istio-system replicaset.apps/istiod-77f9c8bf68 0 0 0 442d istio-system replicaset.apps/istiod-7fc54f9bd 1 1 1 442d kube-system replicaset.apps/calico-kube-controllers-5cdbddfc9 0 0 0 449d kube-system replicaset.apps/calico-kube-controllers-75b6ddb7b8 0 0 0 408d kube-system replicaset.apps/calico-kube-controllers-79568db7f8 0 0 0 449d kube-system replicaset.apps/calico-kube-controllers-7c8c9d85c9 1 1 1 48d kube-system replicaset.apps/calico-kube-controllers-7c9cdf4bfc 0 0 0 396d kube-system replicaset.apps/calico-kube-controllers-7fcc7d8496 0 0 0 167d kube-system replicaset.apps/calico-kube-controllers-b5fcf779f 0 0 0 60d kube-system replicaset.apps/calico-kube-controllers-df49bb54 0 0 0 364d kube-system replicaset.apps/coredns-6f5f9b5d74 1 1 1 449d kube-system replicaset.apps/hostpath-provisioner-69cd9ff5b8 1 1 1 449d metallb-system replicaset.apps/controller-9556c586f 1 1 1 449d NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE ejbca-k8s ingress.networking.k8s.io/ejbca-node1 public ejbca-node1.ejbca-k8s 127.0.0.1 80, 443 24d NAMESPACE NAME TYPE DATA AGE cert-manager secret/cert-manager-webhook-ca Opaque 3 151d cert-manager secret/sh.helm.release.v1.cert-manager.v1 helm.sh/release.v1 1 151d ejbca-cert-manager secret/ejbca-ca-secret Opaque 1 151d ejbca-cert-manager secret/ejbca-secret kubernetes.io/tls 2 151d ejbca-cert-manager secret/sh.helm.release.v1.ejbca-cert-manager-issuer.v1 helm.sh/release.v1 1 137d ejbca-csr-signer secret/ejbca-client-cert kubernetes.io/tls 2 442d ejbca-csr-signer secret/ejbca-credentials Opaque 1 442d ejbca-k8s secret/ejbca-mariadb-secret Opaque 1 449d ejbca-k8s secret/mariadb-secret Opaque 1 449d ejbca-k8s secret/mtls-trust-chain Opaque 1 47d ejbca-k8s secret/tls-ejbca-node1 kubernetes.io/tls 2 60d istio-system secret/external-ca-cert Opaque 1 442d istio-system secret/istio-ca-secret istio.io/ca-root 5 442d metallb-system secret/memberlist Opaque 1 449d metallb-system secret/webhook-server-cert Opaque 4 449d vault secret/tls-api-vault kubernetes.io/tls 2 160d vault secret/vault-ha-tls Opaque 3 160d NAMESPACE NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE persistentvolume/pvc-031c7c9d-63dd-42b7-8c20-83e9e57782d7 100M RWO Delete Bound ejbca-k8s/mariadb-pvc microk8s-hostpath <unset> 449d persistentvolume/pvc-230714e3-ed56-498f-8386-c84b941e9b00 1Gi RWO Delete Bound vault/data-vault-1 microk8s-hostpath <unset> 160d persistentvolume/pvc-3d0cc22e-3c9a-4e8a-a867-9f99b9f97424 1Gi RWO Delete Bound vault/audit-vault-2 microk8s-hostpath <unset> 160d persistentvolume/pvc-4cacc08e-2802-4b54-9af5-b386750fa715 1Gi RWO Delete Bound vault/audit-vault-0 microk8s-hostpath <unset> 160d persistentvolume/pvc-5be3658f-abda-4512-873a-38b4e9e46556 1Gi RWO Delete Bound vault/audit-vault-1 microk8s-hostpath <unset> 160d persistentvolume/pvc-6d665f7a-c89e-4536-8b3a-f8f5a79a4b00 1Gi RWO Delete Bound vault/data-vault-0 microk8s-hostpath <unset> 160d persistentvolume/pvc-903289f8-db05-44c2-8ded-fdb93a34fb21 1Gi RWO Delete Bound vault/data-vault-2 microk8s-hostpath <unset> 160d persistentvolume/pvc-b1d34ec9-10db-4b8d-ab3d-d8ba7810be14 30M RWO Delete Bound ejbca-k8s/ejbca-pvc microk8s-hostpath <unset> 449d NAMESPACE NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE ejbca-k8s persistentvolumeclaim/ejbca-pvc Bound pvc-b1d34ec9-10db-4b8d-ab3d-d8ba7810be14 30M RWO microk8s-hostpath <unset> 449d ejbca-k8s persistentvolumeclaim/mariadb-pvc Bound pvc-031c7c9d-63dd-42b7-8c20-83e9e57782d7 100M RWO microk8s-hostpath <unset> 449d vault persistentvolumeclaim/audit-vault-0 Bound pvc-4cacc08e-2802-4b54-9af5-b386750fa715 1Gi RWO microk8s-hostpath <unset> 160d vault persistentvolumeclaim/audit-vault-1 Bound pvc-5be3658f-abda-4512-873a-38b4e9e46556 1Gi RWO microk8s-hostpath <unset> 160d vault persistentvolumeclaim/audit-vault-2 Bound pvc-3d0cc22e-3c9a-4e8a-a867-9f99b9f97424 1Gi RWO microk8s-hostpath <unset> 160d vault persistentvolumeclaim/data-vault-0 Bound pvc-6d665f7a-c89e-4536-8b3a-f8f5a79a4b00 1Gi RWO microk8s-hostpath <unset> 160d vault persistentvolumeclaim/data-vault-1 Bound pvc-230714e3-ed56-498f-8386-c84b941e9b00 1Gi RWO microk8s-hostpath <unset> 160d vault persistentvolumeclaim/data-vault-2 Bound pvc-903289f8-db05-44c2-8ded-fdb93a34fb21 1Gi RWO microk8s-hostpath <unset> 160d NAMESPACE NAME STATUS ROLES AGE VERSION node/microk8-01 Ready <none> 449d v1.29.4 microk8s-hostpath <unset> 159d
You have now reviewed what has been deployed in MicroK8s, continue to the next step to backup the EJBCA database.
Step 2 - Backup the EJBCA database
Time was invested to configure EJBCA following the tutorials and it would be great to not lose the EJBCA configuration. Follow these steps to backup the EJBCA database before the MicroK8s cluster is reset:
Review what directories are present in the VM host:
BASH$ ls -lThe output is similar to the following:
BASHtotal 20 -rw-rw-r--. 1 user user 3612 Apr 24 07:56 ca.crt drwxrwxr-x. 3 user user 4096 Jan 26 06:30 cert-manager drwxrwxr-x. 3 user user 88 Mar 25 2023 csr-api drwxrwxr-x. 2 user user 4096 May 17 11:58 k81 drwx------. 3 user microk8s 22 Mar 18 2023 snap drwxrwxr-x. 2 user user 4096 Jun 9 15:34 sql drwxrwxr-x. 2 user user 4096 Jan 2 12:01 vault
Install the MariaDB client using the DNF package manager:
BASH$ sudo dnf install -y <https://dlm.mariadb.com/2690116/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-client-10.10.2-1.el8.x86_64.rpm> \ <https://dlm.mariadb.com/2690045/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-backup-10.10.2-1.el8.x86_64.rpm> \ <https://dlm.mariadb.com/2690100/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-common-10.10.2-1.el8.x86_64.rpm> \ <https://dlm.mariadb.com/2690079/MariaDB/mariadb-10.10.2/yum/rhel8-amd64/rpms/MariaDB-shared-10.10.2-1.el8.x86_64.rpm>Backup the database to the SQL directory:
BASH$ mariadb-dump -h 10.152.183.44 -u root -p --all-databases --quick --lock-tables=false --system=users --insert-ignore --force > sql/20240610_BackupToMigrate.sqlThe output is similar to the following:
BASH# mariadb-dump: Couldn't execute 'SELECT CONCAT(QUOTE(u.user), '@', QUOTE(u.Host)) AS u FROM mysql.user u /*!80001 LEFT JOIN mysql.role_edges e ON u.user=e.from_user AND u.host=e.from_host WHERE e.from_user IS NULL */ /*M!100005 WHERE is_role='N' */': Illegal mix of collations (utf8mb4_general_ci,COERCIBLE) and (utf8mb4_uca1400_ai_ci,COERCIBLE) for operation '=' (1267)
You have now backed up the EJBCA MariaDB database using the mariadb-dump utility. Continue to the next step to reset the MicroK8s cluster.
Step 3 - Reset the MicroK8s cluster
When MicroK8s is reset, the cluster is cleaned up by removing the entire configuration and restoring the cluster to the default install from when MicroK8s was first installed. To reset the cluster follow these steps:
Uninstall the ejbca cert-manager Issuer Helm deployment:
BASH$ helm uninstall -n ejbca-cert-manager ejbca-cert-manager-issuerThe output is similar to the following:
BASHrelease "ejbca-cert-manager-issuer" uninstalled
Uninstall the cert-manager Helm deployment:
BASH$ helm uninstall -n cert-manager cert-managerThe output is similar to the following:
BASHrelease "cert-manager" uninstalled
Uninstall istio using istioctl:
BASH$ csr-api/ejbca-csr-signer/istio-1.16.3/bin/istioctl uninstall --purge --forceThe output is similar to the following:
BASHfailed to get proxy infos: failure running port forward process: failure running port forward process: error upgrading connection: error dialing backend: dial tcp 172.16.170.187:10250: connect: connection refused All Istio resources will be pruned from the cluster Proceed? (y/N)
Enter y to continue.
The output is similar to the following:
BASHRemoved IstioOperator:istio-system:installed-state. Removed Deployment:istio-system:istio-egressgateway. Removed Deployment:istio-system:istio-ingressgateway. Removed Deployment:istio-system:istiod. Removed Service:istio-system:istio-egressgateway. Removed Service:istio-system:istio-ingressgateway. Removed Service:istio-system:istiod. Removed ConfigMap:istio-system:istio. Removed ConfigMap:istio-system:istio-sidecar-injector. Removed Pod:istio-system:istio-egressgateway-66fc798fb8-6cmc4. Removed Pod:istio-system:istio-ingressgateway-668c8c965b-s7725. Removed Pod:istio-system:istiod-7fc54f9bd-qhwxg. Removed ServiceAccount:istio-system:istio-egressgateway-service-account. Removed ServiceAccount:istio-system:istio-ingressgateway-service-account. Removed ServiceAccount:istio-system:istio-reader-service-account. Removed ServiceAccount:istio-system:istiod. Removed ServiceAccount:istio-system:istiod-service-account. Removed RoleBinding:istio-system:istio-egressgateway-sds. Removed RoleBinding:istio-system:istio-ingressgateway-sds. Removed RoleBinding:istio-system:istiod. Removed RoleBinding:istio-system:istiod-istio-system. Removed Role:istio-system:istio-egressgateway-sds. Removed Role:istio-system:istio-ingressgateway-sds. Removed Role:istio-system:istiod. Removed Role:istio-system:istiod-istio-system. Removed EnvoyFilter:istio-system:stats-filter-1.13. Removed EnvoyFilter:istio-system:stats-filter-1.14. Removed EnvoyFilter:istio-system:stats-filter-1.15. Removed EnvoyFilter:istio-system:stats-filter-1.16. Removed EnvoyFilter:istio-system:tcp-stats-filter-1.13. Removed EnvoyFilter:istio-system:tcp-stats-filter-1.14. Removed EnvoyFilter:istio-system:tcp-stats-filter-1.15. Removed EnvoyFilter:istio-system:tcp-stats-filter-1.16. Removed PodDisruptionBudget:istio-system:istio-egressgateway. Removed PodDisruptionBudget:istio-system:istio-ingressgateway. Removed PodDisruptionBudget:istio-system:istiod. Removed MutatingWebhookConfiguration::istio-revision-tag-default. Removed MutatingWebhookConfiguration::istio-sidecar-injector. Removed ValidatingWebhookConfiguration::istio-validator-istio-system. Removed ValidatingWebhookConfiguration::istiod-default-validator. Removed ClusterRole::istio-reader-clusterrole-istio-system. Removed ClusterRole::istio-reader-istio-system. Removed ClusterRole::istiod-clusterrole-istio-system. Removed ClusterRole::istiod-gateway-controller-istio-system. Removed ClusterRole::istiod-istio-system. Removed ClusterRoleBinding::istio-reader-clusterrole-istio-system. Removed ClusterRoleBinding::istio-reader-istio-system. Removed ClusterRoleBinding::istiod-clusterrole-istio-system. Removed ClusterRoleBinding::istiod-gateway-controller-istio-system. Removed ClusterRoleBinding::istiod-istio-system. Removed CustomResourceDefinition::authorizationpolicies.security.istio.io. Removed CustomResourceDefinition::destinationrules.networking.istio.io. Removed CustomResourceDefinition::envoyfilters.networking.istio.io. Removed CustomResourceDefinition::gateways.networking.istio.io. Removed CustomResourceDefinition::istiooperators.install.istio.io. Removed CustomResourceDefinition::peerauthentications.security.istio.io. Removed CustomResourceDefinition::proxyconfigs.networking.istio.io. Removed CustomResourceDefinition::requestauthentications.security.istio.io. Removed CustomResourceDefinition::serviceentries.networking.istio.io. Removed CustomResourceDefinition::sidecars.networking.istio.io. Removed CustomResourceDefinition::telemetries.telemetry.istio.io.
Reset microk8s to restore MicroK8s to the default install:
BASH$ sudo /var/lib/snapd/snap/bin/microk8s resetThe output is similar to the following:
BASHDisabling all addons Disabling addon : core/cert-manager Disabling addon : core/dashboard Disabling addon : core/dns Disabling addon : core/gpu Disabling addon : core/helm Disabling addon : core/helm3 Disabling addon : core/host-access Disabling addon : core/hostpath-storage Disabling addon : core/ingress Disabling addon : core/kube-ovn Disabling addon : core/mayastor Disabling addon : core/metallb Disabling addon : core/metrics-server Disabling addon : core/minio Disabling addon : core/observability Disabling addon : core/prometheus Disabling addon : core/rbac Disabling addon : core/registry Disabling addon : core/storage All addons are disabled. Deleting the CNI Cleaning resources in namespace bookinfo Cleaning resources in namespace cert-manager Cleaning resources in namespace default Cleaning resources in namespace ejbca-cert-manager Cleaning resources in namespace ejbca-csr-signer Cleaning resources in namespace ejbca-k8s Cleaning resources in namespace ingress Cleaning resources in namespace istio-system Cleaning resources in namespace kube-node-lease Cleaning resources in namespace kube-public Cleaning resources in namespace kube-system Cleaning resources in namespace metallb-system Cleaning resources in namespace pkirules Cleaning resources in namespace vault Removing CRDs Removing PriorityClasses Removing StorageClasses Removing namespace/bookinfo Removing namespace/cert-manager Removing namespace/ejbca-cert-manager Removing namespace/ejbca-csr-signer Removing namespace/ejbca-k8s Removing namespace/ingress Removing namespace/istio-system Removing namespace/metallb-system Removing namespace/pkirules Removing namespace/vault Restarting cluster Stopped. Setting up the CNI
You have now reset MicroK8s back to the default. Continue to the next step to enable the required services in MicroK8s for deploying MariaDB and EJBCA.
Step 4 - Enable required services on MicroK8s
Enable DNS, Helm, and hostpath storage on the MicroK8s cluster which are used to deploy EJBCA and MariaDB and provide persistent storage for MariaDB. Follow the steps below to prepare the cluster:
Enable the DNS, Helm, and hostpath-storage addons on the cluster:
BASH$ microk8s enable dns hostpath-storage helmThe output is similar to the following:
BASHInfer repository core for addon dns Infer repository core for addon hostpath-storage Infer repository core for addon helm WARNING: Do not enable or disable multiple addons in one command. This form of chained operations on addons will be DEPRECATED in the future. Please, enable one addon at a time: 'microk8s enable <addon>' Addon core/dns is already enabled Enabling default storage class. WARNING: Hostpath storage is not suitable for production environments. A hostpath volume can grow beyond the size limit set in the volume claim manifest. deployment.apps/hostpath-provisioner created storageclass.storage.k8s.io/microk8s-hostpath created serviceaccount/microk8s-hostpath created clusterrole.rbac.authorization.k8s.io/microk8s-hostpath created clusterrolebinding.rbac.authorization.k8s.io/microk8s-hostpath created Storage will be available soon. Addon core/helm is already enabled
Enable the Metal LB addon on the cluster using the IP range of the VM network the VM is on:
BASH$ microk8s enable metallb:172.16.170.240-172.16.170.245The output is similar to the following:
BASHInfer repository core for addon metallb Enabling MetalLB Applying Metallb manifest customresourcedefinition.apiextensions.k8s.io/addresspools.metallb.io created customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created namespace/metallb-system created serviceaccount/controller created serviceaccount/speaker created clusterrole.rbac.authorization.k8s.io/metallb-system:controller created clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created role.rbac.authorization.k8s.io/controller created role.rbac.authorization.k8s.io/pod-lister created clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created rolebinding.rbac.authorization.k8s.io/controller created secret/webhook-server-cert created service/webhook-service created rolebinding.rbac.authorization.k8s.io/pod-lister created daemonset.apps/speaker created deployment.apps/controller created validatingwebhookconfiguration.admissionregistration.k8s.io/validating-webhook-configuration created Waiting for Metallb controller to be ready. deployment.apps/controller condition met ipaddresspool.metallb.io/default-addresspool created l2advertisement.metallb.io/default-advertise-all-pools created MetalLB is enabled
Create an alias for helm command:
BASH$ sudo snap alias microk8s.helm helmThe output is similar to the following:
BASH#Added: # - microk8s.helm as helm
Review what is deployed in the cluster after the microk8s reset:
BASH$ kubectl get all,secret,pv,pvc,svc,no --all-namespacesThe output is similar to the following:
BASHNAMESPACE NAME READY STATUS RESTARTS AGE kube-system pod/calico-kube-controllers-7cb4fd5784-xs4db 1/1 Running 0 3m39s kube-system pod/calico-node-s42fj 1/1 Running 0 3m39s kube-system pod/coredns-85db789d58-2pfxz 1/1 Running 0 2m3s kube-system pod/hostpath-provisioner-7df77bc496-xcdbm 1/1 Running 0 50s metallb-system pod/controller-5f7bb57799-wxcvx 1/1 Running 0 50s metallb-system pod/speaker-rdn59 1/1 Running 0 50s NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 6m31s kube-system service/kube-dns ClusterIP 10.152.183.10 <none> 53/UDP,53/TCP,9153/TCP 2m3s metallb-system service/webhook-service ClusterIP 10.152.183.225 <none> 443/TCP 102s NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE kube-system daemonset.apps/calico-node 1 1 1 1 1 kubernetes.io/os=linux 4m44s metallb-system daemonset.apps/speaker 1 1 1 1 1 kubernetes.io/os=linux 102s NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE kube-system deployment.apps/calico-kube-controllers 1/1 1 1 4m44s kube-system deployment.apps/coredns 1/1 1 1 2m3s kube-system deployment.apps/hostpath-provisioner 1/1 1 1 114s metallb-system deployment.apps/controller 1/1 1 1 102s NAMESPACE NAME DESIRED CURRENT READY AGE kube-system replicaset.apps/calico-kube-controllers-7cb4fd5784 1 1 1 3m39s kube-system replicaset.apps/coredns-85db789d58 1 1 1 2m3s kube-system replicaset.apps/hostpath-provisioner-7df77bc496 1 1 1 50s metallb-system replicaset.apps/controller-5f7bb57799 1 1 1 50s NAMESPACE NAME TYPE DATA AGE metallb-system secret/memberlist Opaque 1 49s metallb-system secret/webhook-server-cert Opaque 4 102s NAMESPACE NAME STATUS ROLES AGE VERSION node/microk8-01 Ready <none> 449d v1.29.4
You have now enabled the required services to deploy EJBCA and MariaDB using Helm. Continue to the next step to deploy MariaDB using Helm.
Step 5 - Deploy MariaDB with Helm & restore the EJBCA SQL backup
Before EJBCA is deployed you need to have the database (MariaDB) deployed and the SQL backup restored. That way when EJBCA is deployed you begin right where you left off from the last backup taken. Follow these steps to deploy MariaDB with Helm and restore the EJBCA database backup:
Install Helm repository for MariaDB:
BASH$ helm repo add groundhog2k https://groundhog2k.github.io/helm-chartsThe output is similar to the following:
BASH"groundhog2k" has been added to your repositories
Create namespace for the EJBCA MariaDB deployment:
BASH$ kubectl create namespace ejbca-k8sThe output is similar to the following:
BASHnamespace/ejbca-k8s created
Create the MariaDB root password secret:
BASH$ kubectl -n ejbca-k8s create secret generic mariadb-secret --from-literal=MARIADB_ROOT_PASSWORD=ejbcaThe output is similar to the following:
BASHsecret/mariadb-secret created
Change to the k81 directory:
BASH$ cd k81Create the
mariadb-override.yamlfile with settings to deploy MariaDB:BASH$ cat > mariadb-override.yaml << EOF ## Default values for MariaDB deployment ## MariaDB docker image image: registry: "docker.io" repository: "mariadb" pullPolicy: IfNotPresent # Overrides the image tag whose default is the chart appVersion. tag: "latest" ## Optional service account serviceAccount: # Specifies whether a service account should be created create: true # Annotations to add to the service account annotations: {} # The name of the service account to use. # If not set and create is true, a name is generated using the fullname template name: "" # Resource limits and requests resources: limits: cpu: 2 memory: 2048Mi requests: cpu: 1000m memory: 1024Mi ## Use Kubernetes Deployment instead of StatefulSet useDeployment: false ## Database configuration settings: ## Arguments for the container entrypoint process arguments: - --character-set-server=utf8mb4 - --collation-server=utf8mb4_unicode_ci ## Optional existing secret for the MariaDB root password existingSecret: mariadb-secret ## Set true to skip loading timezone data during init skipTZInfo: false ## Storage parameters storage: ## Alternative set requestedSize to define a size for a dynamically created PVC requestedSize: 1Gi EOFDeploy MariaDB using helm chart with override:
BASH$ helm install -n ejbca-k8s ejbca-mariadb groundhog2k/mariadb -f mariadb-override.yamlThe output is similar to the following:
BASHNAME: ejbca-mariadb LAST DEPLOYED: Sun Jun 9 13:31:31 2024 NAMESPACE: ejbca-k8s STATUS: deployed REVISION: 1 TEST SUITE: None
Check on the deployment after waiting for 1 min:
BASH$ kubectl -n ejbca-k8s get allThe output is similar to the following:
BASHNAME READY STATUS RESTARTS AGE pod/ejbca-mariadb-0 1/1 Running 0 85s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/ejbca-mariadb ClusterIP 10.152.183.95 <none> 3306/TCP 85s NAME READY AGE statefulset.apps/ejbca-mariadb 1/1 85s
Connect to the database using the
service/ejbca-mariadbto make sure the DB is accessible, enter the password for therootaccount when prompted:BASH$ mariadb -h 10.152.183.234 -u root -pEnter the password for the
rootaccount.The output is similar to the following:
BASHEnter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 51 Server version: 11.4.2-MariaDB-ubu2404 mariadb.org binary distribution Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]>
Enter
exitto exit the database shell:BASHMariaDB [(none)]> exitThe output is similar to the following:
BASHMariaDB [(none)]> exit Bye
Import the EJBCA SQL backup into MariaDB, enter the password for the root account when prompted:
BASH$ mariadb -h 10.152.183.234 -u root -p < ../sql/20240610_BackupToMigrate.sqlEnter the password for the
rootaccount.Connect to the
ejbcadatabase as theejbcauser:BASH$ mariadb -h 10.152.183.234 -u ejbca -p ejbcaEnter the password for the
ejbcaaccount.The output is similar to the following:
BASHEnter password: Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 185 Server version: 11.4.2-MariaDB-ubu2404 mariadb.org binary distribution Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [ejbca]>
Verify the tables are present:
BASHMariaDB [ejbca]> show tables;The output is similar to the following:
BASHMariaDB [ejbca]> show tables; # Output similar to the following: +-------------------------------+ | Tables_in_ejbca | +-------------------------------+ | AccessRulesData | | AcmeAccountData | | AcmeAuthorizationData | | AcmeChallengeData | | AcmeNonceData | | AcmeOrderData | | AdminEntityData | | AdminGroupData | | AdminPreferencesData | | ApprovalData | | AuditRecordData | | AuthorizationTreeUpdateData | | Base64CertData | | BlacklistData | | CAData | | CRLData | | CertReqHistoryData | | CertificateData | | CertificateProfileData | | CryptoTokenData | | EndEntityProfileData | | GlobalConfigurationData | | IncompleteIssuanceJournalData | | InternalKeyBindingData | | KeyRecoveryData | | NoConflictCertificateData | | OcspResponseData | | PeerData | | ProfileData | | PublisherData | | PublisherQueueData | | RoleData | | RoleMemberData | | SctData | | ServiceData | | UserData | | UserDataSourceData | +-------------------------------+ 37 rows in set (0.001 sec)
Enter
exitto exit the database:BASHMariaDB [ejbca]> exitThe output is similar to the following:
BASHMariaDB [ejbca]> exit Bye
You have now deployed MariaDB using Helm and restored the EJBCA database backup. You then verified the tables created from restoring the backup. Continue to the next step to deploy EJBCA using Helm.
Step 6 - Deploy EJBCA with Helm
EJBCA is deployed after MariaDB is deployed and the database backup is restored. Follow the steps below to deploy EJBCA:
Install the Helm repository for EJBCA:
BASH$ helm repo add ejbca-ce https://keyfactor.github.io/ejbca-community-helmThe output is similar to the following:
BASH"ejbca-ce" has been added to your repositories
Create the
ejbca-override.yamlfile:BASH$ cat > ejbca-overrides.yaml << EOF ejbca: useEphemeralH2Database: false useH2Persistence: false importExternalCas: false importAppserverKeystore: false importAppserverTruststore: false importEjbcaConfFiles: false env: TLS_SETUP_ENABLED: "simple" DATABASE_JDBC_URL: "jdbc:mariadb://ejbca-mariadb.ejbca-k8s:3306/ejbca?characterEncoding=utf8" LOG_AUDIT_TO_DB: true LOG_LEVEL_APP: "INFO" LOG_LEVEL_SERVER: "INFO" OBSERVABLE_BIND: "127.0.0.1" METRICS_ENABLED: false OCSP_CHECK_SIGN_CERT_VALIDITY: false OCSP_NON_EXISTING_IS_GOOD: true envRaw: - name: DATABASE_USER valueFrom: secretKeyRef: name: ejbcadbcredentials key: database_user - name: DATABASE_PASSWORD valueFrom: secretKeyRef: name: ejbcadbcredentials key: database_password # - name: PASSWORD_ENCRYPTION_KEY # valueFrom: # secretKeyRef: # name: ejbca-confidentiality-secrets # key: password_encrpytion_key envFrom: [] initContainers: [] sidecarContainers: [] volumes: [] volumeMounts: [] services: directHttp: enabled: false proxyAJP: enabled: false proxyHttp: enabled: true type: LoadBalancer bindIP: 0.0.0.0 httpPort: 80 httpsPort: 443 sidecarPorts: [] nginx: enabled: true host: "ejbca-node1.ejbca-k8s" proxy_url_host: localhost mountInternalNginxCert: true secretInternalNginxCert: "internal-nginx-credential-secret-ca" service: enabled: false type: NodePort httpPort: 30080 httpsPort: 30443 ingress: enabled: false replicaCount: 1 updateStrategy: {} image: repository: keyfactor/ejbca-ce pullPolicy: IfNotPresent tag: latest imagePullSecrets: [] nameOverride: "ejbca-ca" fullnameOverride: "ejbca-ca" serviceAccount: create: true annotations: {} name: "" podLabels: {} podAnnotations: {} podSecurityContext: {} securityContext: {} resources: limits: cpu: 4 memory: 4096Mi requests: cpu: 1000m memory: 2048Mi autoscaling: enabled: false minReplicas: 1 maxReplicas: 5 targetCPUUtilizationPercentage: 80 behavior: {} podDisruptionBudget: create: false minAvailable: 1 maxUnavailable: "" nodeSelector: {} affinity: {} tolerations: [] topologySpreadConstraints: [] priorityClassName: "" schedulerName: "" terminationGracePeriodSeconds: "" EOFCreate the secret for the EJBCA Database username and password:
BASH$ kubectl -n ejbca-k8s create secret generic ejbcadbcredentials --from-literal=database_user=ejbca --from-literal=database_password=ejbcaThe output is similar to the following:
BASHsecret/ejbcadbcredentials created
Create the TLS secret for the certificate, key, and CA cert used by NGINX that is deployed in the EJBCA pod:
BASH$ kubectl -n ejbca-k8s create secret generic internal-nginx-credential-secret-ca \ --from-file=ejbca-node1.ejbca-k8s.pem=ejbca-node1.ejbca-k8s.crt \ --from-file=ejbca-node1.ejbca-k8s-Key.pem=ejbca-node1.ejbca-k8s.key \ --from-file=ejbca-node1.ejbca-k8s-CA.pem=../ca.crtThe output is similar to the following:
BASHsecret/internal-nginx-credential-secret-ca created
Deploy EJBCA using the Helm chart and the override values:
BASH$ helm install -n ejbca-k8s ejbca-ca ejbca-ce/ejbca-community-helm -f ejbca-overrides.yamlThe output is similar to the following:
BASHNAME: ejbca-ca LAST DEPLOYED: Sun Jun 9 15:29:16 2024 NAMESPACE: ejbca-k8s STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: Thank you for installing EJBCA CE! Now that you have deployed EJBCA, you should look over the docs on using EJBCA available here: <https://doc.primekey.com/> Your release is named ejbca-ca. To learn more about the release, try: $ helm status ejbca-ca $ helm get manifest ejbca-ca
Review what has been deployed:
BASHkubectl -n ejbca-k8s get allThe output is similar to the following:
BASHNAME READY STATUS RESTARTS AGE pod/ejbca-ca-84649f7f7b-ck5bn 1/2 Running 0 15s pod/ejbca-mariadb-0 1/1 Running 0 83m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/ejbca-ca LoadBalancer 10.152.183.78 172.16.170.240 80:31313/TCP,443:30725/TCP 15s service/ejbca-mariadb ClusterIP 10.152.183.95 <none> 3306/TCP 83m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/ejbca-ca 0/1 1 0 15s NAME DESIRED CURRENT READY AGE replicaset.apps/ejbca-ca-84649f7f7b 1 1 0 15s NAME READY AGE statefulset.apps/ejbca-mariadb 1/1 83m
Check the EJBCA container logs and verify the container has properly started:
BASH$ kubectl -n ejbca-k8s logs pod/ejbca-ca-84649f7f7b-spwh7 -c ejbca-caThe output is similar to the following:
BASH2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) ####################################################################################################### 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # Whenever you are ready for production: # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # Try out the commercially supported EJBCA Enterprise Cloud on AWS or Azure, fully featured with: # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - Additional enrollment APIs such as a complete REST API, EST, ACME and more. # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - Support for external Registration Authority and OCSP responders # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - Hardware Security Module support # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - Application updates # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # - ...and more! # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # Feel free to contact us directly for a cloud, on-prem, SaaS, embedded or hybrid solution that fits # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # your specific needs. # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # <https://aws.amazon.com/marketplace/seller-profile?id=7edf9048-58e6-4086-9d98-b8e0c1d78fce> # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # <https://azuremarketplace.microsoft.com/en-us/marketplace/apps/primekey.ejbca_enterprise_cloud_2> # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # <https://www.keyfactor.com> # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # sales@keyfactor.com # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) # # 2024-06-09 19:29:50,009+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) ####################################################################################################### 2024-06-09 19:29:50,015+0000 INFO [/opt/keyfactor/bin/start.sh] (process:1) Health check now reports application status at /ejbca/publicweb/healthcheck/ejbcahealth
Open a web browser and access EJBCA Administration.
You have now deployed EJBCA using Helm and logged into the AdminWeb to verify that all your configuration is present from the database restore. At this point, you can continue using EJBCA and try out some of the the other EJBCA integrations.
Next steps
In this tutorial, you learned how to clean up/reset the MicroK8s cluster and deploy MariaDB along with EJBCA using Helm.
Here are some next steps we recommend:
If you are interested in EJBCA Enterprise, read more on Keyfactor EJBCA Enterprise.
If you are interested in EJBCA Community, check out EJBCA Community vs Enterprise or read more on ejbca.org.
If you are an EJBCA Enterprise customer and need support, visit the Keyfactor Support Portal.
Discuss with the EJBCA Community on GitHub Discussions.