Skip to main content
Skip table of contents

Monitor EJBCA host using Monit

The following instructions describe how to set up Monit to monitor EJBCA host variations.

Monit is an open-source monitoring utility that can be used to monitor servers and trigger alerts in case of errors. By integrating EJBCA and Monit, you can monitor the health of your EJBCA application, application server, database management system (DBMS), as well as Apache HTTP Server (HTTPD), if applicable. With Monit, you can view the status of your monitored services in the web interface (dashboard) and configure email notifications for when issues arise.

List of the technology stack used in this guide:

  • AlmaLinux 9.0
  • EJBCA version 7.10.0.1
  • WildFly24
  • MariaDB version 10.5.16
  • Apache version 2.4 front-end reverse proxy running on ports 80 and 443

Step 1 - Configure EJBCA health check service

The EJBCA health check servlet offers several application-level points to be monitored, see Monitoring and Healthcheck for more information. This guide outlines how to enable and monitor the CA health check. 

Enable CA HealthCheck

In EJBCA, use the CA Activation page or the Edit CA page to enable CA health check for monitoring. Note that a testKey needs to be available in the crypto token, see Certificate Authority Overview.

To enable the CA health check service on the CA Activation page:

  1. In EJBCA CA UI, under CA Functions, click CA Activation.
  2. Select the desired CAs to be monitored.
  3. Point your browser to the following URL to reach the health check status over HTTP:

    TEXT
    http://<ejbca-host>:<port>/ejbca/publicweb/healthcheck/ejbcahealth 

Track status output

Add a script to track the output of the health check URL. The output will be processed and displayed in the Monit interface. Polling intervals and alerts can be configured in the control file, see Step 3 - Configure Monit.

To add a bash script for the EJBCA health check:

  1. On your server, create ejbcahealthcheck.sh:

    CODE
    [root@alma90 ~]# vi /opt/ejbcahealthcheck.sh
  2. In the editor, add the following contents to ejbcahealthcheck.sh and then save:

    CODE
    #!/bin/bash
    status=$(curl -s http://localhost:8080/ejbca/publicweb/healthcheck/ejbcahealth | sed -e 's/<[^>]*>//g')
    if [ "$status" == "ALLOK" ] ; then
            echo "EJBCA is OK!"
            exit 0
    else
            echo "NOT OK!"
    		echo "${status}"
            exit 1
    fi
  3. Add execute permissions:

    CODE
    chmod +x /opt/ejbcahealthcheck.sh

Step 2 - Install Monit

To install Monit, you can either build it from source or install it via OS package managers.

Build Monit from source

To build Monit from source, refer to Monit on Bitbucket.

Install Monit using OS package managers

RHEL-based systems (provided via the EPEL release, refer to Extra Packages for Enterprise Linux):

CODE
sudo yum update
sudo yum install monit

MacOS:

CODE
brew install monit

Debian-based systems:

CODE
sudo apt-get update
sudo apt-get install monit

To display version details, run:

TEXT
[root@alma90 ~]# monit --version
This is Monit version 5.32.0
Built with ssl, with ipv6, with compression, with pam and with large files
Copyright (C) 2001-2022 Tildeslash Ltd. All Rights Reserved.

Step 3 - Configure Monit

By default, Monit provides a default configuration control-file named monitrc located at /etc/monitrc (Linux) that contains sample configuration examples.

Follow the steps below to create a new customized configuration file and update the configuration file to align with the services you want to monitor. Depending on your needs, this can range from being minimal without alerts to including additional services and email notifications.

Notes on control file configuration

  • Run a syntax check for the control file before running using monit -t.

  • Reload Monit after updating the control file using monit reload (or optionally restart the service via systemd).

  • Keep a backup copy of your customized file.
  • The control file permissions should be chmod 700.

For more information on customizing the control file, additional settings, and updating the monitored service details, refer to the Monit documentation.

To create a customized Monit control file, using a minimal or additional services example configuration, follow these steps:

  1. On your server, run the following:

    CODE
    [root@alma90 ~]# vi ~/.monitrc
  2. Update your new custom configuration file with one of the following example configurations, and then save the file:
    1. Minimal configuration: The following provides a minimal example configuration:

      CODE
      # Global
      set daemon  30              # check at 30 seconds intervals
      set log syslog
      
      # Monit host and port
      set httpd port 2812 and
          allow admin:monit      # require user 'admin' with password 'monit'
      
      # EJBCA Healthcheck
      check program EJBCA[Healthcheck] with path /opt/ejbcahealthcheck.sh
              if status != 0 then alert
    2. Additional monitored services configuration: The following provides an example configuration including additional services to monitor, and an option to trigger outbound email notifications via SMTP. Note that the example values for memory, usage, and so on are just sample values and may need to be adjusted for your environment and server resources.

      CODE
      # Global
      set daemon  30              # check services at 30 seconds intervals
      set log syslog
      
      # Monit host and port
      set httpd port 2812 and
      #    use address localhost  # only accept connection from localhost (drop if you use M/Monit)
      #    allow localhost        # allow localhost to connect to the server and
          allow admin:monit      # require user 'admin' with password 'monit'
          #    with ssl {            # enable SSL/TLS and set path to server certificate
          #    pemfile: /etc/ssl/certs/monit.pem
          #    }
          #
      
      # EJBCA Healthcheck
      check program EJBCA[Healthcheck] with path /opt/ejbcahealthcheck.sh
              if status != 0 then alert
      
      # Network interface
      check network Interface[Bridge] with interface ens160
              if link down then alert
              if link down then alert
              if changed link then alert
              if saturation > 90% then alert
              if download > 10 MB/s then alert
              if total uploaded > 1 GB in last hour then alert
      
      # System resources
      check system localhost[$HOST]
              if loadavg (1min) per core > 2 for 5 cycles then alert
              if loadavg (5min) per core > 1.5 for 10 cycles then alert
              if cpu usage > 95% for 10 cycles then alert
              if memory usage > 75% then alert
              if swap usage > 25% then alert
      
      # MariaDB
      check process MariaDB
        with pidfile /var/run/mariadb/mariadb.pid
        start program = "/usr/sbin/service mysql start" with timeout 60 seconds
        stop program = "/usr/sbin/service mysql stop" with timeout 60 seconds
        if totalmem > 400 MB for 5 cycles then alert
        if totalmem > 600 MB for 5 cycles then restart
        if cpu > 60% for 3 cycles then alert
        if cpu > 80% for 3 cycles then restart
        if 2 restarts within 7 cycles then timeout
      
      # httpd
      check process HTTPD with pidfile /var/run/httpd/httpd.pid
       group apache
       start program = "/usr/bin/systemctl httpd start"
       stop program = "/usr/bin/systemctl httpd stop"
       if cpu usage > 70% for 10 cycles then restart
       if 5 restarts within 10 cycles then alert
      
      # Wildfly
      check process Wildfly matching 'standalone.sh'
        start program = "/usr/bin/systemctl wildfly start"
        stop program = "/usr/bin/systemctl wildfly stop"
        if totalmem > 3 GB for 2 cycles then restart
        if cpu usage > 90% for 5 cycles then restart
        if 5 restarts within 10 cycles then alert
      
      # Filesystem storage
       check filesystem localhost[/boot] with path /dev/nvme0n1p1
        if space usage > 70% for 5 times within 15 cycles then alert
       check filesystem localhost[/root] with path /dev/mapper/almalinux-root
        if space usage > 80% for 5 times within 15 cycles then alert
      
      ############################################################################
      
      set eventqueue
           basedir /var/monit  # set the base directory where events will be stored
           slots 99999           # optionally limit the queue size
      
      ############################################################################
      
      # SMTP settings
      set mailserver smtp.yourdomain.tld
        port 465
        username "noreply@yourdomain.tld" password "pa$$wOrdHere"
        using auto
      
      # Email recipient
      set alert name.surname@company.com
      
      # Message settings
      set mail-format {
         from:    Monit <noreply@yourdomain.tld>
         subject: monit alert --  $EVENT $SERVICE
         message: $EVENT Service $SERVICE
                       Date:        $DATE
                       Action:      $ACTION
                       Host:        $HOST
                       Description: $DESCRIPTION
      
                  Regards,
                  MonitBot
       }
      
      ###############################################################################
      ## Includes
      ###############################################################################
      ##
      ## It is possible to include additional configuration parts from other files or
      ## directories.
      #
      include /etc/monit.d/*
      #
  3. Update the control file permissions:

    CODE
    chmod 700 /root/.monitrc
  4. Once you have configured the services that you want to include for monitoring, and before launching the Monit web interface in the next step, run a syntax check for the control file:

    CODE
    monit -t

Step 4 - Start Monit

Once you have configured the control file, you need to start (and enable) the monit service.

To start Monit as a service and enable it to start automatically on boot, run the following:

CODE
<sudo> systemctl start monit
<sudo> systemctl stop monit // to stop
<sudo> systemctl enable monit // to enable on boot

Step 5 - Access monitored data

The monitored data can be reached via the Monit web interface or the command line.

Access Monit web interface

To access the Monit web interface and view the status of your monitored services, navigate to the host URL and port defined in the configuration control file in Step 3 - Configure Monit.

  1. Open a browser and go to the host URL and port:

    CODE
    http://<host-or-IP>:2812
  2. Sign in using the default HTTP Basic Authentication username=admin and password=monit.

The following displays examples of monitored data and the information displayed in the Monit Service Manager.

Monitoring examples - Minimal monitored services

  • Displaying minimal monitored services:
  • With minimal monitored services and the EJBCA health check in status failed:

Monitoring examples - Additional monitored services

  • Displaying additional monitored services:
  • Additional monitored services and the EJBCA health check in status failed:
     
  • Click EJBCA[Healthcheck] under Program to view details of the failed event, including native messages returned from the EJBCA health check. Example of Monit Program status:

Monitor services from the command line

You can also monitor the services from the command line and view the status of all monitored services, for example.

To print service status information:

  1. On your server, run:

    CODE
    monit status
  2. The output will look something like the following:

Email notification examples

The following provides examples of email notifications:

  • Status succeeded notification:
  • CA status failed notification:
  • Status failed notification (with more than one failed CA):

Useful Monit commands

The following lists some useful Monit commands and options. For a list of all available arguments and options, refer to the Monit documentation.

Print service status information (in the terminal):

CODE
monit status

Reload all daemon(s):

CODE
monit reload

Stop all running daemon(s):

CODE
monit stop all

Run syntax check for the control file:

CODE
monit -t

Start the service using a different control file:

CODE
monit -c /path/to/control-file

Next steps

Monit also allows you to monitor files, directories, and filesystems for changes, such as timestamp changes, checksum changes, or size changes. For more information, refer to the Monit documentation.

To learn more about the EJBCA health check service used for health monitoring, see Monitoring and Healthcheck.

To learn how to get started with EJBCA Community as a Docker Container, you can follow the Quick Start Guide - Start EJBCA Container with Unauthenticated Network Access

To find out more about EJBCA use cases, see Solution Areas.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.