EJBCA Security

Security is critical for a CA and protection of the CA's private key is essential. Compromising the CA's private key allows anyone issue false certificates, which can then be used to gain access to systems relying on the CA for authentication and other security services.

Study the following sections and take a pragmatic approach to security, suitable for your policy, application and environment.

Note that the information on this page provides details for a pure software installation of EJBCA. It may not be relevant for packaged deployments such as containers, appliances and cloud because many of the details are then built into the platform, or pre-applied.

Required Firewall Ports

Only a few ports need to be open to the outside to allow a functional EJBCA installation:

Depending on your needs, you can choose to have both port 8080 and 8442, or only one of them open.

Granting certain functionality for port 8080 and 8442 can be done by creating an administrator role matching the corresponding "Public Access Authentication Token" and apply desired access rules to it. For example, one might allow enrollment through the RA Web without requiring a client certificate. Use this feature with caution!

Additional ports that you may need to open are SSH, outgoing SMTP, outgoing LDAP etc, depending on your specific setup and services used. You do not need outgoing SMTP if you do not use email notification, for example.

Locally on the host, a number of ports are used in order for your application server to function.

Front-end Server and URL Filtering

When exposing EJBCA (or any web application) to an untrusted network (such as the Internet) it is a good and common architecture to use a front-end server that performs additional filtering. For example, if only OCSP should be available to the Internet, and not the Admin UI, SCEP, EST, and so on, you can allow only those URLs to be accessed from the untrusted network. Other URLs can be accessible from other internal networks. Typical front-end servers are Apache and Nginx.

The following displays an example Apache mod_ajp for EJBCA to only pass on EJBCA URLs from the Apache front-end to the JBoss/WildFly server where EJBCA is running:

ProxyPass /ejbca/ ajp://ejbca_app:8009/ejbca/ keepalive=On ping=500ms retry=1 timeout=300
ProxyPass /.well-known/ ajp://ejbca_app:8009/.well-known/ keepalive=On ping=500ms retry=1 timeout=300

For an example Apache configuration for passing generic OCSP URLs, such as http://ocsp.example.com/, to EJBCA but dropping unwanted requests, see OCSP GET.

Secure Configuration of Front-end Server

If a front-end server (also known as a reverse proxy) is used together with EJBCA, it is crucial that it is configured correctly to avoid security issues.

Many URLs in EJBCA may return a response containing state from the request (such as the Host header) and/or from the session. Depending on the platform, the root URL path / may be a redirect containing an absolute URL based on the request domain. These must not be cached. In these cases, EJBCA will either send no-cache HTTP headers or use a non-cacheable HTTP status code. You must ensure that the front-end server either respects the non-cacheability of such responses, or that it performs no caching at all.

Securing JBoss/WildFly

Configuring for Security

The easiest way to keep the installation secure is to block all default JBoss ports (1099, 1476, 4444, 8082, 8083) from the outside and only allow traffic to Tomcat ports (8442 and 8443). This is because the public end-user actions can be performed through the public servlets, while administration tasks are performed directly on the beans. EJBCA comes with a ready made script for setting up IPTables in Linux, and it can be found in docs/howto/ejbcafirewall.sh

For information on ports used in JBoss and for information on how to set up TLS with JBoss, see documentation on jboss.org.

Securing JBoss Management Interfaces

In addition to the services made available by the running application server or servers, JBoss also makes two management interfaces available to allow remote clients to connect for the purpose of managing the running JBoss installation. For information on how these interfaces are used and how they can be secured, refer to documentation on jboss.org or docs.redhat.com.

To remove the JMX console completely, remove the following from the standalone.xml file:

<subsystem xmlns="urn:jboss:domain:jmx:1.3">
  <expose-resolved-model/>
  <expose-expression-model/>
  <remoting-connector/>
</subsystem>

To remove the welcome pages, remove the following:

<location name="/" handler="welcome-content"/> 

By default, the management web console is only accessible if you configure user access to it, and it is not configured by default.

Disable Specific HTTP Methods

EJBCA by default limits the use of PUT|DELETE|TRACE|OPTIONS in its web applications, but if you are unsure you can further enforce it.

You can use an Apache front-end, or you can use a RewriteValve in standalone.xml according to the following example:

<subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" native="false">
    <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http"/>
    <virtual-server name="default-host" enable-welcome-root="true">
        <rewrite pattern=".*" substitution="-" flags="F">
            <condition test="%{REQUEST_METHOD}" pattern="^(PUT|DELETE|TRACE|OPTIONS)$" flags="NC" />
    </rewrite>
    </virtual-server>
</subsystem>

Content Security Policy

CSP is implemented in browsers to mitigate cross site scripting attacks. For more information, refer to Mozilla Developer Network and Content Security Policy Reference.

CSP has been added to EJBCA Web Interfaces as of ECA-3519 in EJBCA 6.2.0.

To implement CSP globally if you use an Apache front-end with mod_headers:

<IfModule mod_headers.c>
Header set Content-Security-Policy "default-src 'self'"
</IfModule>

The above apache configuration is just an example and will probably not work well. See the headers returned by EJBCA for a working configuration.

Write Protect Deploy

After deploying EJBCA, you can write protect the standalone/deployments directory. In runtime files are only read from this directory, and write protecting it (owned by root, JBoss runs as jboss user) can prevent malicious code to be deployed.

For more information on how to secure a default installation of JBoss, refer to the Red Hat documentation.

You should also ensure that sensitive files (like the server.xml file storing password for the TLS key stores) only can be read by the JBoss user.

Setting up TLS

To set up TLS communication for all HTTP traffic to the server, follow the instructions for installing EJBCA, this will set up HTTPS for the CA UI automatically.

This will set up an TLS port open for the public at 8442 and an TLS port which require client certificate to access the CA UI at 8443.

File Permissions

The application server should be run as a special user. Files should be protected so the ONLY the user running the application server can access them.

JBoss is most likely by default unpacked with read access to all. To make the files readable only by the JBoss user, run the following in the JBOSS_HOME directory to also make this the default permission for files copied to this location:

umask 077
chmod -R go-rwx *

If PKCS12 files are generated for users, the subdirectory (p12) where they are stored and the generated files should be protected in the same way.

Encrypting Datasource Passwords

With JBoss/WildFly you can set up encryption of datasource passwords so they are not human-readable in standalone.xml. In many ways, this is more masking than high-security encryption, but it is nevertheless worth looking into, and you may also integrate with external secrets management systems.

The primary way, since the Elytron subsystem was introduced in JBoss EAP 7.x, to encrypt datasource passwords should be by using a Credential Store, which can even be FIPS compliant. For more information, refer to the JBoss Documentation on Securely Storing Credentials. There are also older methods such as using a Vault or en encoded password in a file, referenced by a security domain. For more information, refer to the WildFly tutorial How to encrypt WildFly Datasource password.

Using 2048 bit Ephemeral Diffie-Hellman Keys

By default Java 8 uses 1024 bit ephemeral Diffie-Hellman keys. Some audits and certifications require the use of larger keys. You can customize the key size in Java as documented in JSSE Reference Guide: Customizing Size of Ephemeral Diffie-Hellman Keys.

In JBoss/WildFly, add the following to your bin/standalone.conf, in the section where you specify options to pass to the Java VM.

JAVA_OPTS="$JAVA_OPTS -Djdk.tls.ephemeralDHKeySize=2048"

Not Exposing Internal IP/Hostname in JSESSIONID Cookie

Like most web applications, browser sessions are managed using JSESSIONID cookies. By default JBoss/WildFly sets a cookie like:

Cookie: JSESSIONID=AxpTtbt42hlB95sk51DrvNJHr0LiT-BTRLGatXh0.ip-172-31-19-190

When you are running JBoss behind a proxy, like an Apache or Nginx, the IP at the end of the cookie exposes the internal IP of your server.

To modify the default value of JSESSIONID that is by default appending hostname (and/or) server group name, define an instance-id attribute in the Undertow Subsystem in standalone.xml (and/or domain.xml).

<subsystem xmlns="urn:jboss:domain:undertow:3.1" instance-id="${mynodename}">
     <buffer-cache name="default"/>
      <server name="default-server">
                <http-listener name="default" socket-binding="http" redirect-socket="https"/>
        .
        .
        .
</subsystem>

<system-properties>
        <property name="mynodename" value="myvalue"/>
</system-properties>

After making the changes, the JSESSION value in standalone\domain mode will be like:

JSESSIONID=FsYvt_nZcyDV2gKpQ_4ZsSYeu41JycphvMdHcYeT.myvalue

User Authentication

Default user authentication for enrollment in EJBCA is done with a one-time password scheme. When a user has enrolled for a certificate his status is set to GENERATED and the password cannot be used again to enroll for a new certificate. An administrator must re-set the user's status and preferably set a new password.

If you are exposing your public web pages to a wider audience, you can use the following functions to counter perceived threats such as brute-force attacks:

• Enable password expiry using the User Password Expire Service. If a user forgets to use his/her one-time password it will be automatically disabled. For more information, see User Password Expire Service in Services.
• Use password expiry with the Maximum number of failed login attempts setting when registering users. For more information, see End Entity Profile Operations.
• Increase the number of BCrypt rounds, making password hashing slower. See the setting of ejbca.passwordlogrounds in conf/ejbca.properties.sample.

If implementing other user authentication scenarios you should remember that certificate authentication is stronger than password based authentication (for example LDAP). If EJBCA users authenticate with some other (not one-time) password instead of usual one-time password, a strong authentication mechanism will be built on a weaker one.

Passwords Defined when Configuring EJBCA

The configuration files (in $EJBCA_HOME/conf) contain some passwords. It is not considered to be a security risk to declare all these passwords in clear text. Anyone that can log on to the server with EJBCA can, apart from reading these files, also do anything he wants with the CLI of EJBCA. If an unauthorized person can use the CLI, then this is a severe security risk. Access to these passwords is itself not much of a problem, since they have no use outside of the server (except for password.encryption.key which can be used to decrypt encrypted passwords stored elsewhere, for example, external database server).

It is very important to restrict the access to the server to only a very few trusted individuals and promote separation of duties and the principle of least privilege. For example, a DBA should not have access to the application server and will thus (if password.encryption.key has been customized in the application server) not be able to decrypt passwords stored in the database he has access to.

If you want to do something about these passwords the subsections of this section describe what should be done:

Passwords Used by EJBCA Taken from Property Files

Some of the passwords are used directly by the EJBCA code. All these passwords may be configured encrypted in the same way as PINs used for auto activation could be encrypted (except for password.encryption.key which is itself used for encrypting the other passwords). For more information, see Auto-activation of Crypto Tokens in Hardware Security Modules (HSM).

List of these passwords across several configuration files:

• password.encryption.key - used for encrypting passwords in the database
• ca.keystorepass - only used for default soft CA tokens, not used at all if HSMs are used. Deprecated and not used since EJBCA 8.3
• databaseprotection.tokenpin - if database protection is enabled (signed database records) this is used to activate the database protection crypto token during startup
• ejbca.cli.defaultpassword - used during installation when creating the ejbca CLI user, and read by the CLI if CLI username/password is not provided to the command line
• ca.cmskeystorepass - deprecated and not used since EJBCA 8.0
• ca.tokenpassword - only used during a manual installation of a SW installation, if left out you will be prompted

For a production installation it is therefore password.encryption.key that is the most important. This property should use any password you consider safe, but it is strongly recommended that you use a randomly generated alphanumeric (using full character set) password.

Passwords Used by the Application Server

Some of the passwords are not used by EJBCA but by the application server. If these passwords should be encrypted, it must be in a way so that they can be decrypted by the application server. You have to consult the documentation of the application server to find out how to encrypt them (example: datasource password in JBoss).

These passwords are:

email server password

• There is no need to set mail server password in mail.properties in EJBCA.
• Note that the mail server password is configured in JBoss/WildFly during installation (needed only if email notifications should be enabled).

database password

• There is no need to set database password in database.properties for EJBCA to run, it can be left out from EJBCA server build.
• The password in database.properties is needed for the local EJBCA database CLI,
• Note that the database password is configured in JBoss/WildFly during installation.

Passwords Prompted for by 'ant runinstall'

If you do not define superadmin.password in web.properties, then ant runinstall will prompt for it. Since this password does not have to be known by EJBCA after the super admin token has been created, it will not exist in any file after the installation.

The passwords java.trustpassword and httpsserver.password, also in web.properties, are used to generate keystore files at ant runinstall. If either of these passwords are not predefined, then they will be prompted for during deploy and install.

If you let 'ant' prompt for these passwords you must set them (encrypted, if possible) in the application server configuration.

Database Integrity Protection

Database integrity protection consists of an additional rowProtection column in all protected tables.

For flexibility, versions and keyid are embedded in the rowProtection line. Versions allow the tables themselves to be modified safely because the entity class can create "version 2" of the protect string with new fields, but still verify "version 1" and other legacy versions rows without the additional field. ProtectData can update the algorithm used, etc. by iterating the version number. The protection key can be changed because the user defines the keyid of the crypto token and always verifies using a specific keyid. The keyids are specified in configuration and multiple keyids can be active at any moment.

Configuration

Database integrity protection should be configured before EJBCA is installed and used the first time, otherwise unprotected data will be inserted into the database; once the system is configured for database protection, using that data will fail. Database protection is configured in the file: conf/databaseprotection.properties.

Limitations

• One limitation on the application using this protection scheme is that we can now not do bulk updates using a JPQ query with UPDATE where xyz. This is because the @Pre/Post events are not triggered for JPA beans when doing that. This is a limitation that can be severe in very large installations;
• We cannot do direct database updates, this is of course the whole idea of the database protection, but it is also a limitation nonetheless;
• When using an HSM for the protection, only digital signature protection is currently available, not HMAC.

Integrity Protected Security Audit

The IntegrityProtectedDevice security audit implementation stores the log entries to the database and relies on the Database Integrity Protection in EJBCA for protecting the authenticity of the log events. Each log event is given an identifier consisting of a "nodeId" and a "sequenceNumber" that is part of the integrity protected data. The nodeId is configurable and must be unique for each JVM in a cluster with shared database access. The "sequenceNumber" is unique per nodeId and starts with 0. The sequenceNumber for a node (JVM) is read when the first log entry is written to this device and then kept in memory for duration of the JVM's lifetime.

The security of this implementation relies on:

• Database integrity protection token.
• The sequence number in memory for each node (JVM).

Using a sequence number will prevent that a single log entry is removed without detection being possible. Keeping the sequence number in the JVM will prevent that all log entries up to a previous point in time is removed without detection being possible.

This implementation cannot detect:

• Removal of all the latest log entries for a node up to a previous point in time if the node's JVM is not running.
• Forged log entries if the database integrity protection token is compromised.

The motivation for this design is that each node (JVM) does not have to wait for other nodes in a (shared database only) cluster. Internally in each JVM the only locking between threads happens when the sequence number is atomically updated. This will allow horizontal scaling to a very high degree without any JVM-to-JVM communication when the shared database supports row-locking.

When using this implementation, the integrity of each fetched log entry is always validated when loaded from the database, but full validation with checks for missing sequenceNumbers are not performed.

Configuration of integrity protected log device is in the files:

conf/cesecore.properties
conf/databaseprotection.properties

Using the Local Database CLI tool, you can verify a whole table and the tool indicates if a row in the table cannot be verified. For the AuditRecordData table. it will also be checked with the sequence number that no row is missing. For more information on the Local Database CLI tool, see Command Line Interfaces.

Repairing Sequence Gaps

The audit log sequence number per node is a counter. Whenever there is a need to write an audit log, the counter is fetched and incremented (as an atomic operation). If the write to the database fails:

• The counter will not be decreased (since multiple threads might be operating on it there is no way of knowing what the missing entry is).
• There will be a sequence gap in the log.

This could also happen if an attacker has removed certain entries in the database to cover his or her tracks. Since failing to write audit log to the database will roll back the transaction that initiated the security event, the security event will not be performed. Most likely you could correlate this with server logs from the time this happen to find that the database was unavailable for technical reasons.

For example, a MariaDB+Galera cluster that is in the process of reforming a quorum could experience a few failed transactions before being operational again.

There is currently no user-friendly way to "repair" such sequence, but it should be documented why it happened (or rather that the cause was technical and not malicious).

A non-user-friendly way to repair sequence gaps follows this principle:

• In a copy of the database, delete all records except 1 (or as many as you have holes)
• Update the record(s), using sql to change sequenceNumber to the ones missing, and primaryKey to something unique
• Run ejbca-db-cli export from the copy database. This will result in a dump with only these records, matching the holes in the original database
• Run ejbca-db-cli import to import this dump in the original database. This will insert the missing sequenceNumbers, using databaseprotection as configured in your ejbca-db-cli

External Log Signing 

Signing logs as they are being archived (rotated) can be done in several ways, for example:

• Logs are sent to syslog and the syslog rotation can sign files when they are rotated, using OpenSSL or a Time Stamp Server (TSA).

Log Signing using Logrotate and OpenSSL

Save and enable the below logrotate configuration for syslog, normally stored as the file syslog in the directory /etc/logrotate.d. If a previous syslog configuration exists, remove it and merge changes if required. Syslog is normally not rotated in most Linux distributions.

After enabling the configuration, ensure that logrotate is run as often as you require. Normally logrotate is run every night and to run more frequently for syslog, add a crontab entry to run logrotate hourly for example. Call logrotate with the specified configuration file logrotate syslog.conf.

/var/log/syslog {
  rotate 5
  postrotate
  /usr/bin/killall -HUP syslogd
  endscript
  lastaction
  OPENSSL=/usr/bin/openssl
  LOGFILE=/var/log/syslog
  FILE="$LOGFILE.`date +%F.%H:%M:%S`.log"
  SIGNATUREFILE="$FILE.sign"
  cp $LOGFILE.1 "$FILE"
  PRIVATEKEY="/etc/logsigner/qc1.priv"
  $OPENSSL dgst -sign $PRIVATEKEY -sha1 $FILE > $SIGNATUREFILE
  endscript
  }

Note that when stored in logrotate.d, the syslog configuration will be called when logrotate is normally run. qc1.priv is the private key used for signing the logfile.

A simple shell script below for signing and verifying using OpenSSL, as well as to convert a pkcs12 file to the public key and private key files that OpenSSL uses.

To verify, you can issue the command: log-sign.sh verify qc1.pub logfile.log lofile.log.sign.

#!/bin/bash

# Openssl command for signing and verifying
#openssl dgst -sign superadmin.key -sha1 test.txt > test.txt.sign
#openssl dgst -verify superadmin.pubkey -signature test.txt.sign -sha1 test.txt

# Openssl command to convert a p12 file to cert and key files in pem
# First cert:
#openssl pkcs12 -in qc1.p12 -nodes -nokeys -clcerts -out qc1.pem
# Then public key
#openssl x509 -in qc1.pem -pubkey -noout > qc1.pub
# Then private key:
#openssl pkcs12 -in qc1.p12 -nodes -nocerts -out qc1.priv

OPENSSL=/usr/bin/openssl
SCRIPTNAME=`basename $0`
OPTION=$1
DATE=`date +"%Y-%m-%d"`

if [ "$OPTION" = "sign" ]; then
 PRIVATEKEY="$2"
 FILE="$3"
 SIGNATUREFILE="$4"
 $OPENSSL dgst -sign $PRIVATEKEY -sha1 $FILE > $SIGNATUREFILE

 exit 0

elif [ "$OPTION" = "verify" ]; then
 PUBLICKEY="$2"
 FILE="$3"
 SIGNATUREFILE="$4"

 $OPENSSL dgst -verify $PUBLICKEY -signature $SIGNATUREFILE -sha1 $FILE
 exit 0

else
 echo "Usage:"
 echo "To sign files you have to edit the script and add the files you want signed."
 echo "$SCRIPTNAME sign   "
 echo "$SCRIPTNAME verify   "
 exit 0
fi

Log Signing using Logrotate and TSA

The same approach is taken for timestamping using a Time Stamp Authority (TSA) server instead of using OpenSSL. The following example uses the Client CLI SignClient and TSA from SignServer.

The new logrotate configuration:

/var/log/syslog {
  rotate 5
  postrotate
  /usr/bin/killall -HUP syslogd
  endscript
  lastaction
  TSACLIENTDIR=/opt/signserver
  LOGFILE=/var/log/syslog
  FILE="$LOGFILE.`date +%F.%H:%M:%S`.log"
  SIGNATUREFILE="$FILE.sign"
  cp $LOGFILE.1 "$FILE"
  $TSACLIENTDIR/bin/signclient timestamp -url "http://127.0.0.1:8080/signserver/tsa?signerId=1" -infile $FILE -outrep $SIGNATUREFILE -base64
  endscript
}

Replace the ip-address and port (127.0.0.1:8080) with the ip and port of your actual TSA and the new shell script for verification (or manual signing), log-sign-tsa.sh:

#!/bin/bash
 TSACLIENTDIR=/opt/signserver
 SCRIPTNAME=`basename $0`
 OPTION=$1
 DATE=`date +"%Y-%m-%d"`
 if [ "$OPTION" = "sign" ]; then
 FILE="$2"
 SIGNATUREFILE="$3"
 TSAURL="$4"
 $TSACLIENTDIR/bin/signclient timestamp -url $TSAURL -infile $FILE -outrep $SIGNATUREFILE -base64
 exit 0
elif [ "$OPTION" = "verify" ]; then
 PUBLICKEY="$2"
 FILE="$3"
 SIGNATUREFILE="$4"
 $TSACLIENTDIR/bin/signclient timestamp -verify -inrep $SIGNATUREFILE -signerfile $PUBLICKEY
 sha1sum $FILE
 exit 0
else
 echo "Usage:"
 echo "To sign files you have to edit the script and add the files you want signed" 
 echo "$SCRIPTNAME sign"
 echo "$SCRIPTNAME verify"
 exit 0
fi

When verifying the time stamp token, the sha1 hash from the time stamp token (signed) and the calculated sha1 hash of the file is printed. You must compare them manually.

Database Privileges

To allow JBoss to create the required database tables during the EJBCA installation process, the EJBCA database user requires CREATE TABLE privileges. During upgrades, EJCBA needs CREATE and ALTER TABLE privileges in addition to SELECT, UPDATE, INSERT, and DELETE privileges.)

After EJCBA is installed, only regular SELECT, UPDATE, INSERT and DELETE commands are needed during normal operations. The table LogEntryData will only be used with SELECT and INSERT.

Instead of changing the privileges of the EJBCA user, it is recommended to have the following two users:

• ejbca: Used for regular operations.
• ejbca-admin: For installation and upgrades, EJBCA is re-deployed with ejbca-admin configured in conf/database.properties.

The script doc/howto/mysql-privileges.sh creates an SQL script, run to limit privileges on a MySQL database. The script sets restricted privileges for every table in the EJBCA database. See the script for in-line documentation.

Denial of Service

Due to Large Data Packages or Slow Connections

There is no way to limit the data packages reaching JBoss through a HTTP request from within JBoss. This is due to the way the TCP protocol works. The best way to avoid this kind of DOS attacks is to use a firewall or proxy that can limit the size of request and configure it properly. For example, mod_reqtimeout can be used in an APache proxy.

SSL Connections in MariaDB

For information on usage of SSL connection for the database connection, refer to MariaDB documentation on Secure Connections.

Other Precautions

Switching to Production Mode

Switching to production mode (default) by setting ejbca.productionmode in conf/ejbca.properties will prevent ant from starting JUnit tests and deploying the CA build on an OCSP responder and vice versa.

Database Transaction Logs in MySQL

For more information, refer to MySQL Reference Manual - The Binary Log.

System Accounting in Linux

See your distribution for details about the package and or refer to general information on Linux Security - User, System, and Process Accounting.