Add an Upstream Load Balancer

This guide describes how to add an upstream Load Balancer to distribute external traffic across multiple Frontend Nodes.

Overview

An upstream load balancer distributes external traffic across your frontend nodes, providing a single stable entry point for the platform. You can use either a Layer 4 (TCP) or Layer 7 (Application/HTTPS) load balancer depending on your requirements.

Note: The examples in this section use port 8443 or 443, but Frontends can be configured to use different ports. Adjust the port numbers according to your configuration.

Prerequisites

Prior to configuring a Load Balancer, users will need the following:

• An existing, functional multi-node cluster with at least 2 Backend Nodes and 2 Frontend Nodes.

• Ensure Load Balancer meets firewall rules listed in Multi-Node Linux Install Prerequisites.

Layer 4 (TCP) Load Balancer

A Layer 4 load balancer operates at the TCP level, forwarding encrypted traffic as-is to the backend without inspecting or modifying it. No TLS certificate is needed on the load balancer itself. The backend's TLS certificate is presented directly to the client. This is the simpler option when you do not need URL-based routing or TLS inspection at the load balancer.

Generic Configuration Steps for AWS NLB

1. Create a target group or backend pool with your frontend node(s) registered on the frontend port (e.g., 8443 or 443).

2. Configure health checks - TCP port check (simple, confirms port is open) or HTTP/HTTPS check to /health-check (preferred, confirms HAProxy is responding).

3. Create the L4 load balancer with a TCP listener on the frontend port.

4. Forward the listener to the target group/backend pool.

5. Lock down frontend firewall rules - only allow traffic from the LB's security group or subnet, so clients cannot bypass the LB and hit HAProxy directly.

6. Enable Client IP Preservation - configure settings to allow backend servers to see the true client IP address.

7. Point DNS to the LB's endpoint.

8. Validate - log in and run a smoke test through the LB.

Key Considerations

• No TLS certificate is needed on the LB. Traffic passes through encrypted as-is to the backend.

• The backend's TLS certificate (self-signed or public CA) is presented directly to the client. The browser will see the certificate configured at HAProxy.

• TCP health checks are simpler but less informative. They only confirm the port is open, not whether the application is responding correctly. Use HTTP/HTTPS health checks to /health-check for better reliability.

• Ensure the OS-level firewall (firewalld on RHEL) is disabled or allows traffic on the frontend port.

Example: AWS Network Load Balancer with port 8443

The following steps demonstrate configuring an AWS NLB to forward TCP traffic to HAProxy on the frontend nodes. The same concepts apply to other cloud providers or on-premises Layer 4 TCP load balancers.

Prerequisites on the Frontend Nodes

1. Confirm HAProxy is listening on 0.0.0.0:8443 (or the instance's private IP on 8443).

2. Ensure each node is reachable on port 8443 from within the network.

3. Decide how you want health checks to work:

   • Easiest: TCP health check on 8443 (checks if port is open)

   • Better: HTTP/HTTPS health check to an HAProxy endpoint (checks if HAProxy is working). HAProxy frontends can respond to the /health-check endpoint.

Step 1: Create an AWS Target Group for the Frontend Nodes

In EC2 Console → Target Groups → Create target group:

1. Target type

   • Select Instances (typical for EC2) or IP (if you want to register IPs directly)

2. Protocol / Port

   • Protocol: TCP

   • Port: (e.g., 8443 or 443)

3. Health checks

   • Protocol: TCP (simple) or HTTP/HTTPS (preferred, since we have the /health-check URL)

   • Port: Traffic port (same as your frontend port)

4. Create the target group, then Register targets:

   • Add your frontend instances (or IPs)

   • After registering, check the Targets tab → Health status to ensure they become healthy

Step 2: Create the Network Load Balancer (NLB)

In EC2 Console → Load Balancers → Create load balancer → Network Load Balancer:

1. Scheme: Internet-facing (public) or Internal (private-only) based on your organizational policy and needs

2. IP address type: IPv4

3. Network mapping:

   • Select the VPC

   • Select the subnet in which your frontend VMs reside

4. Optional: Create/choose an NLB security group to allow inbound TCP traffic on your frontend port from the sources you want (0.0.0.0/0 for public, or your corporate CIDRs, etc.)

Step 3: Add the Listener and Attach the Target Group

While creating the NLB (or afterward):

1. Listener

   • Protocol: TCP

   • Port: (e.g., 8443 or 443)

2. Default action

   • Forward to the target group you created in the previous step

Step 4: Lock Down the HAProxy Instances' Security Group (Important)

On the HAProxy instances' security group, ensure inbound rules allow:

• TCP traffic on your frontend port (e.g., 8443 or 443) from the NLB security group (recommended), so clients cannot hit HAProxy directly

• The health check port (same port if using traffic-port health checks)

Step 5: Enable Client IP Preservation

Follow AWS instructions to enable client IP preservation: https://docs.aws.amazon.com/elasticloadbalancing/latest/network/edit-target-group-attributes.html.

Step 6: Validate

1. Either point your external FQDN to the NLB (recommended) or update your /etc/hosts to point to the NLB IP address for local testing.

2. Log in to https://<analytics_hostname>.:<your_frontend_port> and run a network scan as a smoke test. For smoke test execution details, see either the single-node or multi-node installation guide.

Layer 7 (Application / HTTPS) Load Balancer

A Layer 7 load balancer operates at the HTTP/HTTPS level, providing SSL termination, URL-based routing, HTTP-aware health probes, and optional WAF (Web Application Firewall) capabilities. This is recommended when you need TLS inspection, path-based routing, or advanced health checking.

When using a Layer 7 load balancer with AgileSec platform, you must decide how SSL/TLS is handled between the load balancer and the backend frontend nodes:

Recommended: End-to-end SSL: The AgileSec backend continues to run on HTTPS, and the LB re-encrypts traffic to the backend. This requires the LB to trust the backend's TLS certificate by uploading the AgileSec root CA certificate (only needed with Self-signed / private CA cert).

Generic Configuration Steps

Regardless of the cloud provider or load balancer product, the following steps are required:

1. Provision a public IP for the load balancer's frontend endpoint.

2. Obtain a TLS certificate for the LB's public-facing listener. Use a public CA certificate for production, or a self-signed certificate for testing.

3. Export the AgileSec root CA certificate (for self-signed/private cert only) from the backend. This is located at <installation-dir>/certificates/ca/agilesec-rootca-cert.pem and is needed so the LB can trust the backend's self-signed TLS certificate during the HTTPS handshake.

4. Create the Layer 7 load balancer with:

   • A frontend HTTPS listener on port 443 with the frontend TLS certificate

   • A backend pool pointing to the frontend node(s) private IP(s) on the configured frontend port (e.g., 443 or 8443)

   • Backend protocol set to HTTPS (for end-to-end SSL)

5. Upload the AgileSec root CA as a trusted backend root certificate on the LB and associate it with the backend HTTP settings.

6. Configure an HTTPS health probe to a known endpoint (e.g., /signin). Important: The probe's Host header must match the backend certificate's CN (typically <analytics_hostname>.<analytics_domain>), not the backend's IP address. A mismatch causes the SSL handshake to fail and the backend to appear unhealthy.

7. Update firewall rules to allow traffic from the LB's subnet/security group to the frontend nodes on the frontend port.

8. Open the frontend port in the OS-level firewall on each frontend node. On RHEL, disable firewalld or allow traffic for 443/tcp:

9. Point DNS to the load balancer's public IP address.

10. Validate access to the platform URL, log in, and run a smoke test.

Key Considerations

• 502 Bad Gateway: This typically means the health probe is failing. Check the backend health status on the LB dashboard or via CLI. The most common cause is a certificate CN mismatch (see step 6 above).

• Certificate CN mismatch: The LB connects to backends using their private IP address, but the backend TLS certificate has a hostname-based CN (e.g., agilesec.kf-agilesec.com). The health probe and backend HTTP settings must send the correct Host header matching the certificate CN. It should not be the IP address.

• Two-layer firewall: Both the cloud-level firewall (security groups, NSGs) AND the OS-level firewall (firewalld on RHEL) must allow traffic on the frontend port.

• Dedicated subnet: Some cloud L7 load balancers (e.g., Azure Application Gateway) require their own dedicated subnet that cannot be shared with other resources.

• Health probe endpoint: Use /signin (returns HTTP 200-399) or /health-check available on HAProxy.

Example: Azure Application Gateway with port 443

The following example concretely demonstrates the test of setting up an Azure Application Gateway (Standard_v2) with end-to-end SSL (self-signed). The same concepts apply to AWS ALB, GCP HTTPS Load Balancer, or on-premises solutions like NGINX and F5.

Variables

Set the following variables based on your Azure environment:

RESOURCE_GROUP="<resource-group>"
LOCATION="<azure-region>"
VNET_NAME="<vnet-name>"
SUBNET_APPGW="<dedicated-appgw-subnet>"
NSG_NAME="<nsg-name>"
APPGW_NAME="<appgw-name>"
VM_PRIVATE_IP="<frontend-node-private-ip>"
BACKEND_CERT_CN="<analytics_hostname>.<analytics_domain>"  # e.g., agilesec.kf-agilesec.com

Step 1: Create a public IP for the Azure Application Gateway

az network public-ip create \
  --resource-group $RESOURCE_GROUP \
  --name ${APPGW_NAME}-pip \
  --sku Standard \
  --allocation-method Static

APPGW_PIP=$(az network public-ip show --resource-group $RESOURCE_GROUP --name ${APPGW_NAME}-pip --query ipAddress -o tsv)

Step 2: Create a frontend TLS certificate (PFX)

Azure Application Gateway requires the frontend listener certificate in PFX (PKCS#12) format.

# Self-signed for testing (use a public CA cert for production):
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout appgw.key -out appgw.crt \
  -subj "/CN=$BACKEND_CERT_CN"
openssl pkcs12 -export -out appgw.pfx -inkey appgw.key -in appgw.crt -password pass:AppGwPass123

Step 3: Export the AgileSec root CA for backend trust

scp <user>@<frontend-node>:<installation-dir>/certificates/ca/agilesec-rootca-cert.pem ./backend-trusted-root.cer

Step 4: Create the Application Gateway

az network application-gateway create \
  --resource-group $RESOURCE_GROUP \
  --name $APPGW_NAME \
  --location $LOCATION \
  --vnet-name $VNET_NAME \
  --subnet $SUBNET_APPGW \
  --sku Standard_v2 \
  --capacity 1 \
  --public-ip-address ${APPGW_NAME}-pip \
  --http-settings-port 443 \
  --http-settings-protocol Https \
  --frontend-port 443 \
  --servers $VM_PRIVATE_IP \
  --cert-file appgw.pfx \
  --cert-password "AppGwPass123" \
  --priority 100

Step 5: Configure the backend trusted root certificate

# This uploads the AgileSec root CA certificate to the Application Gateway as a trusted root certificate. This tells the App GW "trust any backend certificate signed by this CA" which is needed for the end-to-end SSL handshake when the App GW connects to the backend over HTTPS. Without it, the App GW would reject the backend's self-signed/private CA certificate and return 502.
az network application-gateway root-cert create \
  --resource-group $RESOURCE_GROUP \
  --gateway-name $APPGW_NAME \
  --name agilesec-backend-ca \
  --cert-file backend-trusted-root.cer

# Update HTTP settings to use the trusted root cert
az network application-gateway http-settings update \
  --resource-group $RESOURCE_GROUP \
  --gateway-name $APPGW_NAME \
  --name appGatewayBackendHttpSettings \
  --protocol Https \
  --port 443 \
  --host-name-from-backend-pool false \
  --root-certs agilesec-backend-ca

Step 6: Configure the HTTPS health probe

The --host value must match the backend certificate's CN to avoid certificate mismatch errors.

az network application-gateway probe create \
  --resource-group $RESOURCE_GROUP \
  --gateway-name $APPGW_NAME \
  --name agilesec-health-probe \
  --protocol Https \
  --host $BACKEND_CERT_CN \
  --path "/signin" \
  --interval 30 \
  --timeout 30 \
  --threshold 3 \
  --match-status-codes "200-399"

# Associate probe with backend HTTP settings
az network application-gateway http-settings update \
  --resource-group $RESOURCE_GROUP \
  --gateway-name $APPGW_NAME \
  --name appGatewayBackendHttpSettings \
  --probe agilesec-health-probe

Step 7: Update firewall rules

# Get the App Gateway subnet CIDR
APPGW_SUBNET_CIDR=$(az network vnet subnet show \
  --resource-group $RESOURCE_GROUP \
  --vnet-name $VNET_NAME \
  --name $SUBNET_APPGW \
  --query addressPrefix -o tsv)

# Allow App GW subnet to reach frontend nodes on port 443
az network nsg rule create \
  --resource-group $RESOURCE_GROUP \
  --nsg-name $NSG_NAME \
  --name AllowAppGW \
  --priority 120 \
  --source-address-prefixes $APPGW_SUBNET_CIDR \
  --destination-port-ranges 443 \
  --access Allow --protocol Tcp --direction Inbound

# Allow Azure App GW infrastructure management traffic (required)
az network nsg rule create \
  --resource-group $RESOURCE_GROUP \
  --nsg-name $NSG_NAME \
  --name AllowAppGWInfra \
  --priority 130 \
  --source-address-prefixes GatewayManager \
  --destination-port-ranges 65200-65535 \
  --access Allow --protocol Tcp --direction Inbound

Step 8: Open port 443 in the OS firewall (if firewalld is enabled)

On each frontend node:

sudo firewall-cmd --add-port=443/tcp --permanent
sudo firewall-cmd --reload

Step 9: Validate

Check backend health:

az network application-gateway show-backend-health \
  --resource-group $RESOURCE_GROUP \
  --name $APPGW_NAME \
  --query "backendAddressPools[0].backendHttpSettingsCollection[0].servers[0]" -o json

The health field should show Healthy. Then:

1. Point your DNS A record to the Application Gateway public IP ($APPGW_PIP).

2. Access https://<analytics_hostname>.<analytics_domain> in a browser and verify the login page loads.

3. Log in and run a network scan as a smoke test. For smoke test details, see the single-node or multi-node installation guide.

4. Test sensor connectivity:

   Bash

   curl -k https://<analytics_hostname>.<analytics_domain>/v1/oauth2/token
   # Should return 405 Method Not Allowed (needs POST) — confirms ingestion routing works