Linux Multi-Node High-Availability Deployment Topology

Multi-Node deployment technical reference detailing components, network traffic flow, and high-availability (HA) considerations.

Overview

In a multi-node deployment, AgileSec components are distributed across specialized node roles to improve scalability, performance, and availability. This topology is recommended for production and for any environment that requires high availability, larger scan volumes, higher ingestion throughput, or greater data retention than a single node can reliably support. It also enables better isolation between user-facing access, core processing, datastores, and (optionally) scan execution so that heavy workloads do not impact overall platform responsiveness.

Node roles and Components

There are currently 4 types of nodes available for AgileSec configurations:

• Frontend Nodes

• Backend Nodes

• Scan Nodes

• Coordinator Nodes

Each of these nodes takes charge of running different components of the AgileSec platform.

Frontend Nodes

Role: Terminate external traffic and provide user access.

Note: Frontend nodes are stateless for user access.

Resourcing: Minimum: 1 | Recommended for HA: 2

Frontend Nodes Components

Frontend Nodes run traffic management and user access services:

Backend Nodes

Role: Host platform services and the data plane, including storage and messaging.

Resourcing: Minimum 2; there must always be an even number of Backend Nodes due to clustering considerations.

Backend Components

Backend Nodes run platform services and data management services:

Scan Nodes

Optional, recommended for large-scale scanning

Role: Isolate scan execution from the core platform and datastores.

Description: Scan nodes are asynchronous, stateless worker nodes subscribing to Kafka topics to get scan requests, execute the scan, and publish data back to Kafka. Scan nodes only run HAProxy and Scheduler services.

Scan Node Components

Scan Nodes run the following:

Note: Scan nodes may act as a dedicated place for scanning. After installing, users may stop the scheduler service on any other nodes to free up resources.

Scan-node HA Proxy usage

Scan nodes run HA Proxy for east–west routing to Backend nodes. This provides fault tolerance by routing requests to healthy Backend services and maintaining stable internal endpoints for dependencies required during scan execution.

Dedicated scan nodes isolate resource-intensive scanning operations from core platform services and datastores. This separation prevents scan workloads from impacting user-facing responsiveness and processing pipeline throughput. Scan nodes execute scans via the Scheduler and rely on HA Proxy routing to reach required Backend services with failover. Scan nodes can be scaled horizontally to meet scanning volume and concurrency requirements.

Coordinator Nodes

Utilized for Stretch Cluster scenarios.

Role: Provide a quorum tiebreaker node for stretch clusters spanning multiple regions, availability zones, or data centers.

Resourcing: Replaces Primary Frontend Node frontend-1. Only 1 is needed in Stretch Clusters for tiebreaking.

Coordinator Node Components

A coordinator node runs only quorum-participating services:

Network traffic flow

Multi-node traffic is best described as external ingress/egress and internal (east–west) traffic. In this reference, external ingress/egress refers to traffic crossing the platform boundary (clients and target systems), and east–west refers to internal node-to-node and service-to-service traffic within the deployment network, regardless of diagram orientation. The key intent is that Frontend nodes terminate external traffic, while Backend and Scan nodes handle internal processing, persistence, and scan execution.

External traffic distribution (Load Balancer or DNS)

Client traffic can be distributed across the Frontend nodes using either an external Load Balancer in front of the Frontend tier or DNS-based load balancing (for example, multiple A/AAAA records pointing to the Frontend nodes). In both cases, all external access terminates at the Frontend HA Proxy, and either Frontend node can serve requests.

External ingress (UI, Advanced Dashboards and APIs)

All user-facing access terminates at the Frontend HA Proxy:

• UI access: User browser → Frontend HA Proxy → Web UI
  Web UI calls the platform through the same external endpoint.

• Scan API: API client → Frontend HA Proxy → Web API
  The Web API then interacts with internal services on Backend nodes as needed.

• Search API : API client → Frontend HA Proxy → OpenSearch
  Today, Search API requests are routed directly to OpenSearch (typically on Backend nodes). (If you later change this to route through Web API, you can update this flow without changing the overall topology.)

• Advanced Dashboards: User browser → Frontend HA Proxy → OpenSearch Dashboards → OpenSearch (internal)

External ingress (remote sensors → Ingestion only)

Remote sensors submit results/events to the platform through the Frontend entry point, but the destination is Ingestion on Backend nodes:

Remote sensor → Frontend HA Proxy → Ingestion service (Backend)

Internal service and pipeline traffic (east–west)

Once traffic enters the platform, internal services communicate primarily within the Backend tier:

• Web API (Frontend) → Backend services (Scheduler/Manager/Secrets Management/CBOM as applicable)

• Services → Kafka (messaging/event pipeline)

• Services → MongoDB (metadata/execution state) and OpenSearch (indexed findings/events), as applicable

• Advanced Dashboards (Frontend) → OpenSearch (Backend)

Service routing (prefer local; failover remote)

For east–west service calls, requests are routed through the local HA Proxy with a preference for local service instances when available. If a local instance is unavailable, traffic automatically fails over to healthy instances on other nodes.

Scan execution and scan egress (backend-executed or scan-node-executed)

Scans can be initiated from the Platform UI (on-demand or scheduled) or via the Scan API. In all cases, the Web API publishes scan requests/events to Kafka, and the Scheduler executes the scan by consuming scan events from Kafka. Where the Scheduler runs depends on whether Scan nodes are deployed:

• If Scan nodes are not used (scans run on Backend nodes):
  Web UI / Scan API → Web API → Kafka → Scheduler (Backend) → Sensor executes scan (egress to target systems)→ Sensor submits results/events → Ingestion (local Backend preferred; failover to other Backend Ingestion nodes)

• If Scan nodes are used (recommended at scale):
  Web UI / Scan API → Web API → Kafka → Scheduler (Scan Node) → Sensor executes scan (egress to target systems)→ Sensor submits results/events → Scan-node HA Proxy → Ingestion (Backend)

In the Scan-node model, result/event submission routes directly to Backend Ingestion via the local HA Proxy on the Scan node, so scan traffic does not need to traverse the Frontend HA Proxy.

Datastore clustering and replication traffic (Backend-heavy)

The following internal traffic is required for HA and distributed operation:

• MongoDB replication: Backend ↔ Backend (primary/secondary replication; arbiter participates if used)

• OpenSearch cluster traffic: OpenSearch nodes ↔ OpenSearch nodes (cluster membership, shard replication)

• Kafka replication and broker traffic: brokers ↔ brokers and clients (services) ↔ brokers

High Availability Considerations

For production deployments requiring high availability:

Frontend tier

• Deploy at least 2 Frontend nodes.

• Use an external Load Balancer (preferred) or DNS-based load balancing to distribute traffic.

• Terminate external access at the Frontend HA Proxy and use health checks so traffic is only sent to healthy Frontend nodes.

Backend tier

• Deploy at least 2 Backend nodes for redundancy of platform services and the data plane.

• AgileSec’s internal service routing is designed to prefer local; failover remote by default to reduce cross-node traffic while maintaining resiliency.

Scan tier (recommended at scale)

• Deploy at least 2 Scan nodes if scans are executed on the platform and scan availability is required.

• Scale Scan nodes horizontally to meet peak scan concurrency. If you choose to use auto-scaling, it must be configured and managed based on platform and infrastructure capabilities.

Datastore HA and quorum guidance

OpenSearch

• Use an odd number of cluster-manager-eligible voting nodes (recommended minimum: 3) to maintain quorum and avoid split-brain.

• If you place a cluster-manager role on a Frontend node, treat it as a coordination/voting node and ensure overall voting quorum remains 3+.

MongoDB

• Use a replica set with an odd number of voting members (recommended minimum: 3 voting members).

• Common patterns: 3 data-bearing nodes, or 2 data-bearing nodes + 1 arbiter (arbiter is voting-only and does not store data).

Kafka

• For HA, deploy at least 3 brokers and configure topic replication to tolerate a broker failure.

• Ensure controller/quorum requirements are met for your Kafka mode (for example, a 3-node controller quorum when using KRaft) to avoid loss of quorum during node failures.