On-Prem Guide to Upgrade to v3.5 (Single-Node)

These instructions apply to all upgrade paths from v3.4.x to v3.5.1. The examples in these instructions upgrade from v3.4 to v.3.5.0. However, the same steps apply for all the following upgrade paths:

v.3.4.x >> v3.5.0
v.3.4.x >> v3.5.1
v3.5.0 >> v3.5.1

Simply substitute the version-specific paths and installer package for your upgrade path in the instructions.

Prerequisites

A running AgileSec v3.4 or v3.5.0 single-node installation.
The v3.5.0 or v3.5.1 installer package.
SSH access to the node with sudo privileges.
rsync installed on the system (used by the upgrade process to sync files).

Path References

The following placeholders are used throughout this guide. Replace them with the actual paths and versions for your environment.

Placeholder	Description	Example
`<v3.4-installation-dir>`	Directory where v3.4 is currently installed.	`/home/ec2-user/agilesec-341`
`<v3.4-installer-dir>`	Directory where the v3.4 installer package is extracted.	`/home/ec2-user/installer_rhel9_3.4`
`<v3.5.0-installer-dir>`	Directory where the v3.5.0 installer package is extracted.	`/home/ec2-user/installer_rhel9_3.5.0`

Pre-Upgrade Checklist

Before starting the upgrade, verify the current installation is healthy by running ./scripts/manage.sh status and confirming all services are in Running state.

Verify rsync is installed:

BASH

rsync --version

If not installed, install it with:

BASH

sudo dnf install -y rsync

Upgrade Steps

Step 1: Disable the health check cron job

The health check cron job automatically restarts stopped services. If you have health check cron enabled, it must be disabled before stopping services to prevent restarting services during the upgrade.

BASH

crontab -l

Expected output:

JAVA

*/5 * * * * /bin/bash <v3.4-installation-dir>/scripts/health_check.sh

Comment out the health check entry by running the following:

BASH

crontab -l | sed 's|^\(\*/5 .*health_check\.sh\)$|#DISABLED# \1|' | crontab -

Verify the health check entry is commented out with a #DISABLED# marker after running the following command:

BASH

crontab -l

Expected output:

JAVA

#DISABLED# */5 * * * * /bin/bash /home/ec2-user/agilesec-341/scripts/health_check.sh

Step 2: Stop all services and perform backup of installation directory

Stop all running services on the current v3.4 installation:

BASH

cd <v3.4-installation-dir>
./scripts/manage.sh stop

Expected output:

JAVA

2026-03-27 03:37:26 [SUCCESS] opensearch-dashboards stopped successfully.
2026-03-27 03:37:26 [SUCCESS] td-agent stopped successfully.
2026-03-27 03:37:27 [SUCCESS] haproxy stopped successfully.
2026-03-27 03:37:28 [SUCCESS] webui stopped successfully.
2026-03-27 03:37:29 [SUCCESS] api stopped successfully.
2026-03-27 03:37:30 [SUCCESS] cbom stopped successfully.
2026-03-27 03:37:31 [SUCCESS] sm stopped successfully.
2026-03-27 03:37:32 [SUCCESS] analytics-manager stopped successfully.
2026-03-27 03:37:33 [SUCCESS] ingestion stopped successfully.
2026-03-27 03:37:38 [SUCCESS] scheduler stopped successfully.
2026-03-27 03:37:39 [SUCCESS] opensearch stopped successfully.
2026-03-27 03:37:40 [SUCCESS] mongodb stopped successfully.
2026-03-27 03:37:41 [SUCCESS] kafka stopped successfully.

All services should show stopped successfully.

Note: If HAProxy is configured to run on a privileged port (e.g., 443), you will need sudo to stop it: sudo ./scripts/manage.sh stop haproxy

Backup the installation directory before moving forward for easier rollback:

BASH

du -hs <v3.4-installation-dir>
cp -r <v3.4-installation-dir> <v3.4-installation-dir>.bak

Backing up is optional but highly recommended if there is enough storage.

Step 3: Generate environment configuration from the v3.4 config

Note: Pre-Upgrade Configuration Change: org_domain with Subdomains

If you are using subdomains (e.g., company.kf-agilesec.com) in the analytics_domain config value in <v3.4-installer-dir>/generate_envs/multi_node_config.conf, then you will need to make a change to the file <v3.4-installer-dir>/generate_envs/multi_node_config.conf before upgrading.

Change the value of org_domain="${analytics_domain}" to the base domain. For example, if your analytics_domain is company.kf-agilesec.com, change

CODE

org_domain="${analytics_domain}"

to

CODE

org_domain="kf-agilesec.com"

This ensures the migration tool, OpenSearch roles, and Kafka topics use the correct base domain matching your existing indices and API expectations.

The v3.5.0 or v3.5.1 installer will merge your existing v3.4 configuration into the new v3.5.0 or v3.5.1 environment file. This preserves your custom settings (organization name, domain, ports, secrets, etc.) while picking up new v3.5.0 or v3.5.1 defaults and updated version strings.

BASH

unzip -d <v3.5.0-installer-dir> <v3.5.0-installer-dir>.zip
cd <v3.5.0-installer-dir>
./generate_envs/generate_envs.sh -t single-node --old-config <v3.4-installer-dir>/generate_envs/single_node_config.conf

The --old-config flag tells the generator to:

Preserve customer-specific values from v3.4 (organization name, domain, ports, secrets, certificates, etc.)
Skip version/URL fields (uses new v3.5.0 values for sensor URLs, API versions, etc.)
Add new v3.5.0 settings with their defaults (e.g., indexing service config, Kafka replication settings)

Expected output summary:

JAVA

...
[SUCCESS] Merged 69 values from old config, 8 skipped (whitelist), 17 new defaults kept
...

The generated .env file is placed at <v3.5.0-installer-dir>/.env.

Step 4: Run upgrade precheck

Run the upgrade precheck to verify the system is ready for upgrade. This validates prerequisites, syncs certificates from the existing installation, and checks for compatibility issues.

BASH

cd <v3.5.0-installer-dir>
./install_analytics.sh upgrade-precheck -p <v3.4-installation-dir> \
  --vars <v3.4-installation-dir>/scripts/.vars

Expected output:

JAVA

Syncing certificates from installed system to installer...
Certificates synced from <v3.4-installation-dir>/certificates
Running pre-requisite checks...
Mode: upgrade

Running upgrade pre-checks...

==================================
Pre-requisite Checks Summary
==================================
Checks Passed:   13
Warnings:        1
Checks Failed:   0

The precheck should show 0 failures. A warning about "State file missing (reconstruction will be attempted)" is expected when upgrading from v3.4 and can be safely ignored.

This step also generates <v3.5.0-installer-dir>/versions.json, which contains the version manifest used by subsequent upgrade steps.

Note: If any checks fail, review the errors and resolve them before proceeding with the upgrade.

Step 5: Run upgrade plan (dry run)

Review the upgrade plan before executing. This is a dry run showing all changes to be made without modifying anything.

BASH

cd <v3.5.0-installer-dir>
./install_analytics.sh upgrade-plan -p <v3.4-installation-dir> \
  --vars <v3.4-installation-dir>/scripts/.vars --force

The --force flag is required for upgrade from legacy installations (v3.4) without a state file.

The upgrade plan output includes:

Component Upgrades - lists all components and their version changes:

BASH

Installer Version: unknown → 3.5.0
  ✓ analytics_manager: unknown → 3.5.0
  ✓ cbom: unknown → 3.5.0
  ✓ indexing: unknown → 3.5.0
  ✓ ingestion: unknown → 3.5.0
  ✓ kafka: unknown → 3.9.1
  ✓ mongodb: unknown → 7.0.30
  ✓ opensearch_dashboards: unknown → 2.19.5
  ✓ opensearch: unknown → 2.19.5
  ✓ platform_initialization: unknown → 3.5.0
  ✓ scheduler: unknown → 3.5.0
  ✓ sm: unknown → 3.5.0
  ✓ webapi: unknown → 3.5.0
  ✓ webui: unknown → 3.5.0
  ✓ nodejs: unknown → 22.21.1
  ✓ openjdk: unknown → 17.0.14+7
  ✗ fluentd: unknown → removed (no longer in node profile)

Note: unknown implies we are upgrading from legacy installations (v3.4). For 3.5.0 > 3.5.1 upgrades, replace with 3.5.0 → 3.5.1.

Configuration Changes - shows diffs for each modified config file. Key changes include:
- MongoDB: log path updated, Unix domain socket added
- OpenSearch: SAML config updated to use local metadata file, new security roles added
- HAProxy: v3 ingestion paths added, bandwidth limiting added
- API: new token expiry settings, CORS config added
- WebUI: version string updated to 3.5.0
- Indexing: new service (replaces Fluentd/td-agent)
- Kafka: configuration changes
Backup Plan - shows what will be backed up before changes are applied
Upgrade Sequence - the order of operations:
1. Create backup directory and manifest
2. Backup binaries and configuration files (changed components)
3. Upgrade database components (mongodb, opensearch, kafka) if changed
4. Upgrade microservice services if changed
5. Upgrade proxy service (haproxy) if changed.
6. Remove deprecated components (stop service, uninstall, cleanup)
7. Update installation state
8. Cleanup temp files

Review the diffs carefully, especially for files marked as USER EDITED. The upgrade will overwrite these with new templates including your preserved settings from Step 3.

Step 6: Run the upgrade

Execute the upgrade. The --no-restart flag prevents services from auto-starting after upgrade (we will start them manually in the next step). The --no-validate flag skips post-upgrade validation since services are not running yet.

BASH

cd <v3.5.0-installer-dir>
./install_analytics.sh upgrade -p <v3.4-installation-dir> \
  --vars <v3.4-installation-dir>/scripts/.vars \
  --non-interactive --force --no-restart --no-validate

Flag	Purpose
`--non-interactive`	Skip confirmation prompts
`--force`	Required for legacy v3.4 installations without state file
`-v`	Verbose output for troubleshooting
`--no-restart`	Do not auto-start services after upgrade
`--no-validate`	Skip post-upgrade validation (services are stopped)

The upgrade proceeds in tiers:

JAVA

Tier 1 (Dependencies):   nodejs, openjdk
Tier 2 (Databases):      kafka, mongodb, opensearch
Tier 3 (Microservices):  analytics_manager, cbom, indexing, ingestion,
                         opensearch_dashboards, scheduler, sm, webapi, webui
Tier 4 (Proxy):          haproxy

Expected final output:

JAVA

All tiers completed successfully
post-upgrade hook completed successfully
Installation version updated to: 3.5.0
Upgrade completed successfully

A backup of the previous installation is created at <v3.4-installation-dir>/backups/<timestamp>/ before any changes are applied.

Step 7: Start all services except ingestion, analytics-manager, scheduler, api, webui

Use the following command to start all services except ingestion, analytics-manager, scheduler, api, webui:

BASH

cd <v3.4-installation-dir>
./scripts/manage.sh start --exclude ingestion,analytics-manager,scheduler,api,webui

Verify the specific services are running:

BASH

./scripts/manage.sh status

Note: If HAProxy is configured to run on a privileged port (e.g., 443), you will need sudo to start it: sudo ./scripts/manage.sh start haproxy

Step 8: Run Metadata Migration

Note: MongoDB, OpenSearch, and Kafka must be running before running migrate.

BASH

cd <v3.5.0-installer-dir>
./install_analytics.sh migrate -p <v3.4-installation-dir> --apply-security -v

Key lines in the expected output:

JAVA

...
2026-03-29 03:45:02 [SUCCESS] config.yml applied successfully.
2026-03-29 03:45:04 [SUCCESS] roles.yml applied successfully.
2026-03-29 03:45:06 [SUCCESS] roles_mapping.yml applied successfully.
2026-03-29 03:45:06 [SUCCESS] OpenSearch security configuration applied successfully.
2026-03-29 03:45:08 [SUCCESS] Database migration completed successfully.
2026-03-29 03:45:15 [SUCCESS] API migration script completed successfully.
2026-03-29 03:46:33 [SUCCESS] v3 Kafka topics created successfully
2026-03-29 03:46:33 [SUCCESS] Kafka topics created successfully.
2026-03-29 03:46:33 [SUCCESS] Migration completed successfully.

Step 9: Start all services

Start all services and re-enable the health check cron job:

BASH

cd <v3.4-installation-dir>
./scripts/manage.sh start --enable-healthcheck

The --enable-healthcheck flag re-enables the health check cron job disabled in On-Prem Upgrade Guide: v3.4 to v3.5.0 (Single-Node).

Verify all services are running:

BASH

./scripts/manage.sh status

All services should show Running status. If any service shows Not running, start it individually:

BASH

./scripts/manage.sh start <service-name>

Step 10: Upgrade Enrichment Profile Policy to Version 3.5.0

Login to the AgileSec Platform as the default admin user.
On Left menu, navigate to: Platform Management -> Analytics Configuration -> Profiles
Click on the Create New button.
Fill in the form fields:
- Profile Name: AgileSec Enrichment Policies v3.5.0
- Description: Default Analytics app policies version 3.5.0
- Binary Location: policy.bin
- Execution Type: Scheduled
- Schedule Frequency: Hourly
Click Upload the Policy Zip file, then attach the package: 3.5.0-policy.zip (provided to you as an upgrade artifact).
Click Save to create the profile.
Navigate to: Analytics -> App Policies
Locate the newly created profile AgileSec Enrichment Policies v3.5.0.
Under the vertical 3-Ellipses menu, Click Action -> Sync to synchronize the policy.
After synchronization completes, under the vertical 3-Ellipses, click Action -> Active to make it active.

Post-Upgrade Verification

Step 11: Validate the upgrade

Login to the Web UI at https://<analytics_hostname>.<analytics_domain>:<analytics_port> using your existing admin credentials.
Verify existing data is preserved - confirm scan configurations from before the upgrade are still present in the UI. Navigate to your scan history and verify previous scan configurations are intact.
Run a new scan - perform a Network Scan to validate the upgraded platform is functioning correctly:
- Go to Sensors -> Network Scan.
- Enter an HTTPS URL to scan (e.g., https://www.google.com).
- Click Scan and wait for it to complete.
- Verify scan completes successfully with findings and risk scores.
Check the Advanced Dashboard - navigate to the OpenSearch Dashboards to confirm they are loading correctly.

If all of the above checks pass, the upgrade from v3.4 to v3.5.0 is complete.

Rollback Procedure

If post-upgrade verification reveals issues, use this procedure to restore the v3.4 installation from the backup created in Step 2.

Note: This procedure requires the backup (<v3.4-installation-dir>.bak) to have been created during Step 2.

Disable the health check cron job:

BASH

crontab -l | sed 's|^\(\*/5 .*health_check\.sh\)$|#DISABLED# \1|' | crontab -

Stop all services:

BASH

cd <v3.4-installation-dir>
./scripts/manage.sh stop

Note: If HAProxy is configured to run on a privileged port (e.g., 443), you will need sudo to stop it: sudo ./scripts/manage.sh stop haproxy.

Restore the backup. This preserves the v3.5.0 state for investigation and restores the v3.4 backup:

BASH

mv <v3.4-installation-dir> <v3.4-installation-dir>-3.5.0-upgrade-state
cp -r <v3.4-installation-dir>.bak <v3.4-installation-dir>

Start services:

BASH

cd <v3.4-installation-dir>
./scripts/manage.sh start
./scripts/manage.sh status

Note: If HAProxy is configured to run on a privileged port (e.g., 443), you will need sudo access to start it.

Re-enable health check cron:

BASH

crontab -l | sed 's|^#DISABLED# \(.*health_check\.sh\)$|\1|' | crontab -

Verify:

BASH

crontab -l

All services should now be running on v3.4. Verify by logging into the Web UI and confirming the platform is operational.