Staged setup engine (AegirSetup*)

AegirSetup{A,B,C,M}.sh.txt in aegir/scripts/ are the four staged scripts that build an Aegir instance on a BOA box. A, B and C are the Satellite chain, driven by OCTOPUS.sh.txt through satellite.sh.inc; M is the Master installer, driven by BARRACUDA.sh.txt through master.sh.inc. They exist as separate scripts for one reason: privilege boundaries. Each stage runs as a different user via su -s /bin/bash - <user>, which wipes the environment — so every stage re-bootstraps itself from a settings snapshot on disk, and every failure travels through marker files, not exit codes. If you edit any of the four scripts, or the _satellite_* / _hostmaster_* / _provision_* functions they call, this page is the execution model your change runs inside.

Line counts at current HEAD: A = 268, B = 221, C = 1672, M = 167. C is the only stage that carries its own function library; A, B and M are thin drivers over lib/functions/.

The user split is enforced in-script with id -u guards that touch the FAIL marker and exit: A refuses to run as non-root (AegirSetupA.sh.txt:188-197); B, C and M refuse to run as root (AegirSetupB.sh.txt:186-195, AegirSetupC.sh.txt:201-206, AegirSetupM.sh.txt:113-118).

The .sh.txt mirror/fetch convention

The staged scripts are never fetched individually and never renamed to .sh. The .sh.txt suffix is a mirror-serving convention — the BOA mirror cluster serves them as plain text — and the scripts execute straight out of the staged build tree via explicit bash <path> <arg>:

1. The octopus wrapper (aegir/tools/bin/octopus) resolves the tree from the verb (up-dev|up-pro|up-lts → _tRee), sets _rgUrl="https://files.boa.io/versions/${_tRee}/boa" (octopus:1149) and curls two files into /var/backups/: OCTOPUS.sh.txt and lib/settings/octopus.sh.cnf (octopus:1154-1155). Both fetches use the standard flag set _crlGet="-L --max-redirs 3 -s --fail --retry 9 --retry-delay 9 -A iCab" — the iCab UA is the fetch identity the mirror cluster expects (see the serial & fetch pipeline).
2. OCTOPUS.sh.txt sources its settings snapshot, then _download_helpers_libs (defined in the settings file itself, lib/settings/octopus.sh.cnf:1090) stages the whole source tree at _bldPth="/opt/tmp/boa", branching on _DL_MODE (:1096-1111): the default BATCH calls _download_boa_code (:1072), which pulls boa.tar.gz via _get_dev_ext; GIT mode runs git clone --branch ${_BRANCH_BOA} ${_BOA_REPO_GIT_URL}/boa.git ${_bldPth} (:1113, _BOA_REPO_NAME=boa). The four AegirSetup* scripts arrive as part of that tree, at /opt/tmp/boa/aegir/scripts/.
3. _satellite_make() does chmod 700 ${_bldPth}/aegir/scripts/* (satellite.sh.inc:4375) and runs stage A with bash — no rename, no copy to $PATH. _satellite_prepare_child_scripts() merely chowns B and C to the instance user so the su'd children may read them (satellite.sh.inc:2620-2631).

Inside the stages, all mirror fetches derive from two exported bases (identical exports in A, B and C — AegirSetupA.sh.txt:103-104, AegirSetupB.sh.txt:103-104, AegirSetupC.sh.txt:103-104; both https):

export _urlDev="https://${_USE_MIR}/dev"
export _urlHmr="https://${_USE_MIR}/versions/${_tRee}/boa/aegir"

C additionally fetches build artefacts from two dedicated mirror paths with its own retry loop (10 attempts, 9 s apart): _get_core_ext → https://${_USE_MIR}/core/<tarball> (AegirSetupC.sh.txt:355-385) and _get_distro_ext → https://${_USE_MIR}/distro/<tarball> (:389-419). A failed download after all attempts is an OOPS: + return 1, not a fatal — platform builds degrade per-item rather than aborting the stage.

Where staging state lives during a run

There is no environment inheritance across stages — su - resets it. Five locations carry all cross-stage state:

• /var/backups/octopus.sh.cnf.${_USER} — the settings snapshot and the only variable bus for A, B and C. The wrapper copies the fetched octopus.sh.cnf to the per-instance name (octopus:538-544, :616-622) and seds run-mode toggles into it (_HM_ONLY, _PLATFORMS_ONLY, _DEBUG_MODE, autopilot pins). _satellite_make() then appends the entire computed runtime state as export lines — _AEGIR_VERSION, _ALL_DISTRO, _BRANCH_*, all _DRUPAL*/_SMALLCORE* version pins, _PHP_*, _THIS_DB_HOST, _USE_MIR, _USER, _xSrl, _X_VERSION, … — immediately before launching A (satellite.sh.inc:4196-4343). Every stage re-sources this file on entry, keyed by its positional argument: _tocIncO="octopus.sh.cnf.$1" with fallback to the unsuffixed file, panic exit if neither exists (AegirSetupA.sh.txt:49,73-81; identical in B and C). A variable a child stage needs must be added to the _satellite_make() append block — anything else is invisible past the su.
• /var/backups/barracuda.sh.cnf — the equivalent bus for M (no per-instance suffix; there is one Master). _aegir_master_install_upgrade() appends its state lines (_AEGIR_VERSION, _ESC_PASS, _THIS_DB_HOST, _USE_MIR, …) before the su (master.sh.inc:1199-1219); AegirSetupM.sh.txt:66-67 sources it or panics.
• /opt/tmp/status-<Stage>-{OK,FAIL} markers — the failure channel. Every guard and _panic_exit() in a stage touches /opt/tmp/status-AegirSetup<X>-FAIL; the parent waits, tests the marker, and cascades: B-FAIL → A-FAIL (satellite.sh.inc:3222-3225), C-FAIL → A-FAIL (:3416-3420), A-FAIL → status-Octopus-FAIL (:4386-4390), M-FAIL → status-Barracuda-FAIL (master.sh.inc:1255-1260). Success markers are touched by the last function of each chain: status-AegirSetupA-OK at the end of _satellite_display_url_finalize() (satellite.sh.inc:5450), status-AegirSetupB-OK at the end of _satellite_child_b_redis_enable_finalize() (:6399). An abort path that exits non-zero without the touch is invisible to the orchestrator.
• /opt/tmp/make_local and /opt/tmp/files — build inputs. _satellite_download_for_local_build() stages the Hostmaster, Hosting and Drupal sources for the local build at /opt/tmp/make_local (make_local.tar.gz in BATCH mode; satellite.sh.inc:3125); _SRCDIR=/opt/tmp/files is the tarball scratch dir every stage sets (A/B assign, C exports).
• ${_ROOT} markers (_ROOT=/data/disk/${_USER}) — INIT-vs-UPGRADE witnesses and run history: aegir.sh, log/setupmail.txt, log/date-init.txt / log/date-upgrade-<core>.txt and log/octopus_log.txt (written by _satellite_log_update(), satellite.sh.inc:5273), log/email.txt, log/option.txt, log/cores.txt (seeded by _satellite_make() after A succeeds, satellite.sh.inc:4393-4400).

_STATUS (INIT vs UPGRADE) is re-derived per stage from the filesystem, never passed down: A and B flip to UPGRADE when ${_ROOT}/aegir.sh exists (AegirSetupA.sh.txt:176-182, AegirSetupB.sh.txt:177-180); C keys off the setupmail log markers instead (AegirSetupC.sh.txt:188-193). A's UPGRADE path also deletes legacy ${_ROOT}/AegirSetup{B,C}.sh.txt* copies left by pre-build-tree BOA generations (AegirSetupA.sh.txt:179-180) — mirrored in _satellite_child_scripts_cleanup() (satellite.sh.inc:3466-3476).

The shared prologue (A ≡ B ≈ C)

The first ~170 lines of A, B and C are deliberately near-identical — do not attribute prologue behaviour to any single stage. Each one:

1. Defines the helper variables: _bldPth, _crlGet/_wgetGet, _filIncO="octopus.sh.cnf", _tocIncO="${_filIncO}.$1", _libFnc="${_bldPth}/lib/functions", _vBs="/var/backups", export _tRee=dev (re-pinned per published tree copy) (AegirSetupA.sh.txt:41-52).
2. Sources the per-instance settings snapshot, falling back to the unsuffixed file; _panic_exit() touches the stage's FAIL marker on a miss (:58-81).
3. Dumps env when _DEBUG_MODE=YES (:87-97).
4. Exports _urlDev/_urlHmr and sources _FL="helper satellite" from ${_libFnc} — panic per missing include (:103-114). This is the Satellite-side pairing; M sources _FL="helper master" instead (AegirSetupM.sh.txt:73-77). The master/satellite libraries are parallel copies that are never co-loaded; a fix in one is mirrored by hand into the other.
5. Resolves _THIS_DB_HOST=FQDN to the box hostname, sets _ROOT=/data/disk/${_USER}, _HM_ROOT, _DISTRO_ROOT, and walks the PHP-CLI ladder — _PHP_CLI_VERSION → first existing /opt/php{85..56}/bin/php → _T_CLI — then pins _DRUSHCMD="${_T_CLI}/php ${_ROOT}/tools/drush/drush.php" and prepends _T_CLI to PATH (:120-170). C exports these where A and B assign them locally, because C's platform functions run in subshell-heavy loops.

One prologue oddity to leave alone: the _OS_CODE = excalibur branch assigns _DB_SERVER=Percona in both arms (AegirSetupA.sh.txt:208-212, OCTOPUS.sh.txt:248-253) — a placeholder axis for a per-OS DB split that currently has one value. The boa info | grep -c ${_DB_SERVER} test beside it counts Percona mentions in live boa info output as a "full BOA stack present" heuristic (with /usr/sbin/csf existence as the second signal) and only decides whether _satellite_download_for_local_build may be skipped when /opt/tmp/make_local is already complete (AegirSetupA.sh.txt:213-221).

Stage A — root-side driver

Entry: OCTOPUS.sh.txt runs its check chain (_satellite_check_php_compatibility … _satellite_detect_vm_family, OCTOPUS.sh.txt:315-323), then _satellite_cnf (writes /root/.<user>.octopus.cnf on first run, satellite.sh.inc:713), _satellite_if_init_or_upgrade, _satellite_checkpoint, _satellite_pre_cleanup, _satellite_make, _satellite_post_cleanup (OCTOPUS.sh.txt:336-343). _satellite_make (satellite.sh.inc:4180) exports the engine serial _xSrl (:4204) — the build-generation stamp that keys the one-time _CORE dot-dir cleanup (dot-files-ctrl-${_xSrl}-${_X_VERSION}, so a serial bump re-triggers it), appends the state block to the settings snapshot, installs the per-instance task runner (run-xdrago → /var/xdrago/run-${_USER}, :4371-4374), forces the webserver up, and launches A as root.

A's body is a linear chain of _satellite_* procedures (AegirSetupA.sh.txt:203-259):

Stage B — instance build (non-root)

B's body is exactly twelve calls (AegirSetupB.sh.txt:201-212), all defined in satellite.sh.inc:

_satellite_child_b_prepare_dirs_permissions   # :5454
_satellite_child_b_install_drush              # :5514 — Drush into ${_ROOT}/tools/drush
_satellite_child_b_drush_xts_cleanup
_satellite_child_b_drush_xts_install          # drush extensions (registry_rebuild, safe_cache_form_clear, …)
_satellite_child_b_drush_test                 # :5668 — aborts with B-FAIL if drush is broken
_satellite_child_b_aegir_build                # :5687 — the core step, below
_satellite_child_b_aegir_health_check         # :5972
_satellite_child_b_letsencrypt                # :6003
_satellite_child_b_aegir_ui_enhance
_satellite_child_b_vhosts_hotfix
_satellite_child_b_symlink_global_inc         # :6363
_satellite_child_b_redis_enable_finalize      # :6374 — touches status-AegirSetupB-OK

_satellite_child_b_aegir_build is where the Satellite becomes an Aegir: on INIT it writes the instance DB credential PHP snippet (${_ROOT}/.${_USER}.pass.php), resolves the DB host topology (localhost / 127.0.0.1 / ProxySQL / remote FQDN → _USE_DB_USER aegir_root), builds the Provision backend, and runs the frontend install: ${_DRUSHCMD} hostmaster-install ${_DOMAIN} … (satellite.sh.inc:5730-5749). On UPGRADE the same function migrates the existing frontend instead. Several interior guards touch status-AegirSetupB-FAIL directly (satellite.sh.inc:1634, :5682, :5865, :5998) — B can fail from deep inside a build step, and A still sees it through the marker.

Stage C — the platform factory

C is the only stage with a real in-file function library (AegirSetupC.sh.txt:253-1632), and the only one whose work is driven by data tables rather than a call chain.

Shared-core selection (:211-243): the same _HOT_SAUCE / _USE_CURRENT logic as A's check resolves _CORE/_THIS_CORE to a numbered generation under /data/all/ (001, 002, …), with the D6/D7 pristine cores at _D6_CORE_DIR/_D7_CORE_DIR = /data/all/000/core/<version>. The per-instance platform root is _pthDst="${_ROOT}/distro/${_THIS_CORE}", chmod 0711. _ALLOW_ALL=NO plus _D_8_ALLOW=NO when _CLIENT_CORES < 1 (:246-250) — a zero-core instance builds no platforms at all.

The distro catalogue (:855-1126): eight parallel declare -A _distros_* maps keyed by 30 three-letter codes — _distros_names (:855), _versions (:889), _drupal_cores (:923), _profiles_names (:957), _profiles_paths (:991), _web_dirs (:1025), _urls (:1059), _php_versions (:1093). Core-only codes (DL6/DL7/DL9, DX0–DX6 for D10.x, DE1–DE3 for D11.x) pin their versions to the _SMALLCORE*_V variables from the settings snapshot; distro codes (CMS, CK1–CK3, THR, VBX, …) pin explicit release versions. Adding a platform = adding one key to all eight maps — there is no other registration point.

Dispatch — _d_dist_loop (:1611-1632) iterates _distros_names and routes each code by pattern:

• DL* | DX* | DE* → _d_core_platform_install (:1478) — builds the dev/stage/prod triad of plain-core platforms (_DRUPAL*_D/_S/_P names).
• CK1 | UC6 | UC7 → _d_dist_custom_platform_install (:1390) — legacy distros with bespoke build paths (_commerce_7_2_install, _ubercart6_install, _ubercart7_install).
• everything else → _d_dist_platform_install (:1423) — the generic path: gated on _ALLOW_ALL=YES and _PLATFORMS_LIST containing ALL or the code (:1452-1454), prompts per distro (_prompt_yes_no, auto-yes on autopilot), fetches the pre-built platform tarball <profile>-<version>-<core>.tar.gz via _get_distro_ext, then registers it.

_PLATFORMS_LIST originates in /root/.<user>.octopus.cnf; on upgrades it can be overridden per instance by the operator control file ${_ROOT}/static/control/platforms.info, which is read into the variable before C runs (satellite.sh.inc:1204-1205). Note the loop iterates an associative array — build order is not the declaration order.

Registration — _save_verify_this_platform (:289-351) is the frontend hook: for a built platform directory that has no drushrc.php yet, it cds into the Hostmaster site dir (_THIS_HM, resolved by _prepare_for_save_verify_platforms from .drush/hostmaster.alias.drushrc.php, :259-285) and runs

${_DRUSHCMD} php-script make_platform "<description>" <profile> <platform-path>

make_platform.php is copied in from /opt/tmp/boa/aegir/helpers/make_platform.php.txt (:277-279); it creates the platform node in the Hostmaster frontend, which is what makes hosting tasks and site installs possible on the new codebase.

Core preparation — _prepare_drupal6_core (:572) and _prepare_drupal7_core (:602) build the pristine shared cores once per version: fetch <core>.tar.gz from the mirror core/ path, normalize permissions (0755 dirs / 0644 files via _fix_dirs_files, :423), apply the BOA taxonomy patches from _pthPch=/opt/tmp/boa/aegir/patches, and symlink the shared contrib layers (o_contrib / o_contrib_seven from _CORE) into the core tree. Supporting cast: _remove_default_core_seven_profiles (:690), _init_this_distro_root (:698), _upgrade_contrib_less (:714), _create_drupal_basic_version (:789) and _create_drupal_core_basic (:806), which maps the version codes onto the right builder.

Main flow (:1634-1664): _prepare_drupal6_core → _prepare_drupal7_core → _prepare_for_save_verify_platforms → _d_dist_loop → cleanup (strips scripts/, *.make, tarballs from /data/all/* trees and removes the make_*.php helpers from the Hostmaster site dir).

Stage M — the Master Aegir

M belongs to the Barracuda run, not to Octopus. _aegir_master_install_upgrade() (master.sh.inc:981) appends the master state block to /var/backups/barracuda.sh.cnf (:1199-1219), stages the local build (_master_download_for_local_build), then launches M as the aegir user with the frontend domain and drush flags as arguments:

su -s /bin/bash - aegir -c "/bin/bash ${_bldPth}/aegir/scripts/AegirSetupM.sh.txt ${_THIS_FRONT} \
  --http_service_type='nginx' \
  --aegir_db_host='${_THIS_DB_HOST}' \
  --client_email='${_MY_EMAIL}' -y"

Output goes to /var/log/boa/aegir_install.log, deliberately chmod 0600 — the log captures the Aegir one-time-login URL, and a world-readable copy would let a local tenant race the admin to redeem it (master.sh.inc:1231-1237).

M itself (AegirSetupM.sh.txt) sources barracuda.sh.cnf (:66-67) and _FL="helper master" (:73-77), sets _ROOT="${HOME}" (= /var/aegir) and a short PHP ladder (8.4 / 8.5 / 8.3 only — the Master runs on current CLI versions, :86-95), then:

1. _hostmaster_dr_up + _provision_backend_up (:124-125, both in master.sh.inc) — Drush and the Provision backend into /var/aegir.
2. Drush sanity gate: ${_DRUSHCMD} help | grep "^ provision-install" — FAIL marker + exit if Provision's commands are not visible (:127-136).
3. Pins the chosen mirror into the build recipe: sed -i "s/files.boa.io/${_USE_MIR}/g" ~/.drush/sys/provision/aegir.make (:138).
4. ${_DRUSHCMD} hostmaster-install --aegir_host=… --aegir_db_user=… --aegir_db_pass=… --aegir_root=${_ROOT} --root=${_ROOT}/hostmaster-${_AEGIR_VERSION} --version=${_AEGIR_VERSION} $@ (:147-153) — $@ forwards the wrapper's arguments, so the frontend domain rides in positionally and -y/-d propagate.
5. Writes the master DB password to /var/aegir/backups/system/.${_AEGIR_DB_USER}.pass.txt, 0600 in a 0700 dir (:155-159).

Debugging a staged run

All four stages gate their tracing on _DEBUG_MODE=YES (env dumps in the prologue, PROC:/DEBUG: lines in every _satellite_* function via _debug_proc). The switch is file-based:

touch /root/.debug-boa-installer.cnf        # barracuda + octopus chains
touch /root/.debug-octopus-installer.cnf    # octopus chain only
touch /root/.debug-barracuda-installer.cnf  # barracuda / BOA.sh.txt (BOA.sh.txt:220)

The octopus wrapper checks the first two and seds _DEBUG_MODE=YES into the staged per-instance settings copy (octopus:554-556, :630-633), which is how the flag survives the su boundary into A, B and C. On the dev tree the barracuda wrapper auto-touches the boa/octopus debug files (barracuda:1814-1815). For M, debug mode additionally switches the launch to tee so the install log also streams to the terminal (master.sh.inc:1239-1243).

Stage progress is visible in the run log as the ${_STATUS} A: / ${_STATUS} B: / ${_STATUS} C: message prefixes; on a hang, the last prefix plus the /opt/tmp/status-* markers tell you which script — and with _DEBUG_MODE, which PROC: — to read.

Maintainer contract

• New cross-stage variable? Add it to the _satellite_make() append block (satellite.sh.inc:4207+) for A/B/C, or the _aegir_master_install_upgrade() block (master.sh.inc:1199+) for M. Exporting it in the parent does nothing — su - drops it.
• New abort path? Touch the stage's status-*-FAIL marker before exiting. A non-zero exit without the marker is silently treated as success by the parent.
• Prologue edits must be applied to A, B and C together (and considered for M) — the duplication is deliberate, like the master/satellite library split. Don't DRY it into an include; the prologue is what runs before includes are provably available.
• New platform = one new key in all eight _distros_* maps in C, a pre-built tarball on the mirror distro/ path matching <profile>-<version>-<core>.tar.gz, and (for core-only codes) the _SMALLCORE*_V / _DRUPAL* pins in the settings files.
• The staged scripts ride the whole-release transport — they are part of boa.tar.gz / the git tree, with no per-file fNN serial. A fix here reaches boxes at their next octopus up-* / barracuda up-* run, not on the 5-minute SKYNET tick; see the serial & fetch pipeline before choosing where a fix lands.

Related

• Hostmaster upgrade orchestrator (AegirUpgrade) — the fifth staged script: the Master frontend's upgrade lifecycle, numbered host_master/NNN roots and hostmaster-migrate.
• The serial & fetch pipeline — how the build tree, wrappers and settings files reach a box, and which transport a given change rides.
• Code style & conventions — the bash rules (_msg prefixes, _SNAKE_CASE, no _alrt in main scripts) these scripts follow.
• Variables and Commands — consolidated tables for the control variables (_HOT_SAUCE, _PLATFORMS_LIST, _DL_MODE, …) and the octopus/barracuda CLI verbs referenced above.