Provision backend internals

Provision is the Drush-8 command package that does every server-side action Aegir takes — write vhosts, create databases, run installs, take backups, deploy tarballs. It is a Drush commandfile set, not a Drupal module at runtime: provision.info declares module identity (version=7.x-3.x, read back by provision_version(), provision.inc:1371) but nothing bootstraps it inside Drupal. On a BOA box it lives at /var/aegir/.drush/sys/provision on the Master and /data/disk/<oN>/.drush/sys/provision per Octopus satellite — cloned from omega8cc/provision.git at the branch selected by ${_BRANCH_PRN} (boa-private/lib/functions/master.sh.inc:793, satellite.sh.inc:3242-3244). The historical bare .drush/provision path is actively removed by the same code paths; never reference it.

Everything here runs as the unprivileged instance user (aegir or oN) under the omega8cc Drush 8 fork (Drush fork internals), and must stay parse- and runtime-safe on PHP 5.6 — the CLI floor on legacy servers. No ??, no scalar types, no arrow functions, no 7.x-only stdlib.

Entry point: the task round-trip

The Hostmaster frontend reaches provision through exactly one door. drush_hosting_task() (hosting/task.hosting.inc:189) re-exports the node's state as a backend context via provision-save (task.hosting.inc:208), defaults the verb to 'provision-' . $task->task_type (task.hosting.inc:223), and dispatches it with provision_backend_invoke() (task.hosting.inc:226). That helper is provision's own (provision.inc:1274) — a drush_invoke_process() wrapper with integrate + dispatch-using-alias, also used backend-side whenever one provision command needs another (e.g. deploy unlocking the old platform). Results and log lines ride the Drush backend channel back to the task node. The frontend half of this contract is on the Hosting API leaf.

Two gates run before any provision command, from provision_drush_init() (provision.drush.inc:51):

• _provision_drush_check_user() (:67) — refuses to run any provision-* command as root (PROVISION_IS_ROOT). Root work (BOA's own scripts, autosymlink) is always delegated, never done in-process.
• _provision_drush_check_load() (:80) — exit(1) when provision_load_critical() (:489) trips: load₁ above ncpus × critical_load_multiplier (default 5, :466) or a flat critical_load_threshold of 10 when the CPU count is unknown. provision_count_cpus() (:445) prefers BOA's daily-generated /data/all/cpuinfo over /proc/cpuinfo.

provision_drush_init() also seeds the context registry: it loads the invocation alias into d() as the root object, honouring the #name option so provision-save can address a context whose alias file does not exist yet (provision.drush.inc:55-56).

Repo layout: four commandfiles, per-verb orchestrators

Drush discovers four commandfiles inside the clone, and that split is the architecture:

Per-command logic lives in orchestrator files loaded by Drush's command-include rule (drush/includes/command.inc:1715,1723): for command a-b-c the parts are reversed and dot-joined, and any c.b.a.inc sitting next to a registered commandfile is include_onced at dispatch. So provision-verify pulls in platform/verify.provision.inc; provision-login-reset pulls in platform/reset.login.provision.inc; provision-backup-delete pulls in platform/delete.backup.provision.inc; hostmaster-install pulls in install.hostmaster.inc at the repo root. The same verb name can exist in several subtrees and all get included — deploy.provision.inc exists in platform/, db/ and http/, each implementing its own slice as hooks (next section).

Counts at HEAD: platform/ carries 17 *.provision.inc orchestrators plus provision_drupal.drush.inc and the platform/drupal/ engine set (38 *.inc engines + README.md); db/ and http/ carry 7 orchestrators each.

The Drush command registry

provision_drush_command() (provision.drush.inc:91) is the single registry. The file-head docblock (provision.drush.inc:2-39) is the canonical command-and-hooks enumeration — there is no api.drush.inc in this fork; that filename is an upstream-doc fiction.

Details that matter when adding or touching a command:

• provision-save merges its option table from the three context classes' static option_documentation() (provision.drush.inc:100-102) and sets allow-additional-options — unknown options are accepted and persisted, which is how extension modules smuggle their per-context properties in without touching the registry.
• Bootstrap levels are load-bearing. provision-verify and provision-delete run at DRUSH_BOOTSTRAP_DRUSH because the target may not be a bootstrappable site; the site-level bootstrap is then performed manually inside the validate hook (see verify below). provision-dlock/provision-dunlock were explicitly lowered to DRUSH with an in-callback root bootstrap, because dispatching at DRUPAL_ROOT on an unlocked D10+ platform fatals (modern typed symfony/console vs Drush 8's untyped adapter).
• Most provision-* verbs have no owner callback at all. The work happens entirely in hooks contributed by the other commandfiles — that is the pluggability model.
• provision-stats from the upstream docblock is an optional external module; it is not registered here. Do not document it as a shipped verb.

Hook choreography per command

For command provision-X, Drush 8 calls, per commandfile CF, in order: drush_CF_provision_X_validate() → drush_CF_pre_provision_X() → drush_CF_provision_X() → drush_CF_post_provision_X(), with matching *_rollback() functions unwound in reverse on failure. Every orchestrator file in this repo is just a bag of these implementations — drush_provision_drupal_pre_provision_verify() parses as commandfile provision_drupal, phase pre, command provision-verify. Third-party provision extensions hook the same chain the same way; that surface is enumerated in provision.api.php and worked through on the Provision API leaf.

verify

• drush_provision_drupal_provision_verify_validate() (platform/verify.provision.inc:13) — for site contexts, performs the bootstrap the DRUSH-level dispatch skipped: DRUSH_BOOTSTRAP_DRUPAL_SITE, write settings.php (_provision_drupal_create_settings_file(), provision_drupal.drush.inc:277), push site files (provision_drupal_push_site(), :204), then FULL bootstrap below D12 and DRUPAL_CONFIGURATION for D12+.
• drush_provision_drupal_pre_provision_verify() (:36) — server contexts: create config/, backups/, clients/ under aegir_root, seed config/includes/global.inc via Provision_Config_Global_Settings; platform and site branches follow in the same function.
• Owner callback drush_provision_verify() (provision.drush.inc:385) re-runs provision-save for the context, then d()->command_invoke('verify') — the context/service fan-out (verify_{type}_cmd on every subscribed service; the Nginx driver's server pass is http/Provision/Service/http/nginx.php:102). This is where vhosts and server configs are actually re-rendered.
• drush_provision_drupal_post_provision_verify() (:485) caches the verified state back into the site/platform drushrc.

install

All in platform/install.provision.inc: validate :17, pre :66 (rollback :74), implementation :102, post :442.

• The implementation branches on Drupal major. D8+ installs run site-install through the platform's site-local Drush (vendor/drush/drush/drush.php under the app root found by findDrupalAppRoot()), because Aegir's Drush 8 cannot drive a modern installer. The Aegir core/console patches are reverted for the installer and re-applied after — gated by the /data/conf/_no_codebase_lock_unlock.ctrl kill-switch — and a failed or degraded install re-locks before returning so a shared platform vendor is never left in the state the patched web cannot serve. A non-zero site-install exit registers PROVISION_DRUPAL_INSTALL_FAILED and returns FALSE, so the frontend marks the task failed and rollback runs.
• The post hook rebuilds caches (_provision_drupal_rebuild_d8plus(), provision_drupal.drush.inc:665, on D10+ via site-local drush cr; _provision_drupal_rebuild_caches(), :629, below), saves the packages map (provision_drupal_system_map(), :961), creates the client symlink, and includes the cron_key engine.
• BOA-specific tail: _provision_drupal_native_symlink() (provision_drupal.drush.inc:503), called at the end of the post hook, moves the new site's files/ and private/ into the per-account static store and symlinks them back — by delegating to the root autosymlink tool through a hardened wrapper, since the store is root-managed and provision is unprivileged. Scoped to /data/disk/ accounts only, warn-not-fail, with box-wide (/data/conf/disable_native_files_symlink.cnf) and per-account (static/control/no_native_files_symlink.info) kill-switches, and fail-closed token validation before anything reaches the root tool. The clone (platform/clone.provision.inc) and migrate (platform/migrate.provision.inc) paths call the same helper with force_unshare to re-home inherited stores after cross-name copies.

deploy — the engine under clone / migrate / restore

platform/deploy.provision.inc: validate :24, pre :53 (unpack the backup; rollback :149), implementation :174, post :182. Clone, migrate and restore all end up here — deploy is "take a site package, land it at this URI on this platform, run the upgrade path".

The post hook is the dangerous part on modern Drupal, and three mechanisms in it are deliberate:

• Chmod-only exec window. updatedb on D10+ runs through the site-local Drush and recompiles the service container. The container must compile in the same patched state the web serves, so the post hook never reverts the Aegir patches for it — it flips only the exec bits via _provision_drupal_chmod_drush_vnd_for_cr() (provision_drupal.drush.inc:754) and restores the at-rest 0400 posture afterwards. Compiling in the reverted state is what used to bake Drush\Log\DrushLog into the persisted container and fatal every site on the platform.
• Old-platform unlock/re-verify. Cross-platform moves unlock the source platform before updatedb and re-verify (re-lock) it after, both via provision_backend_invoke() against the old_platform option (deploy.provision.inc:209-217 and :339-343).
• Update-tracker clear. _provision_drupal_clear_update_fetch_tasks() (deploy.provision.inc:467) drops the re-imported update_fetch_task key-value collection after every deploy-updatedb path, so update.module never wedges on orphaned tracker rows. Non-fatal on every path.

When a D10+ platform has no executable site-local Drush, the post hook skips updatedb with an explicit recovery warning instead of falling back to Aegir's Drush 8 (which cannot bootstrap D10+ and would only fail noisily).

Version engines

provision_drupal_drush_engine_drupal() (provision_drupal.drush.inc:32) declares the drupal engine types clear, cron_key, deploy, import, install, packages, verify. drush_include_engine('drupal', 'install') and friends resolve to platform/drupal/<type>[_<major>].inc — 38 engine *.inc files that run inside the target site's own bootstrap, so each must be written for its Drupal major (see platform/drupal/README.md). This is where per-major divergence belongs; the orchestrators stay version-agnostic.

Finally, provision_drupal_drush_exit() (provision_drupal.drush.inc:52) persists successful option state to the site's drushrc.php after any provision-* command, unless provision_save_config is off.

The context system: d() and provision-save

Contexts are provision's data model: named server/platform/site objects persisted as Drush alias files. There is no database backend-side — the alias files under ~/.drush/ are the store, and the frontend refreshes them through provision-save before every task.

• d() (provision.context.inc:23) — the accessor. A static registry keyed by normalised alias name (provision_normalise_context_name(), provision.inc:1416). d('@name') returns (by reference) the context, creating it on first access via provision_context_factory() (provision.context.inc:113); d() returns the root object set at init; d('all') returns the whole registry. A guard refuses to run before Drush finishes bootstrapping (:41) — calling d() at file scope in a commandfile is a classic three-day bug. On creation the factory result gets method_invoke('init'), type_invoke('init'), then the provision_context_alter Drush hook (:74), the supported way to swap in a context subclass.
• Factory — the alias record (cached via provision_sitealias_get_record(), :89) or the CLI context_type option picks the type; the class is literally "Provision_Context_{$type}" (:130).
• Provision_Context (Provision/Context.php:12) — magic __get/__set back a $properties array, which is exactly what write_alias() (:240) hands to Provision_Config_Drushrc_Alias to write the alias file. is_oid($name) (:83) marks a property as a context reference: reading it returns d($value), which is how d('@site')->platform->server->aegir_root chains lazily across alias files. setProperty($field, $default) (:215) is the init-time idiom: take the CLI/alias option if present, else the default.
• Invocation plumbing — method_invoke() (:136) calls the method if it exists (via the static provision::method_invoke(), provision.inc:1449 — silent no-op otherwise) and then mirrors the call onto all subscribed services (services_invoke(), :290). type_invoke('save') → save_site() etc.; command_invoke('verify') → verify_site_cmd() etc. These name conventions are the extension points service classes implement.
• Subclasses (Provision/Context/): server.php derives backup_path, config_path, include_path, clients_path from aegir_root (:63-66) and spawns services; platform.php sets parent_key = 'server' and owns root/makefile; site.php sets parent_key = 'platform', owns uri, computes site_path = root . '/sites/' . uri, and runs provision_auto_fix_platform_root() so a Composer-managed web root is never mistaken for the platform root. Each class's static option_documentation() feeds the provision-save option table — adding a persisted property means adding it there.
• drush_provision_save() (provision.drush.inc:371) — type_invoke('save') + write_alias(), or unlink() of the alias on --delete.

Services: subscription, drivers, dispatch

The service layer decouples "a site needs a database" from "this server runs Percona":

• Declaration. hook_provision_services() — db (db/db.drush.inc:16) and http (http/http.drush.inc:24). The hook returns array('<service>' => <default type>).
• Driver spawn (server side). Provision_Context_server::load_services() (Provision/Context/server.php:76) iterates declared services and spawn_service() (:86) instantiates Provision_Service_{$service}_{$type} from the server's {$service}_service_type property — http_service_type=nginx → Provision_Service_http_nginx, db_service_type=mysql → Provision_Service_db_mysql. No type → Provision_Service_null. The type value is set by the frontend when it saves the server context.
• Subscription (consumer side). Static subscribe_<type> methods run during context init (Provision/Context.php:201): Provision_Service_db::subscribe_site() (db/Provision/Service/db.php:10) wires db_server (default @server_master) and service_subscribe('db', …); Provision_Service_http::subscribe_platform() (http/Provision/Service/http.php:36) wires web_server the same way. Context::service('db') (Provision/Context.php:270) then resolves through the subscription, else up the parent_key chain, ending at @server_master's spawned drivers.
• Base classes. Provision_Service (Provision/Service.php:10) extends Provision_ChainedState (Provision/ChainedState.php) — the ->succeed('…')->fail('…') chaining idiom shared with Provision_FileSystem, reachable as the provision_file() singleton (provision.file.inc:12). A service carries a $configs registry of config classes per context level, restart() (:276) with a driver-supplied default_restart_cmd(), and the verify_server_cmd/verify_platform_cmd/verify_site_cmd family the verify fan-out calls.
• Driver chains. DB: Provision_Service_db → db/pdo.php → db/mysql.php — MySQL/Percona is the only driver BOA runs (db/mysql/mysql_service.inc is an empty legacy shim). HTTP: http/public.php → http/nginx.php (registers Provision_Config_Nginx_{Server,Inc_Server,Site}, sets satellite_mode = 'boa' and the nginx_has_* capability properties in init_server(); http/nginx/ssl.php on top for SSL). The apache*, cluster and pack drivers are pre-fork upstream residue — retained, not exercised by BOA; do not treat their behaviour as authority.

Class loading

provision.inc:10 requires vendor/autoload.php (Composer-pinned deps — Symfony Process for provision_process(), provision.inc:1307). The Provision_* classes themselves resolve through a second, dedicated \Composer\Autoload\ClassLoader created by provision_autoload() (provision.inc:19): provision_autoload_register_prefix('Provision_', …) (:42) is called with the repo root at provision.inc:66, and again by db_provision_register_autoload() (db/db.drush.inc:55) and http_provision_register_autoload() (http/http.drush.inc:13) to add db/ and http/ as additional roots for the same prefix. That multi-root PEAR-style mapping is how Provision_Service_http_nginx resolves to http/Provision/Service/http/nginx.php. The psr-0 entry in composer.json:28-31 maps the prefix one directory too deep (Provision/Provision/…) and never matches — the runtime registration is the operative mechanism. Extensions register their own class roots the same way (the db/http commandfiles are the template).

Config render pipeline

Provision_Config (Provision/Config.php) is the template writer every generated file goes through. write() (:195): ensure directory → load_template() → process() → render_template() (:169) → atomic-ish write with $mode/$group applied. Three alter points, all plain Drush hooks:

• hook_provision_config_load_templates() / …_load_templates_alter() (Config.php:120,126) — replace the template file outright; otherwise the class walks its own inheritance chain looking for $this->template next to each class file.
• hook_provision_config_variables_alter() (:173) — mutate the variable bag just before extract() + buffered eval('?>' . $template).
• The Nginx config classes additionally append drush_command_invoke_all('provision_nginx_server_config' / 'provision_nginx_vhost_config') output into $data['extra_config'] (http/Provision/Config/Nginx/Server.php:10, Site.php:10, and the Ssl/ twins) — free-form config injection without touching templates.

The template set itself (server.tpl.php, vhost*.tpl.php, vhost_include.tpl.php) and what BOA renders into it are web-stack material, indexed from the topic chapter.

The API surface: provision.api.php

provision.api.php (577 L) is documentation-as-code: the $options recognised from drushrc (provision_backup_suffix, provision_composer_install_platforms, provision_mysqldump_suppress_gtid_restore, …) in the file head (:11-57), then every hook — hook_provision_services() (:82), hook_provision_context_alter() (:96), the Nginx/Apache render hooks (:220, :239, :260), the template/variable alters (:274-:311), the directory/permission/settings alters (:354-:407), hook_provision_deploy_options_alter() (:420), hook_provision_prepare_environment() (:458), and the DB name/user/options alters (:505-:552). If you are extending provision rather than maintaining it, start on the Provision API leaf, which pairs this surface with a working extension skeleton; the frontend counterpart hooks are on Hosting API, and the paired-module pattern is on Extending Aegir.

Codebase lock state (BOA-fork addition)

The D10/D11 "Aegir-patched platform" model adds a state machine the upstream never had, and several commands consult it:

• provision_check_codebase_status() (provision.inc:153) — sets d()->appRoot, d()->symfonyDir and the isCorePatched / isConsolePatched / isValidatorPatched sentinels by inspecting the actual tree.
• provision_drush_local_lock_state() (provision.inc:243) — derives locked | unlocked | partial | unknown purely from on-disk reality: patch sentinels plus the 0400 (locked) / 0775 (unlocked) bits on vendor/drush and the symfony/console Input/Style dirs, resolving local-vs-above vendor layouts. No marker files.
• Lock/unlock bodies at provision.inc:677 / :300; the provision-dlock/provision-dunlock commands are thin wrappers over the same canonical functions (platform/dlock.provision.inc:21-60), so the UI task and Platform Verify share one battle-tested path, and re-running in the already-target state is a clean no-op.

Supporting and legacy files

• provision.context.inc (133 L) — d(), the factory, the alias cache; included from the tail of provision.inc (:1456-1458) together with provision.service.inc (5-line compat shim) and provision.file.inc.
• parse.backend.inc — drush_provision_backend_parse(), the backend-parse callback that renders --backend JSON from stdin human-readable.
• Pre-fork residue retained in the fork but unused by BOA: the aegir*.make family (BOA builds Hostmaster from its own make chain — see Hostmaster frontend internals), upgrade.sh.txt, debian/, scripts/ci-aegir-*, Vagrantfile, provision-tests/, and the Apache config/service classes. Verify against BOA code before citing any of them; retired pieces are tracked in Discontinued features.

Related

• Aegir backend APIs — topic chapter — the task round-trip diagram this page zooms into.
• Provision API — the hook surface for extension authors, with a skeleton.
• Hosting module suite map — the frontend modules whose task nodes become these commands.
• Drush fork internals — the Drush 8 fork provision runs under, including the extension deny-filter.
• Install & staged-setup internals — how this clone lands on a box during AegirSetup.
• Commands reference and Variables reference — consolidated CLI and _VAR tables.