Files-symlink tools — autosymlink, sudo wrapper, symlinkinfo

Three executables do the work described in the subsystem overview: autosymlink (the root worker), fix-drupal-site-symlinks.sh (the hardened sudo entry point Provision calls), and symlinkinfo (the read-only forensic query). autosymlink and symlinkinfo live in /opt/local/bin; the wrapper lives in /usr/local/bin with the rest of the fix-drupal-*.sh family. All three run as root — the per-account static store is root-managed by design.

autosymlink (and its scheduler updatesymlinks, owned by Nightly automation) predate the native subsystem as batch-only precursors; the narrow single-site mode, --force-unshare, the --batch-if-clean discipline and the safety properties below are what turned them into the subsystem's tools. Sources: aegir/tools/bin/autosymlink, aegir/tools/bin/fix-drupal-site-symlinks.sh, aegir/tools/bin/symlinkinfo.

autosymlink — the worker

Converts a site's real sites/<url>/files and sites/<url>/private directories into symlinks pointing at the owning account's store (/data/disk/<account>/static/files/<url>/{files,private}), detects and breaks accidental cross-site sharing, and reports orphaned stores. Everything it does — plans, applies, warnings, reports — goes to /var/log/boa/autosymlink.log.

Global modes

The batch gate is the state file /var/log/boa/autosymlink.state: batch refuses to run (exit 1) unless the file records _LAST_MODE=DRY with _LAST_STATUS=CLEAN, and it invalidates that status the moment it is consumed, so one clean DRY authorises exactly one batch. A DRY run flagged NOT CLEAN (any [WARN]/[ERROR] or unresolved edge case) blocks batch until the cause is fixed and DRY re-run.

Global sweeps iterate /data/disk/* accounts that carry tools/drush, skipping /data/disk/arch. The master hostmaster account is never touched — only Octopus accounts are in scope, so the master keeps plain directories.

Narrow single-site mode

autosymlink --site <url> [--account <oNNN>] [--apply] [--force-unshare]

This is what the Provision install, clone and migrate/rename hooks reach through the sudo wrapper, and the mode to use for one-off fixes:

• --site <url> scopes the run to one site. Without --apply it is a read-only per-site DRY.
• --account <oNNN> — with an explicit account the tool trusts the Drush alias alone as the "site exists" marker. That is deliberate: the alias is written by provision-save, but the nginx vhost only appears at the site's first verify, so the install hook runs before any vhost exists. Without --account, the account is auto-resolved by locating the one /data/disk/* account holding both the alias and the vhost for that site; no pair, no action.
• --apply mirrors the global discipline at site scope: it runs its own per-site clean dry-run first and applies only if that dry-run is clean — without ever reading or writing the global CLEAN-DRY state file, so a narrow run can never authorise or poison a later global batch.
• --force-unshare breaks an inherited cross-site/cross-account link even when a share.files.<site>.info control file exists — used by cloning and migrate/rename, because a freshly cloned/renamed site never opted into sharing.

Both tokens are validated fail-closed before anything runs (site ^[a-z0-9][a-z0-9._-]*$, account ^[A-Za-z0-9._-]+$, no .., no /).

A narrow --apply defers while the nightly maintenance pause marker (/etc/boa/.pause_tasks_maint.cnf) is present, so a task-time conversion never interleaves with the batch sweep on the same store; the nightly batch in turn bails while a provision/clone process runs, so the two cannot overlap. Exit codes: always 0 (done or safe-skip), except 3 — a non-fatal advisory meaning a requested clone unshare did not complete (deferred, dry-run not clean, or the copy failed) and the site may still share the source's files. That is the case behind the [ALERT] Clone unshare for <site> not completed ... manual review required log line; the recovery command is in the wrapper section below.

Safety properties

• Crash-safe conversion. The move is rc-checked: on any mv failure the real directory stays in place and no symlink is created. The only non-atomic window (between a successful move and the ln -s) is closed by self-heal on the next run: store holds the data, in-site link missing → the link is recreated. Concurrent converters are corruption-safe — the losing mv fails because the source is already gone.
• Owner-matched links. Each new symlink's own owner:group is set with chown -h --reference to its store target. On a clone the unshare's rm + ln -s is the last step with no perms-fix behind it, so without this the link would linger root:root until the next verify.
• Filesystem-aware space checks. Source and target devices are compared via df -P: same filesystem means a move is a free rename and needs no headroom; cross-filesystem moves and break-sharing copies get a real du-measured check, with cumulative per-filesystem reservations summarised at the end of the run so a planned batch cannot overcommit storage. The target device is resolved by walking up to the nearest existing ancestor — a brand-new account with no store base yet previously produced a false "Unable to determine available space" DRY failure that permanently blocked its nightly auto-convert.
• Stale stores archived, never overwritten. If a conversion finds an existing store target from a prior lifecycle of a reused site name, it is moved aside to static/files/.archived/<stamp>/<site>/ before the move — a defined, non-destructive remedy, so the narrow install/clone apply proceeds instead of skipping. Layout, accumulation alert and pruning are on Orphans & archiving.

fix-drupal-site-symlinks.sh — the privileged entry point

Provision runs unprivileged, and the store is root-managed, so the install/clone/ migrate hooks cannot call autosymlink directly. This wrapper (/usr/local/bin/fix-drupal-site-symlinks.sh, 700 root:root) is the NOPASSWD sudo entry point that lets aegir and the per-account oN users trigger the root-level conversion for one site without any path to the global modes:

sudo /usr/local/bin/fix-drupal-site-symlinks.sh --site=<url> [--account=<oNNN>] [--force-unshare]

Hardening, all fail-closed:

• The autosymlink path is hardcoded (/opt/local/bin/autosymlink); the caller can only influence which site/account is passed through, never the mode.
• Only --site=, --account=, --force-unshare and --help are accepted; any other argument exits 1.
• Both tokens are validated with the same charset allowlists as autosymlink itself (no traversal, no path separators) before the value reaches a root tool, and autosymlink re-validates them.
• It forwards only the narrow single-site --apply — it cannot be coaxed into batch or live, which operate on every account on the box.

This is also the documented manual recovery when a clone or migrate logged the [ALERT] Clone unshare ... not completed line (typically low disk at clone time): rerun sudo /usr/local/bin/fix-drupal-site-symlinks.sh --site=<clone> --force-unshare once space is available — see Site cloning.

Wiring on the box:

• Sudoers — NOPASSWD entries for aegir and each per-account user are written to /etc/sudoers.d/fix-drupal-site-symlinks (0440) during install/upgrade, alongside the rest of the fix-drupal-*.sh family (lib/functions/satellite.sh.inc, lib/functions/master.sh.inc).
• AppArmor — every PHP profile (opt.php56.bin.php through opt.php85.bin.php) carries mrix exec edges for both the wrapper and /opt/local/bin/autosymlink, so confined FPM/CLI PHP can exec them. See AppArmor.

symlinkinfo — read-only history query

Answers, for one or more sites, was it symlinked and when, was its store archived (and where), and what state is it in right now — from the autosymlink logs plus the live filesystem. It changes nothing.

symlinkinfo bar.tkm.cc                    # one site
symlinkinfo foo.tkm.cc clone.foo.tkm.cc   # several

Per site it reports:

• Symlinked — every files/private conversion, with timestamp and store target.
• State NOW — active (alias + real-docroot vhost), disabled (alias + placeholder vhost — files kept live, never archived), deleted (neither alias nor vhost: an orphan), or partial (only one of the two survives).
• Store NOW — the current store dir, and whether the in-site path is a live symlink right now.
• Archived — each archive event (name-reuse vs deleted-site orphan) with its static/files/.archived/<stamp>/… path, plus archived copies still present on disk with sizes, so they can be located or pruned if not yet purged.

It reads /var/log/boa/autosymlink.log* and autosymlink.verbose.archive.log* including rotated .gz copies, plus /data/disk/*/static/files[/.archived]. Sites are matched as whole path tokens, so foo.tkm.cc never pulls in clone.foo.tkm.cc; malformed site names are skipped fail-closed.

Troubleshooting — "a new site is not getting symlinked"

Check, in order:

1. Wrapper installed? ls -l /usr/local/bin/fix-drupal-site-symlinks.sh — expect 700 root:root. Missing means the box has not fetched it yet (see Deployment below).
2. Sudoers entry? grep -r fix-drupal-site-symlinks /etc/sudoers.d/ should list the account user (and aegir).
3. Kill-switch present? Box-wide /data/conf/disable_native_files_symlink.cnf or the account's static/control/no_native_files_symlink.info disables the subsystem on purpose — see Configuration.
4. Account in scope? Only /data/disk/<account> (Octopus) accounts are symlinked; the master hostmaster account keeps plain directories.
5. Tool ran? Look for the [native-symlink] line in the install task log and a matching entry in /var/log/boa/autosymlink.log. symlinkinfo <site> sums this up in one shot.
6. Store ownership? The store dir should be <account>:www-data; the in-site symlink itself is owner-matched to the store target by autosymlink, so it should never be left root:root.

Deployment

autosymlink, updatesymlinks and symlinkinfo reach every box through the regular SKYNET self-update: serial-gated _fetch_versioned fetches in _update_agents (mirrored with matching serials in _update_boa_tools, which runs even before BOA is installed and when _SKYNET_MODE=OFF exits before _update_agents is reached, so it must fetch them independently), landing in /opt/local/bin with chmod 700. The privileged wrapper is fetched per-file by _update_boa_tools and also copied into place with the rest of the fix-drupal-*.sh family by _aegir_bin_extra_check_fix during install/upgrade (700 root:root). If a freshly bumped tool does not appear, the box simply has not run its daily self-update yet.

Related

• Overview — store layout, event table, privilege model, kill-switches and deliberate sharing.
• Nightly automation — updatesymlinks --auto-fix / --orphan-report, the cron cadence and --debug.
• Configuration — the .barracuda.cnf toggles and every control file the tools honour.
• Orphans & archiving — .archived/ layout, the accumulation alert, operator-only pruning.
• Site cloning — the clone unshare and the [ALERT] line these tools back.
• AppArmor — the mrix exec edges for the wrapper and autosymlink.
• Task failures — the task-log side of a failed install/clone.
• Reference appendix — the auto-generated command catalogue these commands: entries feed.