Relocating stores to attached storage — migratefs

migratefs moves large data off the root partition onto a separate attached filesystem and replaces each moved directory with a symlink, so nothing else on the box has to change. It relocates two things:

• each Octopus account's files store — /data/disk/<oN>/static/files — to <mount>/files/<oN>/static/files, and
• the shared archive — /data/disk/arch (SQL dumps, cluster backups) — to <mount>/files/system/arch.

It is the modern, safe replacement for the old ad-hoc migratefs.sh (service cron stop + kill -9 + mv -f), reusing BOA's proven safe-mover pattern: separate-device gate, self-healing task-queue pause, provision drain, two-pass rsync, ownership preservation, idempotent re-runs, and never-destructive failure handling — on any error the real directory is left in place and no symlink is created.

migratefs relocates only the base. Once an account's static/files sits on the attached disk, the next nightly run finishes the job automatically: autosymlink converts that account's sites into the (now cross-filesystem) store, and the backups mover relocates its backups + backup-exports onto the same filesystem. The files symlinking subsystem owns that machinery.

The tool lives at /opt/local/bin/migratefs (on root's PATH), shipped and auto-updated by BOA.sh.txt via serial-gated _fetch_versioned. Every run — DRY or apply — appends to /var/log/boa/migratefs.log.

Two hard requirements — read first

Operator-only, never automated. There is no guarantee the attached storage has enough free space for what you are about to move, so the decision and the action are 100% yours: run it by hand, in a maintenance window, after reviewing the DRY plan. It is never scheduled and never run by SKYNET. A space check exists as a safety net, but you own the call.

BOA supports exactly ONE attached mountpoint under /mnt. migratefs and the rest of BOA (notably the lshell FTP-jail allow-list built by manage_ltd_users.sh / satellite.sh.inc — see lshell & ltd users) locate the attached files mount as the single real mountpoint under /mnt. With more than one, BOA cannot tell which is the files disk and refuses to guess: migratefs exits 1 when it detects multiple mounts — even with an explicit --target, because the rest of BOA would still disagree — and the FTP-jail wiring fails closed (wires no mount path) in the same situation. Unmount the extras first.

The mount is detected by what it is, not by its name: mountpoint -q, with an st_dev-differs fallback. Any name works (e.g. /mnt/extra); there is no dot-in-the-name convention.

Usage

migratefs [--target <mount>] [--account <oN>] [--no-arch] [--apply] [--yes]
          [--grace <sec>] [--help]

Typical run:

migratefs --target /mnt/extra            # review the DRY plan first
migratefs --target /mnt/extra --apply    # then perform it

Afterwards, convert the sites into the now-attached stores (or wait for the nightly run):

autosymlink --batch-if-clean

What lands where

/data/disk/<oN>/static/files   ->  <mount>/files/<oN>/static/files
/data/disk/arch                ->  <mount>/files/system/arch

<mount>/files/<oN>/static/files is the canonical _MNT_STATIC_FILES layout the lshell FTP jail whitelists, so the relocated store stays inside the account's <oN>.ftp allow-list. Because the relocation happens at the base (static/files itself becomes the symlink), existing per-site symlinks (.../sites/<url>/files -> .../static/files/<url>/files) keep resolving through it unchanged.

Safety properties

• Separate-device gate, evaluated first. Every account/arch step confirms the target is a genuinely different filesystem before any create/move branch runs. If the attached disk is not mounted (an empty /mnt/<x> directory sits on the same device as /data/disk), the step is skipped — data is never created on or moved onto the root partition, not even by the already-symlinked branch re-creating a missing destination.
• Two-pass move for a live store. static/files is web-served, so the copy runs as a non-removing rsync -a first (the live store stays complete during the long transfer), then a fast --remove-source-files reconcile, then rmdir + ln -s. Peak usage on the target is one copy of the store; the source filesystem never grows.
• Non-destructive. The only deletions are rsync --remove-source-files (file by file, after each is copied) and empty-directory cleanup (find … -empty -delete plus a final rmdir) that fails closed on anything non-empty. The symlink is created only after the source empties cleanly; on any failure the real directory is left in place and the copied data also exists at the target.
• Idempotent. Re-running converges: an already-relocated store is a no-op, a partial move merges and completes, a fresh account with no static/files yet just gets an empty store created on the attached disk (ownership preserved, chown -h on the link).
• Interlocks. A single-instance lock; the self-healing /run/boa_queue_stop.pid queue pause (records the tool's PID — runner.sh honours it, clear.sh purges a leaked one by PID-liveness, and /run clears on reboot, so it can never freeze the queue permanently) followed by the --grace wait and a bounded drain of in-flight provision tasks — if a task is still active after grace + drain, nothing is applied that run; and, for arch, a defer while a backup writer (duplicity, mydumper, dbackup, mysql_cluster_backup, sequential_backups) is active, so a backup file is never moved mid-write.

Notes

• Run arch relocation when backups are idle. migratefs defers arch if it detects an active backup writer, but the safest window is one with no scheduled backups.
• migratefs pauses the Aegir task queue, not web traffic. For a fully quiescent move of a busy account, put the site(s) into maintenance mode first.

Relocated arch and host migration

Once /data/disk/arch (or an account's static/files) is a symlink to the attached mount, the host-migration tools xoct/xcopy (transfer shared / transfer) and xmass handle it storage-aware, per the target's disk reality — never copied as a bare (dangling) symlink; the Storage-aware transfers section of that page owns the mechanics. Before migrating a host whose arch/static/files is relocated, make sure those tools are current on both hosts: the handling requires the storage-aware xoct/xcopy/xmass that ship with this BOA release (a current barracuda up-* refreshes them automatically). Older versions copied or skipped the bare symlink and transferred no SQL dumps or cluster backups. Consumers that read a path under arch (e.g. copydbackup, mysql_cluster_backup) resolve through the symlink transparently and need nothing.