Backup-task failures

Symptom — an Aegir Clone, Migrate, or Backup task aborts with:

Could not back up sites directory for drupal.

This is a fault at the source site, not the target: the same backup step runs first in Clone and Migrate, so a backup failure shows up across all three task types. Six distinct root causes, in rough order of frequency.

Cause 1 — permissions / ownership

The backup process must read every file under sites/<domain>/ and write the tarball. Ownership drift (Composer wrote files with the wrong owner, a manual copy left wrong perms) breaks it. Repair with the fix-drupal-* scripts — equals-form flags only — against the real platform path under /data/disk/<USER> (the --site-path target must contain settings.php or the script exits 1):

# As oN.ftp
fix-drupal-site-permissions.sh --site-path=/data/disk/<USER>/static/<platform>/sites/<domain>
fix-drupal-site-ownership.sh   --site-path=/data/disk/<USER>/static/<platform>/sites/<domain>

# Or whole-platform if the drift is broad
fix-drupal-platform-permissions.sh --root=/data/disk/<USER>/static/<platform>
fix-drupal-platform-ownership.sh   --root=/data/disk/<USER>/static/<platform>

Then re-run the failing task.

Cause 2 — disk full

df -h /data/disk/<USER>
df -h /home/<USER>.ftp

Near 100 %, the backup can't write the tarball. Common space to reclaim:

# Old backup tarballs
ls -lt /data/disk/<USER>/backups/ | head
# Old per-platform site trees
ls -lt /data/disk/<USER>/static/
# Tenant DB dumps under quota
du -sh /data/disk/<USER>/static/files/dbackup/* 2>/dev/null

A tenant can shorten DB-backup retention via /data/disk/<USER>/static/control/dBackupCycle.info. For headroom rather than cleanup, resize the data disk.

On a box with an attached data disk, migratefs frees the root partition by moving data, never destroying it: it relocates each account's /data/disk/<oN>/static/files store and the shared /data/disk/arch (SQL dumps, cluster backups) onto the single real mountpoint under /mnt and symlinks them back — on any error the real directory is left in place. The default run is a DRY plan; migratefs --apply performs the relocation (it pauses the Aegir queue via /run/boa_queue_stop.pid and drains tasks first; --grace default 15 s). Operator-only, never automated: it assumes exactly one attached mountpoint under /mnt and refuses to run when more than one is present (even with an explicit --target), and the arch relocation defers while a backup writer (duplicity, mydumper, sequential_backups, dbackup, mysql_cluster_backup) is active. Full runbook: migratefs storage relocation.

Cause 3 — stray sites/<domain>.restore from a prior failure

A previous failed Clone/Migrate may have left sites/<domain>.restore behind, and the next backup tries to include it and fails:

ls -la /data/disk/<USER>/static/<platform>/sites/ | grep '\.restore'
rm -rf /data/disk/<USER>/static/<platform>/sites/<domain>.restore

Then re-Verify and retry.

Cause 4 — symlinks pointing outside the site dir

Some modules create symlinks (Compass-generated CSS, etc.) pointing outside the site directory; the backup tar can't follow them.

# As oN.ftp — inspect, then remove links pointing outside the site tree
find /data/disk/<USER>/static/<platform>/sites/<domain>/files -type l -ls
rm /path/to/offending/symlink

Then retry the task.

Cause 5 — MyQuick mode breaks the Aegir Restore task

If the site uses MyQuick mode (per-table mydumper parallel dumps), the Aegir Restore task won't work — the tarball carries no conventional single .sql file. This is not a backup failure, but the "Restore doesn't work" symptom is often mistaken for one. Recover by hand from the split dumps under ~/backups/<site>/<timestamp>/ via myloader (see Dumps & mydumper).

Cause 6 — mysqldump: Permission denied (dump/transfer binary mode)

Distinct from Cause 1 (site-tree ownership). Here the dump or transfer binary itself is non-executable for the unprivileged aegir user, with a task-log line like:

mysqldump: Permission denied

(or rsync permission-denied during Clone/Migrate). On affected installs /usr/bin/mysqldump and /usr/bin/rsync were left mode 750 root:root, so aegir could not execute them and every Backup/Clone/Migrate failed. 5.10.1 restored mode 755. The re-apply lives in autoupboa, not in barracuda: autoupboa runs chmod 755 on both binaries whenever it executes, on the weekly auto-update cron and from periodic maintenance — and autoupboa is the orchestrator that drives barracuda as a leaf process, never the reverse.

Fix

chmod 755 /usr/bin/mysqldump /usr/bin/rsync

This is the immediate fix; re-run the failed task. On an auto-updating box the correction re-applies on the next autoupboa pass with no operator action.

autoupboa reads /root/.restrict_this_vm.cnf (a restricted-VM marker that would otherwise lock both binaries to 700), but it removes that marker first and then always takes the 755 branch — so on a box that auto-updates the binaries end up 755 regardless of whether the marker was present, and a 700 lock is never something an autoupboa run leaves behind. A bare barracuda up-<tier> system run by hand does not restore the mode — barracuda has no such logic and never calls autoupboa. If you set the mode by hand on a box that does not auto-update, re-apply the chmod 755 after any maintenance that may reset it.

Diagnostic checklist

  • Disk space (df -h).
  • Permissions/ownership fix scripts.
  • sites/<domain>.restore directory.
  • Stray symlinks in files/.
  • /usr/bin/mysqldump and /usr/bin/rsync are mode 755 (Cause 6).
  • Site Verify passes.
  • Re-attempt the failing task.

If all of the above check out and the task still fails, the cause is deeper (DB connectivity, an Aegir code-level bug) — escalate with the task log attached.

Related