Troubleshooting & recovery

Root-side diagnosis and recovery for the failure modes that actually recur on a busy BOA host: stuck Aegir tasks, the APCu/Valkey/cron plugin-not-found cascade, backup-task aborts, 502/504 upstream failures, SMTP TLS rejections, resolv.conf drift after DHCP lease renewals, and getting yourself unblocked by the CSF/Nginx ban pipeline. Each page is a procedure against the live interface — the actual scripts, control files, log paths, and SQL — not a generic "have you tried turning it off and on again" walkthrough.

Diagnosis here assumes you are root on the box (or hold provider console access) and are fluent in the BOA layout: Octopus instances under /data/disk/<USER>, the SFTP-chroot mirror under /home/<USER>.ftp, the cron-driven task runner, the monitor stack under /var/xdrago, and the CSF→Nginx ban chain. The tenant-facing "how do I" framing for these same symptoms lives in the Using and Self-Hosting areas; this is the operator's recovery reference.

Pages

  • Aegir task failures — task spinning, "could not delete", .restore directories, the CiviCRM *.drush.inc block, permission-drift Verify failures, and pulling a task log off disk.
  • Recovery cycles & cache faults — the Verify/Unlock/Verify platform recovery cycle, and the three-cause APCu/Valkey/cron PluginNotFoundException diagnosis (memory starvation, CLI-drush cache poisoning, HDD I/O latency) plus the self-service graceful FPM reload.
  • SMTP TLS rejection — certificate verify failed from the shell down: cert chain, CA bundle, SECLEVEL, SNI, IPv6.
  • IP-blocked recovery — the CSF temp-ban / Nginx nginx_banned_ips.conf mirror, removal and allowlisting, console-only recovery, and the IDS successful-login exemption.
  • 502 / 504 upstream failures — Nginx-reported, but downstream-caused: FPM pool state, the per-instance shared-pool saturation model, _PHP_FPM_TIMEOUT, and the firewall-block look-alike.
  • Backup-task failures — "Could not back up sites directory": ownership drift, disk-full, .restore cruft, escaping symlinks, MyQuick/Restore incompatibility, and the mysqldump/rsync 755-mode regression.
  • Operator FAQ — non-Aegir vhosts, surviving-upgrade package and module pins, log locations, instance enumeration, and version/completion markers.
  • DNS & resolver recovery — BOA's resolv.conf ownership (chattr +i, unbound-first with public fallbacks), the nodnsupdate dhclient hook that stops DHCP lease renewals from rewriting it, and the safe dhcpfix recovery tool.

Cross-references

The subsystem internals these procedures touch are documented in their own topics: the task queue in Architecture, pool sizing in PHP-FPM performance, the ban chain in Abuse Guard and Security, the dump tools in Database, and the off-site engine in Backups.