multiback operations — dcysetup, config tree, cron

The operator surface for the modern backup suite is dcysetup (install + config generation) plus the /root/.remote_backups/ tree it materialises. There is no /etc/boa/multiback.conf, no /etc/boa/backup-config.env, and no BACKUP_* variables — those do not exist. Everything lives under /root/.remote_backups/ and /root/.barracuda.cnf.

dcysetup — install and setup

dcysetup (/opt/local/bin/dcysetup) takes exactly one of three actions (update is an alias for setup):

dcysetup install     # install/upgrade Duplicity + Python + deps (idempotent)
dcysetup setup       # generate config tree, credential templates, secret, cron
dcysetup update      # alias for setup

dcysetup is license-gated (_verify_boa_keys, tier pro or dev); on a host that fails the check it exits before doing anything. install stops cron, runs install_dependencies.sh, restarts cron. setup runs six numbered steps from /root/.remote_backups/run/:

  1. create_global_paths_config.sh — write global_paths.txt, data_paths.txt, custom_paths.txt; generate the root secret.
  2. create_user_paths_config.sh — per-tenant paths.txt.
  3. create_credentials_templates.sh — one <service>.txt template per backend.
  4. create_cron_entries.sh — write the wrapper, the schedule, the cron entry.
  5. create_readme.sh — global README files.
  6. create_config_readme.sh — per-user config README files.

The config tree

Root-side (global / data / custom path-sets):

/root/.remote_backups/
  .secret.txt                       # Duplicity PASSPHRASE (chattr +i, mode 600)
  credentials/<service>.txt         # per-backend secrets + retention vars
  paths/global_paths.txt            # _SOURCE / _INCLUDE_PATHS / _EXCLUDE_PATHS / _INCLUDE_LIST / _EXCLUDE_LIST
  paths/data_paths.txt
  paths/custom_paths.txt
  paths/.backboa.*                  # generated include/exclude fragments + merge state
  run/                              # the six setup scripts + sequential_backups.sh wrapper
  schedule/backup_schedule.txt      # "<service> <user>" lines the wrapper iterates
  logs/                             # OK-<bucket>.log / ERR-<bucket>.log per-run copy

Per-tenant:

/data/disk/<user>/remote_backups/
  .secret.txt
  paths/paths.txt
/data/disk/<user>/static/control/remote_backups/
  credentials/<service>.txt
  logs/

<service> is one of the eleven backend keys: aws, aws_one_zone, aws_standard_ia, azure, b2, cloudflare, do_spaces, gcs, ibm, linode, wasabi.

Credential files

create_credentials_templates.sh writes a per-backend template named exactly <service>.txt — so Cloudflare R2 is cloudflare.txt and DigitalOcean Spaces is do_spaces.txt (not r2.txt / do-spaces.txt). Each template is mode 600 and carries the backend's own variable names plus the two retention vars. Example, cloudflare.txt:

export R2_ACCOUNT_ID="your_account_id"
export R2_ACCESS_KEY_ID="your_access_key_id"
export R2_SECRET_ACCESS_KEY="your_secret_access_key"
export KEEP_WITHIN="3M"
export FULL_BACKUP_FREQUENCY="28D"

A credential file is only scheduled once its your_ placeholders are replaced — create_cron_entries.sh skips any file still matching your_. Each backend has its own variable set: AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_REGION; B2_ACCOUNT_ID / B2_APPLICATION_KEY; AZURE_STORAGE_ACCOUNT / AZURE_STORAGE_KEY; DO_SPACES_KEY / DO_SPACES_SECRET / DO_SPACES_REGION; GCS_PROJECT_ID / GCS_SERVICE_ACCOUNT_KEY; IBM_API_KEY_ID / IBM_SERVICE_INSTANCE_ID / IBM_REGION; LINODE_ACCESS_KEY / LINODE_SECRET_KEY / LINODE_REGION; WASABI_ACCESS_KEY / WASABI_SECRET_KEY / WASABI_REGION.

_validate_credentials reads each file line-by-line, strips export, rejects any value containing shell metacharacters, and (for b2 only) URL-encodes the value. A malformed line is logged to /var/log/backup_validation_issues.log and skipped — one bad entry does not abort the run.

The paths.txt variables

A paths.txt is a plain shell file sourced by multiback before each run. The variables carry a leading underscore (a name without it is read by nothing):

Variable Role
_SOURCE space-separated absolute paths emitted as --include directives
_INCLUDE_PATHS merged --include directive string (pre-built)
_EXCLUDE_PATHS merged --exclude directive string
_INCLUDE_LIST path to a --include-filelist file
_EXCLUDE_LIST path to an --exclude-filelist file

global_paths.txt ships, for example, _SOURCE="/etc /opt/solr4 /var/aegir /var/solr7 /var/solr9 /var/www /var/xdrago". The data set ships an empty _SOURCE and a generated per-tenant include set. To change what is captured, edit the relevant paths.txt; the next run picks it up.

How secrets reach Duplicity

The encryption passphrase is never in a credential file or the target URL. _load_credentials reads it from the secret file into the PASSPHRASE env var:

  • root-side: /root/.remote_backups/.secret.txt
  • tenant: /data/disk/<user>/remote_backups/.secret.txt

create_global_paths_config.sh generates the root secret with openssl rand -base64 32, sets mode 600, and chattr +i (immutable). A missing or empty .secret.txt aborts the run (exit 1). Provider secrets, by contrast, are interpolated into the Duplicity target URL per backend (see Regions & buckets). Lose the passphrase and the archives are unrecoverable — back up .secret.txt out of band.

Upgrade note (5.10.1). An earlier security-audit "hide credentials" change moved provider secrets out of the target URL; that broke Duplicity (the B2 backend has no key env var, and the boto3/S3 endpoint form mis-parsed for some providers, risking writes to the wrong bucket). The in-URL credential form was restored and the updater serial bumped. If a host ran a broken interim build, re-fetch multiback (barracuda up-<tier> system) and re-test.

Email notifications

Per-run summaries are sent by s-nail to _MY_EMAIL (from /root/.barracuda.cnf) when _MY_EMAIL is set and _INCIDENT_REPORT is not OFF. _INCIDENT_REPORT (normalised to OFF/ALL/MINI/CRIT, default MINI; legacy NO becomes OFF, YES becomes MINI) controls verbosity — ALL also emits waiting-queue and validation-alert mails. There is no BACKUP_EMAIL variable.

The cron path — interval-based, not nightly

create_cron_entries.sh writes an interval cron, not a fixed nightly slot. It emits /etc/cron.d/duplicity_backup:

0 */$((_BACKUP_INTERVAL/60)) * * * root /root/.remote_backups/run/sequential_backups.sh

With the default _BACKUP_INTERVAL=360 (minutes) this is 0 */6 * * * — every six hours. _BACKUP_INTERVAL is overridable in /root/.barracuda.cnf; the script also auto-widens the interval if the last total runtime (/var/log/backup_runtime.log) would not fit. The wrapper sequential_backups.sh iterates schedule/backup_schedule.txt (one <service> <user> line per configured backup) and calls multiback backup <service> <user> for each, one at a time, with per-run PID files in /run/duplicity_<service>_<user>_sequential.pid. There is no 3:01 AM nightly entry and no Sunday-weekly full entry — those belong to the separate legacy crontab path (see Legacy backboa).

To inspect the live schedule:

cat /etc/cron.d/duplicity_backup
cat /root/.remote_backups/schedule/backup_schedule.txt

Manual invocation

Every multiback action is <action> <SERVICE> <USER> (plus restore positionals). Full verb table in CLI reference:

multiback test wasabi data       # verify connectivity for the data path-set
multiback backup b2 data         # run a backup pass now
multiback cleanup aws john       # apply KEEP_WITHIN retention now
multiback list b2 jane           # list current files in the set

Related