DNS & resolver recovery

Symptoms: /etc/resolv.conf keeps flipping away from nameserver 127.0.0.1 after DHCP lease renewals; intermittent lookup failures on a box that resolves fine right after a BOA run. Root cause is almost always a fight between the DHCP client and BOA over /etc/resolv.conf. Since 5.10.3 BOA settles the fight permanently: BOA owns /etc/resolv.conf, the DHCP client is told to keep its hands off, and the content self-heals on every agent run. This page covers the ownership model, the two self-healing tiers (they restore different things — do not conflate them), the operator toggles, and the dhcpfix recovery tool.

The canonical file

_fix_dns_settings writes one canonical /etc/resolv.conf:

### BOA-DNS-Config ###
nameserver 127.0.0.1
nameserver 1.1.1.1
nameserver 8.8.8.8
nameserver 9.9.9.9

The 127.0.0.1 line (the local unbound resolver) is included only when unbound is installed and running — the test is /usr/sbin/unbound executable plus /run/unbound/unbound.pid present. Then the three public fallbacks in fixed order: 1.1.1.1, 8.8.8.8, 9.9.9.9. After every rewrite the function reloads unbound via unbound-control reload (when installed) and keeps a reference copy at /var/backups/resolv.conf.vanilla.

The BOA.sh.txt bootstrap copy additionally chmods the file 0644 and locks it with chattr +i, so nothing short of chattr -i can modify it. The copies in the fetched agents (boa, barracuda, octopus, lib/functions/dns.sh.inc) write the same content but leave the file 0644 and mutable — see the two-tier split below.

Trade-off carried from the source: the public fallbacks let lookups survive unbound being down, but they also let queries bypass unbound (cache, RPZ, split view) whenever unbound is unreachable or times out. That is an intentional availability-over-purity choice.

DHCP overwrite protection: the nodnsupdate hook

Whenever the BOA.sh.txt copy rewrites resolv.conf, it also installs an executable hook at /etc/dhcp/dhclient-enter-hooks.d/nodnsupdate:

make_resolv_conf() { :; }

That one no-op is sufficient on its own: dhclient-script sources the enter-hooks directory after defining its own make_resolv_conf, so the no-op wins and resolv.conf is never written by dhclient regardless of which DNS options the lease carries. No dhclient.conf request-line surgery is needed, and the hook is designed to take effect at the next natural lease RENEW — no lease churn at install time.

Self-healing: two tiers

Both tiers run _check_dns_settings, which forces a rewrite of resolv.conf on the same content triggers — but they restore different things:

In practice: every boa, barracuda or octopus run self-heals the resolv.conf content (the check runs early, before mirror selection), so a box that lost its 127.0.0.1 line recovers on the next agent run. But a lost nodnsupdate hook — and the immutable lock — are restored only by the next BOA.sh.txt-driven install/upgrade run, or immediately by dhcpfix (below).

Operator toggles

Quick diagnosis

cat /etc/resolv.conf        # header + 127.0.0.1 first (when unbound runs)?
lsattr /etc/resolv.conf     # 'i' flag = bootstrap-locked; absent = agent/dhcpfix wrote it last
ls -l /etc/dhcp/dhclient-enter-hooks.d/nodnsupdate   # hook present + executable?
host files.boa.io 127.0.0.1                          # is unbound answering?

Missing hook + file flipping after lease renewals = run dhcpfix.

dhcpfix — recovery on a live box

dhcpfix (rewritten in 5.10.3) is safe to run on a production server. It detects the active DHCP client (dhclient, dhcpcd, NetworkManager — the target stack is Devuan/SysVinit + ISC dhclient; the others are handled for completeness) and then does exactly two things:

1. Stops the client from writing resolv.conf. For dhclient it installs only the same nodnsupdate no-op hook (0755) — no awk surgery on /etc/dhclient.conf's request line, no lease release/renew, and it never touches /etc/network/interfaces. The hook is designed to take effect at the next natural RENEW. For dhcpcd it ensures nohook resolv.conf in /etc/dhcpcd.conf (applied on the next dhcpcd cycle, no forced restart); for NetworkManager it sets dns=none under [main] and reloads (not restarts) the service.
2. Writes the canonical nameserver set once — 127.0.0.1 plus the three public fallbacks — 0644 and deliberately mutable (no chattr +i), so BOA stays the owner. The file carries no ### BOA-DNS-Config ### header, so the next agent run re-stamps it into full canonical form — expected, not drift. The file is correct immediately, even before the hook bites.

The configs it edits (/etc/dhcpcd.conf, NetworkManager.conf) and the replaced resolv.conf are first backed up as <file>.bak.<epoch>. Set _DEBUG_MODE=YES in the environment for verbose [dhcpfix] output. The tool is fleet-deployed to /opt/local/bin/dhcpfix via serial-gated _fetch_versioned with a --guard pgrep check that never clobbers a running copy.

Related

• Auto-healing — the unbound.sh watchdog independently restarts unbound and rewrites a BOA-tagged resolv.conf when its liveness lookup fails.
• OS lifecycle — the install/upgrade runs that carry the full bootstrap tier (hook + chattr +i).
• IP-blocked recovery — the other "box unreachable / half-working" recovery procedure in this chapter.