The ban pipeline

The scan_nginx scoring engine decides who should be banned and writes two log files. This page documents what turns those files into an actual 444 on the next request. Three separately-scheduled BOA monitor scripts carry an offender from a log line, through the firewall, and back into the Nginx $is_banned geo:

scan_nginx.sh
   │  writes offenders
   ▼
web.log  +  scan_nginx.archive.log
   │                      │
   ▼                      ▼
guest-fire.sh        guest-water.sh
web.log → csf -td     archive ≥12 → csf.deny
900 (80/443)          "Brute force Web Server"
(temporary, 15 min)   (persistent; ≥24 = do-not-delete)
   │                      │
   └──────────┬───────────┘
              ▼
   /etc/csf  (csf.tempban 80/443  +  csf.deny)
              │
              ▼
   nginx_deny.sh  — regenerate the geo from current csf state
              │
              ▼
   /data/conf/nginx_banned_ips.conf
              │
              ▼
   geo $remote_addr $is_banned  (Nginx)
              │
              ▼
   return 444   (silent drop, zero backend cost)

The defining property of this stage is the same split that defines the whole Abuse Guard: detection is post-hoc, enforcement is real-time. scan_nginx analyses log lines that already happened; the pipeline turns its verdict into a CSF ban, and the Nginx geo — a derived mirror of CSF, fully regenerated every pass — makes that ban bite on every subsequent request and self-expire the moment CSF drops it.

Stage 1 — temporary ban (guest-fire.sh)

guest-fire.sh is the fast path: a 15-minute temporary CSF ban for every fresh offender scan_nginx wrote to web.log. For the web tier it reads /var/xdrago/monitor/log/web.log, takes the IP field (cut -d '#' -f1 | sort | uniq), and for each unique IP:

  1. Looks the IP up in CSF (csf -g) and checks csf.allow for an explicit tcp|in|d=80|s=<ip> allow entry.
  2. If allowed (in csf.allow, or showing an ALLOW … ACCEPT … dpt:80 rule), it is cleared — csf -dr and csf -tr remove any stray block — and never banned. The allow list always wins.
  3. If already denied on 80 or 443, nothing is done.

  4. Otherwise it issues a 15-minute temporary ban on both web ports:

    csf -td ${_IP} 900 -p 80
csf -td ${_IP} 900 -p 443

The same routine runs over three log files with identical allow-first / already-denied / temp-ban logic: the web block above (web.log, ports 80/443) plus an SSH twin (ssh.log, port 22) and an FTP twin (ftp.log, port 21).

Looping and the water interlock

The guard runs five times per invocation with a 10 s pause between passes, so an IP that appears mid-run is still caught within the same tick:

for _iteration in {1..5}; do
  [ ! -e "/run/water.pid" ] && _guest_guard
  sleep 10
done

The [ ! -e "/run/water.pid" ] test is the interlock with Stage 2: while guest-water.sh runs (it touches /run/water.pid on start and removes it on exit), guest-fire.sh skips its guard passes, so the two never fight over CSF at the same time.

Self-healing watchdog

Under a real flood web.log can hold thousands of IPs and a single run can stretch from its normal ~50 s (5 × 10 s) into minutes, blocking the following cron slots. A watchdog records the PID in /run/fire.pid, and on the next invocation kills any predecessor still alive past _FIRE_TIMEOUT (180 s), logging to /var/log/boa/fire_stuck.log. A normal overlap (within the timeout) is left to the single-instance lock, which exits early if more than two copies run. The full watchdog stack is on Operations + tuning.

Stage 2 — persistent escalation (guest-water.sh)

guest-fire.sh only ever applies temporary bans. Repeat offenders are escalated to a persistent CSF deny by guest-water.sh, which runs on its own (slower) schedule and works from the cumulative archive /var/xdrago/monitor/log/scan_nginx.archive.log (_WA) rather than the per-tick web.log. For each unique IP it counts how many times that IP appears across the whole archive:

_NR_TEST=$(tr -s ' ' '\n' < ${_WA} | grep -cF "${_IP}")

That count drives a two-tier escalation (after the csf.allow / /root/.local.IP.list exemptions):

Archive hits Action CSF comment
≥ 12 csf -d persistent deny Brute force Web Server N attacks
≥ 24 csf -d persistent deny, do-not-delete do not delete Brute force Web Server N attacks

The 12-hit deny lasts until the next CSF limits rotation (routine csf.deny trimming reclaims it). The 24-hit deny carries the literal do not delete tag, which CSF's routine trimming respects — so a heavy repeat offender stays banned across the housekeeping that would otherwise expire it. The one thing that does clear it is the operator-triggered full cleanup: when /etc/boa/.full.csf.cleanup.cnf exists, guest-water.sh strips every do not delete line from csf.deny (sed -i "s/.*do not delete.*//g") — the explicit "wipe even the permanent bans" override.

guest-water.sh runs the same escalation over three archives in parallel structure:

Tier Archive var Archive file CSF label
Web _WA scan_nginx.archive.log Brute force Web Server
SSH _HA hackcheck.archive.log Brute force SSH Server
FTP _FA hackftp.archive.log Brute force FTP Server

For the whole run guest-water.sh holds the Stage 1 interlock — it touches /run/water.pid early (after its initial /root/.local.IP.list unblock pass, not first) and rms it at the very end — and once escalation is done it clears the per-tick web.log / ssh.log / ftp.log so the next scan_nginx window starts clean.

Under the hood. guest-water.sh also refreshes the csf.allow provider ranges (Cloudflare, Googlebot, Bingbot, Pingdom, and — behind /root/.extended.firewall.exceptions.cnf — Imperva, Sucuri, Auth0, Site24x7), with a diff-guard that reverts an unexpected csf.allow change and per-provider backups under /var/backups/csf/water/. That allow-list maintenance is what makes the keystone _is_whitelisted_ip guard reliable across the whole pipeline.

Stage 3 — geo regeneration (nginx_deny.sh)

A CSF deny stops traffic at the origin firewall — but on a Cloudflare-proxied vhost the origin only ever sees the Cloudflare edge IP, so an iptables/CSF ban on the real client never bites. nginx_deny.sh closes that gap: it mirrors the web bans into an Nginx realip-keyed geo deny set, so the banned real client is dropped with a 444 at Nginx even when it arrives via Cloudflare.

It rebuilds /data/conf/nginx_banned_ips.conf from current CSF state on every run, collecting web bans only from two sources:

# Active web temp bans: csf.tempban rows whose port field is 80 or 443
awk -F'|' '($3 == "80" || $3 == "443") { print $2 }' /var/lib/csf/csf.tempban
# Persistent web offenders: csf.deny lines tagged by guest-water
grep -F "Brute force Web Server" /etc/csf/csf.deny | awk '{ print $1 }'

This is deliberately web-scoped: it picks up the 80/443 temp bans from Stage 1 and the Brute force Web Server denies from Stage 2, and it excludes the SSH/FTP csf.deny entries and the broad CIDR blocklists — those work at the network layer (they are not Cloudflare-proxied) and have no place in the Nginx geo.

Each collected token is then validated as a bare IPv4 address or an IPv4 CIDR — every octet 0-255, the prefix 0-32:

_ipv4_octet="(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])"
[[ "$1" =~ ^(${_ipv4_octet}\.){3}${_ipv4_octet}(/(3[0-2]|[12]?[0-9]))?$ ]]

A loose pattern would let a malformed token (999.1.1.1, /99, IPv6, junk) reach the geo map and fail the Nginx configtest for the whole box. Only IPv4/CIDR tokens become IP 1; lines here — nginx_deny.sh stays IPv4-only because its source (csf) is IPv4-only; IPv6 offenders travel a separate path.

Stage 3b — IPv6 geo regeneration (nginx_deny6.sh)

csf cannot hold an IPv6 ban, so an IPv6 offender never reaches nginx_deny.sh. Instead scan_nginx's _block_ip writes it to an nginx-native store (/var/xdrago/monitor/log/web6.tempban, lines <ip6>|<expiry-epoch>), and nginx_deny6.sh (*/2, same shared lock) prunes the expired entries, collapses the refreshed duplicates to the max expiry, validates each as a strict IPv6 address, and emits the survivors as <ip6> 1; into /data/conf/nginx_banned_ips.conf6. That file is pulled by the same geo $remote_addr $is_banned set (its wildcard nginx_banned_ips.c* include already covers it), so an IPv6 attacker gets the same 444. The store self-expires (default 900s, _NGINX_V6_BAN_TTL) — the nginx equivalent of csf's 15-minute web temp ban — and a re-offence refreshes it. IPv6 can only arrive via the trusted realip proxy, so a banned v6 is always the real client. The whole IPv6 arm is gated by _NGINX_V6_BAN_DETECT (default YES).

Safe, idempotent reload

The rebuild only disturbs Nginx when something actually changed:

  • Change-gate. The freshly-built file is compared to the live one with cmp -s; if identical, the run exits with no reload.
  • Atomic install + validate. The new file is written to a leading-dot temp in the same directory (so the .c* include glob never sees it) and mv-d into place; the current file is backed up to .nginx_banned_ips.last_good.conf first.
  • Revert on failure. After install it runs service nginx configtest and service nginx reload; on any failure it restores the last-good file (and reloads), so a bad ban set can never take Nginx down.
  • Shared lock. The whole run holds the advisory lock /run/boa_nginx_config.lock (flock -w 30), shared with the other BOA Nginx-config writers (ip_access, user_admin_access, cloudflare_realip, ai_policy, nginx_deny6, migration_proxy_realip) so their configtest+reload cycles never overlap. If the lock can't be taken within 30 s the run skips and retries next tick.

The Provision templates own the consuming side — the geo $remote_addr $is_banned map, the if ($is_banned) return 444; guard, and the wildcard include of the generated file — documented in request guards.

Why the geo self-expires

The geo file holds no state of its own. Each nginx_deny.sh run rebuilds it from scratch out of the current CSF tables, so the lifecycle is entirely CSF's:

  • A Stage 1 temp ban lives in csf.tempban for 15 minutes. While it is there it appears in the geo; once CSF drops the expired row, the next nginx_deny.sh pass simply doesn't emit it and the geo reload removes the 444.
  • A Stage 2 persistent deny stays in the geo until CSF removes the csf.deny line — at limits rotation for a ≥12 ban, or never (until manual action) for a do not delete ≥24 ban.

So bans self-expire without any manual cleanup: the operator manages CSF, and the Nginx geo follows automatically. To lift a ban early, clear it in CSF (csf -dr / csf -tr) and the next geo regeneration drops it — see Operations + tuning.

Release valve: clearwebbans

Everything this page documents can be unwound in one command: clearwebbans removes the Stage 2 csf.deny Brute force Web Server escalations (csf -dr), then the temporary 80/443 bans (csf -tr), clears the per-tick web.log plus both archive generations (scan_nginx.archive.log and the guest-water-rotated scan_nginx.archive.x3.log — so guest-fire.sh cannot re-apply and guest-water.sh cannot re-escalate from history), resets the /var/log/scan_nginx_lastpos byte offset to the current end of access.log, and finally regenerates the geo via /var/xdrago/nginx_deny.sh. It is strictly WEB-scoped: SSH/FTP bans and their archives are untouched. The run is idempotent, supports --dry-run, and holds a lock at /run/clearwebbans.lock. The full command reference — flags, output, when to run it — lives on Operations + tuning.

Allow list wins at every stage

None of the three stages trusts the previous stage's output blindly — each one re-checks the CSF allow list before it acts:

  • guest-fire.sh clears (never bans) any IP in csf.allow / showing an ALLOW rule.
  • guest-water.sh skips any IP listed in /root/.local.IP.list, never re-denies an IP already shown denied or allowed in the live csf -g output, and unblocks (csf -dr/csf -tr) only the IPs it finds in csf.allow.
  • nginx_deny.sh only ever emits IPs that are currently banned in CSF, so an allow-listed IP that was never banned never reaches the geo.

This is the same keystone guarantee scan_nginx enforces with _is_whitelisted_ip: to exempt a service fleet-wide, put its IPs on the CSF allow list and every stage of the pipeline honours it. See scan_nginx scoring engine — Whitelisting and Operations + tuning.

Related