fix route-health: bulk route flush + wider health window

flush_gw_routes() deleted routes one-by-one in a loop (ip route del x127k), which hung the oneshot service for 20+ minutes under netlink contention and froze ALL gateway health monitoring. Replace with a single kernel-level "ip route flush table T via GW" -- the same idiom route-update.sh already uses for stale-gateway cleanup -- guarded by timeout 60. Also widen the ping and vpn_status InfluxDB query windows from 1m to 3m. The queries use last(), so a wider window does not delay reaction to a real outage; it only stops an empty result (=> false "dead") when telegraf momentarily starves under load. Together both changes break the flush->load->stale->flush feedback loop. Verified on igw: cycle time ~18s (was 20+ min), table 202 v6 routes via the ikev2.vdska peer restored to 127259. Co-Authored-By: Claude <noreply@anthropic.com>

fix route-health: bulk route flush + wider health window
5f8a5bc3 · Vitaly Lipatov · 22d3fa85 · 5f8a5bc3
Commit 5f8a5bc3 authored Aug 01, 2026 by Vitaly Lipatov
Show whitespace changes
Inline Side-by-side

Showing with 10 additions and 7 deletions

route-health.sh router/route-health.sh +10 -7

No files found.
--- a/router/route-health.sh
+++ b/router/route-health.sh
@@ -47,12 +47,12 @@ trap 'rm -f "$HEALTH_DATA" "$VPN_DATA" "$IPERF_DATA" "$PROXY_DATA"' EXIT
 response=$(curl -sG "$INFLUXDB_URL" \
    --data-urlencode "db=$INFLUXDB_DB" \
-    --data-urlencode "q=SELECT last(percent_packet_loss) FROM ping WHERE time > now() - 1m GROUP BY gateway" \
+    --data-urlencode "q=SELECT last(percent_packet_loss) FROM ping WHERE time > now() - 3m GROUP BY gateway" \
    --max-time 5 2>/dev/null)
 vpn_response=$(curl -sG "$INFLUXDB_URL" \
    --data-urlencode "db=$INFLUXDB_DB" \
-    --data-urlencode "q=SELECT last(connected) FROM vpn_status WHERE time > now() - 1m GROUP BY gateway" \
+    --data-urlencode "q=SELECT last(connected) FROM vpn_status WHERE time > now() - 3m GROUP BY gateway" \
    --max-time 5 2>/dev/null)
 iperf_response=$(curl -sG "$INFLUXDB_URL" \
@@ -279,11 +279,14 @@ flush_gw_routes()
        [ -d "$list_state" ] || continue
        [ -f "$list_state/table" ] || continue
        local t ; read -r t < "$list_state/table"
-        # TODO: routes are deleted one-by-one in a loop; consider separate
+        # Bulk-remove only this gateway's routes from the shared table.
-        # tables per gateway so flush_gw_routes becomes a single "ip route flush table"
+        # Same idiom route-update.sh uses for stale-gateway cleanup (a single
-        $ipcmd route show table "$t" 2>/dev/null | grep "via $dead_gw" | while read -r route ; do
+        # "ip route flush table T via GW"), not a per-route delete loop — the
-            $ipcmd route del $route table "$t" 2>/dev/null
+        # loop was O(n) over 120k+ routes and hung the service via netlink
-        done
+        # contention. Verified on igw: "ip -6 route show table 202 via <gw>"
+        # selects exactly that gateway's routes (same count as grep "via <gw>");
+        # a different gateway returns 0 (it filters, not a full-table dump).
+        timeout 60 $ipcmd route flush table "$t" via "$dead_gw" 2>/dev/null
    done
 }