Commit 5f8a5bc3 authored by Vitaly Lipatov's avatar Vitaly Lipatov

fix route-health: bulk route flush + wider health window

flush_gw_routes() deleted routes one-by-one in a loop (ip route del x127k), which hung the oneshot service for 20+ minutes under netlink contention and froze ALL gateway health monitoring. Replace with a single kernel-level "ip route flush table T via GW" -- the same idiom route-update.sh already uses for stale-gateway cleanup -- guarded by timeout 60. Also widen the ping and vpn_status InfluxDB query windows from 1m to 3m. The queries use last(), so a wider window does not delay reaction to a real outage; it only stops an empty result (=> false "dead") when telegraf momentarily starves under load. Together both changes break the flush->load->stale->flush feedback loop. Verified on igw: cycle time ~18s (was 20+ min), table 202 v6 routes via the ikev2.vdska peer restored to 127259. Co-Authored-By: 's avatarClaude <noreply@anthropic.com>
parent 22d3fa85
......@@ -47,12 +47,12 @@ trap 'rm -f "$HEALTH_DATA" "$VPN_DATA" "$IPERF_DATA" "$PROXY_DATA"' EXIT
response=$(curl -sG "$INFLUXDB_URL" \
--data-urlencode "db=$INFLUXDB_DB" \
--data-urlencode "q=SELECT last(percent_packet_loss) FROM ping WHERE time > now() - 1m GROUP BY gateway" \
--data-urlencode "q=SELECT last(percent_packet_loss) FROM ping WHERE time > now() - 3m GROUP BY gateway" \
--max-time 5 2>/dev/null)
vpn_response=$(curl -sG "$INFLUXDB_URL" \
--data-urlencode "db=$INFLUXDB_DB" \
--data-urlencode "q=SELECT last(connected) FROM vpn_status WHERE time > now() - 1m GROUP BY gateway" \
--data-urlencode "q=SELECT last(connected) FROM vpn_status WHERE time > now() - 3m GROUP BY gateway" \
--max-time 5 2>/dev/null)
iperf_response=$(curl -sG "$INFLUXDB_URL" \
......@@ -279,11 +279,14 @@ flush_gw_routes()
[ -d "$list_state" ] || continue
[ -f "$list_state/table" ] || continue
local t ; read -r t < "$list_state/table"
# TODO: routes are deleted one-by-one in a loop; consider separate
# tables per gateway so flush_gw_routes becomes a single "ip route flush table"
$ipcmd route show table "$t" 2>/dev/null | grep "via $dead_gw" | while read -r route ; do
$ipcmd route del $route table "$t" 2>/dev/null
done
# Bulk-remove only this gateway's routes from the shared table.
# Same idiom route-update.sh uses for stale-gateway cleanup (a single
# "ip route flush table T via GW"), not a per-route delete loop — the
# loop was O(n) over 120k+ routes and hung the service via netlink
# contention. Verified on igw: "ip -6 route show table 202 via <gw>"
# selects exactly that gateway's routes (same count as grep "via <gw>");
# a different gateway returns 0 (it filters, not a full-table dump).
timeout 60 $ipcmd route flush table "$t" via "$dead_gw" 2>/dev/null
done
}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment