mirror of
https://github.com/rubenhensen/k8scd.git
synced 2026-11-16 06:29:09 +01:00
docs: record the backup check
This commit is contained in:
1 parent
d836e4608c
commit
f2e4c5aff6
1 file changed
+9
@@ -177,6 +177,15 @@ fails). The dqlite stalls were tracked down by hand from the API server's
|
|||||||
metrics, and Velero's scheduled backups failed every night from August
|
metrics, and Velero's scheduled backups failed every night from August
|
||||||
until 2026-10-21 without anyone noticing.
|
until 2026-10-21 without anyone noticing.
|
||||||
|
|
||||||
|
- [x] A stopgap for the worst blind spots (2026-10-25,
|
||||||
|
`gatus/backup-check.yaml`): an hourly CronJob reads the API and pushes
|
||||||
|
four external endpoints to s2 (group "cluster"): the newest daily and
|
||||||
|
weekly Velero backup Completed and recent, every attached Longhorn
|
||||||
|
volume healthy with a backup younger than 26h, and no Vault pod sealed.
|
||||||
|
Gatus alerts through ntfy after two failed reports, or when no report
|
||||||
|
arrives for 3 hours. The token comes from an ESO Password generator.
|
||||||
|
Tested with made-up data for each failure (a Failed or old backup, a
|
||||||
|
degraded or unbacked-up volume, a sealed pod)
|
||||||
- [ ] After the second SSD (a time series database writes all the time
|
- [ ] After the second SSD (a time series database writes all the time
|
||||||
and would add to today's stalls): metrics-server, then Prometheus or
|
and would add to today's stalls): metrics-server, then Prometheus or
|
||||||
VictoriaMetrics with Grafana, alerting to ntfy. Loki later
|
VictoriaMetrics with Grafana, alerting to ntfy. Loki later
|
||||||
|
|||||||
Reference in new issue
Block a user