Commit Graph
24 Commits
Author SHA1 Message Date
Ruben Hensen 72f49bed51 Set resource requests from measured usage 2026-10-25 07:08:16 +01:00
Ruben Hensen 9014898fc5 velero: run kopia maintenance daily and throttled
Each of the 15 kopia repositories ran maintenance every hour, and about
once a day a full run of several minutes that writes its cache to the
node's only SSD. Those runs line up with dqlite stalling for seconds and
controllers losing their leases (vault 09:57-10:07, argocd and mbgwp
09:17-09:40 today). Make the default frequency 24h and cap the jobs at
half a CPU, so a full run writes slower and longer instead of in one
burst.
2026-10-24 14:43:38 +02:00
Ruben Hensen 1926ccc42a velero: upgrade to 1.18.2 (chart 12.2.1)
No breaking changes for this setup: the chart renders the same resources and server flags, and chart 12 only dropped the Bitnami kubectl image from the CRD hook, which stays off because Argo CD applies the CRDs. The AWS plugin goes to v1.14.4, the line for Velero 1.18.
2026-10-23 22:47:27 +02:00
Ruben Hensen e3074b43ad velero: upgrade to 1.17.1 (chart 11.4.0)
1.17 drops restic, which nothing here uses (all 411 pod volume backups and 20 repositories are kopia), and moves the maintenance job settings from server flags into the velero-repo-maintenance ConfigMap, which the chart now renders with keepLatestMaintenanceJobs: 3. The AWS plugin goes to v1.13.2, the line for Velero 1.17.
2026-10-23 22:45:00 +02:00
Ruben Hensen bd2606591e velero: correct the upgradeCRDs comment
Argo CD applies the chart's crds/ directory on every sync, so CRD updates don't depend on the hook; the 1.16 upgrade confirmed it.
2026-10-22 15:32:19 +02:00
Ruben Hensen ce1115d4f9 velero: upgrade to 1.16.2 (chart 10.1.3)
Velero 1.15 is only tested up to Kubernetes 1.31; 1.16 is tested on 1.33.
The AWS plugin moves along to v1.12.2. Argo CD applies the chart's crds/
directory itself, so the CRD schema update doesn't need the disabled
upgrade hook. No backup was running.
2026-10-22 15:30:11 +02:00
Ruben Hensen ff73f99d43 velero: name the release resources "velero" so repo maintenance runs
Velero 1.15 builds kopia repository maintenance jobs from a Deployment
called "velero" in its namespace. Ours was velero-helm, so every
BackupRepository reported 'error to build maintenance job: Deployment.apps
"velero" not found' and the repos on the NAS were never cleaned up, some
not since March 2025. fullnameOverride renames the Deployment, Service,
ServiceAccount and RBAC; the old velero-helm objects get deleted by hand
since this app doesn't prune.
2026-10-22 09:23:51 +02:00
Ruben Hensen 6d9e405126 Point Velero and the NAS services back at 192.168.1.99
Reverts 8c2bd89. The NAS had fallen back to a DHCP address (.10), which
also moved the UniFi controller it hosts away from where the gateway
and access points inform, and left the Longhorn backup target at
nfs://192.168.1.99 unreachable. It now has 192.168.1.99 set statically
in DSM again, so everything that pointed at .99 works without changes.
2026-10-21 15:25:16 +02:00
Ruben Hensen dfdc4451fb velero: raise the server memory limit to 2Gi
At 512Mi the server has been OOMKilled 314 times. With the bucket
reachable again it died 23 seconds into a manual backup of ~14.7k items,
which then got marked Failed on restart. opti1 has 32GB.
2026-10-21 12:33:31 +02:00
Ruben Hensen e19f4b8db5 velero: skip the CRD upgrade hook
The hook's init container pulls docker.io/bitnami/kubectl:1.32, which
Bitnami has removed, so the sync for the new NAS address hung on
ImagePullBackOff. The chart stays at 8.4.0 and the CRDs are already in
place.
2026-10-21 12:22:57 +02:00
Ruben Hensen 8c2bd89444 Point Velero and the NAS services at 192.168.1.10
The NAS moved from 192.168.1.99 to .10, so the backup storage location
has been Unavailable ("no route to host"). Every backup since at least
26 July failed, and 154 expired ones are stuck in Deleting because Velero
can't reach the bucket to remove them. MinIO answers on .10:9000 and DSM
on .10:5000.
2026-10-21 12:16:19 +02:00
Ruben Hensen c83d5c87d7 Enable CSI 2 2025-03-01 13:26:12 +01:00
Ruben Hensen f08886c10f Enable CSI 2025-03-01 13:15:07 +01:00
Ruben Hensen 2780dd80f3 Change to handmade credfile 2025-03-01 03:20:04 +01:00
Ruben Hensen c7f861b21f Add env aws 2025-03-01 03:12:59 +01:00
Ruben Hensen 420fd5a5b3 remove existing secret 2025-03-01 03:06:51 +01:00
Ruben Hensen 2df1b88146 Add extraSecretRef 2025-03-01 03:03:01 +01:00
Ruben Hensen b35f3cdd11 Change back to server ip 2025-03-01 02:10:51 +01:00
Ruben Hensen 88d5d8fcf4 Add service to nas 2025-03-01 01:48:18 +01:00
Ruben Hensen a07191f964 Change provider location 2025-03-01 01:18:26 +01:00
Ruben Hensen 5e15039e68 Change restic to nodeagent 2025-03-01 01:15:47 +01:00
Ruben Hensen d46a6e96c1 Update Velero config from map to slice 2025-03-01 01:05:40 +01:00
Ruben Hensen 7de7c82dd3 Change name velero helm 2025-03-01 01:01:38 +01:00
Ruben Hensen 6a0fa2f4da Add velero 2025-03-01 00:55:08 +01:00