Each of the 15 kopia repositories ran maintenance every hour, and about
once a day a full run of several minutes that writes its cache to the
node's only SSD. Those runs line up with dqlite stalling for seconds and
controllers losing their leases (vault 09:57-10:07, argocd and mbgwp
09:17-09:40 today). Make the default frequency 24h and cap the jobs at
half a CPU, so a full run writes slower and longer instead of in one
burst.
No breaking changes for this setup: the chart renders the same resources and server flags, and chart 12 only dropped the Bitnami kubectl image from the CRD hook, which stays off because Argo CD applies the CRDs. The AWS plugin goes to v1.14.4, the line for Velero 1.18.
1.17 drops restic, which nothing here uses (all 411 pod volume backups and 20 repositories are kopia), and moves the maintenance job settings from server flags into the velero-repo-maintenance ConfigMap, which the chart now renders with keepLatestMaintenanceJobs: 3. The AWS plugin goes to v1.13.2, the line for Velero 1.17.
The nightly backup of 2026-10-23 ended PartiallyFailed because a repository maintenance job pod finished while Velero was reading its emptyDir volumes. emptyDir holds nothing worth restoring, so a volume policy now skips it for both schedules.
Velero 1.15 is only tested up to Kubernetes 1.31; 1.16 is tested on 1.33.
The AWS plugin moves along to v1.12.2. Argo CD applies the chart's crds/
directory itself, so the CRD schema update doesn't need the disabled
upgrade hook. No backup was running.
Velero 1.15 builds kopia repository maintenance jobs from a Deployment
called "velero" in its namespace. Ours was velero-helm, so every
BackupRepository reported 'error to build maintenance job: Deployment.apps
"velero" not found' and the repos on the NAS were never cleaned up, some
not since March 2025. fullnameOverride renames the Deployment, Service,
ServiceAccount and RBAC; the old velero-helm objects get deleted by hand
since this app doesn't prune.
Reverts 8c2bd89. The NAS had fallen back to a DHCP address (.10), which
also moved the UniFi controller it hosts away from where the gateway
and access points inform, and left the Longhorn backup target at
nfs://192.168.1.99 unreachable. It now has 192.168.1.99 set statically
in DSM again, so everything that pointed at .99 works without changes.
At 512Mi the server has been OOMKilled 314 times. With the bucket
reachable again it died 23 seconds into a manual backup of ~14.7k items,
which then got marked Failed on restart. opti1 has 32GB.
The hook's init container pulls docker.io/bitnami/kubectl:1.32, which
Bitnami has removed, so the sync for the new NAS address hung on
ImagePullBackOff. The chart stays at 8.4.0 and the CRDs are already in
place.
The NAS moved from 192.168.1.99 to .10, so the backup storage location
has been Unavailable ("no route to host"). Every backup since at least
26 July failed, and 154 expired ones are stuck in Deleting because Velero
can't reach the bucket to remove them. MinIO answers on .10:9000 and DSM
on .10:5000.
Ruben Hensen