Architecture¶
The operator reconciles Kubernetes resources and Garage's Admin/RPC APIs as one system. Kubernetes owns the desired workload and identity boundary; Garage owns the distributed object layout and data placement.
Resource relationships¶
flowchart LR
GC[GarageCluster]
GN[GarageNode]
B[GarageBucket]
K[GarageKey]
T[GarageAdminToken]
G[GarageReferenceGrant]
Garage[(Garage Admin / RPC API)]
Workloads[Kubernetes Pods, PVCs, Services]
GC --> GN
GC --> Workloads
GN --> Workloads
GC --> Garage
GN --> Garage
B --> GC
K --> B
K --> GC
T --> GC
G -. authorizes .-> GC
G -. authorizes .-> B
G -. authorizes .-> K
The API group is garage.rajsingh.info. GarageCluster is served as v1beta2 and the deprecated v1beta1; the other operator CRDs are currently v1beta1.
Controllers and ownership¶
| Resource | Primary responsibility | Kubernetes artifacts |
|---|---|---|
GarageCluster |
Cluster config, tier topology, services, layout coordination, health, federation, operations | StatefulSets, node-local-pool DaemonSets, Services, ConfigMaps, PDBs, endpoint Services, generated GarageNodes |
GarageNode |
One Garage identity and layout role | Usually one single-replica StatefulSet, PVCs, ConfigMap, and RPC Service; node-local pool members run in the parent pool DaemonSet |
GarageBucket |
Bucket, aliases, quotas, website, lifecycle, grants | Generated Secret relationships and Garage-side bucket state |
GarageKey |
S3 key material and permissions | Generated Kubernetes Secret and Garage-side key/grants |
GarageAdminToken |
Static Admin API bootstrap token Secret | Generated Kubernetes Secret; it does not create a revocable Garage token row |
GarageReferenceGrant |
Cross-namespace reference authorization | No workload; status reports users |
Reconciliation boundaries¶
The operator uses the Garage Admin API v2 for cluster, node, layout, bucket, key, token, lifecycle, repair, and health operations. It does not shell out to a Garage CLI in the controller.
Kubernetes admission is part of the safety design. The webhooks validate topology, immutable identity-sensitive fields, cross-namespace references, reserved environment variables, scale requests, node-local prerequisites, and prepared deletion. Running without them can remove important safety boundaries even when the controller binary starts.
Leader election is required by supported installations because layout mutation is serialized through process-local and cluster-level coordination. A disabled leader election is an explicitly unsupported single-manager mode and must be acknowledged in Helm values.
Workload shapes¶
- Default Auto storage: one
GarageNodeand one single-replica StatefulSet per storage slot. - Unified Auto gateway: one gateway
GarageNodeand one single-replica StatefulSet per gateway identity. - Edge gateway: one cluster-level StatefulSet, because the remote storage cluster owns its layout roles.
- Node-local pool: one DaemonSet per named pool plus generated
GarageNodeidentities for selected Kubernetes Nodes. - External
GarageNode: no workload; the operator manages a role for a process that exists elsewhere. - Management handle: no workload; the operator manages the external Admin API target.
The storage identity guide explains why these shapes cannot be freely converted or scaled interchangeably.