agatha
bf27c97deb
Adds complete k8s/ manifest tree: Namespace, VaultAuth + VaultStaticSecret CRDs (VSO secret sync from Vault KV v2), API and UI Deployments and Services, nginx Ingress with cert-manager TLS, MinIO StatefulSet with PVC and init Job, and Alembic init container on the API Deployment for automatic schema migrations. Includes .yamllint.yml config and validate-k8s Makefile target. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
64 lines
6.2 KiB
Markdown
64 lines
6.2 KiB
Markdown
# Research: Kubernetes Production Manifests
|
|
|
|
## Decision 1: VSO CRD chain (VaultConnection → VaultAuth → VaultStaticSecret)
|
|
|
|
**Decision**: Use three CRDs — `VaultConnection`, `VaultAuth`, and `VaultStaticSecret` — all under `apiVersion: secrets.hashicorp.com/v1beta1`.
|
|
**Rationale**: This is the required VSO resource chain. `VaultConnection` points to the Vault server address. `VaultAuth` declares the Kubernetes auth method (role, service account, mount path). `VaultStaticSecret` references a `VaultAuth` via `vaultAuthRef` and declares the Vault KV path and the destination K8s Secret name. VSO syncs all Vault keys to the K8s Secret 1:1 by default — no explicit key mapping needed.
|
|
**Alternatives considered**: `VaultAuthGlobal` for cross-namespace sharing — not needed; all resources are in the same `reactbin` namespace.
|
|
|
|
Key fields:
|
|
- `VaultStaticSecret.spec.type`: `kv-v2` (standard for modern Vault)
|
|
- `VaultStaticSecret.spec.refreshAfter`: `1h` (Go duration string)
|
|
- `VaultStaticSecret.spec.destination.create: true` — VSO creates the K8s Secret if absent
|
|
- `VaultAuth.spec.kubernetes.role` — a Vault role the operator must pre-create and bind to the `reactbin` namespace service account
|
|
|
|
## Decision 2: MinIO as StatefulSet (not Deployment)
|
|
|
|
**Decision**: Run MinIO as a `StatefulSet` with `volumeClaimTemplates`.
|
|
**Rationale**: StatefulSet gives the pod a stable name (`minio-0`) and automatically reattaches its PVC on pod recreation. A Deployment would require a manually-created PVC and is prone to PVC binding issues on reschedule. The marginal complexity of a StatefulSet over a Deployment is acceptable. `ReadWriteOnce` PVC is correct for single-replica MinIO.
|
|
**Alternatives considered**: Deployment with explicit PVC — works but PVC lifecycle is decoupled from the pod, creating operational risk.
|
|
|
|
MinIO health probes:
|
|
- Liveness: `GET /minio/health/live:9000`
|
|
- Readiness: `GET /minio/health/ready:9000`
|
|
|
|
MinIO env vars: `MINIO_ROOT_USER`, `MINIO_ROOT_PASSWORD` (injected from a K8s Secret synced by VSO).
|
|
|
|
## Decision 3: Bucket initialisation via Kubernetes Job with `minio/mc`
|
|
|
|
**Decision**: A one-off `Job` using `minio/mc:latest` runs `mc mb --ignore-existing` to create the bucket idempotently.
|
|
**Rationale**: This is the standard in-cluster pattern. `--ignore-existing` makes the job safe to re-apply (exits 0 if bucket already exists). `restartPolicy: OnFailure` retries transient failures (e.g. MinIO not yet ready).
|
|
**Alternatives considered**: Init container on the API pod — tightly couples bucket creation to API startup; a Job is cleaner and independently rerunnable.
|
|
|
|
## Decision 4: Ingress — single resource, `/api/` path before `/`
|
|
|
|
**Decision**: One `Ingress` resource with `ingressClassName: nginx`, two path entries in a single rule: `/api/` (Prefix) → API Service, `/` (Prefix) → UI Service; `/api/` must be listed first.
|
|
**Rationale**: nginx ingress evaluates paths in declaration order; the more specific `/api/` prefix must appear before `/` or all traffic is routed to the UI. No path rewriting annotation is needed — the API already handles full `/api/v1/...` paths.
|
|
**TLS**: cert-manager annotation `cert-manager.io/cluster-issuer: letsencrypt-prod` triggers automatic certificate provisioning into a K8s Secret named in `spec.tls[].secretName`. HTTP→HTTPS redirect is on by default when TLS is configured (`nginx.ingress.kubernetes.io/ssl-redirect: "true"` is explicit but redundant).
|
|
**Alternatives considered**: Two separate Ingress resources (one per service) — works but harder to reason about routing order; single Ingress is canonical.
|
|
|
|
## Decision 5: Alembic init container — same image, workdir `/app`
|
|
|
|
**Decision**: The API Deployment includes an init container with the same image as the main container, `command: ["alembic", "upgrade", "head"]`, and `workingDir: /app`. It shares the API's env secret via `envFrom` so it can read `DATABASE_URL`.
|
|
**Rationale**: Alembic needs `DATABASE_URL` to connect and `alembic.ini` + `alembic/` to find migrations. Both are available in the production image once `Dockerfile.prod` is updated. Using the same image guarantees the migration files match the running version.
|
|
**Dockerfile.prod update required**: Add `COPY --chown=appuser:appgroup alembic/ ./alembic/` and `COPY --chown=appuser:appgroup alembic.ini .` in the runtime stage (not the builder stage — no compilation needed).
|
|
**Alternatives considered**: Separate migration image — adds a second image to build and push on every release; unnecessary when the source image already has everything.
|
|
|
|
## Decision 6: Image tag strategy — placeholder `latest`, substituted at deploy time
|
|
|
|
**Decision**: Manifests reference image tags using `latest` as a documented placeholder. The operator substitutes the real tag with `kubectl set image` or a `sed` one-liner before applying.
|
|
**Rationale**: Kustomize's `images` transformer is the clean alternative, but introduces a tooling dependency. For a personal single-operator deployment, `sed` or `kubectl set image` after `kubectl apply` is simpler and requires no additional setup. The placeholder is documented in the operator guide (quickstart.md).
|
|
**Alternatives considered**: Kustomize overlays — appropriate for multi-environment setups; over-engineered for one environment.
|
|
|
|
## Decision 7: Two VaultStaticSecrets (API env and MinIO credentials)
|
|
|
|
**Decision**: Separate VaultStaticSecret resources for API env vars and MinIO root credentials, syncing into `api-env` and `minio-credentials` K8s Secrets respectively.
|
|
**Rationale**: The API's env secret contains database, JWT, and S3 access credentials. MinIO's root credentials are a different concern with a different rotation lifecycle. Keeping them separate makes Vault policies simpler (least privilege) and avoids giving the API pod access to MinIO's root password.
|
|
**Vault paths assumed**: `reactbin/api/config` (KV v2) for API env; `reactbin/minio/credentials` (KV v2) for MinIO root credentials.
|
|
|
|
## Decision 8: Namespace manifest included in `k8s/`
|
|
|
|
**Decision**: `k8s/namespace.yaml` creates the `reactbin` namespace as part of the manifest set.
|
|
**Rationale**: Makes the full deployment self-contained — operator runs `kubectl apply -f k8s/` without a prerequisite namespace creation step.
|
|
**Note**: If the namespace already exists, `kubectl apply` is idempotent.
|