docs / deployment

Deploying Fred on GCP / GKE

A typical Google Kubernetes Engine deployment — the workloads, the in-cluster services, the managed Google Cloud services, and how you know it is correct, not just running.

This documents a typical GKE deployment. Fred is cloud-agnostic — the same shape runs on AWS (EKS) or Azure (AKS), swapping the managed pieces (object storage, image registry, managed TLS, model API) for each cloud's equivalents. The specifics below are GCP. Project, region, and host names are placeholders — PROJECT_ID, REGION, studio.example.com.

Scope & audience

For operators and platform engineers standing up Fred on GKE Autopilot. This page is the map: the shape of a deployment, what talks to what, and its definition of "done". It is deliberately not the environment-specific runbook (exact hostnames, secrets, and commands) — that ships with the Helm chart and stays in sync with it.

Companion pages: Platform Architecture covers the logical component model and data flows; this page is the operational deployment view.

Topology at a glance

A Fred instance runs in a single Kubernetes namespace on a GKE Autopilot cluster. Four stateless application workloads share a set of in-cluster services (Postgres, Keycloak, OpenFGA, OpenSearch, Temporal). The only reach outside the cluster is to a few managed Google Cloud services — reached keylessly through Workload Identity.

GKE AUTOPILOT CLUSTER — ONE NAMESPACE users HTTPS GKE Ingress (gce) · Managed TLS · reserved global IP + DNS FRED WORKLOADS · GITOPS fred-frontend fred-agents — runtime · SSE control-plane-backend knowledge-flow · backend + worker IN-CLUSTER SERVICES · HELM FOUNDATION PostgreSQL (pgvector) OpenSearch Keycloak (OIDC) OpenFGA (ReBAC) Temporal one Postgres, a database per component durable state — no ephemeral pod storage databases: keycloak · openfga · temporal · fred MANAGED GOOGLE CLOUD Cloud Storage (GCS)object storage Vertex AIembeddings · LLM Artifact Registrycontainer images Cloud Build objects embed

Fred's stateless workloads (deployed via GitOps) and its in-cluster services run in one namespace. Only Knowledge Flow reaches managed Google Cloud — Cloud Storage and Vertex AI — authenticated keylessly through Workload Identity (no service-account keys). Images come from Artifact Registry, built by Cloud Build.

What runs in the cluster

Fred workloads (stateless)

app
control-plane-backend
Teams, users, policies and permissions, agent-template and instance lifecycle, session metadata, and ExecutionGrant issuance. The sole authority for agent discovery and enrollment.
app
fred-agents
The agent execution surface (/agents/execute, /agents/execute/stream, plus an OpenAI-compatible endpoint). Runs the agent definitions; conversation state persists to Postgres.
app
knowledge-flow (backend + worker)
Ingestion, documents, tags/libraries, vectors, and retrieval — the only component that uses object storage. The Temporal worker drains the ingestion/processing queue.
app
fred-frontend
Nginx serving the static React UI; proxies to the backends, so it comes up last, after their Services exist.

In-cluster services

data
PostgreSQL (pgvector)
One StatefulSet, a database per component (keycloak, openfga, temporal, fred). Backed by a persistent volume — never ephemeral pod storage.
data
OpenSearch
Single-node store for hybrid/vector search and the control-plane KPI / analytics data.
infra
Keycloak · OpenFGA · Temporal
OIDC identity (a realm with a public frontend client and confidential per-backend clients), relationship-based authorization (ReBAC), and durable workflow scheduling (with a gated Temporal UI).
Postgres, OpenSearch, Keycloak, OpenFGA and Temporal run as workloads in the cluster in a typical deployment — not as managed services. You can move any of them to a managed equivalent (e.g. Cloud SQL) later; the apps only need the connection string.

Managed Google Cloud services

A typical GKE deployment leans on a small set of managed services:

gcp
Cloud Storage (GCS)
Object storage for the content store (documents, extracted objects, files) and the virtual filesystem. Selected with a type: gcs discriminator; used only by Knowledge Flow.
gcp
Vertex AI
Embeddings and model inference for ingestion and retrieval, granted via roles/aiplatform.user.
gcp
Artifact Registry + Cloud Build
Container image registry and builder. Images are tagged per release and referenced by the workloads.

Keyless object storage — Workload Identity

Fred authenticates to Google Cloud with no service-account keys. It uses GKE Workload Identity: clients call Application Default Credentials, and the pod's Kubernetes service account (KSA) is federated to a Google service account (GSA).

  1. A GSA (e.g. fred-knowledge-flow-gcs@PROJECT_ID.iam.gserviceaccount.com) is granted roles/storage.objectAdmin on the buckets and roles/aiplatform.user.
  2. The GSA is bound to the workload's KSA with roles/iam.workloadIdentityUser.
  3. The KSA is annotated iam.gke.io/gcp-service-account: <GSA>. That is the whole wiring — no JSON key is ever created, committed, or mounted.
No secret for storage. Because access is keyless, object storage needs no entry in the deployment's secret. Rotation and revocation happen entirely through IAM.

Edge, TLS & secrets

Ingress, TLS and DNS

All external traffic enters through a single GKE Ingress, fronted by a reserved global static IP. Public hostnames (frontend, Keycloak, and the Temporal UI) resolve via DNS A-records you create outside Helm. TLS uses GKE ManagedCertificate resources — one per host — which provision automatically once DNS points at the IP.

Certificates take time. A managed certificate goes Active only after DNS resolves to the load-balancer IP — typically 15–60 minutes. A ready pod behind a still-provisioning certificate is unreachable over HTTPS; don't mistake that for a failure.

Secrets & configuration

Sensitive values live in a single Kubernetes Secret, rendered from a git-ignored values file (a committed example shows the shape with empty values). It carries the Postgres passwords, the Keycloak admin password and confidential client secrets, the OpenFGA API token, and any optional model-provider API key. Object storage is absent by design — it uses Workload Identity. Non-secret configuration (hostnames, feature flags, per-team quotas) is plain Helm values.

How it is deployed

The deployment splits cleanly into two layers with different change cadences:

foundation
Foundation — imperative Helm
The slow-moving base: Postgres, Keycloak, OpenFGA, OpenSearch, Temporal, the shared Ingress, the managed certificates, and the one Secret. Provisioning runs as Helm hook Jobs — create databases and users, seed the Keycloak realm and clients, apply schema migrations, register the Temporal namespace.
apps
Apps — GitOps via Argo CD
The four stateless workloads, delivered continuously. A single Argo CD Application tracks a multi-workload chart; per-app enable flags and pinned image tags live in a values file. Sync is deliberately manual (server-side apply), so a rollout is an explicit, reviewable action.
Autopilot-friendly. Every workload sets explicit CPU/memory requests and runs unprivileged. App Deployments replace in place (maxSurge: 0) rather than surging a second copy of all four at once — which would spike memory and node auto-provisioning on a small cluster. Postgres uses a modest persistent volume on a standard storage class.

Deployment steps

  1. Prerequisites. gcloud, kubectl, and helm authenticated to the target project; a GKE Autopilot cluster; the target namespace.
  2. Reserve the edge. Reserve a global static IP and create DNS A-records for the public hostnames pointing at it.
  3. Google Cloud prerequisites. Create the Artifact Registry repository and enable Cloud Build; create the GCS buckets, the GSA, the Workload Identity binding, and enable Vertex AI.
  4. Fill secrets. Copy the example values file, fill the required secret values, and keep it out of version control.
  5. Deploy the foundation. Install the foundation chart. Its hook Jobs provision the databases and users, seed the Keycloak realm and clients, and register the Temporal namespace. Confirm the provisioning Jobs completed.
  6. Build images. Build the component images at one shared, date-stamped tag via Cloud Build, so a release round is one coherent version.
  7. Run migrations. Apply the control-plane and knowledge-flow schema migrations (each tracks its own migration version).
  8. Deploy the apps. Bring up control-plane-backend, then knowledge-flow (backend + worker) and fred-agents, and the fred-frontend last — it resolves backend DNS at startup.
  9. Provision identity. Create the initial users, teams, and groups so the groups claim is populated.
  10. Verify. Run the status/completeness check until it is fully green (see below), then log in over HTTPS and run an ingestion smoke test.
Steady state is a GitOps loop: build and bump image tags, commit and push, trigger the Argo CD sync, and re-run the status check. A config-only change is a plain rollout restart — no full release.

Verifying the deployment — complete and correct

The hardest deployment bugs are silent: every pod is 1/1 Ready, yet the product is subtly broken. Liveness is not correctness. Treat "done" as an executable checklist — a single status command green across every dimension:

presence
Workloads & expected components
Every desired replica Ready — and every expected component present, so a disabled or missing service is flagged, not silently absent.
deps
Dependency health
Each service's /ready probe confirms its real backends (Postgres, OpenSearch, OpenFGA, object storage) — not just that the pod started.
edge
TLS certificates Active
Managed certificates must be Active, not provisioning — a ready pod behind a half-issued certificate is unreachable over HTTPS.
correct
Config & schema correctness
The semantic checks liveness can't see: durable (not ephemeral) conversation storage, the analytics store wired, keyless object-storage access working, migrations applied.

Gotchas worth knowing

Durable conversation storage. The agent runtime must persist to a real database, not a pod-local file on an emptyDir — otherwise conversations vanish on every restart while still appearing in the session list.
Analytics needs its store wired. The KPI dashboard reads from OpenSearch; if that sink is not enabled and connected, its endpoints return errors even though everything else is healthy.
Don't tear down the edge to "restart". Disabling the frontend deletes its managed certificate; bringing it back triggers a fresh 15–60 min re-provision. Roll the pod instead.