Skip to content

GCP

How indx runs on GCP: one Terraform stack, one dispatch-only workflow, one Cloud Run service closed behind a global load balancer that is the public origin, a DNS-proved Google-managed certificate, and the Vertex AI lanes as variables.

One Cloud Run service runs the image with its run.app URL closed, so a global load balancer is the only way in, and one DNS-only A record the operator creates by hand is the public hostname. Everything here is infra/gcp/ and .github/workflows/deploy-gcp-prod.yml, a dispatch-only workflow shaped like the AWS one and sharing no variable, secret, identity or state with it by name (ADR-0054). The stack can still be applied by hand, and it is the same stack either way.

Demonstrated on 2026-09-18 at everything.g.indx.jp: the deploy run is green through the public smoke, the @deployed scenarios and the web app’s specs.

The service has no database and no bucket, and no secret at all: Vertex AI is reached through LiteLLM on the instance’s own service account, so there is no key to hold. Its weights are in the image and read offline. The model lanes are reached only when the switch below is on; otherwise they report themselves unavailable and the runtime identity carries no aiplatform role.

The service takes bytes inline and nothing else. INDX_URI_SCHEMES=none disables every URI source, so no caller can name a file on the container’s disk or a URL inside the project, and INDX_LOADER_FILE_ROOTS is closed to the samples directory as a second layer (ADR-0042) — the three fixed environment lines the AWS task carries beside AWS_REGION.

Stack Applied by Holds
infra/gcp/terraform the workflow’s deploy, or an operator by hand the enabled APIs, Artifact Registry, the runtime service account, the Cloud Run service, the load balancer, the DNS authorization and its Google-managed certificate, the budget

State lives in a Cloud Storage bucket the operator creates first, named by a partial backend, so nothing about it is in the repository:

bucket = "indx-everything-prod-tfstate"
prefix = "indx-everything/gcp"

Nothing secret lands in that state: the origin certificate is Google-managed and its key never leaves Google, so the bucket holds resource addresses and the lock, and nothing more. just infra::terraform::validate formats, initializes without a backend and validates this stack with no credentials, and .github/workflows/infra.yml runs it on every pull request.

There is no perimeter. The A record is DNS-only, so a visitor reaches the balancer directly, and the balancer admits everyone. The other cloud behind Cloudflare’s proxy is AWS; this one is not, because the hostname is two labels below the zone and Cloudflare’s free certificate covers one, and because the operator holds no Cloudflare token: the two records this stack needs are made by hand (ADR-0065). The stack knows no DNS provider. The API is unauthenticated (ADR-0039), and Cloudflare Access is not available on a DNS-only record: authentication, when it comes, is the app’s or Google’s.

The service’s invoker check is disabled (invoker_iam_disabled), which is what lets the balancer forward unauthenticated. It is not an allUsers invoker grant, because the organization’s domain-restricted-sharing policy refuses that member on any IAM policy, and the switch names no member at all. ingress = INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER keeps the run.app URL closed, so the one hostname the balancer certifies is the only name the service answers to.

  • A project the operator owns, and gcloud. The stack takes the project as a plain variable and asserts nothing about it. A dedicated project is the right answer here; two settings below are project-wide.

  • Terraform >= 1.9, < 2.0.

  • A state bucket, created once:

    Terminal window
    gcloud storage buckets create gs://indx-everything-prod-tfstate \
    --project=<project id> --location=asia-northeast1 --uniform-bucket-level-access
    gcloud storage buckets update gs://indx-everything-prod-tfstate --versioning
  • Write access to the zone’s DNS, in a dashboard. No credential is read: the two records the hostname needs, an A record to the balancer and the _acme-challenge CNAME Google issues the certificate against, are created by hand, and their values exist only once the stack does, so the first deployment prints them and stops.

The stack enables the APIs it needs itself — run, artifactregistry, compute, certificatemanager, iam, logging and billingbudgets, plus aiplatform and cloudresourcemanager with the switch on — because a fresh project answers every resource with a “not enabled” error otherwise, which is the harder error to read. They are not disabled on destroy: the project is the operator’s and may hold more than this stack.

The Vertex AI quota itself is not requested here. It belongs to the project’s owner, not to a template (ADR-0053).

The API is public, and every call to a Vertex AI-backed lane is billed to the project, so those lanes are off unless enable_vertex_ai is true. Off means the container carries no model variable, so llm, ner-llm and enrich-llm are not advertised and a request naming one is refused with a 422, and generic-vlm and hosted-text report themselves unavailable. A model named while the switch is off fails terraform plan rather than being ignored, and so does the switch turned on without a billing account or without an address to notify.

On, these variables become these lines on the container:

Variable Becomes
gcp_project_id VERTEXAI_PROJECT
vertex_location VERTEXAI_LOCATION, asia-northeast1 by default; model availability is per region and is not the region the service runs in, which is why it is its own variable
vertex_llm_model INDX_CLASSIFIER_LLM_MODEL, INDX_NER_LLM_MODEL, INDX_ENRICH_LLM_MODEL and INDX_VLM_MODEL, each vertex_ai/<model>
vertex_embed_model INDX_EMBED_MODEL, vertex_ai/<model>
vertex_embed_dimension INDX_EMBED_DIMENSION

There is no key and no secret: the switch adds roles/aiplatform.user to the runtime service account and turns aiplatform.googleapis.com on, and LiteLLM takes the credential from the instance. The dimension is declared and not discovered: a hosted embedding space with no dimension is not advertised at all, silently, so the plan refuses an embedding model without one.

The switch also creates a monthly Cloud Billing budget on billing_account (vertex_monthly_budget_usd, 50 by default, whole USD), which notifies alert_emails through one monitoring channel per address at 80 percent forecast and 100 percent actual. It is a horn and not a brake: a Cloud Billing budget cannot detach a role the way AWS Budgets can (ADR-0040 is AWS-only), so the operator turning the switch off is what stops the spend, and the overshoot is whatever is billed between the alert and that apply. Its scope is the whole project, not Vertex AI: the Cloud Run instance, the load balancer and the registry are all counted with the model calls.

The workflow holds no key. GitHub mints an OIDC token for the Environment a job runs in, and a Workload Identity Federation provider in the project exchanges it for a token on one of two service accounts. Creating that is a one-time sequence with gcloud, by hand, like the state bucket above.

One pool and one provider, on GitHub’s issuer, admitting exactly the two subjects this workflow runs as. The subject carries the immutable owner and repository IDs rather than their names, so a repository rename cannot transfer the trust, which is the form the AWS roles trust too.

Terminal window
repo="repo:INDXDev@209891251/indx-everything@1301159823"
gcloud iam workload-identity-pools create github \
--project=<project id> --location=global --display-name="GitHub Actions"
gcloud iam workload-identity-pools providers create-oidc github \
--project=<project id> --location=global --workload-identity-pool=github \
--issuer-uri=https://token.actions.githubusercontent.com \
--attribute-mapping=google.subject=assertion.sub \
--attribute-condition="assertion.sub in ['${repo}:environment:production-gcp','${repo}:environment:production-gcp-plan']"

Two service accounts, because the plan identity is read-only and the deploy identity is not. Each is bound to the principal for its own subject alone, so a token minted for the plan Environment cannot impersonate the deploy account.

Terminal window
gcloud iam service-accounts create indx-everything-prod-deploy --project=<project id>
gcloud iam service-accounts create indx-everything-prod-plan --project=<project id>
number=$(gcloud projects describe <project id> --format='value(projectNumber)')
principal="principal://iam.googleapis.com/projects/${number}/locations/global/workloadIdentityPools/github/subject"
gcloud iam service-accounts add-iam-policy-binding \
indx-everything-prod-deploy@<project id>.iam.gserviceaccount.com \
--project=<project id> --role=roles/iam.workloadIdentityUser \
--member="${principal}/${repo}:environment:production-gcp"
gcloud iam service-accounts add-iam-policy-binding \
indx-everything-prod-plan@<project id>.iam.gserviceaccount.com \
--project=<project id> --role=roles/iam.workloadIdentityUser \
--member="${principal}/${repo}:environment:production-gcp-plan"

The deploy account’s roles are derived from what the stack creates. roles/serviceusage.serviceUsageAdmin enables the APIs, roles/artifactregistry.admin creates the repository, sets its IAM member and pushes the image, roles/iam.serviceAccountAdmin creates the runtime account and roles/iam.serviceAccountUser is what lets the Cloud Run service run as it, roles/run.admin creates the service with its invoker check disabled, roles/compute.loadBalancerAdmin covers the NEG, the backend service, the URL map, the HTTPS proxy, the global address and the forwarding rule, roles/certificatemanager.editor the DNS authorization, the certificate and the map the proxy reads it through, roles/logging.admin the _Default bucket’s retention, and roles/browser the project read the budget’s filter makes. The last two lines are needed only with the Vertex AI switch on, roles/resourcemanager.projectIamAdmin for the roles/aiplatform.user binding on the runtime account and roles/monitoring.notificationChannelEditor for the channel the budget notifies through.

Terminal window
for role in \
roles/serviceusage.serviceUsageAdmin \
roles/artifactregistry.admin \
roles/iam.serviceAccountAdmin \
roles/iam.serviceAccountUser \
roles/run.admin \
roles/compute.loadBalancerAdmin \
roles/certificatemanager.editor \
roles/logging.admin \
roles/browser \
roles/resourcemanager.projectIamAdmin \
roles/monitoring.notificationChannelEditor; do
gcloud projects add-iam-policy-binding <project id> \
--member="serviceAccount:indx-everything-prod-deploy@<project id>.iam.gserviceaccount.com" \
--role="$role" --condition=None
done

The budget is a billing-account resource, not a project one, so its role goes on the account, and only with the switch on.

Terminal window
gcloud billing accounts add-iam-policy-binding <billing account> \
--member="serviceAccount:indx-everything-prod-deploy@<project id>.iam.gserviceaccount.com" \
--role=roles/billing.costsManager

The plan account gets roles/viewer and nothing else. Both accounts get roles/storage.objectAdmin on the state bucket, because the GCS backend writes a lock object even to plan.

Terminal window
gcloud projects add-iam-policy-binding <project id> \
--member="serviceAccount:indx-everything-prod-plan@<project id>.iam.gserviceaccount.com" \
--role=roles/viewer --condition=None
for account in indx-everything-prod-deploy indx-everything-prod-plan; do
gcloud storage buckets add-iam-policy-binding gs://indx-everything-prod-tfstate \
--member="serviceAccount:${account}@<project id>.iam.gserviceaccount.com" \
--role=roles/storage.objectAdmin
done

That list is derived from the resources above and is untested until the first run, so read a refusal on the first deploy as a missing role rather than a broken stack. The stack asserts nothing about any of it. A narrower policy, a custom role per resource family, is the operator’s to write.

Then two Environments. production-gcp-plan with no reviewer and a deployment branch rule of main only, and production-gcp with a required reviewer and the same branch rule. That reviewer is the approval for every mutation.

Then the repository variables and secrets, which are the names the workflow reads and no others:

Name Kind Value
GCP_PROD_DOMAIN_NAME variable, optional the public hostname, any name under indx.jp; gcp.indx.jp by default
GCP_PROD_PROJECT_ID variable the project every resource is created in; no default
GCP_PROD_WORKLOAD_IDENTITY_PROVIDER variable the provider’s full resource name, projects/<number>/locations/global/workloadIdentityPools/github/providers/github; no default
GCP_PROD_SERVICE_ACCOUNT variable the deploy account’s email; no default
GCP_PROD_PLAN_SERVICE_ACCOUNT variable the plan account’s email; no default
GCP_PROD_PROJECT_NAME variable, optional the name every resource is prefixed with, and the registry repository; indx-everything-prod by default
GCP_PROD_REGION variable, optional the region the service and the registry live in; asia-northeast1 by default
GCP_PROD_TF_BACKEND_BUCKET variable, optional the state bucket; indx-everything-prod-tfstate by default
GCP_PROD_TF_BACKEND_PREFIX variable, optional the prefix in it; indx-everything/gcp by default
GCP_PROD_MIN_INSTANCES variable, optional 0 or 1; 0 by default
GCP_PROD_VERTEX_ENABLED variable, optional true turns the Vertex AI lanes on; false by default
GCP_PROD_VERTEX_LLM_MODEL variable, optional e.g. gemini-2.5-flash; no default, and refused by the plan while the switch is off
GCP_PROD_VERTEX_EMBED_MODEL variable, optional e.g. text-multilingual-embedding-002; no default
GCP_PROD_VERTEX_EMBED_DIMENSION variable, required with the embedding model the model’s vector size; 0 by default, which the plan refuses beside a model
GCP_PROD_VERTEX_LOCATION variable, optional the Vertex AI region, which is not the region the service runs in; asia-northeast1 by default
GCP_PROD_BILLING_ACCOUNT variable, required with the switch the account the budget is created on, e.g. 012345-6789AB-CDEF01; no default
GCP_PROD_VERTEX_MONTHLY_BUDGET_USD variable, optional whole USD the budget alerts on; 50 by default
GCP_PROD_ALERT_EMAILS variable, required with the switch the addresses the budget notifies, as a JSON list, e.g. ["ops@example.com"]; [] by default

There is no secret in that list at all. Vertex AI is reached on the instance’s own service account, the certificate is Google’s, and this workflow writes no DNS record.

GCP_PROD_DOMAIN_NAME is any name under indx.jp, so the hostname need not say which cloud serves it. The three stacks write records in one zone, so every job refuses a name equal to PROD_DOMAIN_NAME or AZURE_PROD_DOMAIN_NAME, the other two clouds’ hostnames, before anything else runs. That is the one check that keeps two workflows from naming one record. deploy runs only when the operator has typed the name into confirm.

Every operation is Deploy to GCP (production) on main, dispatched by hand. No push deploys. There is no cutover, because the record is the operator’s and is never moved. There is no rollback job either, because a rollback is a deploy of a tag the registry already holds.

action Environment What happens
plan production-gcp-plan A read-only plan of the stack. Its summary is the artifact a deploy must name.
deploy production-gcp Builds the image into Artifact Registry if the tag is absent, re-plans and refuses when the plan differs from the approved run, applies, waits for the service, checks the two DNS records and waits for the certificate, and proves the public host.

Every run starts with the static validation job, which validates the stack, lints the shared scripts and runs actionlint over the workflow, with no production credentials. Then the one job the action names.

Runs in production-gcp-plan, so no reviewer, but only from main and only as GCP_PROD_PLAN_SERVICE_ACCOUNT, which can read and cannot write.

  1. Guards the hostname, and the image_tag input’s shape when one is given.
  2. Exchanges the Environment’s OIDC token for application-default credentials, which are what the google provider and the GCS backend read; nothing else is exported for them.
  3. Initializes against GCP_PROD_TF_BACKEND_BUCKET and GCP_PROD_TF_BACKEND_PREFIX and plans with the image reference <region>-docker.pkg.dev/<project id>/<project name>/indx-everything:sha-<commit>.
  4. Uploads core-plan-summary.json as gcp-prod-plan-summary-<run id>, kept 7 days. It is the resource addresses and their actions plus the commit, ref and repository it was planned from, and nothing secret.

Note the run ID. It is the plan_run_id a deploy must cite, and it is valid for that commit only.

Dispatch with confirm set to the hostname and plan_run_id set to that plan run. The reviewer approves the production-gcp Environment, then:

  1. Guards the ref, the hostname, confirm, the numeric plan_run_id and the image_tag shape, and downloads the approved plan summary from that run.
  2. Applies google_artifact_registry_repository.main alone, so the repository exists before the push. -target pulls the enabled APIs in with it.
  3. Frees runner disk, logs into GHCR and ensures the weights image, signs in to the registry, and builds Dockerfile for linux/amd64 and pushes sha-<commit>, skipped when that tag is already in the registry.
  4. Plans again and compares that plan with the approved summary. The provenance must be this repository, main and this commit, and the change set must be identical except for the registry and the enabled APIs the bootstrap in step 2 created. Any other difference fails the run before anything is applied.
  5. Applies that plan, then waits up to 900 seconds for the Cloud Run service to report Ready.
  6. Resolves the hostname’s A record and the _acme-challenge CNAME and compares them with the balancer’s address and the DNS authorization’s record. When they differ, the step writes the two records to create into the job summary and fails, which is where the first deploy ends. When they match, it waits up to 30 minutes for the certificate to become ACTIVE; Google issues it once it has read the challenge record, usually within minutes, and never reaches the origin.
  7. Runs the public smoke. It asserts /health, the capabilities each hosted lane is supposed to have, that a file: URI is refused, that both pages are served, and that the balancer’s address answers a direct client naming the hostname with a 200, because it is the public origin. With min_instances = 0 the first request is expected to fail while the instance wakes, so the smoke retries /health for up to 900 seconds before it asserts anything.
  8. Runs just test::bdd::deployed against the hostname, the same @deployed scenarios an operator runs by hand.
  9. Runs just frontend::e2e-deployed against the hostname, so a deploy whose UI cannot run the demo is a red workflow and not a green one.

The first deploy is two cycles. The balancer’s address and the challenge record exist only once the stack does, so the first run applies it, prints the A record and the CNAME to create in its summary and fails at step 6. Create them, DNS-only, then run plan and deploy again: that second run waits for the certificate and runs the three suites.

The run’s own plan summary is uploaded as gcp-prod-deploy-summary-<run id> whether it passed or not.

Set image_tag to the sha-<commit> of an image already in the registry, on the plan and again on the deploy. The plan is then the plan of that image, and it is what the reviewer reads; the deploy refuses a named tag the registry does not hold, rather than building this commit under an earlier commit’s name. That is the rollback. An earlier image is redeployed through the same approval, and the records never move, because the address they point at is in the stack the apply reconciles.

This is the same stack the workflow applies, for an operator who does not have the GitHub setup above. Sign in — Terraform reads application-default credentials, which gcloud auth login alone does not write:

Terminal window
gcloud auth login
gcloud auth application-default login
gcloud config set project <project id>

Write the backend and the variables from the two examples, then initialize:

Terminal window
cd infra/gcp/terraform
cp backend.hcl.example backend.hcl
cp terraform.tfvars.example terraform.tfvars
terraform init -backend-config=backend.hcl

The registry has to exist before the image can be pushed to it, and the Cloud Run service cannot start without the image, so the registry is applied alone first. -target pulls in the enabled APIs it needs:

Terminal window
terraform apply -target=google_artifact_registry_repository.main

Build the image and push it under the tag terraform.tfvars names:

Terminal window
just infra::image::build
tag="sha-$(git rev-parse HEAD)"
repo="asia-northeast1-docker.pkg.dev/<project id>/indx-everything-prod/indx-everything"
docker tag indx-everything:local "$repo:$tag"
gcloud auth configure-docker asia-northeast1-docker.pkg.dev
docker push "$repo:$tag"

Apply the rest with the switch off, so nothing is billed to a model, and read the two values the records need:

Terminal window
terraform apply
terraform output -raw load_balancer_ip
terraform output -json acme_challenge_record

Create the two records in the zone, DNS-only: an A record at the hostname whose content is the address, and a CNAME at the challenge record’s name whose content is its data.

Google reads the _acme-challenge record and issues the certificate against it in the minutes after, and until then the balancer answers nothing on 443. Watch it become ACTIVE:

Terminal window
gcloud certificate-manager certificates describe indx-everything-prod-origin \
--project=<project id> --format='value(managed.state)'

Then the hostname must answer, and so must the address itself when the client names the hostname, because the balancer is the public origin:

Terminal window
curl -sf https://<hostname>/health | jq -e '.status == "ok"'
curl -s -o /dev/null -w '%{http_code}\n' \
--resolve "<hostname>:443:$(terraform output -raw load_balancer_ip)" \
https://<hostname>/health

Then check the rest of the host:

Terminal window
curl -sf https://<hostname>/ | grep -q '<html'
curl -sf https://<hostname>/en/deploy/ | grep -q '<html'

Then warm the service once and run the two suites that drive a real host:

Terminal window
curl -sf https://<hostname>/health >/dev/null
just test::bdd::deployed https://<hostname>
just frontend::e2e-deployed https://<hostname>

The warm-up is not ceremony; see below. Last, turn the switch on and apply again, with enable_vertex_ai = true, billing_account, the models, the dimension and alert_emails set in terraform.tfvars:

Terminal window
terraform apply

Flipping the switch either way is a new revision, because the container’s environment changes.

min_instances is 0 by default and validated to 0 or 1; max_instance_count is 1 either way. Zero costs nothing while idle. One keeps an instance warm and is billed for it around the clock.

Zero is not free of consequence. The cold start is an image pull plus an engine import, which the startup probe allows 240 seconds for — 24 failures at 10 seconds, Cloud Run’s ceiling. Cloud Run’s own request timeout (request_timeout_seconds, 120 by default) is measured from when the request arrives, so it bounds the wait for a cold instance too: a start that outlasts it ends in a 504 while the probe still has time to finish. The first request after an idle period is therefore expected to fail, and a request a minute later to succeed against the instance it woke. A scripted client must allow for that first failure, or min_instances = 1 is the setting for anyone who cannot.

The cold start has not been measured on a real deployment yet; fill the number in here after the first one.

  • Both records must stay DNS-only. Google resolves _acme-challenge.<hostname> to issue and renew the certificate, and that record must be the only one at its name. A proxied A record would put Cloudflare’s edge in front of a hostname its free certificate does not cover, so the browser fails the handshake before the balancer is reached. A CAA record on indx.jp would have to allow pki.goog; there is none today.
  • The certificate is issued after the records exist. DNS authorization is asynchronous: Google issues once it has read the challenge record, usually within minutes. managed.state on the certificate is the thing to watch; the workflow waits for it before the smoke. Until then the balancer answers nothing on 443.
  • The record check reads DNS with the runner’s resolver. A record created a moment before the run can still be absent there, and the step then fails with the records printed as if they did not exist. Wait a minute and dispatch again; nothing to edit.
  • _Default log-bucket retention is project-wide. log_retention_days has nowhere else to go: Cloud Run writes to Cloud Logging and there is no per-service store. In a shared project this silently changes another workload’s retention. A dedicated project is the real answer.
  • cpu_idle = false bills the instance’s whole lifetime, not its requests. CPU stays allocated between requests because the engines warm their caches on first use and a throttled instance would do that arbitrarily slowly. With min_instances = 1 that is container_cpu — two vCPU by default — billed around the clock.
  • The budget watches the project, not the model. Narrowing it to the Vertex AI service is a budget_filter.services entry and is work this template does not do.
  • monitoring.googleapis.com is not in the enabled list. It is on by default in a new project. One that has it disabled fails at the notification channel, with the switch on. cloudresourcemanager.googleapis.com is in the list, but only with the switch: the budget’s filter takes the project number, which is a Cloud Resource Manager read. The count on that read is what removes it while the switch is off, and its depends_on is what holds it until apply, so the first plan with the switch on does not call an API that is not enabled yet.
  • deletion_protection is false. The google provider defaults it to true, which makes terraform destroy fail until an extra apply clears it — a trap in a stack one operator applies and tears down by hand. Nothing stops a destroy here but the operator.