Skip to main content

Automations — a separate Terraform root

This is not the main Terraform root

The automations department is an independent Terraform root under terraform/automations/. It matters operationally:

Main rootAutomations root
State keyprod/terraform.tfstate · dev/terraform.tfstateautomations/terraform.tfstate
CI laneterraform.yml (excludes this subtree)terraform-automations.yml
Default tagsProject=playrollDepartment=automations, Project=ec-automations

They share only the state bucket. An apply here can never touch prod state — the same isolation pattern as terraform/bootstrap/. Do not run a plan or apply from the wrong directory.

Runtime event flow

Unlike the video_indexing GPU in the main root — which has no AWS event trigger at alltranscribe_gpu has a fully declared wake path. The always-on hub instance holds an IAM role policy that lets it start, stop, and dispatch jobs to the GPU.

How the GPU is woken

The wake path is entirely in Terraform, in transcribe_gpu.tf:

  • Desired power state is stopped. aws_ec2_instance_state.transcribe_gpu declares state = "stopped" with lifecycle { ignore_changes = [state] }. The comment in the source says it plainly: "Runtime wake/stop is owned by the automation on the hub — Terraform must never fight a job in flight." So Terraform sets the baseline and then gets out of the way.
  • The hub may start and stop it. aws_iam_role_policy.hub_wake_transcribe_gpu (attached to aws_iam_role.hub) grants ec2:StartInstances and ec2:StopInstances, scoped to that one instance ARN (statement StartStopTranscribeGpu).
  • The hub sends it work. The same policy grants ssm:SendCommand against the instance plus the AWS-RunShellScript document (statement SendJobCommands).

So the lane is:

hub (always on) ──ec2:StartInstances──▶ transcribe_gpu (wake)
──ssm:SendCommand────▶ transcribe_gpu (run job)
──ec2:StopInstances──▶ transcribe_gpu (sleep)
What schedules the hub itself

Nothing in Terraform. The hub is a plain always-on EC2 instance; its user_data provisions systemd units and automation virtualenvs. The when lives on the box, not in the infrastructure code — so this diagram cannot show it, and does not pretend to.

Deploy lane

Deployment is push-based and holds no long-lived secrets: GitHub Actions assumes an OIDC role (github_actions_automations_deploy), pushes a build artifact to the deploy_artifacts S3 bucket, and the hub pulls it (hub_read_deploy_artifacts).

Not in the dependency map

The Dependency map is generated from terraform graph in the main root only. Producing the equivalent for this root requires terraform init against its own backend, which contacts S3. Since the automations root is small and its interesting relationships are the runtime ones above, only the event-flow diagram is generated here.