Advanced: Cloud Infrastructure, Containers, and Infrastructure as Code

Why it matters#

Chapter 14 deployed the Learning Coach on managed services, with the same container image promoted from staging to production. That was the right start. It left one thing implicit: the infrastructure itself, meaning the network, database, storage, identities, secrets, and alerts. Those were created by people, in consoles, from notes.

That works for one environment and two people. It stops working when any of these happen:

• Environments multiply. Staging, production, a load-test environment, a campus that needs its own account. Each one built by hand is one more chance to differ.
• Changes happen under pressure. Incidents produce fixes made at night. Without a shared definition, those fixes stay invisible until something depends on them.
• Someone has to audit it. Northfield's security team will ask how student data is protected. “Here's the code that defines it, and here's the history of every change” is a far stronger answer than screenshots.
• AI helps write it. Agents are good at generating infrastructure definitions. They're also good at generating ones that open a database to the internet or grant every permission. Infrastructure written as code can be reviewed and tested like any other code; clicks can't.

This module goes deeper than chapter 14 into four areas: infrastructure as code, networks and identity, containers, and the decision about orchestration. It's for readers heading toward roles where they'll own production infrastructure, or review someone else's.

Learning objectives#

After this module you will be able to:

1. Explain infrastructure as code, plans, state, and drift, and use a plan as the main review artifact for infrastructure changes.
2. Design a cloud network with private and public parts, and give each workload and person only the permissions it needs, with temporary credentials.
3. Build small, reproducible container images with health checks and clean shutdown.
4. Decide when Kubernetes is justified and when a managed container platform is enough, with a written revisit trigger.
5. Serve several environments and tenants from one definition, choosing between shared and dedicated deployments.
6. Produce an infrastructure module, a drift report, and an environment rebuild drill.

22.1 From clicks to code#

What “infrastructure” covers#

In this module, infrastructure means everything your application needs that isn't your application's code: the private network it runs in, the database server and its settings, storage buckets, the container platform's services, domain names and certificates, secrets, the identities that services and people use, and the alerts that watch them.

For the Learning Coach, Sam's inventory after the incident listed 61 such items in production. Thirty-eight matched the setup notes. Fourteen had settings that differed from the notes. Nine weren't in the notes at all, including a storage bucket that nobody could explain (it turned out to hold an old export from the pilot, which the privacy office then asked to have deleted).

Declarative configuration#

Infrastructure as code (IaC) means describing infrastructure in text files, kept in version control, and letting a tool create and change the real resources to match. The most common style is declarative: you write down the end state you want (“a PostgreSQL database of this size, with these settings, in this private network”), and the tool works out the steps.

The most widely used tool is Terraform. In August 2023 its maker, HashiCorp, moved it from an open-source license to the Business Source License, which isn't open source (HashiCorp, 2023). Community members forked the last open-source version as OpenTofu, now maintained under the Linux Foundation (OpenTofu, 2023). The two remain very similar for the work in this module. Cloud providers also offer their own tools, and some teams prefer general-purpose languages for the same job. The ideas below apply to all of them (tool details last reviewed October 2026).

Plan, review, apply#

Declarative tools work in two steps. A plan compares your configuration with what the tool knows about the real resources and lists what it would create, change, or destroy. Running a plan changes nothing; it exists so people can check that the proposed changes match their intent before anything happens (HashiCorp, terraform plan). Apply carries out a plan.

This gives infrastructure the same review loop as application code. A pull request changes a configuration file; the pipeline runs a plan and posts it on the pull request; a reviewer reads the plan, not just the code; after merge, the pipeline applies exactly that plan. Reading the plan matters because the same small edit can mean “change a setting in place” or “delete the database and make a new one”, and only the plan tells you which.

flowchart LR
    PR["Pull request<br/>edits a module"] --> CHK["Format, validate,<br/>policy checks"]
    CHK --> PLAN["Plan for each<br/>affected environment"]
    PLAN --> REV{"Human reads the plan.<br/>Any destroy of a<br/>stateful resource?"}
    REV -- "No" --> MERGE["Merge"]
    REV -- "Yes" --> HOLD["Second approver,<br/>backup confirmed,<br/>or rework the change"]
    HOLD --> MERGE
    MERGE --> STG["Apply to staging"]
    STG --> SMOKE["Smoke tests"]
    SMOKE --> PROD["Apply the saved plan<br/>to production"]

State: the tool's memory#

To know that a resource in your configuration corresponds to a particular real database, the tool keeps a record called state. Terraform uses it to map each resource in your files to a real object, and it refreshes that record against reality before every operation (HashiCorp, State).

Three practical rules follow:

• Store state remotely, with locking. A state file on one laptop is lost when the laptop is, and two people applying at once can corrupt it. Use a remote backend that supports locking.
• Treat state as sensitive. HashiCorp warns against keeping state in version control, because it can expose secrets (HashiCorp, State). Database passwords and connection strings can end up in it. Restrict who can read it, as you would a password vault.
• Don't edit it by hand. Use the tool's commands to move or remove entries. A broken state file makes the tool believe resources exist that don't, or forget ones that do.

Drift#

Drift is the difference between what your configuration says and what's really deployed. It's what the Learning Coach had: hand changes that nobody recorded. Infrastructure as code doesn't prevent someone from clicking in a console at 2 a.m. It makes the click visible.

Terraform's refresh-only mode produces a plan that only updates the recorded state to match changes made outside the tool, which is useful after an incident to see exactly what changed (HashiCorp, terraform plan). A normal plan on an unchanged configuration should report no changes; if it proposes changes, someone changed reality. Run one on a schedule for every environment, and alert when it isn't empty. Then each emergency change ends with one of two decisions: write it into the code, or let the next apply undo it.

22.2 Networks and identity in depth#

Accounts as boundaries#

Cloud providers let you group resources into separate accounts (some providers call them projects or subscriptions). An account is the strongest boundary most teams have: by default, nothing in one account can reach another, and permissions, bills, and audit logs are separate.

AWS recommends separating workloads into multiple accounts as you grow, and setting organization-wide guardrails that limit what any identity in an account can do (AWS, IAM best practices). For the Learning Coach, that means at least one account each for staging and production, so a mistake in staging can't touch student data, and potentially one per campus that requires it.

Virtual networks and subnets#

Inside an account, your resources sit in a virtual private network, a private address space that's yours alone (AWS calls it a VPC, a virtual private cloud). You divide it into subnets, smaller ranges of addresses. A public subnet has a route to the internet, so things in it can receive traffic from outside. A private subnet has no inbound route from the internet.

The usual layout for a web application:

• A load balancer in public subnets receives HTTPS traffic from students' browsers.
• The API and worker containers run in private subnets. Only the load balancer can reach the API.
• The database sits in private subnets with no public address at all. Only the API and worker can reach it.
• Outbound calls to the model provider leave through a controlled exit point, so you know and can restrict where traffic goes.

Firewall rules attached to each resource (AWS calls them security groups) state exactly who may connect to whom, on which port. “The database accepts connections on port 5432 only from the API's and worker's security groups” is a rule an auditor can read and a policy check can verify.

Zones and regions#

A region is a separate geographic area, such as a cluster of data centers in one part of a country. Each region contains several availability zones, isolated locations far enough apart that one failing rarely affects another. AWS's documentation recommends spreading instances across zones so the application survives the failure of one location (AWS, Regions and Zones).

Chapter 14's ADR-011 already ran the database with a standby in a second zone. Regions are a different question. Running in two regions protects against a whole region failing, but doubles much of the infrastructure and makes data consistency much harder. Choose the region for legal and latency reasons (where student data may live, and where students are), and treat multi-region as a decision that needs a measured reason.

Workload identity and temporary credentials#

Every program that calls a cloud service needs an identity, and every identity needs credentials. The old way was a long-lived access key stored in a configuration file or a CI secret. Keys like that leak, through logs, images, and screenshots (chapter 14's failure clinic was a key baked into an image).

The better way is temporary credentials. AWS recommends that workloads use roles that receive short-lived credentials automatically, and that people sign in through a central identity provider rather than holding personal keys (AWS, IAM best practices). CI pipelines can work the same way: with OpenID Connect (OIDC), a GitHub Actions job requests a token that the cloud provider exchanges for an access token valid only for that job, so no cloud secret is stored in GitHub at all (GitHub, OpenID Connect).

After the move, the Learning Coach had no long-lived cloud keys anywhere. The pipeline's production role could be assumed only by workflows running on the main branch in the production environment.

Least privilege, step by step#

Least privilege means each identity can do only what its task needs, on only the resources it needs, under conditions you choose. AWS suggests a practical route: start from broad managed policies while you learn what a workload needs, then narrow them, using the provider's tools that generate a policy from the actions a role actually used (AWS, IAM best practices).

For the Learning Coach worker, the narrowed policy allowed reading and writing objects under one prefix in one bucket, reading two named secrets, and sending messages to one queue. Nothing else. The API's role was different and smaller. The pipeline's role could deploy, but couldn't read student data. When an agent proposes a policy with a wildcard ("Action": "*" or "Resource": "*"), treat it as a draft to be narrowed, never as done.

22.3 Containers in depth#

Images, layers, and tags#

A container image is a packaged file system plus instructions for starting your program. Images are built in layers, one per build step, and layers that haven't changed are reused, which makes rebuilds fast when the order of steps is sensible (copy the dependency list and install dependencies before copying your code, which changes more often).

Each image has a tag, a human-friendly name like coach-api:2026-10-14, and a digest, a fingerprint of its exact contents. Tags can be moved to point at a different image; digests can't. Deploy by digest, or by a tag your pipeline never reuses, so “the image we tested” and “the image we deployed” are provably the same thing.

Multi-stage builds#

The tools you need to build an application (compilers, package managers, test libraries) aren't needed to run it, and every extra file is something to patch and a possible way in. A multi-stage build uses several stages in one Dockerfile and copies only the finished result into the final image, leaving the build tools behind (Docker, Multi-stage builds).

# Build stage: install dependencies into a virtual environment
FROM python:3.13-slim AS build
WORKDIR /app
COPY requirements.lock .
RUN python -m venv /venv && /venv/bin/pip install --no-cache-dir -r requirements.lock
COPY src/ src/

# Runtime stage: only the environment and the code, run as a non-root user
FROM python:3.13-slim
RUN useradd --create-home coach
COPY --from=build /venv /venv
COPY --from=build /app/src /app/src
USER coach
WORKDIR /app
EXPOSE 8000
CMD ["/venv/bin/uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"]

The Learning Coach's API image shrank by a little over half after this change (the exact size depends on base image versions). More useful than the size: the final image had no compiler and ran as a non-root user, two things a reviewer can check in one glance.

Configuration outside the image#

The same image should run in every environment. What differs (database address, model provider settings, feature flags) comes from outside. The Twelve-Factor App guidelines describe configuration as everything that varies between deployments and suggest a test: could you publish the code at any moment without exposing a credential? (Wiggins, 2017). Secrets come from the platform's secret store at start-up, never from the image, and the IaC defines which secrets each service may read.

Health, readiness, and shutdown#

The platform needs to know whether your container is working. Two kinds of check are common:

• A liveness check answers “is this process stuck?” If it fails repeatedly, the platform restarts the container. Keep it simple: the process responds.
• A readiness check answers “should this container receive traffic right now?” It can include whether the database connection pool is available. During start-up or a database hiccup, the container stops receiving requests without being killed.

Mixing them up causes outages: a liveness check that tests the database restarts every container when the database is slow, which makes things worse.

Graceful shutdown matters as much. When the platform replaces a container during a deploy, it sends a stop signal first. The Learning Coach API now stops accepting new requests, finishes those in flight (a streamed tutor answer can take several seconds), and then exits. The worker finishes or returns its current job to the queue. Without this, every deploy cut off some students mid-answer.

22.4 When orchestration earns its place#

What Kubernetes gives you#

Kubernetes is an open-source system for running containers across a group of machines. Its documentation lists what it provides: service discovery and load balancing, automated rollouts and rollbacks, restarting failed containers, fitting containers onto machines efficiently, secret and configuration management, and horizontal scaling (Kubernetes, Overview). You describe the state you want, and Kubernetes keeps working to make the real state match it.

The same documentation is clear about what Kubernetes isn't. It doesn't build your application, doesn't provide databases or message queues as built-in services, and doesn't dictate logging or monitoring; it provides building blocks for platforms (Kubernetes, Overview). Someone still has to choose, assemble, and operate those blocks.

What it costs#

Managed Kubernetes services run the control plane for you, but your team still owns a great deal: cluster upgrades on the project's release schedule, networking and ingress choices, node sizes and scaling rules, access control inside the cluster, the add-ons for certificates, secrets, and metrics, and the knowledge to debug all of it at night. For a team of three running two services, that is a second product to maintain.

Signals that justify it#

Kubernetes tends to earn its place when several of these are true:

• Many services, owned by several teams, need a common way to deploy, scale, and observe.
• There's a platform team, or at least people with the time and skill to operate clusters.
• Workloads need things managed platforms don't offer well: unusual scheduling (for example GPUs shared across jobs), long-running stateful processes, or a requirement to run the same way across clouds or on a campus's own hardware.
• The organization already runs Kubernetes well, so adding a workload is cheap.

The Learning Coach had none of these. Managed containers already gave it rolling deploys, autoscaling, health checks, and secrets. The honest answer to “should we move to Kubernetes?” was “not yet, and here's what would change that”.

The middle options#

Between “one managed service” and “a Kubernetes platform” there are steps. You can run several services on the same managed container platform. You can add a managed queue and scheduled jobs. Chapter 23 looks at self-hosting a model on GPUs, the case most likely to push the Learning Coach toward orchestration, and why it waited.

22.5 Environments, tenants, and regions#

One module, many environments#

A module is a reusable block of infrastructure code with inputs. The Learning Coach's module, coach_stack, describes one complete environment: network, database, storage, container services, secrets, alerts. Each environment is then a short file that calls the module with its own values.

module "coach" {
  source = "../../modules/coach_stack"

  name               = "coach-production"
  region             = "us-east-2"
  db_instance_class  = "db.r6g.large"
  db_max_connections = 400
  api_min_tasks      = 3
  api_max_tasks      = 12
  tutor_concurrency  = 40
  tenant_ids         = ["main", "riverside", "lakeshore"]
  cost_center        = "learning-coach"
}

Staging uses the same module with smaller numbers. Because every setting is an input, the differences between environments are visible in two short files instead of hidden in two consoles. The 400-connection limit from finals night is now a reviewed number with a comment linking to the incident that justified it.

Shared or dedicated: silo, pool, and bridge#

When you serve several customers (here, campuses), you choose how much to share. AWS's SaaS guidance names three models. In the silo model each tenant gets dedicated resources, possibly a whole stack. In the pool model tenants share resources, and isolation is enforced by the application at run time. The bridge model mixes the two, siloing some parts or some tenants and pooling the rest (AWS, Silo, Pool, and Bridge Models).

The Learning Coach already enforced class membership on every query (ADR-007, chapter 10), so adding a campus identifier to that model was a known kind of change. Pooling Riverside and Lakeshore with the main university was cheap and safe enough. Northfield's rule ruled pooling out for Northfield.

Cost visibility per tenant#

Once several tenants share infrastructure, “what does each campus cost us?” becomes a real question, because the state system would pay per campus. Put a tag on every resource the module creates (cost center, environment, tenant where a resource belongs to one), and record per-campus usage of the expensive shared parts, mainly model tokens, in the application's metrics (chapter 17). The cloud bill then splits by tag for dedicated resources, and by measured usage for shared ones. Neither is exact; both are far better than dividing by three.

Architect's decision: how do we serve three new campuses?#

AI collaboration recipe#

Agents write infrastructure code quickly, and infrastructure mistakes are expensive and sometimes public. Keep agents on the writing side of the line and people on the applying side.

1. Inventory before code.

Here is the output of the provider's resource listing for our production account, and our setup notes. Produce a table of every resource: type, name, key settings, whether it appears in the notes, and any settings that differ from the notes. Don't propose changes yet.

2. Write the module, not the apply.

Write an infrastructure module for the resources in this table. Every setting must be an input variable with a description, or a constant with a comment explaining why. The database must be in private subnets with no public address. No IAM policy may use a wildcard action or resource; list each permission with the reason it's needed. Mark every stateful resource with protection against deletion.

3. Explain the plan.

Here is the plan output for this pull request. Summarize every create, change, and destroy in plain language. Flag any destroy or replacement of a stateful resource, any change that opens network access, and any change to an IAM policy. Say which items you're unsure about.

4. Review for exposure.

Review this module as a security reviewer would. List anything reachable from the internet, any identity with more permission than its task needs, any secret that could end up in state or logs, and any resource without tags. Cite the line for each finding.

Boundaries to state every time:

• “Write code and explanations; never run apply.”
• “No credentials in any file, example, or output.”
• “If a setting's value isn't in the inventory, ask; don't guess.”
• “Treat any destroy of a database, bucket, or key as a stop sign and say so.”

Worked example: bringing production under code#

Sam and Lena split the work over two weeks in October. Maya reviewed every plan.

Step 1: Inventory. An agent turned the provider's listing into the 61-row table described in 22.1. Lena checked each of the 23 rows that didn't match the notes against the incident history. Seventeen were explained by past incidents; four were defaults the notes had simply omitted; two (the old pilot export and an unused firewall rule nobody could trace) were marked for deletion after review.

Step 2: Write the module. The agent drafted coach_stack from the table using the prompt above. The first draft had two problems the review caught: the worker's role allowed every action on the storage service, and the database's deletion protection was set in staging only. Both were fixed before anything ran.

Step 3: Import, don't recreate. The existing production resources had to be adopted by the code, not rebuilt. Terraform's import blocks map an existing real resource to an address in the configuration:

import {
  to = module.coach.aws_db_instance.main
  id = "coach-prod-db"
}

The goal of this step is a dull result. After importing all 59 resources that were staying, the plan for production had to report no changes at all. The first attempt proposed 11 changes, each a place where the module's value differed from reality. For each one, the team decided which was right: production's real value went into the module (the finals-night connection limit), or the module's value was correct and the change was accepted (the memory setting lowered in the spring was restored). The third attempt showed No changes. Your infrastructure matches the configuration. That plan, saved with the pull request, was the evidence that the code described production exactly.

Step 4: Rebuild staging from the module. Lena deleted the hand-built staging environment and created a new one from the same module with smaller inputs. It took 47 minutes, most of it waiting for the database. Then she reran the 25,000-student load test, this time with staging's database settings matching production's ratio of connections to instance size. It passed, with less headroom than the first run had suggested: the p95 time for the plan page rose to 1.6 seconds at the peak, within NFR-3 but not by much. The ADR got the new report and a note explaining why the first one didn't count.

Step 5: Northfield. The Northfield stack was a third environment file calling the same module, with Northfield's account, region, and a single tenant. Northfield's security team reviewed the module itself and asked for one change (a longer retention period for the account's audit log), which became an input. Because the change went into the module, every environment could choose it.

Step 6: Drift detection. A scheduled pipeline now runs a plan for each environment every night and posts to the team channel if any plan isn't empty. In its first month it fired twice: once for a console change Sam made during a database maintenance window (written into the code the next morning), and once for a tag the provider added automatically (ignored explicitly in the module, with a comment).

flowchart TB
    subgraph REPO["One repository"]
        MOD["modules/coach_stack"]
        ENVS["environments/<br/>staging, production, northfield"]
    end
    subgraph TEAM["Team's organization"]
        STG["Staging account<br/>synthetic data only"]
        PROD["Production account<br/>main, Riverside, Lakeshore<br/>(pooled)"]
    end
    subgraph NF["Northfield's organization"]
        NFA["Northfield account<br/>Northfield only (silo)"]
    end
    MOD --> ENVS
    ENVS -->|"plan and apply<br/>via pipeline role"| STG
    ENVS -->|"plan and apply<br/>via pipeline role"| PROD
    ENVS -->|"plan and apply<br/>via cross-account role<br/>Northfield can audit"| NFA

Verification: is the infrastructure what the code says?#

Infrastructure code can be wrong in the same ways application code can, and it can also be right but out of date. Check both.

A rebuild drill is the most honest test of infrastructure code. It answers the question every disaster recovery plan depends on: if this account vanished, could you build it again, and how long would it take?

Failure clinic: the rename that deleted a database#

Symptom. In November, a tidy-up pull request renamed the module's database resource from main to primary, to match the naming in the architecture diagram. The plan comment on the pull request said Plan: 1 to add, 0 to change, 1 to destroy for staging. The reviewer, busy, saw that the code diff was a one-word rename and approved it. The pipeline applied it to staging. Staging's database was deleted and a new, empty one created. The load-test data set, two days of work to generate, was gone.

Root cause. To the tool, a resource with a new name is a new resource, and the old name has disappeared from the configuration. Terraform's documentation says so directly: by default a changed address means destroy the old object and create a new one, unless you record the move with a moved block (HashiCorp, Refactor modules). The code diff looked harmless. The plan said exactly what would happen, and nobody read it.

Deletion protection didn't help either. Staging's database lacked it, and the tool's own prevent_destroy setting only works while the resource's configuration is present; renaming removed the old configuration, so the protection went with it (HashiCorp, lifecycle).

Fix.

• Restore staging from the previous night's snapshot (the module did set automated backups, which is why this was a lost afternoon, not a lost week).
• Add the moved block, so the rename becomes a rename in the plan:

moved {
  from = aws_db_instance.main
  to   = aws_db_instance.primary
}

• Turn on the cloud provider's own deletion protection for every database, in every environment, as a module input that defaults to on. That protection lives on the real resource, not in the configuration, so a rename can't remove it.
• Add a pipeline rule: any plan that destroys or replaces a database, bucket, encryption key, or DNS zone fails automatically unless a second person approves it with a written reason.

Lesson. For infrastructure, the plan is the change; the code diff is only how you asked for it. Review the plan.

Architect's checklist#

• ☐ Every environment is defined in version-controlled code, from shared modules.
• ☐ State is stored remotely, locked, access-controlled, and treated as sensitive.
• ☐ A scheduled plan detects drift in every environment, and someone acts on it.
• ☐ Plans are posted on pull requests and read by a reviewer; destroys of stateful resources need a second approver.
• ☐ Databases and other private resources have no public address; firewall rules name exactly who may connect.
• ☐ Workloads, people, and pipelines use temporary credentials; no long-lived keys remain.
• ☐ Each role's permissions are reviewed against actual use; no wildcards.
• ☐ Images are built in stages, run as non-root, and are deployed by digest.
• ☐ Liveness and readiness checks are separate; shutdown finishes in-flight work.
• ☐ Staging can be rebuilt from empty within a stated time, and that's drilled.
• ☐ Shared versus dedicated deployment is a recorded decision, with tenant isolation tests.
• ☐ Every resource carries cost and ownership tags.
• ☐ The orchestration choice has a written revisit trigger.

Exercises#

Beginner. The Meal Planner runs on one managed container service, one database, and one storage bucket, all created by hand. List every piece of infrastructure you think it has, including the ones people forget (domain, certificate, secrets, alerts, backups). For each, write which settings would need to be the same in staging and production, and which could differ. Then explain, in a paragraph a non-engineer could follow, what a plan shows and why someone should read it.

Intermediate. Design the network and identities for the Trip Planner: a web front end, an API, a worker that calls a flight-price service, and a database. Draw the public and private subnets and write each firewall rule in words (“X may connect to Y on port Z”). Write the permissions for the worker's role as a short list with a reason for each. Then hand your design to an AI assistant with the “review for exposure” prompt, and judge each finding: real, not real, or needs more information.

Advanced. The Shop Help Chat is sold to online shops. Three small shops are happy to share infrastructure; one large shop's contract requires its customer data in its own cloud account. Write the decision record: options (pool, silo, bridge, and one more of your choosing), decision, tradeoffs, validation, and revisit trigger. Sketch the module's inputs so both deployments come from one definition, and describe the tests that prove one shop can never see another shop's orders.

Lab, artifact, and review#

Lab. Put one environment of your Learning Coach (or your capstone) under infrastructure as code. Inventory what exists, write a module, import the existing resources, and work until the plan reports no changes. Then rebuild a second environment from the same module and time it. Finally, make a deliberate change by hand in the console and confirm your scheduled plan catches it.

Deliverable. Add to your portfolio:

• the module, with every input documented;
• the first empty plan for an imported environment, and the list of differences you resolved to get there;
• a rebuild drill report: start and end times, problems hit, and what you changed;
• a decision record for your deployment model (shared, dedicated, or mixed), with its revisit trigger.

Review. Ask someone who didn't write the module to read one plan and tell you, in their own words, what it will do. If their account differs from yours, improve the plan's readability (names, descriptions, smaller changes) before anything else.

Reflection#

• Which setting in your system exists only because someone changed it by hand, and who knows about it?
• If your production account disappeared tonight, what would you lose that no code could recreate?
• What permission does your most powerful automated identity have that it has never used?
• Where would you draw the line between shared and dedicated infrastructure for your users, and what would move it?
• What would make you choose Kubernetes, and how would you know it had happened?

The next module goes deeper into the most expensive and least predictable part of an AI system: finding the right evidence and running the model. Northfield's next request was that no student text leave its account, including to a model provider, and that changed the retrieval and serving design as much as this module changed the infrastructure.

References#

• Amazon Web Services. Regions and Zones. Amazon EC2 User Guide.
• Amazon Web Services. Security best practices in IAM. AWS Identity and Access Management User Guide.
• Amazon Web Services. Silo, Pool, and Bridge Models. AWS Well-Architected SaaS Lens.
• Docker. Multi-stage builds. Docker documentation.
• GitHub. OpenID Connect. GitHub Docs.
• HashiCorp. (2023, August 10). HashiCorp projects changing license to Business Source License v1.1. HashiCorp Discuss.
• HashiCorp. lifecycle reference. Terraform documentation.
• HashiCorp. Refactor modules. Terraform documentation.
• HashiCorp. State. Terraform documentation.
• HashiCorp. terraform plan command reference. Terraform documentation.
• Kubernetes. Overview. Kubernetes documentation.
• OpenTofu. (2023, August 25). OpenTofu announces fork of Terraform. OpenTofu blog.
• Wiggins, A. (2017). The Twelve-Factor App: Config.

Links checked October 2026.