Capstone Studio: Design, Build, and Operate an AI Product

Why it matters#

Each chapter practiced one part of the work. Real systems need all of them at once, and the hard part is the joins: requirements that become tests, an architecture that agents can work within, evidence that survives the next change, a demo that tells the truth.

The capstone is also where your portfolio comes from. Chapter 21 is about the career paths open to you; the artifacts you produce here are what you'll show. A repository with a PRD, decision records, tests, evaluation results, a threat model, and an honest retrospective says more about you as an architect than any screenshot.

Finally, it's practice for the habit this book has argued for throughout: using AI to build quickly, and never treating generated work as correct because it runs.

Completing the capstone independently#

The opening story is a fictional teaching example. The panel, marks, and pass conditions below describe a suggested assessment format that an instructor or peer group can adopt. Reading the book does not enroll you in an assessed program or arrange a reviewer.

If you are studying on your own, use the same evidence requirements as a self-review rubric:

1. Write your project charter and check it against the scope conditions below. Record your decision before building. Where the chapter asks for reviewer approval, document your own review and any questions that still need outside expertise.
2. Keep an evidence index in docs/evidence.md. Link each requirement to its implementation, test result, and the version tested. Mark missing evidence as unfinished work.
3. At each phase gate, rerun the checks and record what passed, what failed, and what you changed. Reading time does not include this project work; adjust the suggested ten-week schedule to your experience and available hours.
4. Record a short demonstration that includes a failed request and recovery. Use the weighted rubric to assess your work, with a short reason and evidence for each score.
5. If possible, ask a peer to follow three claims from your index to the underlying evidence. If no independent reviewer is available, label the result as self-assessed and list the checks you could not independently validate.

Self-review helps you make progress; it does not replace specialist review when the product or its users require it. Your deliverable is a working application and an inspectable portfolio, with limitations stated alongside the results.

Learning objectives#

After this chapter you will be able to:

1. Choose and scope a capstone product that exercises the whole lifecycle in the time you have.
2. Run discovery and write requirements with measurable quality attributes and acceptance tests.
3. Design and document an architecture with diagrams, contracts, ADRs, and security and cost reviews.
4. Plan and orchestrate AI-assisted implementation in vertical slices, with human review and continuous test evidence.
5. Prepare a system for production: deployment, secrets, observability, rollback, performance, security, and AI quality evidence.
6. Present and defend the system honestly, with its limitations, incidents, future plan, and a reflection on what AI did well and poorly.

20.1 Project choices#

Pick one project. The Learning Coach is the guided path: the book has walked through every part of it, so you'll spend your time on judgment rather than discovery. The other options exercise the same skills with fresh decisions.

Approving your own product#

Your own idea can make a stronger portfolio, if it meets these conditions:

• You can reach real users for at least five conversations and a test session.
• An AI feature matters to it, and its errors can be noticed and corrected without serious harm. Avoid products that give medical, legal, or financial advice or make decisions about people's rights; those need expert and legal review beyond a capstone.
• It handles some personal data, so privacy and deletion are real.
• It talks to at least one outside service, so integration and failure handling are real.
• Adults only, and no data scraped from other sites without permission.

Write a one-page charter (problem, users, scope, non-goals, top three risks) and have it approved by your reviewer before starting.

Thin but complete#

The most common capstone failure is scope. Ten weeks part-time (an assumption; adjust to your program) is enough for a narrow product built properly, not a broad one built partly. Aim for a few features, each with requirements, tests, evaluation, monitoring, and a place in the runbook.

Start with a walking skeleton, a term Alistair Cockburn coined in 1996 for a thin implementation that connects the main parts of the architecture and works end to end (Cockburn, Bio). For the Learning Coach, that's: sign in, create a one-task plan, save it, see it again, deployed to staging by the pipeline, with one test. Every later feature hangs on that skeleton.

20.2 Phase 1: Product discovery and requirements#

Weeks 1–2. Chapter 3 is your guide.

User research and constraints#

Talk to at least five potential users before writing requirements. Ask about their current way of solving the problem, the last time it went wrong, and what they'd stop using a product for. Record constraints: time, budget, devices, accessibility needs, rules about their data.

For a Learning Coach capstone, interview students who are preparing for exams now. For the Shop Help Chat, talk to the shop's staff as well as its customers; staff know which questions arrive every day and which ones need a person.

PRD and quality attributes#

Write a short PRD: problem statement, users, jobs to be done, user stories with IDs, non-goals, and quality attributes written as measurable nonfunctional requirements. “Fast” isn't a requirement; “plan page loads in under 2 seconds at the 95th percentile with 200 concurrent users” is. Include at least one requirement each for performance, availability, privacy, accessibility, and AI quality.

User flows and acceptance tests#

Draw the main user flows, including the unhappy paths: the AI is slow, the outside service is down, the user changes their mind. For each story, write acceptance criteria with examples that will become tests. Include edge cases the book's failure clinics taught you to expect: time zones, two users with similar data, a document that contains instructions.

Risk prioritization#

List risks across product, technical, security, privacy, AI quality, and cost. Score each by likelihood and impact (chapter 15's simple 1-to-3 scheme works), and plan how you'll reduce the top five. Identify your riskiest assumption and how you'll test it in the first three weeks.

Gate 1. A peer or reviewer reads your PRD and can explain back what you're building, for whom, and what you're not building. Your top risks each have an owner (you) and a plan.

20.3 Phase 2: Architecture design#

Weeks 2–3. Chapters 5 through 9 are your guides.

Context, container, and component diagrams#

Draw the C4 system context and container diagrams, and a component diagram for the module with the most logic (Brown, C4 model). Keep them as Mermaid text in the repository so they change with the code. Mark every trust boundary and every outside service.

Data and API contracts#

Write the data model with a data inventory: for each field holding personal data, why you keep it, who can see it, and when it's deleted (chapter 7). Write API contracts before implementation as an OpenAPI document (chapter 6), and schemas for any AI tools, each tool getting the user's identity from the server, never from the model (chapters 8 and 15).

Technology choices and ADRs#

Use the book's reference stack unless you have a reason not to; the capstone tests judgment, not novelty. Record significant choices as short ADRs (Nygard, 2011). Expect five to eight: module structure, data storage and search, sign-in, the AI feature's design and model, hosting, and agent autonomy.

Security and cost reviews#

Write a threat model: entry points, actors, trust boundaries, and threats ranked by risk (chapter 15). For an AI feature, include prompt injection, where text in user content tries to redirect the model (OWASP, LLM01:2025), and broken access control, where one user reaches another's data (OWASP, A01:2025).

Write a cost model per active user, with every price labeled as an assumption and a monthly cap set at the provider (chapters 8, 9, and 16).

Gate 2. A design review with a peer playing the panel from chapter 19: five slides, a pre-read, scenarios walked through the design, and a recorded outcome.

20.4 Phase 3: AI-assisted implementation#

Weeks 3–7. Chapters 10 through 13 are your guides.

Agent work packages and ownership#

Break the work into work packages: task cards with a goal, the spec sections and contracts that apply, files in scope, acceptance tests, stop conditions, and what the agent must not touch (chapters 10 and 12). You own every package; agents do the work inside it. Write an AGENTS.md with commands, the repository map, product-wide rules, and pointers to specs and ADRs, and update it in the same pull request as any rule change.

Keep your agent task history: the task cards, pull request descriptions, review notes, and anything you rejected and why. It's a portfolio item, and it's the raw material for your retrospective. You don't need full chat logs; you need the decisions.

Vertical-slice implementation#

Build the walking skeleton first, then add one vertical slice at a time: a feature from screen to database, with its tests, deployed to staging (chapter 6). Order slices by risk: the riskiest assumption from phase 1 goes in the first slice after the skeleton.

Parallel agents are optional. If you use them, follow chapter 12: contracts first, separate files, a merge order, and a person integrating.

Human reviews and integration#

Read every line in the risky zones: authentication, permission checks, database queries, secrets and configuration, dependencies, deletion, and anything sending data to the AI provider (chapter 4). Run an independent review agent on each pull request if you like, but you approve. Merge small pull requests often; integration pain grows with the size of what you merge.

Continuous test evidence#

Every pull request runs the tests in CI, and the main branch stays green. Keep three kinds of test from chapter 13: fast unit tests for rules, integration tests for the API with a real database, and a small set of end-to-end tests for the main flows. Add the golden set for your AI feature early, even with ten cases, and grow it. Keep a defect log: each bug, how it was found, its cause, and the test that now catches it.

If you work in a team#

Some programs run the capstone in teams of two to four. The phases and gates stay the same; ownership needs more care:

• One owner per module and per decision, named in the instruction files and ADRs. Shared ownership tends to mean nobody reads the risky zones.
• Contracts between people, not just between agents. Agree on API and data contracts before splitting work, as chapter 12 did for parallel agents.
• Every pull request reviewed by a teammate, not only by an agent, with the risky-zone gates from chapter 19.
• Individual evidence. Each person keeps their own task history and writes their own retrospective section, so the panel can see who decided what.

Gate 3. All user stories in scope pass their acceptance tests in CI on the main branch. Permission tests use two users and two of everything.

20.5 Phase 4: Production readiness#

Weeks 7–9. Chapters 14 through 17 are your guides.

Deployment and secrets#

Deploy through the pipeline only: build once, test, deploy to staging, then production, with migrations that work with the previous version of the code (chapter 14). Keep secrets in the platform's secret store, out of images and history, and set spending limits at every paid provider.

Observability and rollback#

Add structured logs with request IDs, traces across the AI call, and two or three dashboards (chapter 17). Set SLOs for your main flows and one alert per SLO, with a runbook entry for each. Then practice a rollback: deploy a deliberately broken release to staging, notice it from the alert, roll back, and time it. Do the same for a database restore.

Performance and security testing#

Run load tests against staging with a fake model that waits as long as the real one: at least a smoke test, an average-load test, and a spike test (Grafana Labs, Load test types). Find your first bottleneck and write it down, with the numbers and the version tested.

Security testing for a capstone means: the permission tests, injection tests with hostile content in every place users can put text, a dependency audit, a secrets scan of the repository and images, and a check that logs contain no personal data (chapter 15).

AI quality evaluation#

Run the golden set on the release version: normal cases, edge cases, adversarial cases, and cases the AI must decline (chapters 9 and 13). Report results by category, not just a total, with the model, prompt version, and retrieval settings recorded. If you use an AI grader, show how you checked it against your own labels.

Gate 4. A readiness review: you walk a reviewer through the evidence index (section 20.6) and every item links to a result from the release commit.

20.6 Phase 5: Presentation and retrospective#

Weeks 9–10. Chapters 18 and 19 are your guides.

Working demo and design defense#

Present like chapter 19's second review, not its first. Five slides: the decision you want judged, context, options, risks and evidence, and the plan. Then the demo, scripted to include an unhappy path: the AI declining a request, an outside service failing, a permission check holding. Prepare backup slides for the questions you know are coming: cost, security, what happens when the AI provider is down.

Run the demo on the deployed system, not your laptop. Have a recorded fallback in case the network fails, and say so if you use it.

Document limitations and incidents#

List what doesn't work, what isn't tested, and what you'd need to know before real users depend on it. Honest limitations are a strength in the assessment; hidden ones found by the panel aren't.

Write a short postmortem for at least one real incident or near miss during the project. Keep it blameless: the goal of a postmortem is to understand contributing causes and prevent recurrence, not to find someone to blame (Lunney & Lueder, Postmortem culture). For a solo project, that includes not blaming yourself or the agent; find the missing check.

Future evolution plan#

Write what comes next as an architect would: a debt register with the highest-interest items, the revisit triggers from your ADRs, the next three features in order with why, and what would have to change at ten times the users (chapter 18).

What AI did well and poorly#

Run a retrospective on your own use of AI. Norm Kerth's Prime Directive for retrospectives sets the tone: “Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand” (Retrospective Wiki, The Prime Directive).

Then use your task history, not your memory:

• Which work packages did agents complete with little rework? What did those cards have in common?
• Which needed heavy rework or were rejected? Why?
• Which bugs came from generated code, and which check caught them, or didn't?
• Where did AI make you faster, and where did it make you overconfident?
• What did you learn to do yourself, and what do you still not understand well enough to review?

The evidence index#

Pull everything together in one file, docs/evidence.md, that maps each assessment area to its artifacts, each with a link, the commit it describes, and its date. It's your first slide's backup, your readiness gate, and your reviewer's map.

flowchart LR
    subgraph A["Assessment area"]
        A1["Architecture 25%"]
        A2["Implementation and<br/>integration 20%"]
        A3["Testing and<br/>security 25%"]
        A4["Operational<br/>readiness 15%"]
        A5["Communication and<br/>reflection 15%"]
    end
    subgraph E["Evidence (each with commit and date)"]
        E1["PRD, C4 diagrams, data inventory,<br/>contracts, ADRs, cost model"]
        E2["Repository, task history,<br/>vertical slices, merged PRs"]
        E3["CI results, golden set results,<br/>threat model, permission and injection tests"]
        E4["Runbook, dashboards, load-test report,<br/>rollback and restore drill records"]
        E5["Design review, demo, limitations,<br/>postmortem, retrospective, evolution plan"]
    end
    A1 --> E1
    A2 --> E2
    A3 --> E3
    A4 --> E4
    A5 --> E5

20.7 How the capstone is assessed#

The panel assesses five areas, weighted to reflect what this book has argued matters most: the architecture and the verification each count for a quarter, because together they decide whether a system can be trusted.

The rule: a visually impressive prototype without verification cannot pass the capstone. Whatever the scores elsewhere, a capstone fails if any of these are missing: automated tests running in CI on the main branch; permission tests with two users; AI evaluation results from the release version; a threat model; a load-test report; a demonstrated rollback; a runbook.

How the panel reads your evidence#

Expect reviewers to sample, not to read everything. A typical panel picks a few items from your evidence index and follows each to the bottom: a requirement to its acceptance test and its latest CI run; a threat to the test that covers it; an SLO to its alert and runbook entry. They try one or two attacks during the demo. They ask what you'd do differently.

Three things raise marks more than polish does:

• Traceability. Any claim can be followed to evidence in two or three clicks.
• Judgment shown in tradeoffs. ADRs whose rejected options were real, and a cut list you can defend.
• Honesty under questions. “I don't know, and here's how I'd find out” scores better than a confident guess the panel can disprove.

Architect's decision: how big should the capstone be?#

AI collaboration recipe#

Agents can do much of the capstone's building and a lot of its paperwork. They can't tell you whether your evidence is real, and they can't do your retrospective for you.

1. Phase gate review.

Act as a strict capstone reviewer. Here is the assessment table, the pass conditions, and our evidence index. For each pass condition and each assessment area, list what evidence is missing, stale (older than the latest change to the code, model, or prompt), or doesn't show what it claims. Don't give praise; list gaps with file paths.

2. Work package drafting.

From the PRD story US-3 and the contracts in docs/api/, draft a task card: goal, spec sections, files in scope, acceptance tests as examples, stop conditions, and files the agent must not touch. Flag anything in the story that's ambiguous instead of deciding it.

3. Evidence regeneration.

Write a script that, on the current commit, runs the test suite, the golden set, the permission and injection tests, and the smoke load test against staging, then writes a report with the commit SHA, date, model, prompt version, and each result. Don't run anything against production or the real AI provider.

4. Task history analysis.

Here are our task cards and pull request records. For each, classify the outcome: merged as is, merged after rework, rejected. Summarize patterns in the cards that went well versus badly. Don't speculate beyond the records.

Boundaries to state every time:

• “Report gaps; don't fill them with claims.”
• “Every result must record the commit and configuration it describes.”
• “No production access, no real provider in load tests.”
• “Ambiguities go to me as questions.”

Worked example: a Learning Coach capstone in ten weeks#

Here's one way to plan the guided capstone, following option B. The weeks are an assumption for a part-time schedule of around ten hours a week.

flowchart LR
    P1["Weeks 1–2<br/>Discovery and PRD"] --> G1{"Gate 1<br/>PRD explained back"}
    G1 --> P2["Weeks 2–3<br/>Architecture"] --> G2{"Gate 2<br/>Design review"}
    G2 --> P3["Weeks 3–7<br/>Skeleton, then slices"] --> G3{"Gate 3<br/>Stories pass in CI"}
    G3 --> P4["Weeks 7–9<br/>Production readiness"] --> G4{"Gate 4<br/>Evidence index complete"}
    G4 --> P5["Weeks 9–10<br/>Demo, defense,<br/>retrospective"]
    G3 -. "slipping?" .-> CUT["Cut a feature,<br/>never verification"]
    CUT -.-> P3

Three moments in this plan deserve attention.

The skeleton in week 3. It's ugly and does almost nothing: sign in, create a plan with one task, see it after reloading. But it runs through the pipeline into staging with one end-to-end test, and from then on, every slice adds to a deployed system. Students who skip this usually discover deployment problems in week 9.

The test session in week 5. Rebalancing is the Coach's core promise and its riskiest assumption: will students accept a plan that moves their tasks? Three students using the real slice for twenty minutes answer that better than any amount of building. If they don't confirm moved tasks, it's week 5, and there's time to change the design.

The cut list. Classes, quizzes, summaries, and shared notes appear in the PRD as non-goals and in the evolution plan as the next features, with the ADR each would need. That turns “I didn't build it” into “I chose not to, and here's how I would.”

Verification: is your capstone ready to defend?#

Failure clinic: the evidence from another version#

Symptom. A week before demo day, Lena rehearsed with Maya. Her evidence slide said the chat answered 59 of 61 golden cases correctly and that the load test held p95 response under 3 seconds at 50 concurrent conversations. Maya picked one golden case at random and tried it on the staging chat. The answer was wrong: the chat said a bike bought 35 days ago could be returned, and the shop's policy was 30 days.

Reproduce. Lena ran the case through the release candidate: wrong. She checked out the commit from week 6, when she'd run the golden set: right. Between the two, she had switched to a newer model version, after the provider announced the old one's retirement, and shortened the system prompt to cut cost. Both changes were sensible. Neither had been followed by a new evaluation run. The load-test report was older still, from before she'd added the handoff feature, which made an extra database call on every message.

Diagnose. Every number on the slide was real. Each described a version of the system she was no longer shipping.

1. Evidence didn't record its version. The reports had dates but no commit, model, or prompt version, so nothing showed they were stale.
2. Regenerating evidence was manual. Each report took a different command and some setup, so reruns happened when she remembered.
3. Changes to the model and prompt didn't feel like code changes. Chapter 17's lesson exactly: the recipe behind an AI answer includes the model and prompt, and changing them changes the system.
4. The policy rule lived only in the prompt. Shortening the prompt removed the sentence that stated the 30-day window clearly. A rule that matters was enforced by wording, not by code.

Fix.

1. One command for all evidence. Recipe prompt 3 produced a script that runs the tests, golden set, permission and injection tests, and smoke and average-load tests on staging, and writes one report stamped with the commit SHA, date, model, prompt version, and retrieval settings.
2. Evidence runs on every release candidate, in CI, and the evidence index links to the latest run. The phase gate review prompt flags any item older than the latest change.
3. Rules move into code. The return window is now checked by the returns tool on the server, with tests for days 29, 30, and 31. The prompt explains the policy; the code enforces it, as chapter 15 recommended for anything that must always hold.
4. The rerun found three things. Two golden cases now failed and were fixed; the load test showed p95 at 3.8 seconds under the new handoff call, fixed by adding an index. The slide on demo day showed the new numbers, and the limitations slide mentioned the episode.

Lesson. Evidence is a statement about one version of a system. Stamp every result with what it describes, make regenerating it one command, and rerun it whenever the code, the model, or the prompt changes. Old evidence presented as current is a claim nobody checked, however real it once was.

Architect's checklist#

Throughout the capstone:

• ☐ The project is approved, scoped thin but complete, with written non-goals.
• ☐ A walking skeleton is deployed by the pipeline in the first weeks.
• ☐ Requirements are measurable and acceptance criteria become tests.
• ☐ The riskiest assumption is tested with real users early.
• ☐ Diagrams, contracts, and ADRs are in the repository and match the code.
• ☐ A threat model and cost model informed real decisions.
• ☐ Agents work from task cards inside an architecture I own; I review the risky zones.
• ☐ CI is green on main; permission, injection, and acceptance tests run on every change.
• ☐ AI quality is evaluated by category on the release version.
• ☐ Deployment, secrets, observability, rollback, and restore are practiced, not just configured.
• ☐ All evidence records its commit and configuration and regenerates with one command.
• ☐ The demo includes failure paths and runs on the deployed system.
• ☐ Limitations, a postmortem, a retrospective, and an evolution plan are written honestly.

Exercises#

Beginner. Choose your capstone project and write a one-page charter: problem, users, three to five stories, non-goals, top three risks, and the walking skeleton you'll build first. If you choose the Meal Planner, include how you'll make sure plans never violate a stated allergy, and why a prompt instruction alone isn't enough.

Intermediate. Plan a Trip Planner capstone as work packages. Write the walking skeleton and four vertical slices in risk order, with a task card for the slice that integrates the outside price service: goal, contract, files in scope, acceptance tests (including the service being slow, down, and returning stale prices), and stop conditions.

Advanced. Run a full readiness review of a Shop Help Chat capstone, real or imagined. Write the evidence index against the assessment table and pass conditions, then play reviewer: find the three weakest items, explain what a panel would ask about each, and write the fix plan. Include how you'd test the handoff to a person and the chat's behavior when a customer claims to own an order they don't.

Lab, artifact, and review#

Lab. Build, operate, and defend your capstone through the five phases and four gates in this chapter.

Required portfolio. Product spec; diagrams; schemas; interface contracts; agent task history; code repository; tests; evaluation results; threat model; load-test report; deployment runbook; ADRs; demonstration. Link each from docs/evidence.md, with the commit it describes.

Assessment. Architecture 25%; implementation and integration 20%; testing and security 25%; operational readiness 15%; communication and reflection 15%. A visually impressive prototype without verification cannot pass.

Review. Present to a panel of at least two people, one of whom hasn't seen the project before. Ask them to try to break it during the demo. Afterward, ask each to name the one piece of evidence they found most convincing and the one claim they doubted. Add both to your retrospective.

Reflection#

• What did you cut from your capstone, and how did you decide?
• Which piece of your evidence would you trust least if someone else had produced it?
• Where did AI save you the most time, and where did it cost you time later?
• What part of your system do you understand least, and what would it take to understand it?
• If a real user started depending on your product tomorrow, what would you fix first?

Chapter 21 looks beyond the capstone: what competence as an AI architect looks like, the career directions it opens, and how to keep learning without chasing every new tool. Bring your portfolio; it's the evidence for the next conversation.

References#

• Brown, S. The C4 model for visualising software architecture. c4model.com.
• Cockburn, A. Bio. alistaircockburn.com.
• Grafana Labs. Load test types. Grafana k6 documentation.
• Lunney, J., & Lueder, S. Postmortem culture: Learning from failure. In Site Reliability Engineering. Google.
• Nygard, M. (2011, November 15). Documenting architecture decisions. Cognitect blog.
• OWASP. A01:2025 Broken access control. OWASP Top 10:2025.
• OWASP. LLM01:2025 Prompt injection. OWASP Gen AI Security Project.
• Retrospective Wiki. The Prime Directive. Agile Retrospective Resource Wiki.

Links checked October 2026.