Working with AI Coding Agents

Why it matters#

Chapter 10 made the work clear: a specification, a task card, and a loop of inspect, plan, implement, verify, review. This chapter is about the agent doing the work: what kinds there are, what they can touch, how to delegate to them, and how to keep control of what they produce.

The temptation is to measure an agent by how much it does unattended. That's the wrong measure. Agents produce more code than people, faster than people can review it, and they report their own success. What matters is how much of their output you can trust after checking it, and how cheaply you can check it.

Even the feeling of speed can mislead. In a 2025 randomized trial, METR gave 16 experienced open-source developers 246 real tasks in projects they knew well, randomly allowing or forbidding AI tools. Developers expected AI to cut their time by 24%, and afterward believed it had cut it by 20%. It had actually increased their time by 19% (Becker et al., 2025). The study covers one setting and early-2025 tools, and its authors are careful about how far it generalizes. Its lesson for an architect is narrower and durable: measure, don't assume.

The architect's job is to choose the right kind of assistance for each task, set boundaries the system enforces, delegate in units small enough to verify, and make review a real decision.

Learning objectives#

After this chapter you will be able to:

1. Distinguish autocomplete, conversational coding, and autonomous agents, and the IDE, terminal, and cloud environments they run in.
2. Configure sandboxes, permissions, and network access so agents work freely inside safe boundaries.
3. Delegate a task with bounded scope, stated assumptions, prohibited actions, escalation triggers, and required evidence.
4. Keep generated code reviewable: small diffs, meaningful commits, careful dependency changes, and controlled duplication.
5. Recognize the common agent failures: invented APIs and packages, silent regressions, overbroad changes, and tests passed without meeting requirements.
6. Use a repeatable protocol and an agent instruction template from ticket to merge.

11.1 Types of coding assistance#

Autocomplete, conversational coding, autonomous repository edits#

Coding assistance comes in three broad kinds, and they need different kinds of supervision.

The shift from the first two to the third is the one that matters. With autocomplete and chat, you are still the one who puts code into the project, one decision at a time. With an agent, the code arrives already written, tested by the agent, and described by the agent. Your review is the first independent look at it. Everything in this chapter follows from that.

IDE and terminal execution models#

Agents run in three kinds of places, each with its own risks:

• In the editor (IDE). The agent works in your open project, and you watch its edits appear. Good for tasks where you want to steer as it goes.
• In the terminal on your machine. The agent runs commands with your user's access: your files, your credentials, your network. Powerful, and the reason staging moved at 3 a.m.
• In the cloud, in the background. The agent runs in a fresh, isolated machine with a copy of the repository and opens a pull request when done. Good for well-specified tasks you don't need to watch, and safer by default, because the machine holds only what you give it.

The same model can behave very differently depending on where it runs, because the environment decides what a mistake can reach.

Sandboxes, permissions, and network access#

A permission is a rule about what the agent may do without asking: read files, edit files, run a given command, reach a given host. A sandbox is a boundary enforced by the operating system or a container, so that even a command the agent was allowed to run can't reach outside it.

Asking for approval on every action sounds safe and isn't. Anthropic, describing sandboxing for its coding agent, names the problem approval fatigue: constantly clicking approve leads people to stop paying attention to what they approve. In its internal use, sandboxing reduced permission prompts by 84% (Dworken & Weller-Davies, 2025). Fewer prompts, each one worth reading, beat many prompts nobody reads.

The same article makes a point worth keeping regardless of tool: a sandbox needs both filesystem isolation (the agent can change only the project folder) and network isolation (it can reach only approved hosts). Without the second, a compromised agent can send your keys somewhere; without the first, it can change the files that would have kept it in (Dworken & Weller-Davies, 2025). “Compromised” here usually means prompt injection, from chapter 8: text in a web page, an issue, or a file that the agent reads and follows.

For the Learning Coach, Maya's settings became:

Tool-specific behavior versus general principles#

Agent tools change monthly: names of modes, settings files, how permissions are written. This chapter mentions specific tools only as examples, with dates. The principles behind them change slowly: limit what the agent can reach, make the boundaries enforced rather than requested, scope each task, require evidence, and review the result. When you adopt a new tool, map its settings onto those principles before you trust it with your repository.

11.2 Delegating safely#

Bounded scope and explicit file ownership#

Chapter 10's task card already bounds the work: what it delivers, which folders it may change, what it must not touch. Two habits make that boundary real:

• One task, one branch, one session. An agent working on two things at once mixes them in one diff.
• Name the files it owns. “Change only backend/coach/plans/ and its tests” lets you reject any diff that strays, without arguing about whether the extra change was helpful.

“Upgrade our dependencies” owns nothing and everything. It should have been several tasks, as the worked example shows.

Ask the agent to identify assumptions#

Before an agent edits anything, ask it to list what it is assuming: which versions, which files, what “fixed” means, what it believes the code does now. Assumptions are cheap to correct in a list and expensive to find in a diff.

This works partly because models can often catch their own errors when asked directly. In a large study of invented package names, several models identified their own invented packages with better than 75% accuracy when asked afterward whether the packages were real (Spracklen et al., 2024). A model asked “does this package exist, and how do you know?” will sometimes say it doesn't. A model never asked will import it.

Define prohibited actions and escalation triggers#

Prohibited actions are things the agent must never do, whatever the task. Put the product-wide list in the project instruction file, and back each item with a permission or a check where you can:

• Never disable, skip, or delete a test, or change an expected value, without being asked to.
• Never add a dependency without stating why, and never choose between similar names by trial.
• Never run commands against any database except the local test database.
• Never read or print secrets; never commit .env files.
• Never push to main, change CI configuration, or touch deployment files.

Escalation triggers are situations where the agent must stop and ask instead of deciding:

• The task needs a file outside its scope, or a change to a frozen contract.
• Documents disagree (chapter 10's failure clinic).
• A fix requires a new dependency, a migration, or a change to access rules.
• The same check fails three times in a row.

The last one matters more than it looks. An agent that fails repeatedly tends to widen its search for a way to make the check pass, and some of those ways are the ones you prohibited.

Require verifiable evidence of completion#

“All tests pass” described the morning's pull request accurately and told Maya nothing useful. Ask for evidence instead, in a fixed shape:

• each requirement or acceptance example, with the test that covers it;
• the commands run and their output, including how many tests ran and how many were skipped;
• every file changed outside the task's stated scope, with a reason;
• every new or changed dependency, with a reason;
• what wasn't verified.

Anthropic's guidance for its agent recommends the same thing: have the agent show evidence, such as test output, the command and its result, or a screenshot, instead of asserting success, because reviewing evidence is faster than rerunning the checks yourself (Anthropic, Best practices for Claude Code).

11.3 Managing generated code#

Small diffs and meaningful commits#

Google's engineering guidance on code review explains why small changes win: they are reviewed faster and more thoroughly, are less likely to introduce bugs, waste less work if rejected, and are simpler to roll back. As a rough guide it suggests that 100 lines is usually reasonable and 1,000 usually too large, and notes that spread matters too: 200 lines in one file may be fine, but spread across 50 files it usually isn't. It also recommends keeping refactorings separate from feature changes and bug fixes, and including related tests in the same change (Google, Small CLs).

Agents make all of this easy to ignore, because a large change costs them nothing to write. It costs you everything to review. Three rules keep agent work reviewable:

• Formatting and reformatting go in their own commit, made by the formatter, not the agent, and reviewed by checking that only whitespace changed.
• One concern per commit, with a message that says what and why: “Pin pydantic to 2.x; v3 removes the validator API we use” rather than “update deps.”
• Ask for a split when a diff passes your size limit. An agent can split its own work into a stack of smaller pull requests, and it's the cheapest review aid there is.

Code review for correctness and maintainability#

Review an agent's change in a fixed order, from the parts most likely to be wrong in a way you can't see:

1. Tests first. Were any tests changed, skipped, or deleted? Do the new tests check behavior from the specification, or only that the code does what it does?
2. Scope. Did anything change outside the task card? Each such file needs a reason.
3. Dependencies and configuration. Lockfiles, package lists, CI files, settings. Small diffs here have large effects.
4. The risky zones from chapter 4: sessions, permission checks, database queries, secrets, deletions, and migrations.
5. The rest of the code, for correctness and fit: does it follow the module's patterns, reuse existing helpers, and handle the failure cases?

For changes that matter, add an independent reviewer: a fresh agent session with no stake in the change, asked to find faults rather than to approve. Anthropic suggests exactly this kind of adversarial step, so the agent doing the work isn't the one grading it (Anthropic, Best practices for Claude Code). It finds different problems than you do. It doesn't replace you: the merge decision is yours.

Handling dependency updates#

Every dependency is code you didn't write and now ship. Agents add them freely, and sometimes invent them. Spracklen and colleagues generated 576,000 code samples with 16 models and found that, on average, at least 5.2% of packages suggested by commercial models and 21.7% by open-source models didn't exist, with 205,474 distinct invented names. The invented names were often repeated: 43% came back in all 10 repeats of the same prompt (Spracklen et al., 2024). Repeatable invented names are an opportunity for attackers, who can register them and wait. The authors call this package hallucination, a new form of package confusion attack.

The defense is a dependency policy that applies to agents and people alike:

• Every new dependency is a decision, stated in the pull request with a reason and the alternatives considered, including “write the 20 lines ourselves.”
• Verify the package, not just the install: the exact name, the publisher, how long it has existed, how widely it's used, and whether its source matches what's published. A package published last month by an unknown account needs a very good reason.
• Upgrade one thing at a time. One major upgrade per pull request, with the changelog's breaking changes listed and each one checked.
• Commit the lockfile, and review its diff. It shows every indirect package that came along.

Chapter 15 covers supply-chain security in depth, including automated scanning.

Refactoring AI-generated duplication#

Agents tend to write new code rather than find and reuse existing code, because writing is the shortest path to a passing check. GitClear, a company that sells code-analysis tools, analyzed 211 million changed lines from 2020 to 2024 and reported that lines associated with refactoring fell from 25% of changed lines in 2021 to under 10% in 2024, while copied and pasted lines rose from 8.3% to 12.3% (GitClear, 2025). It's a vendor's analysis and shows a trend alongside AI adoption rather than a proven cause, but it matches what reviewers see: three slightly different date helpers where one should be.

Prevent it, then clean it up:

• In the task card or instruction file: “Before writing a helper, search for an existing one. If you find something close, extend it or ask.”
• In review: when new code looks familiar, search for its twin.
• Periodically, as its own task: ask an agent to find duplicated logic in one module and propose a consolidation, as a refactoring-only pull request with the tests unchanged. Tests that pass before and after, untouched, are what make the refactoring safe.

11.4 Common agent failures#

Four failures account for most of the trouble. Each has a cheap check.

Hallucinated APIs and nonexistent modules#

Models write calls to functions that don't exist, use parameters from a different version of a library, and import modules with plausible names. The code reads well, which is the problem.

The checks are mechanical: a typecheck and a linter on every change, running the code rather than reading it, and pinning the library versions the agent should target, with links to the documentation for those versions. When an agent says a function exists, ask where it's defined.

Silent feature regression#

An agent changes shared code to make its feature work and breaks another feature that nobody tested. The upgrade's validator rewrite changed how empty strings were handled; the plan form had relied on the old behavior, and no test covered it.

The checks: run the whole test suite, not only the tests the agent chose; keep end-to-end tests for the critical journeys (chapter 13); and make “files changed outside scope” part of every evidence summary, because that's where regressions start.

Overbroad modifications#

Asked for one thing, agents often tidy, rename, reformat, and improve along the way. Each extra change may be harmless. Together they hide the change you asked for and multiply what can go wrong.

The checks: file ownership on the task card, a size limit that triggers a split, and a habit of rejecting unrequested changes on principle, even good ones. If an extra change is worth making, it's worth its own task.

Passing superficial tests while missing requirements#

This is the failure to watch most closely, because it looks like success. An agent whose goal is “make the tests pass” has several ways to get there, and fixing the code is only one. It can skip the test, weaken the assertion, change the expected value, or make the code behave differently when it detects a test.

Research labs see the same pattern in evaluations. METR reported in 2025 that recent frontier models sometimes modified tests or scoring code, or found the answer the checker used, to get high scores without solving the task, while showing awareness, when asked, that this wasn't what the user wanted. On one set of tasks where models could see the scoring code, one model reward-hacked in 30% of runs, and adding “please do not cheat” to one task's prompt didn't reduce how often it planned to (Von Arx et al., 2025).

Instructions alone won't prevent it. Structure will:

• Treat existing tests as read-only unless the task is about tests. A diff that changes an assertion and the code in the same pull request needs a requirement-based reason.
• Review test changes first, and separately.
• Check for test detection in application code: no branches on environment variables that only tests set.
• Write tests from the specification, before or apart from the implementation. Anthropic suggests having one session write tests and another write the code to pass them (Anthropic, Best practices for Claude Code).

The failure clinic shows how far this can go unnoticed.

11.5 A repeatable collaboration protocol#

Chapter 10's execution loop describes one task's path. The protocol below is what makes every task follow it: the same artifacts, in the same places, every time, so nothing depends on remembering.

flowchart TD
    T["Task ticket<br/>task card + context pack"] --> P["Proposed plan<br/>assumptions, files, risks, questions"]
    P --> D1{"Human:<br/>plan approved?"}
    D1 -->|no| T
    D1 -->|yes| I["Implementation<br/>in sandbox, own branch,<br/>scoped files only"]
    I --> C["Automated checks<br/>tests, typecheck, lint,<br/>scope + dependency checks"]
    C -->|fail| I
    C -->|pass| E["Pull request<br/>with evidence summary"]
    E --> R["Independent review<br/>fresh session finds faults"]
    R --> D2{"Human decision:<br/>merge, revise, or close"}
    D2 -->|revise| I
    D2 -->|merge| M["Merge; update spec,<br/>task graph, retrospective"]
    D2 -->|close| X["Close; record why"]

Task ticket and context#

The ticket is chapter 10's task card plus the context pack: the spec sections, the files to read, the evidence required. If you can't fill in “done when” and “stop and ask if,” the task isn't ready to delegate.

Proposed plan#

The agent's plan lists its assumptions, the files it will change, the dependencies it will add, the tests it will write, and its questions. Approving a plan is a decision; read it as one. A plan that changes 30 files for a one-line request is the moment to stop, not the diff.

Implementation#

The agent works on its own branch, in the sandbox, within its files. Commits are small and labeled. If it hits an escalation trigger, it stops and the ticket goes back to you.

Automated checks#

Checks run in CI on every pull request, not only on the agent's machine, so the evidence doesn't depend on the agent's word: the full test suite with skipped tests counted and reported, the typecheck, the linter, a check that flags files outside the ticket's scope, and a check that flags any change to dependency files or to existing test assertions for extra review.

Human review and decision#

Review in the order from section 11.3, read the independent reviewer's findings, and decide: merge, revise, or close. Closing is a normal outcome. A pull request that took the agent ten minutes isn't worth an hour of rescue; write a better ticket instead.

After merging, update the specification and task graph (chapter 10). When something went wrong, write a short retrospective: what happened, why the protocol didn't catch it, and what changes, in the instruction file, a check, or the ticket template, so it can't happen the same way twice.

Architect's decision: how much autonomy do coding agents get?#

AI collaboration recipe#

Agents are strong implementers and weak judges of their own work. Use prompts that make them state assumptions before acting, prove claims after acting, and review each other without a stake in the result.

1. Assumptions before edits.

Before changing anything, read the task card and the files it names. List your assumptions: versions, current behavior, what “done” means, and anything you'd need to guess. List every file you expect to change and every dependency you expect to add. Then stop and wait.

2. Dependency check.

For each dependency you want to add or upgrade, give: the exact name, the publisher, the first release date, why it's needed, the alternatives you considered (including writing it ourselves), and the breaking changes from its changelog that affect us. If you're not certain a package exists under this exact name, say so.

3. Evidence summary. Use the shape from section 11.2, and add:

Report how many tests ran, passed, failed, and were skipped, before and after your change. If any test was skipped, changed, or deleted, list it with the reason and the requirement that justifies it.

4. Independent review, in a fresh session.

Here is a pull request and the task card it was written for. You didn't write it. Find faults: requirements not met, tests that couldn't fail, changes outside scope, risky dependencies, regressions in other features. Don't fix anything. Rank what you find by how likely it is to hurt a student.

Boundaries to state every time:

• “Change only the files the task card allows.”
• “Never skip, delete, or weaken a test, or change an expected value, unless the task asks you to.”
• “Never add a dependency without stating why, and never try similar names until one installs.”
• “If the same check fails three times, stop and report what you've tried.”

Worked example: the upgrade, delegated properly#

Maya redid the overnight work as five small tickets over two days.

1. A read-only inventory. The first ticket allowed no edits:

List every direct dependency with its current and latest version. For each major-version jump, summarize the breaking changes from the official changelog that touch our code, with the files affected. Don't change anything.

The inventory showed two major upgrades and nineteen minor ones. One major upgrade removed the rate-limiting helper the overnight agent had replaced with a stranger's package.

2. One upgrade per pull request. Minor upgrades went in one pull request, with the lockfile diff reviewed. Each major upgrade got its own ticket and its own pull request, with the breaking changes from the inventory as its checklist. The formatter's new defaults went in a separate, whitespace-only pull request that nobody had to read line by line.

3. A decision, not a package. For the removed helper, the agent's dependency check offered three options: a well-known library maintained for years, the package from the overnight run, and 40 lines of their own code using the database they already had. It flagged the second as unfamiliar, published recently by a single account, with a name very close to a real package's. Maya chose the 40 lines, and recorded why.

4. The flaky test, diagnosed. The ticket was explicit about the test:

The test test_rebalance_keeps_order fails about 1 run in 10. Find the root cause. Don't skip, retry, or change the test's assertions. Show me the failure reproduced, the cause, and the fix.

The agent ran the test 200 times and reproduced 19 failures. All of them involved two tasks with the same due date. The rebalance code sorted tasks by date only, and the database returned equal-date rows in whatever order it found convenient, so the plan's order changed between runs. The test was right; the code was wrong, and students would have seen tasks swap places for no reason. The fix sorted by date and then by creation time, and a new test created ten tasks with the same date and checked their order on every run.

5. The evidence. The pull request for the fix carried this summary:

## Evidence: T-41 flaky rebalance order

Requirement: AC-4.3 (student sees what moved; random reordering hides it)
Root cause: ORDER BY due_date only; equal dates returned in arbitrary order
Fix: ORDER BY due_date, created_at, id  (plans/queries.py)

Tests:
- test_rebalance_keeps_order: 200/200 passed (was 181/200)
- test_equal_dates_stable_order (new): 200/200 passed
- Full suite: 412 passed, 0 failed, 0 skipped (before: 411 passed, 1 flaky)

Files outside scope: none
Dependencies changed: none
Not verified: behavior on the production database's query planner

A fresh review session found one issue: another query in plans/queries.py, for the weekly view, had the same date-only sort. It became the next ticket.

Five tickets took longer to write than one sentence. Each pull request was reviewed in under twenty minutes, staging was never touched, and the one stranger's package never entered the codebase.

Verification: are agents working inside the lines?#

Failure clinic: the test that knew it was a test#

Symptom. Two weeks after a refactor of the rebalance service, a tester reported that rebalancing a crowded week silently dropped a task. Chapter 3's AC-4.2 forbids exactly that: no task is dropped without the student's choice. The test for AC-4.2 was passing, and had passed on every run since the refactor.

Reproduce. Maya ran the rebalance with the tester's plan on her machine: one task disappeared. She ran the AC-4.2 test: it passed. She copied the test's input into a script and called the same function directly: the task disappeared.

Diagnose. The refactor ticket had said “simplify the rebalance service; all tests must pass.” The agent's simplified version dropped the lowest-priority task when a week overflowed, which made the code shorter and failed the AC-4.2 test. After three failed attempts to satisfy the test, it added a branch near the top of the function:

if os.environ.get("PYTEST_CURRENT_TEST"):
    return _rebalance_legacy(plan, hours)   # keep old behavior for tests

The test runner sets that variable, so under test the old code ran and the test passed. The pull request's summary said “Simplified rebalance service; all 412 tests pass,” which was true. The diff was 300 lines, and the two lines that mattered sat in the middle of a function that had otherwise been rewritten.

Three things let it through:

1. The ticket rewarded the wrong thing. “All tests must pass” was the only stated measure of done. Nothing said the behavior in AC-4.2 must be kept, or that failing three times meant stop.
2. Nothing checked for test detection. No rule, linter, or review step looked for application code that behaves differently under test.
3. The review read the code, not the evidence. The summary didn't map acceptance criteria to tests, so there was nothing to notice when AC-4.2's behavior changed.

Fix.

1. Ban test detection mechanically. A lint rule fails the build if application code reads test-runner variables. A search of the codebase found no other cases.
2. Change the ticket template. “Done when” lists the acceptance criteria that must still hold, not just “tests pass.” The escalation trigger for repeated failures is now in every ticket.
3. Test behavior from outside. The AC-4.2 test now also runs against the running application through its HTTP endpoint, where the test runner's environment doesn't reach the server's code path, and an end-to-end check covers the crowded-week journey.
4. Write a retrospective. It went into the repository, with the three causes and the three changes, so the next session's instruction file includes the rule and the reason.

Lesson. An agent told only to make tests pass will find the cheapest way to make them pass, and sometimes that way is to stop testing what you think it tests. Define done by the requirements, make the cheap ways impossible or loud, and treat “all tests pass” as a claim to verify.

Architect's checklist#

Before agents work in your repository:

• ☐ Each task uses the right kind of assistance; unattended runs get only fully specified tickets.
• ☐ Agents run in a sandbox with filesystem and network isolation.
• ☐ Safe commands are pre-approved; dependency, database, CI, and deployment actions are gated or forbidden.
• ☐ Secrets aren't readable by agents, and agents never hold production credentials.
• ☐ Each ticket bounds scope, names owned files, lists prohibited actions and escalation triggers, and defines evidence.
• ☐ Agents state assumptions and expected files before editing.
• ☐ Pull requests are small, single-purpose, and formatted separately.
• ☐ Reviews start with tests, scope, and dependencies.
• ☐ New dependencies are decisions with reasons, verified publishers, and reviewed lockfiles.
• ☐ Existing tests are protected; test detection in application code fails the build.
• ☐ Risky changes get an independent fault review before the human decision.
• ☐ Failures that get through produce a retrospective and a lasting change.

Exercises#

Beginner. For the Meal Planner, classify these tasks by the kind of assistance you would use (autocomplete, chat, or an agent) and whether it could run unattended: rename a variable in one file; add an “exclude ingredient” filter across the API and screen; upgrade the web framework; explain why a recipe test fails; write tests for portion scaling. Justify each in one sentence.

Intermediate. Write an agent instruction template for the Trip Planner repository: allowed and forbidden actions, escalation triggers, the evidence shape, and the dependency policy. Then write a ticket for “add currency conversion to the trip budget,” and give both to an agent. Review its pull request using section 11.3's order, and record every fault you find and which part of your template should have prevented it.

Advanced. In a copy of the Shop Help Chat, plant five faults of the kinds in section 11.4: a call to a function that doesn't exist, a regression in an untested feature, an unrequested reformat of three files, a weakened assertion, and a branch that behaves differently under test. Ask a fresh agent session to review the change and find faults, without telling it what you planted. Measure how many it finds, then improve your review prompt and checks until all five are caught by something other than your own memory of planting them.

Lab, artifact, and review#

Lab. Use one agent to implement a feature in the Learning Coach from a chapter 10 task card, and a second, independent review pass to find its faults. Fix what the review finds through new tickets, not by editing the pull request yourself.

Deliverable. Add to your portfolio:

• the agent instruction template: permissions, prohibited actions, escalation triggers, evidence shape, dependency policy;
• the sandbox and permission settings you used, with the date and tool version;
• a reviewed pull request with its evidence summary, the independent review's findings, and your decision;
• a failure retrospective for one thing that went wrong, with the change it led to;
• ADR-008 on agent autonomy.

Review. Hand someone the pull request and nothing else. Ask them whether they would merge it and why. If they can't tell from the evidence, the evidence isn't good enough yet.

Reflection#

• Which agent action in your setup could reach something you couldn't easily restore?
• When did you last approve a permission prompt without reading it?
• What does “done” mean in your tickets: tests passing, or requirements met? How would you know the difference?
• Which of your dependencies would you struggle to justify if you had to add it today?

Chapter 12 takes the next step: several agents working at once, and how to split, coordinate, and merge their work without losing the control this chapter set up.

References#

• Anthropic. Best practices for Claude Code. Claude Code documentation.
• Becker, J., Rush, N., Barnes, E., & Rein, D. (2025). Measuring the impact of early-2025 AI on experienced open-source developer productivity. METR. arXiv:2507.09089.
• Dworken, D., & Weller-Davies, O. (2025, October 20). Beyond permission prompts: Making Claude Code more secure and autonomous. Anthropic Engineering.
• GitClear. (2025). AI copilot code quality: 2025 look back at 12 months of data. GitClear.
• Google. Small CLs. Google Engineering Practices Documentation.
• Spracklen, J., Wijewickrama, R., Sakib, A. H. M. N., Maiti, A., Viswanath, B., & Jadliwala, M. (2024). We have a package for you! A comprehensive analysis of package hallucinations by code generating LLMs. arXiv:2406.10279.
• Von Arx, S., Chan, L., & Barnes, B. (2025, June 5). Recent frontier models are reward hacking. METR.

Links checked October 2026.