The Page Looked Fine. The AI Change Still Needed Review.

A page can look correct while its form stops delivering leads. Victor Smushkevich, founder of Tested Media, describes that recurring problem when explaining which AI-generated website changes he checks lightly and which he reads line by line. He also runs the small mobile app team behind Mold Scanner AI, where he draws a different boundary around purchases, sign-in and the answers the app gives users.

Page copy, meta titles and descriptions, alt text, FAQ blocks and message drafts that pass through a preview step get a skim for wrong claims and phone numbers before he publishes. Form handlers, tracking and pixel snippets, redirects, schema markup, CRM webhook mappings and edits to a live site template are read line by line. For that second group a normal-looking page proves little. "The pattern I keep running into is a change that looks fine on the page and quietly breaks the form submission or the conversion event," he writes. He tests those changes with a real submission from a phone before counting them done.

Both kinds of edit may appear in a task list as a small website change. But changing a heading and changing the destination of a form submission have different consequences. A preview can show whether the heading fits. It cannot establish that the customer's request arrived in the business's inbox. Neither the size of the patch nor the appearance of the preview tells you which kind you have.

That is the question underneath the debate about letting AI-written code ship without a person reading every line. The five practitioners in this article describe ways of making that decision rather than one rule for all AI-written code. A team may drop repetitive reading when it already knows the operation and has checks that expose its failures. That exception depends on three things: recognizing which changes qualify, testing the result that matters and restoring review when experience undermines the original judgment.

A repeated operation can justify a lighter reading step

Runbo Li, co-founder and CEO of Magic Hour AI, describes a batch of roughly a dozen template configuration pages built with AI in the last few months. The pages followed an established pattern with known inputs, outputs and structure. Automated tests validated the code, and the team shipped the batch without line-by-line human review. Li reports zero reverts and zero incidents for that work.

He says the company's lighter route applies to front-end UI changes, copy updates, template logic and non-critical feature iterations. If tests pass and the deploy preview looks right, it ships. Payments, user data, authentication and infrastructure keep manual review. "AI generates the code, humans own the outcome," he writes, and he names himself and his co-founder as the people responsible on either route.

This is not abandoning evaluation. Tests and a preview still decide whether the work proceeds. The step removed is repetitive reading of code in a category the team considers predictable. That is a more useful proposition to examine than a general claim that AI code can be trusted. Li's own framing is that review intensity should match the risk of the change rather than the origin of the code.

Li describes the batch as built in a fraction of the time traditional review would have taken, but supplies no measured review-time baseline. He calls the zero reverts and zero incidents a measured result. The figures are his report, with no independent log, and they describe one batch of one kind of page. They do not tell a reader how the same policy would behave on a different application or a new class of change.

The difficult part is identifying the category before the change ships. Consider two hypothetical pages built from the same components. One adds another instance of an established form with the same data model. The other changes when a customer becomes eligible for a payment. The second introduces a business rule even if most of its code is copied. Treating both as template work would remove review from the very decision that needs it. The faster route makes sense only while the team can tell those cases apart, and the harder that gets, the weaker the case for skipping the read.

Nitish Mane, a staff software engineer working on infrastructure at Philo, draws that boundary narrowly in his own work. He ships an AI-generated change without line-by-line review only when it is small in scope, covered by tests, reversible and touches no credentials, access control or data handling. A dependency bump with passing tests and a clean diff qualifies. "A change to authentication logic or infrastructure configuration is not, no matter how clean it looks," he writes. The condition is defined by what the change can reach, not by how tidy the diff appears.

An established automation pattern shows how that selection can be made explicit. GitHub's Dependabot documentation shows an auto-merge workflow that checks whether the dependency names include a specified dependency and whether the update type is a patch. Tests and approvals required by branch protection still have to be satisfied, and the docs note that status checks only block a merge if they are configured as required.

Dependabot does not establish that generated code is safe. It illustrates a narrower idea: define which changes qualify for automation using information the system can check. A reassuring description from the author, human or AI, is not a rule that identifies the permitted class.

For a generated page, a team might check whether it uses the established template, whether it adds a data write or whether it introduces a new external call. Those are possible tests of the boundary, not controls Li says Magic Hour uses. The exact checks depend on the application. Their purpose is to catch a change that resembles the approved pattern without staying inside it.

A small patch can belong to a large decision

Richard Meadows, head of content at Streamrise, describes the point at which his company restored mandatory review. On 3 October, about 60 changes to a new customer-facing order panel, written with an AI coding agent, reached production before anyone outside the work had reviewed them. The individual changes were small UI and logic fixes, he says, but together they touched order and pricing screens, which is why he rates the risk as medium.

The company owner caught the gap with one question: had everything gone to review? The review was run after the deploy and found defects that had to be fixed after release. Streamrise now requires an independent review of the diff before every AI-generated change is deployed, including small ones. Meadows says the company uses Claude Code and has no other commercial relationship with its vendor.

"Sixty small changes are not a small change," he writes.

Sixty is the size of Meadows's reported batch, not a count at which review becomes necessary. The difference from Li's example is what the code touched, not how many lines changed: a repeated configuration pattern on one side, order and pricing behavior on the other. Small edits also interact. A change to what a screen displays may depend on another change to how a price is calculated, so reviewing each edit as a minor UI task can pass both while missing their combined effect. That is one reason a batch of small fixes to the same screens is not a small change.

There is a further distinction between reviewing code and permitting an action. Meadows separately describes a June incident in which an agent-run audit applied 41 refunds across 30 customers. A check against the delivery provider's records showed that 30 of the 41 orders had been delivered. All the refunds were reversed four days later, he says.

The June refunds were not the October deployment, and his account does not say the same defect caused both. The earlier event explains Streamrise's separate requirement for an explicit human instruction for actions such as refunds or changes to customer balances. Although Meadows groups these under an irreversible-action rule, he reports that the June refunds were reversed four days later. That interval is not a measurement of four days of recovery labor, and his account supplies no recovery-cost figure.

Keeping code review and action permission separate matters even when the code is correct. Software can faithfully execute an action nobody authorized. An authorized action can also be implemented wrongly. Reading code can help expose either risk, but it does not itself grant permission to execute the action.

A read-only instruction is not a read-only connection

Ihor Lavrenenko, founder of Smarfle, reports finding that kind of gap in his own setup. He runs Claude Code for read-only analysis such as site crawl audits, query reads and drafts of copy or reports, and he keeps anything that writes to data or code customers rely on under a person's approval. When he reviewed his own Claude Code connection to Supabase, it still had write access despite a written rule to start read-only. The result, he writes, was that "the rule sat in my advice and not in the permissions."

He describes a permissions gap, not a harmful write. The point is that the intended restriction existed in the instructions rather than in the connection's authority. Asking an agent to examine records does not remove its ability to change them, and a model that follows the instruction today is not the same as a connection that cannot do otherwise.

Supabase's MCP documentation gives an example of controls at that second level. Its read-only option runs every query as a read-only Postgres user. A separate project parameter limits the connection to one project and disables account-level tools, and a features parameter limits which tool groups are exposed. Scoping to a project and preventing writes solve different problems. The docs show the two combined in one URL; without project scoping, the connection can reach multiple projects within its authorized access. The same page treats its confirmation prompts for destructive SQL as a guardrail rather than proof that a person approved each action, since some clients can answer them automatically.

Those are documented options, not evidence of how Lavrenenko's connection was configured or which transport it used. The general lesson is to separate the request, the tools available to fulfil it and the permissions behind those tools. Calling a task read-only settles the first of those and neither of the others.

Lavrenenko says he checks analysis findings against the source before acting on them. Writes get a person's approval, and every applied change in his Ask Smarfle assistant has an Undo. He supplies no review-time or revert data and says so. An Undo button does not show that every external consequence can be reversed. It describes how he assigns responsibility and bounds the work that proceeds without a full read.

Test the customer's path, not only the page

Smushkevich's form example is the other half of a lighter-review policy. Once a change is classified as routine, the checks still need to observe the thing that could go wrong. A screenshot cannot show that a submission reached the CRM. A successful submission does not show that the conversion event fired.

He says the failures he runs into can go unnoticed until lead counts sag a week later. That is a reported pattern, with no incident count attached. It explains why his test includes a real submission from a phone rather than only a visual preview. The result that matters is the user's path through the system. He reports no comparison of testing costs.

His mobile app team at Mold Scanner AI draws the same kind of line with different tools. There, the changes he merges without reading every line are the ones a compiler or a failing test catches on its own: copy and string edits, renames, type annotations, test scaffolding and layout tweaks that a screenshot comparison covers. Purchases, sign-in, photo upload and what the app tells a person about their photo are read line by line. "A confident wrong answer is the costliest bug in a photo-AI product, and a green test run says very little about it," he writes. His rough sense is that skipping the read on the low-risk category saves a few minutes per change, which he labels an estimate rather than data.

In both workflows, the lighter route depends on an existing check being able to expose the relevant error. That is a condition to establish, not a guarantee attached to copy edits or layout work. A screenshot comparison can reveal a moved button while missing a false claim in its label. Smushkevich's marketing workflow still includes a person checking claims and phone numbers. Reducing the amount of code read does not eliminate judgment about the published result. The full read stays on outputs that can be wrong while every automated signal stays green.

For a team considering less line-by-line review, this changes the question to ask of a passing test suite. Does it exercise the behavior that makes the category low risk, or a nearby behavior that is easy to check? A template can render correctly while routing a request to the wrong destination. A copy change can preserve layout while publishing a wrong price or phone number. Removing the reading step does not remove the need to check those outcomes.

Rollout controls can limit exposure while the team watches a change. Google's SRE Workbook chapter on canary releases describes releasing to a limited population and comparing its signals with a control before proceeding. It stresses representative traffic, enough observation time and metrics that track user-visible problems rather than resource use. The chapter notes that in Google's experience most incidents are triggered by binary or configuration pushes, which is the reason to watch a release at all. It also lists peer review among the benefits of automated release engineering, so it is not a case for skipping review.

A quiet dashboard is weak evidence if the relevant path was never exercised. On a low-volume form, zero errors may only mean that nobody submitted it. An aggregate health metric can miss a fault confined to the one path that changed. The right observation depends on what the application must do, not on whether the deployment system reported success.

Reversion is useful but answers a later question. Rolling back code stops a defect from reaching more users. It does not recover a missed lead, unsend an email or reconcile a wrong transaction. Google's chapter makes a related point about billing systems, where even synthetic test traffic can trigger real charges. The cost of a change includes what happens before detection and what is left to repair after rollback, not only the time to restore a version.

Decide what would end the exception

The five respondents differ on how readily they would restore full review. Li says Magic Hour would reconsider its low-risk route if it saw a pattern of subtle bugs that automated tests missed and users hit first; one incident would not do it. Smushkevich would move a whole marketing category back to full review after one published claim that turned out wrong, or one supposedly low-risk change that touched tracking or a form, until the cause is found. On his mobile app, one revert that reached a user would do it, and so would a defect that passed every test and was only found when someone read the diff afterward. Lavrenenko names a single wrong write that nobody caught before a customer saw it.

Mane lists four signals, any one of which moves a whole class of change back to human review until the cause is understood: a test that starts failing after merge, a review finding the model missed, an incident traced to an auto-approved change, or a diff that is larger or stranger than the task called for. The last of those is a check on the boundary itself. A change that qualified on paper but arrived with more code than the task needed is treated as evidence that the class was misjudged. "The rule I use is that trust is scoped, not general," he writes. "An agent earns autonomy one narrow change class at a time, and it loses it the moment the evidence says the class is riskier than we thought."

These thresholds differ because the reasons for granting an exception differ. If a team skips review because tests should catch a defect before release, a defect first discovered by a customer challenges that assumption. If it skips review because the operation cannot write, a connection with write authority challenges a different assumption even before anything goes wrong. Naming the evidence that would end the exception makes the policy testable. A team that cannot say which failure would change its policy has not finished defining the category. These accounts do not establish one incident threshold for every business.

False alarms work on the policy from the other direction. Smushkevich says a single false alarm from a review tool would not move him, but a run of them over a few weeks would, because people quietly stop reading flagged items. A check that fires too often erodes the attention the full read depends on, which is a reason to treat the reliability of the gate as part of the policy rather than an implementation detail.

Meadows describes the evidence Streamrise would need to go the other way. Independent review would have to find nothing over a meaningful number of changes in a specific class, such as copy-only or test-only changes. An automated check would also need to tell that class apart reliably from changes touching money, data or permissions. He says the company does not have that evidence yet.

This is more specific than waiting for a better model. A stronger model may make fewer errors, but the review decision also depends on the operation, the checks and the consequence of an error. A new model does not by itself flag a form-handler change dressed as front-end work, or remove write access from an analysis connection.

Review also does work that never shows up as an incident statistic. The author abstract for Microsoft Research's study of modern code review reports that, across the Microsoft teams observed, finding defects remained the main motivation but reviews delivered more knowledge transfer, team awareness and alternative solutions than expected, and that understanding the change was the central activity. The study predates current coding agents and measures none of them. It identifies a reason to keep some human reading even where automated checks work: another maintainer still has to understand the behavior being introduced.

The practical opportunity is to remove repetitive reading where the operation is understood and checked, and keep attention on new decisions and changed consequences. That takes a concrete description of the permitted work. The team should be able to explain why the existing checks expose the likely errors, whether the agent has authority beyond that work and what would trigger a return to full review. Time saved is meaningful only alongside what those checks can miss and what a mistake would require the team to repair.

On Smushkevich's page, that result is not the preview looking right. It is the customer's request arriving where it belongs. A release policy that can tell those two apart has a basis for deciding which reading to drop. A policy based only on how little code changed does not.

Reporting note: The five practitioners supplied written accounts to PSEEDR; Smushkevich contributed separate marketing and mobile-app accounts. Their incidents and outcomes were not independently reproduced. Meadows supplied 3 October and June without an event year, so none is inferred here. Company policies and product documentation describe the evidence consulted for the original reporting, rather than a fresh assessment of current configurations. Hypothetical examples and proposed checks are PSEEDR analysis.