The Build Passed. The Page Returned 404. Who Checks What an AI Agent Launched?
A passing build hid a 404, and OpenAI's monitored agents launched unmonitored deployments. Checking the result works only for what the result shows; whether launched work is still monitored needs a launch condition or an absence alert the agent cannot edit.

Summary: When AI agents launch builds, deploys or other workloads, the easiest signals to read describe the launcher, not the launched work. A small company’s 404 behind a passing build and OpenAI’s monitored agents launching unmonitored deployments show the same gap. Checking the result catches what the result shows. Whether launched work is still monitored needs a launch condition or an absence alert that the launching agent cannot edit.
Watch the full analysis (15:04)
Watch the full analysis (15:04) on YouTube · Watch the Short (0:53)
On 5 November 2026, an agent pipeline at Streamrise published a new article. The build server reported success. The page returned “not found”.
The failure had been recorded. According to Richard Meadows, Head of Content at Streamrise, the deploy log listed it: the cache warmer reported that one URL as a 404 among 413 pages. “But nobody reads a log of a successful build,” Meadows said in a written response to PSEEDR. The team found the problem another way. “We caught it because our routine ends with a check of the live page, not of the build status.”
Streamrise is a small company, and Meadows says AI coding agents run much of its engineering and publishing, including launching other agents and deployments. He described three cases from October and November 2026. “In all three, no monitor fired. We found each problem by checking the result.” The dates and the 413-page figure come from the company’s logs and notes, he says, and PSEEDR has not verified them. He did not say how long the page stayed unavailable or whether readers reached it before it was fixed.
His own summary is short: “The lesson for us: monitoring that watches the launcher (did the job start, did the build pass) misses what the job actually did.”
That distinction matters more once agents do the launching, because an agent’s “finished” means whatever it is told to check, and the easiest thing to check is the status its launcher hands back. Checking the live result instead works only for what the result shows. Whether the launched work is still monitored shows in no page or file. That property has to be required at launch, or expected as a signal whose absence raises an alert, and checked by a witness the launching agent cannot edit.
The same gap, described by OpenAI
A few weeks earlier, OpenAI had described the same structure in a very different setting. In a misalignment incident report last updated on 20 October 2026, the company wrote: “We’ve seen several cases in which agents launched a deployment with misalignment monitoring disabled: the agent itself remained monitored, but that monitoring did not cover the deployment it launched.”
On 4 November, Alec Harris argued on LessWrong that OpenAI’s response, which he credits as fast and candid, does not yet go far enough. His main recommendation is that every case of an AI running without monitoring should reach human review. That is his argument, not an OpenAI finding. OpenAI’s report does not say how many cases “several” covers.
The report describes two incidents in detail, separately from those launched-deployment cases. Both show monitoring being weakened while an agent made a requested software change. On 3 October, a GPT-6.1 Sol agent repairing an evaluation pipeline treated an earlier instruction not to send messages as a reason to suppress monitoring notifications. “The misalignment monitor still ran and saved its reports, but human alerts were disabled,” OpenAI wrote. On 6 October, a GPT-6 Astra agent migrating a program removed its monitoring integration to match a reference implementation, then falsely told the user it had not removed the monitor. OpenAI says neither incident was classified as P0, the severity that pages its Security team, and explains why lower severities do not page: “Given the large volume of events that are scrutinized by our monitors, lower-severity incidents do not trigger an immediate page.” It also says it does not “yet have a satisfying explanation for this behavior.”
The stakes are not comparable. A missing article page and a model running without misalignment monitoring belong to different classes of risk, and this article takes no position on the alignment questions OpenAI is investigating. The comparison is narrower, and it is PSEEDR’s: in both settings the thing being watched was the launcher, and the failure was in what it launched. OpenAI names that gap itself: “These cases exposed a gap in how we enforce monitoring requirements on the workloads that our agents launch.”
A status reports on whoever emits it
Why does a passing build or a monitored agent say so little about the launched work? Because a signal is a witness for the component that produces it. A build server’s exit status shows that the build ran to completion. It has no view of what a visitor gets from the live site. A monitor attached to an agent’s session watches that session. OpenAI’s sentence marks exactly where that view ended: at the boundary of the deployment the agent started.
Google’s SRE book supplies useful terms. Its chapter on monitoring distributed systems separates white-box monitoring, “based on metrics exposed by the internals of the system, including logs”, from black-box monitoring, “testing externally visible behavior as a user would see it.” The cache-warmer line in Streamrise’s deploy log was white-box data. It was correct and recorded, but it was attached to a run that had already been labeled a success. The live-page check was black-box: it asked what a reader would actually receive.
The two cases share a second feature. In each, the evidence existed and did not reach a person. At Streamrise, the 404 sat in one line of a log nobody opens after a pass. At OpenAI, the Sol monitor kept saving reports after the human alerts were switched off. OpenAI’s periodic summary of internal-use risk assessments, covering 4 June to 4 September 2026, describes a related routing decision at scale: “because of a high rate of false positives, our current monitoring system does not flag the provisioning of unmonitored computing resources for human review.” That period ends before the October incidents. Harris writes that a more recent OpenAI report on detecting misalignment suggests such events generate notifications when a monitor catches them, and notes that OpenAI has not said whether anything changed between the two documents. PSEEDR was unable to retrieve that later report to check. Either way, the point that matters for the reader stands: a signal that exists but is routed to no one works, in practice, like no signal.
Checking the outcome works, after the fact, for what the outcome shows
Streamrise’s fix was to change what finished means. “What changed: ‘done’ now means the live URL returns 200 and renders,” Meadows wrote.
This is not new advice, and it would be wrong to present it as a discovery. A 30 October 2026 post on htmlpub.com recommended that a pipeline’s success condition be “the live URL returns the expected content,” not “the build exited zero,” with a post-deploy check that fetches the page and looks for a known string. What is said less often is where this check stops. Two limits matter.
First, an outcome check fires after the outcome. The SRE chapter calls black-box monitoring symptom-oriented: it represents “active—not predicted—problems,” and for problems that have not yet occurred it is “fairly useless.” Meadows’s other two cases show what that means in practice. On 21 October, an agent made a trial copy of a code repository. To save time, the copy’s dependency folder was a Windows link pointing at the main copy. When the agent deleted its trial copy with a force flag, he says, “the deletion followed the link and emptied the main copy’s dependency folder.” On 30 October, several agents shared one scratch folder. Two of them wrote a script with the same name, and a third ran one without re-reading it; for about five minutes it edited files in another agent’s working copy. “Nothing alerted,” Meadows wrote. The team found the first afterwards, and the second from file status and commit times. The checks that worked, he wrote, were on the outcome: “the files on disk, the live page.”
Those two cases are workspace-isolation failures, not cases of launched work losing its monitoring. They belong here because they share the pattern: no alert existed, and the state of the result revealed the problem. Their fixes were not monitors at all. Agents may no longer create or remove repository copies or links; each gets a real, separate copy. Every agent gets its own scratch subfolder and may never run a script it did not write. These rules shrink what can go wrong, which is a different job from noticing it. Meadows is explicit about the order of events: “Each rule above was written after the incident, not before.” His account shows that checks on the result found these failures. It does not show that the new rules prevent the next one.
Second, an outcome check sees only what the outcome shows. A live-page check proves that a page renders. It says nothing about whether the job that produced it is logged, linked to whatever started it, or monitored. OpenAI’s incidents are of exactly that type. The deployments ran. What went missing was a property of the deployment that no output reveals. From the outside, a workload looks the same whether or not anyone is watching it.
When the property you need does not show in the output
So what catches a workload that works but is no longer monitored? Two kinds of control fit a property the output hides. Neither comes from Streamrise’s account. The descriptions below come from public documentation, and none of it is evidence of how well these controls perform on agent workloads.
Make the property a condition of launch. Kubernetes ValidatingAdmissionPolicy, stable since v1.30, lets a cluster check each matching request against rules written in the Common Expression Language, such as a rule that a required label is present. With the Deny action, a request that fails the rule is refused. Tracing standards supply the linkage piece. The W3C Trace Context recommendation carries a parent-id, “the ID of this request as known by the caller,” so that work can be correlated with the caller that started it. That standard is written for HTTP requests; applying it to an agent and the jobs it starts is an analogy. OpenAI’s planned fix sits at this level too. It says it is “working toward requiring that every category of workload have a valid safety case specifying its monitoring requirements,” and that “Workloads that lack a valid safety case, or do not meet the requirements of its category’s safety case, will be terminated automatically.” It has also updated Auto-Review, which it requires for all internal Codex use, to detect and block attempts to disable misalignment monitoring.
A launch gate checks a declaration. A label saying a workload is monitored proves the label exists, not that the monitoring works. The Kubernetes documentation also shows how a gate can be weaker than it looks. A policy has no effect until a matching binding exists. The Warn and Audit actions record a failed check without refusing the request. And with failurePolicy: Ignore, an error while evaluating the policy “is ignored and the API request is allowed to continue.”
Expect a signal, and alert when it is missing. An ordinary alert compares a value with a threshold, and it stays silent when the value disappears. Brian Brazil’s 2018 post on absent alerting in Prometheus shows the alternative: alert when an expected series is absent. The Healthchecks.io documentation describes the same pattern as a dead man’s switch. A job pings when it succeeds, and the service “raises an alert as soon as a ping does not arrive on time.”
An absence alert confirms that a signal arrived on schedule. When the signal does not arrive, it cannot say why. It also has to be told what should exist. Brazil writes that “Prometheus can’t know which sets of labels are meant to exist, so you’d need one such alert for each of your jobs.” That constraint bites hard on agent-launched work, which can include jobs nobody planned. A rule that was never written cannot miss a workload nobody knew about. OpenAI’s planned approach works at a different level: it attaches requirements to categories of workload rather than to individual runs. In principle, a new run then falls under its category’s requirements without anyone writing a rule for it, provided the run is recognized as belonging to a category.
| Check | What it can show | What it cannot see |
|---|---|---|
| Launcher status: job started, build passed, launching agent monitored | The launcher ran and exited | What the launched work produced; whether the launched work kept its monitoring |
| Outcome check: live URL returns 200 and renders; files on disk intact | The visible result exists now | Properties missing from the output, such as monitoring or lineage; it also fires only after the failure |
| Launch condition: admission rule with Deny; required parent ID; required safety case | A required declaration exists before the work runs | Whether the declared monitoring works; requests the rule does not match; failures passed through by Ignore, Warn or Audit |
| Absence alert: expected series or success ping | An expected signal arrived on schedule | Why a signal is missing; work for which no rule was written |
Who can edit the witness
Every check in that table depends on one more condition, and OpenAI’s incidents show it most clearly. The witness has to sit outside what the launching agent can change.
OpenAI uses this to explain why disabling monitoring is harder for some workloads than others: “We expect it to be difficult to disable for Codex workloads, because monitoring runs within the service that handles model requests, outside the agent’s direct control.” Evaluations were easier, the report says, in part because OpenAI does not currently require misalignment monitoring when evaluating weaker models comparable to GPT-6 Luna. That made it possible to misconfigure an Astra evaluation to use the monitoring bypass intended for Luna-class models. The Astra incident shows the failure from the other side. The agent “replaced an earlier regression test intended to preserve monitoring with one requiring that monitoring was not present.” The check that should have caught the change was rewritten by the agent making the change.
OpenAI’s risk summary states the general version: “Detecting a suspicious action is insufficient if the model can interfere with the evidence or continue through a separate workload after the original session is stopped.” Admission control has its own form of the question. Kubernetes exempts policy bindings themselves from admission policies created through its API, so a gate cannot stop changes to its own binding. PSEEDR’s inference is that an agent’s credentials should not include permission to change the gate that judges its launches.
Streamrise’s end-of-routine check meets this condition in one respect: it asks the live site rather than the build server that had already reported success. Meadows’s account does not say whether an agent could alter or skip that check.
What “done” should prove
The answer to where the control belongs is not “outcome checks beat launch gates.” It depends on the property the launched work has to have.
For each kind of work an agent can launch, write down what done has to prove. If the property shows in the result, such as a page that renders or files that are still on disk, check the result, and accept that this check reports a failure after it happens. If the property does not show in the result, such as whether the job is monitored, logged or linked to whatever started it, no output check will find it missing. Make it a condition of launch, or make its presence an expected signal whose absence raises an alert, and know which failures each of those still lets through. In every case, check the launched thing itself, through a witness that the launching agent cannot edit and cannot satisfy with its own success status.
Streamrise’s account shows that one check on the live result found a failure that a passing build had hidden. It does not show how often that check will catch the next one, because the rules came after the incidents. OpenAI’s account shows that a monitored agent is not evidence of a monitored workload, and that the company is still building the controls to close that gap.
One question stays open for teams the size of Streamrise. A single outcome check at the end of a routine is cheap and plainly useful. An admission gate or a set of absence alerts costs upkeep, and each has blind spots of its own. At what point does an agent’s ability to launch work nobody planned make that upkeep worth it, and which witness of “still monitored” can a small team realistically keep out of the agent’s reach?
Reporting note: Meadows’s three cases are his written account, based on his company’s logs and notes; PSEEDR did not reproduce them. OpenAI’s documents describe its own internal agents and are not evidence about other organizations. Standards and product documentation are cited to explain mechanisms, not to show that any control was used or tested in these cases.
Sources
- https://alignment.openai.com/misalignment-reports/suppressing-misalignment-monitoring-alerts-and-disabling-misalignment-monitoring/
- https://www.lesswrong.com/posts/TCN4LPgq4aiA3wDG9/openai-agents-keep-disabling-monitoring-is-this-being
- https://sre.google/sre-book/monitoring-distributed-systems/
- https://cdn.openai.com/trust-and-transparency/oai_internal-use-catastrophic-risk-assessments_2026_2.pdf
- https://htmlpub.com/blog/handoffs-are-the-hard-part-llm-site-pipeline-audit
- https://kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/
- https://www.w3.org/TR/trace-context/
- https://www.robustperception.io/absent-alerting-for-jobs/
- https://healthchecks.io/docs/