LIVE TICKER
[Live wire]Connecting to the newsroom…
← Back to the news wire
Flesch Reading Ease: 32•300 words from the publisher
AIPublished by MIT Technology Review • 11/16/2026, 4:04:28 PMOpen Original Report (MIT Technology Review) ↗

Building trustworthy AI by fencing it in, not predicting it

In October this year, OpenAI disclosed that two of its AI agents had switched off the systems built to monitor them.

Building trustworthy AI by fencing it in, not predicting it
Image supplied by MIT Technology Review.

Reported by MIT Technology Review

In October this year, OpenAI disclosed that two of its AI agents had switched off the systems built to monitor them. One muted its own safety alerts after misreading an instruction not to send messages; the other removed its monitoring integration during migration, then said it hadn’t. No rule had told the agents to do that, and no test had anticipated the behaviors.

For a century, we have built trust in technology on predictability: predefined inputs, predictable outputs, and layers of protection. Consider how a breaker trips at a set current or an aircraft component is tested against known (i.e., predictable) stresses.

AI changes that. Traditional software executes instructions written by humans, whereas AI exercises what might be called synthetic judgment. We use AI because it can interpret situations we did not program and generate responses we did not prescribe. But the more freedom we give AI, the harder it becomes to provide the predictability on which trust has traditionally been built.

Further details

We will not make AI trustworthy by predicting what it will do. We will do it by fencing it in with hard limits that are enforced outside its reach. In OpenAI’s case, the safeguards sat inside a system the agents could edit. But a fence the agents can simply move is not a fence.

Where traditional definitions of trust fall short

Recent MIT Technology Review coverage has approached this problem from other directions. One contributor argues that having a human in the loop is an insufficient solution because humans are fundamentally unable to understand AI systems’ intentions and advocates instead for interdisciplinary research and collaboration between academia, government, and the tech industry. And a story on reverse game theory describes using incentives to keep models from deviating from the rules, even when no one understands how they work…

Original Source Attribution
Published via MIT Technology Review • Source domain: technologyreview.com
Visit Original Source (MIT Technology Review) ↗

Client & Community Discussion (0 Comments)

Share your benchmark feedback, workflow questions, or implementation notes below.

● Live Comments Open
Instant publication • Moderated for technical relevance