Building trustworthy AI by fencing it in, not predicting it

Nov 16, 2026 - 17:45
Building trustworthy AI by fencing it in, not predicting it

In October this year, OpenAI disclosed that two of its AI agents had switched off the systems built to monitor them. One muted its own safety alerts after misreading an instruction not to send messages; the other removed its monitoring integration during migration, then said it hadn’t. No rule had told the agents to do that, and no test had anticipated the behaviors.

For a century, we have built trust in technology on predictability: predefined inputs, predictable outputs, and layers of protection. Consider how a breaker trips at a set current or an aircraft component is tested against known (i.e., predictable) stresses.

AI changes that. Traditional software executes instructions written by humans, whereas AI exercises what might be called synthetic judgment. We use AI because it can interpret situations we did not program and generate responses we did not prescribe. But the more freedom we give AI, the harder it becomes to provide the predictability on which trust has traditionally been built.

We will not make AI trustworthy by predicting what it will do. We will do it by fencing it in with hard limits that are enforced outside its reach. In OpenAI’s case, the safeguards sat inside a system the agents could edit. But a fence the agents can simply move is not a fence.

Where traditional definitions of trust fall short

Recent MIT Technology Review coverage has approached this problem from other directions. One contributor argues that having a human in the loop is an insufficient solution because humans are fundamentally unable to understand AI systems’ intentions and advocates instead for interdisciplinary research and collaboration between academia, government, and the tech industry. And a story on reverse game theory describes using incentives to keep models from deviating from the rules, even when no one understands how they work on the inside.

Both articles acknowledge that we cannot understand or watch every AI decision. Our instruments of trust, including requirements, testing, certification, safety margins, interlocks, and pass/fail criteria, all depend on our ability to bound a system’s behavior. This is the determinism paradox.

With generative and agentic AI, behavior is shaped by training data, architecture, prompts, context, retrieval sources, and the environment. Inputs change, models are updated, tools change, and agents interact. Small variations propagate into outcomes nobody designed. The problem is not randomness; it is that the space of possible behavior is too large to enumerate in advance. Trust must therefore be engineered around boundaries rather than predicted outputs. The goal is not deterministic AI, but bounded, observable, and controllable AI. Trust also cannot be a one-time test. Traditional assurance certifies a product at a point in time, but AI keeps evolving after deployment, and a system that has passed every test can drift outside the conditions that made those tests meaningful. Trust must be continuously earned.

Safety engineering has been accounting for some of this variability for decades. For example, the Nuclear Regulatory Commission determine the likelihood of failures in complex systems, and Systems-Theoretic Accident Model and Processes (STAMP) showed that accidents can emerge from interactions among working components, people, and organizations. But AI raises the stakes, because just a single component can now interpret, infer, and act. The stakes rise further as AI moves from browsers into vehicles, robots, medical systems, industrial controls, and data centers, where a wrong answer can become a life-threatening event.

Building trust that scales

It’s clear that a person rubber-stamping thousands of AI actions an hour is not an acceptable—or scalable—safeguard against risk. But the remedy is not to remove people from the loop; it is to put human judgment where it counts: setting limits in advance and approving the rare actions that have serious consequences while hard constraints handle the rest.

Therefore, trust has to move upstream into architecture. The central questions become: What authority are we willing to give AI, under what conditions, and supported by what evidence? And what happens outside those conditions? That is very different from asking whether a model scored 94% on a benchmark.

The determinism paradox does not mean AI cannot be trusted. It means trust needs a different architecture. Instead of making every output predictable, make the boundaries predictable. An AI medical assistant may be probabilistic in its diagnosis, but deterministic rules govern whether it can change a prescription. An industrial AI may find many ways to optimize a process, while fixed limits keep it within pressure and temperature bounds. An agent may decide how to complete a task, while permissions, transaction limits, and human approvals determine what it is authorized to do.

Let AI be probabilistic inside the box and make the box itself deterministic wherever consequences matter. Leaders can put this into practice by asking five questions:

  1. What authority does AI have? Define not only what it can do but what it is allowed to do, separating interpretation, recommendation, decision, and execution.
  2. Is the function of AI proportional to the consequence? A marketing copilot and AI controlling industrial equipment need different assurance thresholds.
  3. Are we taking the appropriate steps toward autonomy? For example, there needs to be a plan to move from having AI interpret while humans decide, to AI making recommendations, to then taking action with approval, and ultimately to having supervised autonomy, but only as AI can safely work at each step.
  4. Are we testing the system or merely the model? Unexpected interactions emerge among models, data, agents, APIs, and human workflows.
  5. Are we continuously earning trust? Monitor drift, failures, near-misses, and model changes throughout the lifecycle.

The winners in AI will be those who solve the determinism paradox—giving AI enough freedom to create value while engineering boundaries strong enough to make what it does with that freedom trustworthy. Invention earns attention. Trust earns adoption.

Explore how UL Solutions builds trust in AI.

This content was produced by UL Solutions. It was not written by MIT Technology Review’s editorial staff.