Glass-Box Agents Are Not Optional

Somebody in your company is going to ask why did the AI do that?

Not in a demo. In a meeting, after the fact, with a customer, a regulator, or a loss on the table. And the answer you can give in that meeting is decided long before it — by how you built the thing. There are only two kinds of answer. One is "we can't know; the model decided." The other is "step 14 was the model. Here is what it saw, here is what it said, here is the rule that accepted it, and here is where the process went next."

The industry has spent three years trying to make the first answer sound acceptable. It is not, and it won't become so. This post is about the second answer — what it actually takes to be able to give it, why that is more important than most of what we argue about in AI, and why it is not a feature you bolt on but a paradigm you build in.

We are trying to open the wrong box

"Black box" and "glass box" are old words in machine learning. A decision tree is a glass box: you can read it. A neural network is a black box: you can only watch it. And an entire industry — explainability, interpretability, attribution, mechanistic analysis — exists to pry the black box open.

For a foundation model, that is a losing fight. Not a hard fight. A losing one. The weights are billions of numbers, the behaviour is emergent, and the best available techniques give you plausible narratives about a decision, not the decision. You can ask the model to explain itself, and it will produce something fluent — which is exactly the problem, because fluent is not the same as true, and the explanation is generated by the same opaque process as the answer.

So concede it. The model is a black box. It will remain a black box. Every system with a model in it has an opaque component at its centre, and no amount of prompting or tracing changes that.

Once you concede it, the real question surfaces, and it is a much better question:

How wide is the black box, and does it have an address?

Width: what the opacity swallows

Look at what happens inside a typical agent framework.

                ┌──────────────────────────────────────┐
                │              THE AGENT               │
                │                                      │
                │   while not done:                    │
                │       think                          │
                │       pick a tool                    │
                │       call it                        │
                │       read the result                │
                │       decide whether to stop         │
                │                                      │
                └──────────────────────────────────────┘
                   ↑                                ↑
             tools go in                    a result comes out

The model is opaque — fine, we agreed to that. But notice what else lives inside the loop: which tool gets called, in what order, how many times, and when it stops. All of that is decided by the model, at runtime, probabilistically. The routing is inside the box. The retry policy is inside the box. The stop condition is inside the box. The path is inside the box.

The model's opacity didn't stay in the model. It swallowed the whole process.

That is why "we can't know; the model decided" is the only honest answer such a system can give. It is not that the engineers were careless. It is that the unit of opacity is the entire agent. There is no step 14. There is one box, and the decision happened everywhere inside it at once.

Wrapping observability around that box does not help. You get a trace: a list of tool calls and token counts. You can see that the agent called the fraud service and then the payment service. You cannot see why it went that way rather than another, because there was no "way" drawn anywhere — the way was improvised. A trace of an improvised path is a diary, not an explanation.

Address: a black box that is exactly one node wide

Now look at the same work built as a flow.

  [ receive ] → [ enrich ] → [ ★ assess ] → [ rule: amount < 10k ∧ risk = low ]
                                                    │                │
                                                   yes               no
                                                    ↓                ↓
                                              [ auto-approve ]  [ human review ]
                                                    ↓                ↓
                                                    └──→ [ pay ] ←───┘

The star is the model. It is still a black box. Nothing about the weights got clearer. But look at what changed around it:

• Its inputs are on the context. What the model saw at assess is the context document at that moment — a durable object the runtime persisted, not a prompt that vanished after the call.
• Its outputs go back to the context. What the model said is written to a named field. It is not an in-memory variable inside a loop; it is data, and everything downstream reads it from there.
• The path into it and out of it is drawn. The model does not decide what happens after assess. An edge does. The rule on that edge is evaluated outside the model, deterministically, against the context.
• Every policy it is subject to is a node. The amount threshold, the risk gate, the human review — each is a place on the canvas, not a sentence in a system prompt hoping to be obeyed.
• The black box has a name and a position. It is assess. It is the third node. When someone asks why, you point at it.

This is what "glass box" concretely means. Not that the model became transparent — it didn't — but that the opacity was shrunk to one node and pinned to an address. Everything that is not the model's judgment is visible, deterministic, and inspectable: the reads, the writes, the routing, the gates, the loop, the stop.

And here is what falls out of that automatically. Because the runtime executes nodes and persists context between them, the record of a run is not a trace of tool calls — it is a sequence of (node, what it read, what it wrote, which edge fired). That is the shape of an explanation. It is also, not coincidentally, the shape a regulator, an auditor, or an appeal asks for. You didn't build the explanation. You built the process such that the explanation is what running it produces.

The distinction is not autonomy vs control

It is tempting to read this as the old fight — "agents should be free" versus "agents should be governed" — and to predict, as most people do, that the answer will be a hybrid: the workflow sets boundaries, the agent plans inside them, the workflow validates, and so on.

That hybrid is not a prediction. It is what the picture above already is. The model plans and judges inside assess; the flow decides what its judgment is allowed to cause. The question everybody frames as "which decisions can we delegate to the agent?" has a precise answer in a flow, and it isn't a policy setting. You delegate exactly what you put inside a model node. The delegation boundary is the outline of a node. Widen the node and the agent has more room; narrow it and it has less. Either way, the boundary is drawn, not negotiated with a prompt.

The same goes for the objection that glass boxes only suit rigid, regulated processes — that research, strategy, discovery, anything exploratory needs a free agent. An exploratory agent is a loop with a model in it. Draw the loop:

        ┌────────────────────────────────────────┐
        │                                        │
        ▼                                        │
  [ ★ explore ] → [ ★ critique ] → [ rule: good enough? ] ─ no ─┘
                                            │
                                           yes
                                            ↓
                                      [ deliver ]

Inside explore the model can wander wherever it likes. What is constrained is not the exploration — it is the shape around it: what it can read, what it can write, the check it must pass each pass, the condition that ends it, and who sees the result. A wide-open loop inside glass is still glass. You can watch every pass. You can insert a person on pass three. You can cap it at ten. The agent is as free as its node is wide, and no freer.

So the axis is not autonomy. It is where the path lives. In an agent framework the path lives inside the model, decided at runtime, and the black box is the whole system. In a flow the path lives on the canvas, decided at design time, and the black box is one node wide. Agent builders bound what the model can see. Flow engineering bounds where its reasoning can lead.

Why this is bigger than compliance

The obvious buyer for a glass box is the person who has to answer the why question: the risk officer, the compliance team, the regulator's counterpart. In insurance, banking, healthcare, government, the black-box answer is not merely embarrassing — it is disqualifying. "The model decided" is not a sentence you can say to an appeals board.

But if you stop there, you have filed glass-box agents under governance — a feature for regulated industries, an audit log for the rest. That undersells it badly, because the same property serves people who never talk to a regulator:

The engineer at 2 a.m. An agent misbehaved in production. In a black box you re-run it and hope it misbehaves again so you can watch. In a glass box you open the run, walk to the node, read what it saw and what it wrote, and you know. Debuggability is explainability with a different audience.

The domain expert. The claims adjuster, the security analyst, the underwriter — the person who actually knows what the process should be. In a black box they can't read the agent; its logic is in Python and prompts, so they describe what they want to an engineer and wait. In a glass box the process is the drawing. They can read it, argue with it, and often change it. A canvas a domain expert can read is a domain expert who can own the agent.

The next engineer. Six months later someone inherits this. A black-box agent is a prompt, a tool list, and a loop, and its behaviour is an emergent property of all three that nobody can reconstruct. A glass-box agent is a graph. It reads the same on day 400 as on day one.

Same property. Four beneficiaries. That is not a governance feature. That is what it means for software to be legible — and we used to consider legibility non-negotiable, before the model arrived and we were so impressed by what it could do that we stopped asking to see how.

This is a paradigm, not a patch

Here is the uncomfortable part. You cannot get a glass box by adding things to a black box. Tracing, logging, guardrails, evaluations, a "human in the loop" checkbox — all of them wrap the box. None of them shrink it. The box is as wide as it is because of where the control logic lives, and moving control logic out of the model and onto a drawn, executed, persisted structure is not an add-on. It is a different way of building the software.

Concretely, it changes what you consider the unit of development:

Every row on the right is a habit the left side doesn't have. Putting control on edges instead of in prompts is a habit. Treating the context as the source of truth that every node reads from and writes to is a habit. Drawing the loop instead of writing while is a habit. Reaching for a node when you want a policy is a habit. Those habits, together, are what we have been calling flow engineering — and the point of naming it is that it is a discipline you have to learn, not a library you import.

That is the update. Software 1.0 was written logic. Software 2.0 was learned weights. The thing people are calling 3.0 is not "prompts" — it is the realisation that a probabilistic component at the centre of a system forces you to move everything that must be certain out of it and onto a structure that can be read. The model gets narrower and the structure around it gets richer. That is the whole shift, and glass-box agents are its visible face.

The claim, plainly

You will never make the model a glass box. Stop trying. Make the process one.

An agent whose black box is the entire system cannot answer the question that every real deployment eventually gets asked. An agent whose black box is one node wide answers it by existing. That difference is not about how much the model knows or how well it reasons; it is about whether the path was drawn or improvised, and that is decided by how you build.

Glass-box agents are not a feature for regulated industries. They are the precondition for putting a model anywhere a decision has to survive a question — which, if you are honest about your business, is everywhere. Getting there means updating how we develop software: control on edges, state on a durable context, policy as nodes, the loop as a drawing. It is a new set of habits.

FloMorphic is a platform built so those habits are the only way to work in it. The model is a node. Everything else is glass.