Who Verifies the Physical World?

A few years ago, the hard question in AI was whether a machine could think. Then it was whether software could act on its own. Both are being answered, faster than most of us expected.

The question I keep coming back to is the one almost nobody in those conversations asks: when the software decides, who checks that it actually happened?

Not in a database. On a receiving dock at 6 a.m. On a production line where the kit is one part short. On a store shelf where the tag says $3.99 and the system says $4.49. That is where every plan, every forecast, and every AI decision finally meets the physical world. And it is the one place where most companies still run on trust.

The shift from AI that advises to AI that acts is already on the calendar. Gartner expects 60% of enterprises using supply chain software to have adopted agentic AI features by 2030, up from 5% in 2025, and spend on that software to grow from under $2 billion to $53 billion. McKinsey estimates that by 2030 agents could orchestrate up to $1 trillion of US retail revenue, and $3 trillion to $5 trillion globally.

Think about what that means on an ordinary Tuesday. An agent notices a stockout risk and places a reorder. Another reroutes a shipment around a delay. A third answers a customer asking where their order is. Each one acts on a record: what's on hand, what arrived, what shipped.

Gartner's own advice to supply chain leaders is telling. Start, it says, where "high-quality data is accessible" and "behavior is verifiable." In other words, agents are only as good as the facts underneath them. And in physical operations, the most important facts are created by hand, on a floor, by someone with a scanner, a clipboard, or a good memory.

We have known for a long time how far the record drifts from reality. When researchers at Harvard and the University of Chicago examined nearly 370,000 inventory records across 37 stores of one retailer, they found 65% of them inaccurate. An earlier study by the same group called execution "the missing link in retail operations" and estimated that execution problems alone cut retail profits by more than 10%. It hasn't gone away: IHL Group puts the cost of out-of-stocks and overstocks at $1.77 trillion in 2025.

Those studies are from retail, but every operator I talk to recognizes the pattern. The order says 48 cartons and 46 arrive, one of them crushed. The load plan says trailer 4417 and a pallet ends up on 4418. The kit list says 12 parts and the line gets 11, one of them the wrong revision.

None of these is a software failure. The ERP did its job. The WMS did its job. The gap opened in the few seconds when a person, or a machine, did the physical work and nobody checked it against what was supposed to happen.

The cost doesn't show up there. It shows up weeks later. A supply chain leader at a consumer brand told me recently that they budget between 0.5% and 1% of gross revenue for retailer chargebacks. At one distribution warehouse we know well, claims came back on 1 to 3% of outbound loads, worth millions a year, and when the claims landed there was nothing to show what had actually shipped.

It isn't for lack of trying. Operations have had scanners for decades. But a scanner only confirms that a barcode was read, not that the work was right. It can't tell that a pallet is short of what the order asked for, that a part is the wrong version, or that a carton was crushed. Often the scan itself is the problem: a pallet carries several labels, much of what matters isn't in a barcode, and the shipping notice never reaches the dock. Behind all of it is a problem nobody has solved: the physical world has no standard. Every supplier labels and documents its goods its own way, with one barcode, several, or none, and a different packing slip from each.

Everything else has been left to people: paper checklists, tribal know-how, and a supervisor's spot audit. That worked when teams were stable and volumes were predictable. It breaks with high turnover, routing guides that run to hundreds of pages, and peak seasons that double the work overnight.

So the industry made a reasonable trade. It invested in systems that plan and record, and accepted that what happens on the floor would be checked later, by exception, when something went wrong. For a long time, "probably done" was good enough.

It isn't anymore. When a person read the record and made the decision, a bad record got a second look. When an agent reads it and acts in a second, it doesn't.

The answer isn't another dashboard. It's a layer that sits between the systems that decide and the people and machines that do the work, and it does three things at the moment of work: it guides the step, verifies it against what was supposed to happen, and keeps the evidence.

We call this verified execution: every physical step guided, checked against what was supposed to happen, and recorded with evidence at the moment it happens.

In practice that shows up in three ways, and most operations will run all three side by side.

Frontline teams, guided and checked. The task appears on the device the team already carries, step by step. One capture reads the label, the packing slip, and the item together, checks them against the order and your systems of record, and flags the exception on the spot, with a photo and a timestamp. A new hire on day two does the job the way your best person does it.

Hands-free, with fixed vision. On a conveyor, at a dock door, or in a van, fixed vision does the same checks with no one holding a device. Every carton and every pallet is verified as it moves, not sampled at the end of the shift.

People and machines on one floor. Robots and humanoids will start joining frontline teams in the next two to four years. That adds a new question: when a robot moves the pallet, who checks it went to the right lane? The work needs to be orchestrated, which means deciding whether a person or a machine takes each step, and verified the same way, whoever does it.

What ties the three together is simple. Every touch creates a fact, checked at the moment it happens, instead of an assumption reconciled weeks later.

Vision AI over a warehouse floor makes people nervous, and they're right to ask hard questions. Some companies have strapped phones to warehouse associates to collect footage and sell the data. That isn't verification. It's surveillance, and it erodes the trust that frontline work depends on.

The principle we hold ourselves to is narrow on purpose: verify the work, not the worker. Check that the right pallet went to the right lane, that the seal number matches the bill of lading, that the label on the case matches the PO. The evidence is about the goods and the steps, and it protects the team as much as the company. When a customer disputes a delivery, the evidence shows the job was done right.

If you're bringing Vision AI into your operation, write that principle down before you write the RFP.

Here is the part I find most interesting. Once every touch is verified, the floor stops being the place where data goes wrong and becomes the place where good data comes from.

Planning gets inventory it can trust. Finance gets receipts that match what arrived. Customer service gets evidence instead of a he-said, she-said. And agents get what Gartner says they need most: data that is accessible and behavior that is verifiable.

That is why I think of this as ground truth at every touch. It isn't a feature of one system. It's a property of the whole operation, and it becomes more valuable with every layer of AI built on top of it. The companies that get there first won't just make fewer mistakes. Their agents will be allowed to do more, because the facts underneath them hold up.

This is the problem we've spent nine years on at PackageX, across 1,500+ global facilities and 200+ enterprises. When one specialty retailer we work with started verifying store inventory adjustments at the touch, the job went from about five days and three people to about 30 seconds. But it's bigger than any one company. Every operation adopting AI will have to answer it.

If you lead operations, supply chain, or store operations, try these with your team:

1. 1.Pick one shipment from last week. Can you show what physically happened at each touch? Not what the system recorded, but what happened, with evidence.
2. 2.Where does your most expensive mistake get caught today? At the moment of work, or weeks later in finance, claims, or a customer complaint?
3. 3.If an agent started acting on your inventory and receiving data tomorrow, would you trust it? If the answer is "not yet," the problem isn't the agent.

AI can think. Agents can act. The companies that win the next decade will be the ones that can also answer the third question: who verifies the physical world? My bet is that the answer has to be built into the work itself, at every touch.

I'd like to hear where you see this breaking in your own operation.

What is the execution layer in physical operations?

The layer between the systems that decide and the people and machines that do the work. It guides each step, verifies it against what should happen, and keeps the evidence.

Why do AI agents need verified physical data?

Agents act on inventory, receiving, and shipping records in seconds. If those records weren't checked when the work happened, agents act on mistakes faster.

Is Vision AI on the floor surveillance?

It shouldn't be. Verification checks the goods and the steps, such as the pallet, the seal, and the label, not the people doing the work.