What a company must own when machines can do the rest
“The question is not what the machine can attain. It is what we can keep.” Common Wealth, 2026
The short version. A company that can build itself can produce every answer it needs except one. It cannot want anything. And if nobody states what the company wants, the system infers it from the only evidence available, which is what the company has already done. Your history is a record of what you did, not of what you meant, so the decisions you regret get industrialised alongside the ones you are proud of, and nothing in the machinery can tell them apart. Four things decide whether you still hold the wanting: who can be held to the work, who owns the definition of good, who controls the gate that enforces it, and whether anyone can still say no. The first three are recoverable. The fourth, once rented, cannot be bought back.
Nobody is going to acquire your company. No term sheet, no diligence, no premium, no headline. And yet the acquisition may already be underway, one reasonable purchase at a time.
Somewhere in your vendor stack, the agents that do the work are named by someone else’s directory, judged by someone else’s checkers, governed by someone else’s gates, and pointed at goals chosen from someone else’s menu. Each rental improves the quarter. None of it appears in any register of who owns what. Every one of those steps is correct. The sum of the steps is the loss of the company.
But one renewal cycle from now, you could be operating inside somebody else’s structure and paying rent for the privilege, and that is a worse outcome than being bought. At least an acquisition has a price.
It does not announce itself in a strategy review. It shows up as a night like this one.
Two in the morning, some Tuesday next year. A payment run that executed at midnight has sent money to a supplier that was supposed to be on hold, and the person on call is walking the trail backwards.
Every system on the path did its job. The agent had its own identity, correctly issued and correctly scoped. The gateway logged the call. The data platform logged the supplier query. The payment system logged the transfer. Four complete records, each one accurate, and no way to join them into a single account of what happened.
So nothing failed. That is the point. There is no misconfiguration to find, no missing log to add, no team that was careless. Four systems recorded four true things and no system was responsible for recording that they were the same act.
By four in the morning she has the sequence, and it cannot be undone, because the money has settled. What she does not have, and what her director will ask for at nine, is a name. Not the name of the thing that ran. She has that. The name of whoever can be held to it.
That is not an operations problem that grew. It is a company problem that was always there and was hidden by an assumption that has quietly stopped being true.
Boards are asking a version of this every quarter, usually filed as an AI strategy review, and mostly getting an answer about tools. This is my answer, from inside a company.
What a company actually is, once it can build itself
A company is a set of answers.
What we do. Who we do it for. How we do it. What counts as good work. Who decides. And underneath all of them, the one nobody writes down: what we are for.
Those answers have always lived in people. The company was the container that held them together long enough to be useful, and most of the apparatus of a firm exists to keep them consistent across more people than can fit in a room.
A company that can build itself can produce almost all of them. It can propose what to build. It can build it. It can check its own work, and get better at all three. Given a target it can work out the method, and increasingly it can work out who should do what. That is not a forecast. Most of it is already purchasable.
There is one answer it cannot produce. It cannot want anything.
This is the distinction that matters and it is not the one people expect, so let me put it precisely. You can give a system a goal. You cannot give it a want.
The industry word for the first one is intent, and every serious thing being built this year runs on it. Declare the intent, let the system work out the method. That is a real advance and I am not arguing against it. But intent is what you asked for. A want is what you would still choose when what you asked for turns out to have been the wrong thing to ask for, and no amount of better intent handling produces the second, because it was never the same input. A goal is a target and a system will pursue it beautifully. A system optimising towards a goal has no way to tell you the goal is wrong, because that judgement is exactly the input it was not given.
And the obvious objection is right, up to a point. You can write a want down, and you should. But what you write down is the answer you reached the last time you argued it, and the wanting is the standing ability to reach a different one. That is why it needs a named person and a date rather than a document, and why it is the only one of the four rooms that dies of being settled.
Years of watching companies set targets, I have not once seen anybody write down what they would do if the target turned out to be the wrong target. The target gets a number, an owner and a dashboard inside a fortnight. The other half gets discussed, sometimes at length, and then goes nowhere a machine can read.
So a self-building company is not a company that runs itself. It is a company in which everything except the wanting has become producible.
Which changes the question. The strategic question of the decade is not how much of your company a machine can run. It is which parts you refuse to rent.
What happens when nobody says it
Here is what should worry a board, and it has nothing to do with the technology going wrong.
If nobody states what the company wants, the system does not stop. It infers. And the only evidence available to infer from is what the company has already done.
Your history is a record of what you did, not of what you meant.
Take a company that discounts to close quarters. Nobody ever decided to be a discount business. It happened one deal at a time, each concession defensible on the day, most of them regretted by the following Tuesday. Five years of that sits in the record as five years of closed deals with price concessions attached. A system asked to close more deals will read that record, find the pattern that closes them, and industrialise it. Faster, more consistently, at higher volume, improving every quarter.
Nobody chose that. Nobody can point to the meeting where it was chosen. And the record is not lying. The company did discount. What the record does not contain, and cannot contain, is that every one of those discounts was a loss the company took under pressure and would not have taken with time.
That is the mechanism. A company that builds itself becomes more of whatever it has already been. The decisions you regret get industrialised alongside the ones you are proud of, and nothing in the machinery can tell them apart, because nothing in your history is labelled.
And this is not a malfunction. It is the system working exactly as designed, on the only input it was given.
And you do not have to take my word for any of it, because the test takes an afternoon and you already hold the data.
Pick one class of decision your company has made repeatedly for five years. How you price when a quarter is short. Who gets promoted. Which projects get killed and which get one more quarter. Pull the record, and read only what was decided, because the reasons are not in the record and never were. Then ask what pattern a system optimising that outcome would extract from it.
If the answer embarrasses you, that is not a data quality problem. That is your stated intent, as far as any machine can tell. And it is what your history will teach the next system you point at it, whether or not anybody intended to teach it that.
Why this is not a mission statement problem
The reflex answer is that we have a purpose. Most large companies have several, on walls and in decks, and none of them are the thing I am describing.
A purpose statement is a want expressed at an altitude where it cannot adjudicate anything. It will not tell you which to take when speed and correctness conflict on a Thursday, and that is the level at which a company actually gets built. The wanting that matters is a stated position on a trade, owned by somebody, that can be re-stated when it changes. When margin and trust conflict, which one gives. When a customer’s convenience and a customer’s privacy conflict, which one gives. When we are late, what are we allowed to drop.
Every organisation has answers to those. I have not yet seen one that had written them down, and I have asked, because until now the answers lived in the judgement of people who were in the room, and the room was where the trade got made. Nobody was being careless. There was no reason to write down a thing that walked into every meeting on its own legs.
The room is being automated. The answers are not being written down. Everything that follows is about the gap.
The four rooms, and why they are these four
A castle was never one wall. It was layers, and the layers were honest about what they were: the outer walls were replaceable, the keep was not, and the design flowed backwards from one question. When the outer layers go, what must still be ours?
Four things sit in the keep, and each is a way of losing the ability to say what the company wants.
Lose the hands and you cannot say who acted for you. Lose the standard and you cannot say what you meant by better. Lose the gates and your standard is advice while somebody else’s is binding. Lose the ability to say no and you are ratifying rather than choosing.
That is doing, judging, enforcing and wanting, and they run in that order because each one is a step further from the work and closer to the point of it.
They are not equally serious, which is the part most versions of this argument miss. Identities can be re-issued. A standard can be re-authored, painfully and politically, but it can be done. Gates can be re-procured at the next renewal, at a price you will not enjoy. The wanting cannot be recovered, because by the time you notice it is gone the only record of what you wanted has been overwritten by five years of what the system did in its absence.
Where this does not apply
Everything here assumes somebody is coming for you, and most supplier relationships are not that. If your vendors stay good, your contracts stay renewable and the market stays competitive, then holding the keep is an insurance premium you paid every year and never claimed. That is a real cost and I cannot price the risk for you. The narrower version I will defend is that it is not only insurance, because room two says verification is what limits how much you can safely hand over, so the standard you own is also the speed at which you can let go.
It also assumes you have something worth keeping in there. The argument is strongest for firms whose product is judgement and weakest for firms whose product is reach. If your actual advantage is distribution, or capital, or a brand people already trust, the definition of good may not be your scarce asset and I am handing you somebody else’s problem in a well-made box.
And there is a serious position on the other side. I will put it as strongly as I can, though I do not believe it: that speed of adoption beats architectural caution, that the firms who move fastest and worry least will out-learn the careful ones, and that the keep will turn out to have been somewhere to hide. People who think that are not being reckless. They are betting that the ground moves faster than any structure you can build on it, and on a five year view that bet has often been right.
Room one: the hands, or who can be held to it
A company is a legal person. Not because anyone believes a company has a mind, and not because it earned it. A company is a person in law because a company can be obliged. It can be sued, fined, taxed and bound by contract. There is somewhere to send the bill and somebody who has to answer.
That tradition has never been about consciousness or capability. It has always been about who can be held to something, which is why the same status attaches to ships, and in some jurisdictions to rivers, and why the book gets it right in a single line: “every legal person ever created is a glove with a human hand inside it.”
The careful version of this argument, and it is the only one worth holding, does not claim that machines fall short. It allows that a machine might be conscious, says that if it were it would be owed kindness, and then separates the two questions: being owed kindness is a different thing from membership. That separation is what this room rests on. Capability and moral consideration are one question. Obligation is another, and only the second is what a company runs on.
Those are different questions, and every organisation currently conflates them. Identity answers which thing acted. Obligation answers who can be asked, blamed, replaced or sued. Every access control ever built assumed they were the same question, because for the whole history of computing the thing holding the credential was a person or stood for one. Agents break that assumption and leave the mechanism intact. The credential still works perfectly. The person behind it has become optional.
Keep a name behind every act. That is the whole rule, and here is why it is under pressure in a way an identity project will not fix.
An estate that issues an identity per agent per task is generating obligations faster than any human process can attach names to them. That is a design consequence rather than a measurement. Each identity is correct. Each one is scoped. And the human review that used to sit behind a credential, the manager who signed the access request, does not scale to a population that regenerates hourly. Joining the initiating user, the agent and the downstream service account into one auditable chain is beyond most enterprise security stacks today. Test yours before you disagree.
So organisations are shipping three answers, and only one of them holds.
The first is the shared account. Its trail is complete and it ends at nobody. This is the sloppy case and it is the one everybody writes about, which is unfortunate, because it is the easiest to fix and the least interesting.
The second is delegation, and it is the popular answer this year. The agent acts under the standing of the person who asked, reaching exactly what that person can reach. It closes the two in the morning problem completely, because there is always a name at the end. It also makes your most privileged engineer the ceiling on everything they delegate, and it issues copies of a named human’s authority free, as often as anyone asks.
And it fails a second way that is worse, because you cannot see it. The trail ends at a name, and the name belongs to somebody who did not act. Nothing in the record separates the case where they did it from the case where they allowed it. An account that ends at nobody is visibly broken. An account that ends at the wrong person looks correct, and no auditor catches that by inspection.
The third is the one to buy. The agent gets its own identity, scoped to the task and expiring with it, and one named human stands behind it the way a ship has a captain regardless of how much of the sailing is automated. The name attaches when the agent is commissioned, not when it acts, which is the only reason this survives the arithmetic above: identities can regenerate hourly, commissions do not. The captain is not claiming to have steered. The captain is not approving each manoeuvre either, and room four is about what approving is worth. The captain is the answer to who can be held to it.
The philosophical version of this is the one that decides it. Standing that can be copied at zero cost is not standing. The same book puts the rest of it better than I can: “One person, one vote is a rule for things of which there is only one, and the moment the voter can copy itself the franchise is not extended. It is abolished.” An agent can be more capable than any of your employees and it still must not act on an employee’s credentials, not because it is lesser, but because there is no longer one of it.
Room two: the standard, or who owns the definition of good
Firms exist because it costs less to transact inside them than in the open market, and those costs come down to searching, bargaining and enforcing. Enforcement is the cost of establishing that the other party did what they said they did.
Agents collapse searching, bargaining and coordinating almost to nothing, and they raise enforcement, because the work is now done by something that cannot be held to account, that keeps the only record of what it did, and whose output has to be checked before anyone can rely on it.
You will hear it said this year that agents have made coordination free and therefore dissolved the firm. That has the sign wrong. They moved the cost from coordinating the work to verifying it, and the firm re-forms around wherever verifying is cheapest.
Which makes the definition of good the scarce asset. The acceptance criteria, the taste, the worked examples of excellent that your best people carry around and have never written down. That was fine while the work sat with the people who held it. It is fatal the moment the work moves to machines, which know only what you can state.
And here is the part that gets skipped, because it is the part that hurts. Writing down the definition of good is not a documentation exercise. In the strongest published measurement of it, the GDPval benchmark of expert-graded work across forty four occupations, experts asked to rank two or more unlabelled deliverables against each other agreed seventy one percent of the time. Your best people disagree with each other three times in ten, and that is on the easy question of which of these is better rather than the hard one of whether this is right.
On brand is the clearest case of it. A standard nobody has ever written down completely, adjudicated daily by taste, and now being handed to classifiers the brand does not own.
Somebody has to decide whose taste wins. That is an adjudication problem, it is political in every organisation that has ever had a creative department, and it is the reason this work does not get done rather than the reason it is slow.
And a written standard is an asset that depreciates, which most succession plans have not noticed. Taste is trained on junior work reviewed by seniors. If the junior work goes to machines, the reviewing still has to happen somewhere, or the standard you wrote becomes a museum piece: an accurate description of a company that no longer exists. The keep needs a school.
The evidence that it is genuinely scarce is that the people best placed to solve it cannot. One of the largest enterprise software vendors published exactly the right diagnosis this year: tokens measure how much an AI talks, not the work it completes. Correct. Their replacement unit counts prompts processed, reasoning chains completed and tools invoked. Every one of those events is emitted by their own runtime. Nobody was being lazy. The seller’s own output is the only thing that costs nothing to observe, which is precisely why the scarce thing stays scarce.
What would change my mind
All of this rests on one claim: that the definition of good cannot be bought. If somebody ships a general completion oracle, a thing that reliably tells you whether work came back right, across work classes, without you having authored the standard first, then the scarce asset becomes a subscription and most of this essay is wrong.
The weak version of my defence is that nobody has managed it yet, which is an observation rather than an argument. The stronger version is a property such a thing would have to have. It would need to tell you whether your work is right without ever having been told what right means in your business, which is not something an outsider can observe. And anything that learned your standard by watching your own output would be learning what you did rather than what you meant, which is the problem rather than an escape from it.
So the falsifier is specific. Not somebody claiming a general oracle, but somebody demonstrating one that did not learn what good means from the work it is judging. If you see that before I do, stop reading me on this.
Keep the definition of good, and expect the argument about it.
Room three: the gates, or who enforces the standard
Owning a standard is not the same as owning its enforcement, and the gap between those two is a room of its own.
A standard nobody enforces is advice. A standard enforced by machinery you do not control is somebody else’s standard wearing your name. You can hold room two completely and still lose here, and most organisations will.
Enterprises are building machine law right now and calling it policy as code: gates that block a change rather than advising against it. The gates are good and I would not run an agentic estate without them. But a gate is configured force, and the question is not whether it works, it is who can change what it enforces.
The argument I keep returning to here has a name, and the name is the book’s: improvement is capture. Its own sentence is the one worth holding, because the enterprise version of it is already running in production: “machine law will be better on every measure a legal order knows how to keep, more consistent, more accurate, faster, cheaper, more enforced, and that each improvement will, in the order’s own structure, concentrate the force and freeze the reference that the order’s freedom lived in.”
Two things happen and the second is the one nobody watches. Force concentrates, which everybody can see. And the reference freezes. The definition of good stops being adjusted by the small human judgements that used to keep it current, because there is now something that settles it faster than anyone can object.
So a gate does not only enforce your definition of good. It ends the argument about it, and it ends it at a date. Room three quietly freezes room two at whatever your understanding was on the day the rule was written, which is why an organisation can hold a standard it authored and still be operating on a version of its own judgement that is two years old.
If your platform vendor owns the gates, their roadmap is your law, and every improvement they ship deepens the dependence precisely because it works. You do not lose this room in a negotiation. You lose it by upgrading.
And there is a quieter version of this with no captor at all. If your models, your gates and your checkers are the same ones your competitor rents, your estate converges on the industry mean by construction, and the convergence gets reported as transformation. A rented standard is not only somebody else’s. It is everybody’s, which is a strange thing to have paid for.
Here is a small one, which happened this quarter with no meeting at all. One major model provider now instructs developers to remove temperature, top-p and top-k from their configurations. Those settings were how you made two runs deliberately different in order to get a real second opinion, and how you made one run as close to reproducible as the platform allowed, in order to show an auditor the same answer twice. Both were free. Both are now the vendor’s to set. No announcement and no negotiation. An improvement.
And the stranger thing happening to gates this year, which is not capture
Look at where accountability in an agentic estate actually lives today. The best published development playbook I know names its human sign-off points and enforces separation of duties through branch protection, so that the agent which wrote the code has no way to approve it, and it states outright that the pull request is the audit record. That is a real answer and it works.
It also means the one gate carrying a human signature is a code review. Outside a code repository there is no equivalent. There is no pull request behind a campaign change, a pricing rule, a customer record or a data model.
And in the same month, practitioners started arguing that code review itself is what ends. The case is honest and I think it is right: nobody can closely review hundreds of machine-written changes a week, so humans will review the working result rather than the code.
So the one place where the gate already had a name on it is being dismantled, for good reasons, by people who are correct about the arithmetic. Nothing is being built to carry the signature across, because the new review surface is a person looking at a running system and forming an opinion, and an opinion leaves no field.
The usual warning about this room is that somebody else may come to own your gates. The sharper version is that the one gate that currently works is quietly being retired, and nobody has been asked where the name goes next.
Keep the gates portable, keep the policies yours, and treat any gate you cannot take with you as a lease rather than land.
Room four: the wanting, or whether anyone can still say no
The endgame is not a tyrant. It is a system so good at deciding that nobody else bothers, and then nobody else can. Companies survive bad decisions constantly, and recovering from them is most of what management actually is. What a company does not survive is losing the ability to make a different one, because every recovery it has ever managed depended on somebody inside still knowing how.
The enterprise version is the estate that runs while nobody reads. Its failure mode is not malice and not error. It is atrophy: every decision the system absorbs is a muscle the organisation stops exercising, until intervening is no longer a skill you have.
And atrophy does not arrive as a decision. It arrives as workload. A senior architect at a federally funded research institution put the mechanism plainly this month: “we’ve become the reviewers and not the authors.” The volume, in her word, is “crushing”. Nobody voted to stop authoring, and nobody will vote to stop reviewing either, because reviewing is what the job now is.
Somebody has tried to measure what reviewing at that volume is worth. A controlled study slipped a clearly dangerous command into a routine permission prompt, partway through the session, for a large population of paid developers who did not know what was being tested. Most of them let it through.
I am not going to give you the figure, and the reason is worth more than the number would be. No methodology has been published. The vendor that ran it sells the approval interface, and made the automatic setting the default shortly afterwards. A number you cannot locate is not evidence, and the room about whether anyone can still say no is a poor place to lower that bar.
The gap is the finding. The only party positioned to measure whether your approvals are real is the party selling you the thing being approved, which is room two’s problem showing up in a second room. Nobody is going to hand you this number. You will have to produce it, and the rest of this room is how.
So when you ask who is still exercising judgement in your estate, the honest answer is not the person whose name is on the approval. It is whoever set the threshold at which they are asked.
And none of this will be resisted as a mistake. It will be welcomed as relief. A ratified decision is a decision with cover, and “the system recommended it” is the safest sentence available in a large company. Any attempt to hand real decision rights back will feel, to the person receiving them, like being handed personal risk that was previously somebody else’s. Expect the objection to arrive as a workload argument, because that is the respectable form of it.
The model being sold to boards makes this worse while sounding like the fix
The fashionable description of an AI-native company has agents sensing, interpreting, deciding, executing and learning, with each decision going to a human for a yes or no, and the executives positioned above the loop rather than inside it.
It sounds like retained control. Look at what the human is actually holding. The option set was generated, framed and ranked by the system. The only two available actions are accept and reject. And the whole thing runs at machine speed, which means the approval arrives faster than anyone could examine the evidence behind it.
A person in that position is not deciding. They are ratifying a decision made upstream by whatever produced the options. An approval you cannot investigate is a signature, not a decision. And it leaves a record that is complete, well formed and entirely truthful about a human having approved, which is the most dangerous kind of record there is.
Above the loop is not a position of authority. It is a position of ratification.
And that distinction is about to stop being rhetorical. CEN-CENELEC’s joint technical committee on AI is drafting the human oversight requirement for high-risk systems, and the draft that reached public ballot this summer is reported to turn on one quantity: the maximum time a designated person has to respond to a system output before harm occurs. That number decides which oversight measures apply and whether real-time human oversight is feasible at all. The draft has not been published and I have not read it, so treat the mechanism rather than the detail.
But the mechanism is the thing, and you do not need the standard to use it. Oversight is a race between how fast the system acts and how long a person needs to understand what it did. Where the second is larger than the first, the control is decorative. Call it the oversight ratio, time available divided by time required. Your approval logs give you the first number today. The second is a judgement you have to set and defend, per class of decision, and if nobody in your organisation can say how long a competent review of a pricing exception takes, you have found something before you have computed anything. Nobody is waiting for Brussels to tell them how long their people get.
That is usually said as a warning. It is now a number. Accenture surveyed three thousand C-suite leaders this year, in its quarterly Pulse of Change. Eighty two percent are increasing AI investment. The share reporting widespread, sustained business value is twenty three percent, down from thirty two percent earlier in the same year. Investment up, value realisation down, inside six months, and reported by one of the largest firms in the sector, which has every commercial reason to report the opposite. That does not look like a market waiting for better technology. It is the shape atrophy would take in the accounts, before anyone had a word for it.
And why this room is the one you cannot rebuild
The other three rooms fail in ways that produce visible damage. A missing name shows up in an incident review. A wrong standard produces work somebody rejects. A captured gate blocks something you wanted shipped, and somebody complains.
In a company that builds itself, a wrong standard is not a defect. It is a seed. It does not produce one bad output. It produces a generation, then the next generation is trained on the first, and by the time anybody notices, the thing being corrected is not a decision but a lineage.
That is why the ability to say no is the last line rather than the first. Not because refusal is virtuous, but because a system you can still demote is a system whose history remains editable. A system you cannot demote has made your history permanent, including the parts of it you never meant.
So the rule the estates that age well are running, and it is worth naming because a named rule survives a reorganisation: revocable autonomy. Earned in increments against evidence, and every increment stays revocable. A system you can no longer demote is not governed, whatever the dashboard says. Keep the reverse gear.
Why there is nothing underneath this
You can change platform. You can change model provider. You can replace your entire tooling estate and it is a bad quarter rather than an ending, because each of those is a retreat to a layer beneath.
An architecture is the last one when there is no layer left to retreat to. When the hands, the standard, the gates and the ability to say no are all somebody else’s, there is nothing underneath to fall back on. Not final. Remaining.
And there is a second reading. This is not the last architecture because nothing comes after it. It is the last one a human writes. Everything above it will increasingly be built, tested and repaired by the system, and most of that is genuinely better. These four rooms are what remains when that is true.
What to do on Monday
Four rooms, four actions. Three of them cost a decision. The fourth costs an argument.
The name behind every agent is an identity decision your security team can make this quarter, and the test is not whether the agent has an identity. It is whether you can produce, for any action in the last ninety days, the human who can be held to it. If that requires joining four systems by hand, you do not have it.
Portable gates is a clause in your next contract. Can we change what the gate enforces, and can we take it with us. If the answer to either is no, you are leasing your own policy.
The reverse gear is a design rule, written once. Every increment of autonomy is granted against evidence and stays revocable, and somebody owns the demotion decision by name.
The definition of good is the argument, and it costs the thing organisations find hardest: getting two senior people to write down separately what finished looks like for one class of work, and then sitting in a room until the two answers are one answer. And decide where your juniors still touch real work, because that is where the next version of the standard comes from.
So start with one class of work. Not a strategy, not a platform, not a centre of excellence. One class, one written standard, one named owner, and one number: what fraction of that work passes its own check the first time. Almost nobody can produce that number today, which is itself the finding, and the organisations that can are the ones who will still be making their own decisions in five years.
And one question for the executive team, which is not a technology question and should not be delegated to anyone who thinks it is. When margin and trust conflict, which one gives, and who says so. Write the answer down, date it, name the person who can change it, and put the argument back on the calendar. A standard can afford to be frozen between revisions. A want dies of it, and room three’s warning applies to your own answer as much as to a vendor’s gate. It is the only part of your company the machine cannot produce for you, and the moment it is written down it stops being something the system has to guess from your worst quarter.
There is a good test in Common Wealth for the rest of it, and I recommend it over anything I could invent. Is the reference reclaimable, meaning can you still reach the writing and change what the gate enforces. Is the persuasion plural, meaning is more than one thing deciding whether work is right, since two copies of one model in a review chain fail together and a second pass by the same family is a procedure rather than a control. Are the operations apart, meaning does the same supplier write the standard, run the agent and judge the output. And the sharpest form of that last one, because it is the version most estates fail without noticing: a checking agent forked from the agent that did the work, inheriting its whole conversation, has nothing left to disagree from.
And one thing I owe the book. It is arguing against the concentration of power and it puts companies squarely inside the problem. I have taken the mechanism and pointed it outward, on behalf of an enterprise, at its suppliers. The mechanism transfers honestly. The politics does not, and running the test in one direction only is the exact move it warns about. If those three questions are worth asking of your vendors, they are worth asking of your own estate, and the answers there are usually less comfortable.
And one meeting to go back into. Every enterprise has sat through the renewal where the vendor reaches the good part of the demo, the new release can auto-approve low-risk changes, the toggle is on by default, and everybody nods because the feature works. It will approve well. Approving well is the problem, because every approval it absorbs is a decision your organisation stops knowing how to make. Nobody in that room needed to block the feature. Somebody needed to ask who judges the output, whether it is the same party that produced it, and whether we can still turn this off next year.
Enterprise AI. The architecture, the economics, governance and the proof 
Leave a comment