ztzoff.tech

Nov 5, 2026

Who owns the agent at month six

The vendor has left, the model is being deprecated, and the prompt that worked in March does not in September. The operating model an agent needs to survive its first year, and what the owner does.

Month six is when I get the call. The agent shipped in the spring. The vendor's engagement ended. The model it was built on has a deprecation date. A prompt that worked in March does not quite work in September, and nobody can say why, because nobody was watching. The business still depends on it, which is the good news and the problem.

This is the second Scale essay in the Discover, Design, Build, Scale series, and it is about the dullest and most decisive question in the method: who owns this thing, and what does owning it mean.

The owner is a person, not a team

Every agent that survives its first year has one person who can answer three questions without looking anything up: what did it cost last month, what is its pass rate this week, and what changed last. That person has the access to change a prompt, the budget line it runs on, and the authority to switch it off. A team can share the on-call rota. Ownership does not share.

In Google Cloud's framework this role is called the agent owner, and it sits on the client side, which is correct. A vendor can build the agent. A vendor cannot own it, because the vendor does not feel the cost or the benefit.

What the owner does in a normal week

Not much, if the Build stage was done properly. That is the point. The weekly job is to read three numbers and decide whether anything needs to change:

  • The eval pass rate, from the harness that runs on every change and, on a schedule, against production samples. A drop with no deploy behind it means the world changed: a model update, a new document, a tool returning a new shape.
  • The holdout verdict, if the holdout is still running. Helps, hurts, or no evidence yet. When it has said "helps" with a tight interval for long enough, the owner can shrink the holdout to a few percent and leave it on as a canary.
  • The cost per run against the projection from the build. Drift here is usually a prompt that grew, a retry loop, or a model switch nobody announced.

Everything else is exceptions, and the runbook covers those.

Three chart tiles for the owner's weekly numbers. Eval pass rate over twelve weeks holds near ninety percent, then drops below the threshold with no deploy behind it, annotated as a model update, a new document or a tool returning a new shape. The holdout verdict shows an effect estimate with an interval clear of zero on the helps side, with the note that once it holds the holdout shrinks to a few percent as a canary. Cost per run drifts above the dashed projection from the build, annotated as a prompt that grew, a retry loop or an unannounced model switch. A footer shows the budget line: run cost at volume plus the owner's time plus quarterly re-evaluation plus maintenance at fifteen to twenty-five percent of the build cost per year.

The re-evaluation cadence

Models change under you. Providers deprecate versions, quietly adjust behaviour between versions, and change prices. The owner needs a cadence, written down, for re-running the full eval against the current model and against two or three alternatives. Quarterly is typical. The output is the same table from the build: which model is the cheapest that passes, and whether that has changed.

This is also where compressing the model that passes earns its keep: a year in, a smaller or distilled model often clears the same eval at a fraction of the cost, and the owner is the only person positioned to notice.

When to change things, and when to leave them alone

The owner's hardest skill is restraint. A prompt change that "obviously" helps is a hypothesis. It goes through the same loop as every change during the build: proposed against the spec, reviewed by someone else, run through the eval, and, if it touches behaviour, measured against the holdout. I shipped a feature switched off on our own library because the rule I fixed before the sweep said no value qualified. A measured no is a deliverable, and the owner should expect to ship a few of them a year.

The reverse also holds. A lesson or a feature that the holdout measures as harmful gets switched off because a number said so, not because someone lost an argument.

The budget line

An agent in Scale is not an experiment. It needs a line in the operating budget with four parts: the run cost at expected volume, the owner's time, the re-evaluation cadence, and maintenance. For maintenance I use the industry rule of thumb of 15 to 25 percent of the build cost per year, and I tell clients the lower end is for systems that shipped with an eval and a runbook, because changes can be made without fear. The upper end is for systems where every change is a small act of courage.

Put the number next to the holdout's measured benefit and you have the only ROI calculation worth having: a measured gain against a known cost. Hours saved times an hourly rate is not it.

The month-six checklist

If you inherited an agent, or are about to, these are the questions I would ask in the first hour:

  1. Who can change a prompt today, and how would they know by tomorrow whether it helped?
  2. When did the eval last run against production samples, and what was the number?
  3. What model is it on, when is that model's deprecation date, and when was an alternative last tested?
  4. What did it cost last month, and what was projected?
  5. Where is the runbook, and when was it last used?

Two unanswered questions is normal. Four means the agent is running on luck. Working through this list is part of what I do in a free AI readiness assessment, and when the engagement is a system audit rather than a new build, it is the first day of the production system audit.

An agent nobody owns does not fail loudly. It fails the way a garden does: slowly, then all at once, and by then the person who planted it is gone.

Book a discovery call