ztzoff.tech

Nov 5, 2026

What an agent build actually looks like, week by week

Eight weeks from a written eval to an agent a real person depends on: the spec, the first real user in week two, the holdout, independent review, and the handover. What the client receives each week.

When a prospect asks what a build looks like, they usually expect a Gantt chart. What they need is to know what they will be holding at the end of each week, and what they can stop the project over. This is the Build stage of the Discover, Design, Build, Scale method as I actually run it, on a small build of six to eight weeks. A medium build stretches the same shape over ten to fourteen.

A week-by-week chart of a small agent build. Before week one the eval exists. Week one produces the spec and the skeleton. By week two a real operator is inside. Weeks three to five are the loop, where every change is proposed, built in isolation, reviewed by someone else and gated by the eval, and the holdout runs from week three to the end. Week six is the model decision. Weeks seven and eight are the handover of the eval harness, the runbook, the holdout result and a named owner. A green line across every week shows the eval running on every change.

Before week one: the eval exists

A build does not start until Design has produced an evaluation: real cases from the client's own workflow, the expected outcome for each, a threshold, and the check that decides pass or fail without asking the model. If that does not exist, the first week is spent writing it, and I bill it as discovery, because that is what it is. Starting a build without an eval is how you end up with a demo that works for the people who built it.

Week one: the spec and the boundaries

Every build starts with a short written spec: the intent, what the agent may read, write and change on its own, what needs a person, what it must never touch, and how each of those will be verified. This is the permission map turned into something a security team can approve and an engineer can test.

In the same week the skeleton goes up: the agent framework, the tracing, the eval harness wired into the pipeline so it runs on every change. No clever prompting yet. The client receives the spec and a first eval run against a baseline, which is usually a person doing the job by hand.

Week two: a real user is inside

By the end of week two, one person who will live with this system uses it on real work, with real data, under supervision. Not the sponsor. An operator. I have written about why a real user belongs inside by day twelve: they force architecture decisions into the open while changing them is still cheap. They also find the question the spec forgot to ask, every time.

The client receives the first list of things the spec got wrong. That list is a good sign. A build with nothing on it has not met reality yet.

Weeks three to five: the loop

This is where the agent gets good, and the discipline is that every change goes through the same loop. A change is proposed against the spec. It is built in an isolated workspace. Someone who did not build it reviews the diff against the spec. The eval runs. If it drops, the change does not merge.

The reviewer part matters more than it sounds. On our open-source library, AgentExperience.NET, the latest round of work had ten pieces, each reviewed independently, and each review found something real: a measurement that treated its two groups differently, a lock that reached across tenants, erased data that survived in a search index. None of those would have shown up in a demo. A client's build gets the same review, and the client sees the specs.

From week three the holdout is on. A random share of runs gets the old path, or the plain agent without the feature being tested. Both groups are judged by the client's own check. By the end of the build this gives a benefit with an interval, not an anecdote. I explain the design in a holdout for AI pilots.

Week six: the model decision

With a working eval, choosing the model becomes arithmetic. Run the eval across candidates, take the cheapest one that passes, and record the result. In my experience the team's favourite frontier model loses this contest more often than anyone expects, and the run cost drops by a multiple that compounds every month.

The client receives the model comparison table and the projected cost per run at production volume.

Weeks seven and eight: the handover

The last two weeks are not polish. They are the things that turn a project into an operation:

  • The eval harness as a product. Versioned, owned, able to stop a release. It is the client's, not mine. Shipping eval frameworks describes the bar.
  • The runbook. What to do when retrieval returns nothing, when the model provider is down, when quality drifts with no deploy behind it. The 3 a.m. runbook is the template.
  • The holdout result. Helps, hurts, or no evidence yet, with the interval, judged by the client's own check.
  • A named owner. A person inside the client, with the access and the budget line, who can change a prompt on Tuesday and know by Wednesday whether it helped.

The client receives all four. Then the build is done, in the sense I define in when is an agent done.

What this costs and what it does not

A small build like the one above runs $40k to $80k over six to eight weeks; a medium one $80k to $160k over ten to fourteen. The numbers are public because opaque pricing wastes everyone's time. What the shape above does not cost is a surprise in month three, because the eval, the runbook and the holdout exist on day one of operations instead of being written after the first incident.

If your current build has none of the four handover items, or has no eval it runs on every change, it is worth finding out why before the budget runs out. That is a conversation I am happy to have in a free AI readiness assessment.

Book a discovery call