[July 2026 edition]
The Skill of Not Building in the AI Agent Era (4) — Implementations That Don't Repay the Effort, and the Case for Waiting
Published: Jul 25, 2026
Reading time: ~6 min
Part 1 looked at third parties swallowed by the platform, Part 2 at why implementations built on AI don’t last, and Part 3 at leveraging the agent’s loop for the times you must build. Part 4 turns to the implementations that shouldn’t be built at all, and to the case for waiting.
Not starting, waiting, making do with what already ships, declining to generalize: none of that is cutting corners. Each is an engineering judgment about whether the thing should exist at all. Here are ten implementations that rarely repay the effort.
Ten Implementations That Rarely Repay the Effort
1. Model-Specific Prompt Optimization
Hundreds of lines of system prompt per model, layered with examples, prohibitions, output repair, and self-evaluation. Tuning that closely to one model ages badly: when the model changes, some of that scaffolding becomes dead weight, and over-instruction can actively hurt performance. The harness around it lasts much longer. Inputs and expected results, evaluation criteria, failure cases, and regression tests all survive a model change. The prompt wording is a replaceable part, rewritten whenever the model moves.
2. A Homegrown JSON Repairer and Output Parser
Earlier models couldn’t reliably return a fixed JSON format, so teams wrote code to extract, repair, and retry. Structured output is now a standard feature from every major vendor, and its accuracy keeps improving. The larger the repair layer built on the assumption of broken JSON, the more completely the model side overtakes it. Exception handling still earns its place, but a homegrown parser has less and less reason to sit at the center of a product, and the model’s own constraints fill the remaining gaps.
3. A Model Router with No Traffic
Auto-selecting among frontier, small, and local models sounds reasonable. In practice, shaving a modest monthly API bill costs you the judgment calls about quality on each route, plus an evaluation regime to keep running. Measuring cost and quality first, then splitting only the paths that actually matter, gets there sooner. In a market where prices fall this fast, the price you set out to optimize can disappear before the optimization ships.
4. A Premature Semantic Cache
A cache that reuses past answers for similar questions does cut model calls. It also produces stale answers, false similarity matches, permission leaks, and mixed-in personal data, plus inconsistency after every model update. The savings it protects keep shrinking, too: inference prices keep falling, and each model update turns the stored answers into a previous generation’s. It starts to pay only once call volume is high enough that the savings clear both implementation and incident-response costs.
5. Premature, Complex RAG
With a few dozen documents, chunking strategies, multiple embeddings, reranking, knowledge graphs, and hierarchical summarization often arrive all at once. Context windows have expanded fast, and some models now read millions of tokens in a single call. For a small corpus, feeding in the whole text tends to be cheaper, simpler, and more accurate than an elaborate pipeline. Start there, try plain search next, and improve only what you can measure. Build the complex version up front and much of it falls away with each expansion of context.
6. A Homegrown Long-Term Memory
Embedding conversations, scoring importance, applying time decay, summarizing and re-retrieving: human-like memory is a tempting thing to build. It also breaks in practice, because vector search retrieves what looks similar without noticing that a fact has changed. Remembering a user as single long after their wedding is an architecture problem, not a model one. Platforms are absorbing this as a standard feature as well, and ChatGPT already ships saved memories and references to past chats. What the job usually needs is plain state — files, history, tasks, settings — which SQLite or the filesystem handles.
7. Platform-Specific Plugins
Dedicated extensions for ChatGPT, Claude, Gemini, Cursor, and VS Code often get built before a single user asks. A stable API or CLI at the center, with plugins kept as thin adapters, limits the damage, and building only the ones with real users limits it further. As Part 1 described, the official integration path itself can be replaced within a year.
8. A Homegrown Workflow DSL
Custom YAML, a graph editor, or a bespoke syntax for expressing agent procedures. The foundations already exist: LangGraph, CrewAI, n8n, Dify. Building one from scratch means maintaining not just the syntax but the parser, validator, execution engine, and debugger, with version compatibility and migration tooling on top. The premise is shifting, too. The smarter models get, the more procedural work collapses into a single prompt, which leaves more of these cases to an existing framework or ordinary code.
9. Unnecessary Multi-Agent Architecture
A research agent, a design agent, an implementation agent, a review agent: the naming makes a system look sophisticated. Every added agent also adds state, message passing, retries, conflicts, logs, and token consumption. And as models improve, work that once needed a division of labor collapses into one call, so the more elaborate the arrangement, the more of it is wasted a generation later. Where a single model call and ordinary control flow suffice, they break less. An org chart transplanted directly into software structure usually just makes it heavier.
10. A Homegrown General-Purpose Agent Framework
Unifying planning, execution, observation, retries, state persistence, and tool calling into one base layer is a natural urge. Generalizing while there is still only one use case is almost always early. As the model API absorbs tool use, structured output, state management, and long-running tasks, a homegrown framework thins into a compatibility shim. Wiring one process with ordinary code, then consolidating once the same shape has recurred across several implementations, wastes less. “We might need it later” is a weak reason to build.
Waiting Is Not Falling Behind
In ordinary development, delaying a start is easily seen as a competitive disadvantage. In AI, not necessarily. In three months of waiting, the model’s context may widen, the output may stabilize, an official API may be added, a standard protocol may spread, prices may drop, an OSS implementation may mature, a competing service may withdraw, or the need may turn out never to have existed.
So a waiting period is worth imposing on any infrastructure that isn’t urgent. Rather than creating a repository the day the idea lands, substitute an existing model, try the task by hand, and run it once with ordinary code. A few weeks of that shows whether the problem actually recurs. If it survives the next model update, that’s the moment to consider building.
You Can Generate Code, but Not Time
AI lowered the cost of writing code. That’s exactly why you can now build more than before. But being able to build more does not mean you can maintain more. It’s your future self who checks behavior at every model update, replaces retired APIs, tunes old prompts, and removes obsolete mechanisms.
In the AI agent era, the resource an engineer should guard most carefully is their own time. Not GPUs, not tokens. Models keep getting smarter and prices keep falling, and much of what looks necessary today ships as a standard feature tomorrow. So the heavy construction that patches the current model’s immaturity is rarely worth rushing. Use the model in small pieces, wire them with ordinary code, measure the problem, resist generalizing, wait a few months, and delete what stops being needed.
Above all, the choice not to start is an engineering judgment in its own right. Skill in this era isn’t measured only by how intricate a system you built. How much of your own time you kept out of things that never needed building is part of the measure too.
Part 5 covers the six checks worth running before you start.