The Engine Does Not Win the Race

· 12 min read

llm ai elixir erlang programming opinion hacking

Somewhere, right now, a senior engineer is on the verge of a nervous breakdown because his company will not let him switch from Claude to GPT-5.2, or from GPT to Gemini, or from any of them to a locally-hosted llama that smells faintly of the intern’s vape juice. He is convinced that the model is the product, that the difference between shipping and slop is measured in the number of parameters, and that if only management would approve the more expensive API key, all his problems would evaporate like a hallucinated citation.

He is wrong. Spectacularly, expensively, and in a way that will cost him exactly the difference between the two API tiers he is agonising over.

Allow me to explain, in the manner of a man who has spent the last year building the thing he is about to tell you matters more than the model itself.


The Engine Is a Fish With Amnesia

Let us first agree on what a language model actually is, because ninety percent of the industry’s agonising proceeds from a category error. A language model is a finite state machine with the memory span of a goldfish that has been through electroconvulsive therapy. Its entire operation collapses into the unglamorous sequence:

start → context → query → response → stop

There are no sessions. There is no memory. Every single request you make begins from a blank slate, and the model dutifully autocompletes whatever you have managed to cram into the context window in that one trembling moment. The model does not know your codebase, your conventions, your catastrophic past decisions, or even that it spoke to you thirty seconds ago. It knows only the distillate you present it with—and, being a statistical amplifier of whatever it is fed, it will faithfully amplify that distillate into either gold or guano.

Here is the uncomfortable corollary that nobody in the model-comparison industry wants to hear: the model is the least interesting component of the entire stack. The interesting component is the thing that decides what goes into that context window, in what order, at what cost, and with what guard-rails around the output. That thing is the harness. And the harness is where the entire difference between “Dull ∧ LLM = Dull” and “Wise ∧ LLM = Wise²” actually lives.

I wrote that truth table once, in a fit of optimism, and immediately got accused of supporting both Trump and Putin by people whose reading comprehension caps out at three words per paragraph. So let me be plainer this time. The model is the engine. Nobody drives an engine. You drive a car—and the car is the harness.

Nobody Buys a Car for the Crankshaft

Any fool can bolt a thousand-horsepower engine into a shopping trolley. It will do a spectacular burnout for exactly one second before the steering shears off, the brakes fade to nothing, and the whole contraption wraps itself around a lamppost with the shopping basket still full of the groceries it was never going to deliver. Horsepower is the number that sells the brochure; the transmission, the suspension, the brakes, and the fuel injection are what decide whether you actually arrive anywhere.

The industry has this exactly backwards. We have spent three years obsessing over which engine to buy while ignoring the fact that most of these engines are being bolted into shopping trolleys—single-threaded Node.js scripts that crash the instant a tool call returns an unexpected byte, wipe the entire conversation, and force you to re-explain your whole codebase to a model that has already forgotten you exist.

And here is the cruelest part: the engine does not even know it is in a shopping trolley. It cannot feel the lamppost approaching. It will cheerfully keep generating code until the trolley is a smoking ruin, because generating code is all it has ever been trained to do. The harness is the only component with both the information and the authority to say “no.”

So what does a proper harness actually do? Two things, and only two things, and both of them are worth more than any model upgrade you are currently begging your CTO for. It decides what the model sees. And it decides what the model is allowed to do with what it sees. Everything else is decoration.

What the Model Sees: The Miser’s Art of Context

The single most expensive resource in this entire economy is not compute, not GPUs, not even the API bill—it is the context window, and the tokens you waste filling it with noise. Every model has a hard ceiling on how much it can attend to before quality collapses into a puddle of generic autocomplete, and that ceiling is a great deal lower than the marketing department would have you believe. Nobody can actually process a hundred thousand tokens without noticeable quality loss. The ones who claim otherwise are selling subscriptions.

This is why the R in RAG matters hundreds of times more than the model itself, and why every serious player has been quietly burrowing into your computer with an IDE or a CLI. Shipping the entire codebase to the server on every request is not just expensive—it is actively harmful, because it drowns the signal in noise. A proper harness does not ship the world. It ships a distillate : precisely the files, symbols, and idioms the current task requires, nothing more, nothing less.

Let me be concrete, because I built the thing I am about to describe, and I would rather show you the plumbing than wave at the facade. My slowly-simmering harness, DeepSeek Harness ( dsh ) , spends an almost pathological amount of effort on not wasting your tokens:

Selective context expansion. You type @lib/auth.ex or @error or even the lazy phrase “error above,” and the harness expands only that reference into the prompt—fetching the file, the URL, or the relevant error block it has been tracking as open versus resolved. It does not dump your entire repository into the context window on the off-chance that something in there might be relevant. It fetches the one thing you pointed at, and it keeps a running ledger of which issues are still open so that when you say “fix it,” it knows what “it” refers to without you re-pasting the stack trace for the fourth time.

AST instead of raw text. This is the hill I will die on. Raw source code is a terrible format for a context window—verbose, redundant, and full of formatting noise that contributes nothing to understanding. An abstract syntax tree is structured, compact, and far more legible to a model that was trained on code. My Ragex integration lets the harness search and present code as structure rather than as a wall of text. Same understanding, a fraction of the tokens. This is not a micro-optimisation; it is the difference between being able to fit a real codebase in the window and being able to fit a single file.

Compression on demand. When a session has been running long enough that the context window is groaning under its own history, /compact hands the conversation to the model and asks it to produce a condensed summary preserving the core goal, the files touched, the decisions made, and the next steps. The verbose transcript is then replaced by that summary. The model keeps its memory of what matters without you paying to retransmit every failed attempt and every “no, I meant the other thing.”

The miser’s dashboard. /tokens and /cost show you, per turn and cumulatively, exactly what you are spending. You cannot optimise what you cannot measure, and the token gauge on the status bar is the financial equivalent of a fuel gauge on a car that everybody else is driving blind.

Sessions that survive death. Every conversation is persisted to disk as a resumable narrative, so if you walk away for the night—or the harness crashes, or the cat walks across the keyboard and closes the terminal—you resume with dsh -c exactly where you left off. This sounds like a convenience feature. It is actually a token-saving feature of the first order, because the most expensive tokens in any project are the ones you spend re-explaining your codebase to a model that forgot it overnight.

And, most importantly, the harness is the thing that decides not to send anything at all . When it detects that the model is about to call the exact same tool with the exact same arguments it already called in the previous turn—the tell-tale heartbeat of an agent stuck in a loop, burning tokens in a circle—it refuses to execute, injects a notice telling the model to synthesise its answer from the results it already has, and moves on. The cheapest tokens are the ones you never spend. A harness that can say “no” is a harness that saves you money on every single session.

What the Model Is Allowed to Do: The Suspension and the Brakes

But a harness that merely feeds the model well is only half a car. The other half is the part that keeps the thing on the road when the engine, in its infinite statistical confidence, decides to floor it toward a brick wall. This is where quality actually comes from—not from the model’s intelligence, but from the scaffolding that refuses to let the model’s output become the product without passing through a gauntlet of human-shaped checkpoints.

Quality, in my experience, is not something a model produces. It is something a harness enforces by constraining the model’s freedom at every fork in the road. The fewer choices the model has to make between two paths, the cleaner the result—a principle I have found to outweigh the quality of all prompts combined. Consider the mechanisms by which a real harness puts this principle into practice:

Persistent rules, not per-session rituals. The harness carries a scoped rule engine. Rules with the all: scope are injected into every prompt; rules with a narrower scope, like cr: for code reviews, apply only when that command runs. This is the difference between engineering—where your conventions are encoded once, in the harness, and obeyed forever—and penance—where you manually correct the same mistake in every generated file until you retire.

The practices that teach the model your house style. When the harness first meets a project in an unfamiliar language, it asks you to point at exemplary codebases, walks through them, and squeezes out a concise, human-editable list of good practices for that language. Those practices are then injected into the context whenever the model works with that language. Feed it good code, get good code. This is not mysticism; it is pattern matching. The harness is what remembers to feed the model the good code, every single time, without you having to remember.

The plan gate. This is the single most important quality mechanism I have built, and it is almost embarrassingly simple. When a batch of tool calls looks non-trivial—two or more file-modifying actions—the harness pauses, drafts a structured plan, and refuses to execute until a human has looked at it and pressed “approve.” No more watching your agent cheerfully barrel through eight steps in one breath, commit untested code to main , and open a pull request titled “feat: everything.” The gate is the brake pedal. Without it, the engine is a runaway.

The circuit breaker. When a tool fails three times in a row without producing anything, the harness disables it and tells the model to fall back to standard tools. This is the difference between an agent that learns from its mistakes and an agent that bangs its head against the same wall for forty turns while billing you by the token.

The validation gauntlet. The harness is wired to the native linters and static analysers— oeditus_credo , propwise , credo , dialyzer —and can run them against the whole project, a diff, or a branch review. Quality is not asserted; it is executed . The model does not get to declare its work done; the linters get to declare it done, and they are considerably less susceptible to flattery than the model is.

The workflow engine. For the occasions when a task is big enough to deserve ceremony, the harness runs a full pipeline: branch off main , summarise the task, decide whether it can be split into non-clashing sub-tasks, execute those in physically isolated git worktrees under parallel BEAM processes, require tests and documentation, gate on the linters, and only then commit. The whole thing is version-controllable, so your entire team shares the same agent rules, the same quality gates, and the same workflow. The harness is where your engineering standards live in a form that survives the departure of the senior engineer who used to carry them around in his head.

The Architecture Is the Point

I have saved the most important point for last, because it is the one people find hardest to believe. A harness built on the BEAM—on Erlang/OTP and the actor model—is not merely more reliable than a single-threaded script; it is architecturally incapable of the failure mode that plagues every fragile harness out there. The agent’s cognitive state runs in an isolated process, decoupled from the processes that execute tools. When a tool crashes—and tools will crash, because tools run shell commands and shell commands are chaos—the supervision tree contains the damage. The brain survives. The conversation survives. The context, which you have paid for in tokens and in blood, survives intact.

This is not a reliability footnote. This is a quality and token-economy argument wearing a hard hat. Every crash that wipes your session is not merely an annoyance; it is the total loss of every token you spent building that context, followed by the certain expenditure of twice as many tokens rebuilding it from scratch with a model that has already forgotten everything. Fault tolerance is not a luxury feature. Fault tolerance is the cheapest context-compression strategy ever invented, because the cheapest context is the context you never have to rebuild.

The Engine Is Not the Product

So here is the thesis, stated without hedging, in the hope that it survives contact with the comment section: you are arguing about the wrong component. The model is a commodity. They are all autocomplete, they are all statistical amplifiers, and the difference between the best and the merely excellent is a rounding error compared to the difference between a good harness and a bad one. A brilliant model in a fragile shell is a smart idea waiting for a crash. A mediocre model in a disciplined harness will out-ship a brilliant model in a shopping trolley every single time, because the harness is where the wise part of “Wise ∧ LLM = Wise²” lives.

The engine does not win the race. The car does. And the car—the suspension that keeps it on the road, the brakes that stop it before the wall, the fuel injection that meters exactly how much petrol per stroke, and the driver who refuses to let it floor it into a lamppost—is the harness.

Stop shopping for engines. Start shopping for cars.

Happy prompting—and may your harness be the one that knows how to say no.


Previously, on the subject of refusing to lose your mind over autocomplete: