Inference Cost Is Now a Line Item You Engineer Against — But Nobody Sells You the Accounting

Four Projects, One Direction

In a single month, four independent projects aimed at the cost of running models climbed the Hacker News and GitHub front pages.

DeepInfer took 25,086 stars and a Hacker News thread of 2,355 points and 892 comments — the month's most-discussed technical launch. It makes DeepSeek inference faster. gpu-unlock reached 14,928 stars advertising up to 2× FP16/BF16/TF32 tensor throughput on consumer 20/30/40-series cards. Microsoft's TokenEconomizer reached 4,557 stars attacking coding-agent cost as a black box. And Llmspeed earned 331 points and 86 comments helping people feel how fast a local model runs before they buy hardware.

Four projects, four different approaches — and they all point the same direction. Read the descriptions in sequence and the pattern is unmistakable: faster inference, cheaper hardware, black-box control, better pre-purchase measurement. Every single one attacks the supply side of the equation.

Now ask the question a founder actually asks when the invoice arrives: where did the money go?

None of the four answers it. Not because they're badly built — because they're solving a different problem. They make the unit cheaper. None of them tells you which of your own sessions burned the budget, or what a fix would actually save.

The Missing Substrate Arrived Too

There's a second signal in October that makes this concrete rather than philosophical, and it's easy to miss because it looks like routine pricing news.

DeepSeek published per-request search and fetch pricing: $0.75 per 1,000 successful requests (607 points / 282 comments). Cloudflare shipped a Web Search API in the same month (596 points / 286 comments).

Granular, published unit pricing is the substrate optimisation needs. When the only number you have is a monthly total, "tokens are expensive" is a feeling. When the unit price is published per request, that same waste becomes "$18 a month on this pattern" — a number a founder can act on, compare, and budget against.

This is the same shift that happened in cloud computing roughly fifteen years ago. EC2 had a price per instance-hour from day one, but it took years before per-service cost attribution became a product category. The unit price was necessary but not sufficient; someone had to build the accounting.

What Attribution Actually Requires

The reason this gap persists is worth being precise about, because the naive version of the product doesn't work.

You cannot instrument a coding agent from the inside. LangFuse and LangSmith are excellent tools, and they solve a different problem: they trace applications whose developers own the call sites. A coding agent is a third-party binary that a founder runs interactively. There is no source file to edit, no SDK to wrap, no request handler to decorate.

So capture has to happen outside the agent, in two layers:

A local proxy records token counts per request without modifying anything. It sees the cost and nothing about the meaning.

Harness hooks supply what the proxy can't see — which tool was called, which file was read, which task was in progress. They see the meaning and nothing about the cost.

Neither is sufficient alone. Attributed cost requires both, joined on the session.

That's the engineering. It's not exotic, and it's not the hard part. The hard part is the next sentence.

The Number That Kills the Product

Every tool in this space reports consumption. Almost none reports verified reduction.

That distinction matters more here than in almost any other developer tool, because the audience is unusually well-calibrated. Founders running three coding agents and reading cost threads are exactly the population that will check your claim against their own invoice. And the way this category dies is a savings number that doesn't survive that check.

Which means the sequence of building matters as much as the content. The correct order is:

1. Build the measurement harness before the intervention. A/B windows that record cost and quality together. A saving that degrades output is not a saving, and claiming otherwise ends the product's credibility with the one audience it needs.
2. Build interventions that are reversible by default. Output truncation, retrieval-first file reads, model routing by task class — each applied per project, each off by default, each undoable in one click. An intervention that can't be reverted is one the user will remove entirely.
3. Publish negative results. "We cut a team's spend 34% without slowing delivery" is a strong claim. "This intervention saved nothing on this codebase and here's why" is a stronger claim, because it's the one that proves you're measuring rather than selling.

The Structural Reason This Stays Open

The obvious question: why won't the harness vendor just build this?

They will build part of it. Any harness can show you what you spent. But there's a specific part of this product that a vendor cannot credibly sell, and it's the part where the money is.

A harness vendor will never recommend routing work away from its own model. Nor will it recommend cutting the context that makes its product feel capable. The intervention layer — "use a smaller model for this task class," "truncate tool output above N bytes," "use retrieval instead of full-file reads" — is structurally unbeatable by the vendor whose model you'd be using less.

That's a genuinely unusual competitive position. Most developer tools compete on being better at the same job as an incumbent. This one competes at a layer the incumbent can't enter without arguing against its own product.

The supply side, by contrast, is crowded and will stay crowded. Four well-funded projects in one month is not a market you want to enter as a solo developer.

The Pricing Trap

One more thing worth flagging, because it's a mistake this category invites.

The adjacent model is FinOps, where tools price at roughly 1-3% of the spend they manage. That model works at cloud scale, where managed spend is millions and the fee is a meaningful but tolerable slice of an infrastructure budget.

At founder scale it produces an absurd result. A 1% fee on a $400/month agent bill is $4 — too small to be a business — while the value delivered (finding $118 of waste) is identical regardless of how large the bill is.

Flat pricing is the right shape here, and the number should be chosen against the alternative rather than against a percentage. The alternative is not "a cheaper tool," it's "an unexplained bill and a spreadsheet." If the product finds a third of a $400 bill, $15/month is obviously worth it, and it stays obviously worth it whether the bill is $400 or $4,000.

What to Measure This Month

1. Attribute one week of your own agent spend by hand. Take your last invoice, open your session history, and try to assign the total to projects. Most founders can allocate maybe 60% and are guessing on the rest. That gap is the size of the problem in your own workflow.
2. Count your re-reads. In one long session, note how many times the agent read the same file. In our own usage this is consistently the largest single waste pattern, and it's invisible in a transcript because the sixth read looks exactly like the first.
3. Check whether any of your tools can prove a saving. If you've ever adjusted a prompt or a config to cut cost, ask what evidence you have that it worked. If the answer is "it felt faster," you have no evidence — and neither does anyone selling you a fix.
4. Distrust the percentage. Any tool that quotes a savings figure without showing you the baseline and the A/B window is quoting a marketing number. The credible version shows you the before, the after, and where the measurement is uncertain.

The Bottom Line

The supply side of inference is being solved, thoroughly and by well-capitalised teams: faster frameworks, cheaper hardware, per-request pricing. That's good news, and it's also why no solo founder should compete there.

The demand side is empty for a structural reason. Accounting is unglamorous, it requires trustworthy measurement rather than impressive engineering, and the vendors best positioned to build it have a business reason not to. Four projects optimised the cost of a token this month; not one told anybody where their tokens went.

For a one-person company, that's the more interesting half of the market — and the half where being first is still possible.