GrevireDev / My Local AI Cost 10.72× Less Than the API — Until I Recomputed It Properly

Drafted with Mimo v2.5 and corrected by Deepseek V4.1

Correction (2026-10-26). The first version of this article claimed local inference was 10.72× cheaper than the equivalent API. That figure was wrong: I priced every input token at the fresh rate, ignoring cached input, which is billed at roughly a tenth of it and accounts for most of an agent workload. Recomputed properly, the gap is about 2×, not 10.72× — and once you amortize the GPU it disappears entirely. The corrected numbers are below, along with what I got wrong and what still holds. Thanks to the readers on r/LocalLLM who caught it.

TL;DR — I measured what my local AI inference costs over 30 days. At the wall plug: 12.38 CHF (~$14.05) in electricity. Priced naively, the same token volume would have cost 132.77 CHF (~$150.71) through the API — a factor of 10.72×. That number was wrong. With cached input billed correctly, the API equivalent is closer to $28-34, so the real gap is roughly 2×. Against a $1,135 GPU, that does not amortize. Local AI is not cheaper. It is a different purchase.

The Trigger

When people talk about local AI, the first question is almost always: “which GPU should I pick?”

I asked a different question: how much does it actually cost, once it’s running?

Not the retail price of components. Not lab benchmarks. Just the electricity bill at the end of the month, compared to what I would have paid staying on a cloud API.

The answer, measured with a wattmeter over 30 days, surprised me. Then it turned out I had measured the local side honestly and the API side carelessly.

What I Measured

My setup: an AMD processor, 32 GB of DDR4, a 1 TB SSD, and a Radeon RX 7900 XTX with 24 GB of VRAM. A dedicated inference box, powered and monitored at the wall plug for the whole period.

Over a 30-day period (September 25 – October 25, 2026):

What I Got Wrong

Three separate errors, all on the API side of the comparison. The local measurement was fine.

1. I ignored cached input. This is the big one. I multiplied all 964 million input tokens by the fresh input rate of $0.15/M. Providers bill cached input at around $0.016/M — roughly a tenth — and in agent workloads 90-95% of input is cache hits, because an agent re-reads the same files and the same conversation context over and over.

Recomputed, assuming 90% and 95% cache hits:

So the API equivalent is $28-34, not $151. Against $14.05 of electricity, that is a factor of about 2 — and at a lower cache ratio, say 50%, it climbs back to about $86, or 6×. The honest range is 2× to 6×, and the true value depends on a cache ratio I did not measure.

2. My pricing grid did not match the model I ran. I used qwen3.8-flash rates for a qwen3.8-flash-next-coder workload. Those are different variants. This is a deeper problem than any quantization quibble, and it means even the corrected figure is an approximation.

3. The power figure needs an idle split. 30.20 kWh over 720 hours is a 41.9 W continuous average. A box with a 7900 XTX does not idle at 42 W. Two possible explanations: my logging has gaps (reboots, meter offline, logger crash), or my estimate of how much of the month was actual work is too optimistic. Either way, the plug total is not automatically the inference cost — and for a dedicated box it is not automatically not the cost either, since there is no other use to subtract. I need the idle series separated from the inference series before I quote a monthly figure again.

4. The heat argument does not apply to me. Several readers made the fair point that electricity becomes heat, so in winter the cost is partly recovered. The physics is right. But I live in a flat and the rig sits in the cellar, so it heats the communal areas, not my living room. The heat is real; the credit is not mine. Moving it upstairs would fix that, and the noise is the reason it has not moved.

The Benchmark Month Makes It Worse, Not Better

There is a fourth nuance worth stating, because it cuts against my own conclusion.

This was an R&D month, not a production month. A large share of those 964 million input tokens comes from benchmarking: high-context sweeps to find each model’s context ceiling, prefill and decode tests at 256k, dozens of models tried and discarded.

That matters for the cache ratio in a way that hurts my argument. A benchmark sends a fresh prompt every time. There is no repeated context to cache, so the cache-hit ratio in a benchmark month is close to zero — which is precisely the scenario where the API counterfactual is most expensive. My naive calculation was not just arithmetically wrong, it was accidentally describing the one workload where local looks best.

In production, the opposite holds: an agent re-reads the same project files and the same conversation for hours, so the cache ratio goes up and the API bill goes down. A production month with fewer tokens and a high cache ratio could plausibly land below my electricity cost. I do not know yet, because I have not measured a production month — but the direction is clear, and it is not in local’s favour.

Why Local Still Makes Sense — Just Not on Price

The ratio is not spectacular. It is roughly 2× on marginal cost, and negative once you count the hardware. Here is the arithmetic that matters:

The GPU cost about 1,000 CHF (~$1,135). At a 2× marginal advantage — about $20/month saved — it takes roughly 4.6 years to pay back on electricity alone. At 6× it is about 15 months. Neither figure includes the rest of the machine, the depreciation, or the month of learning.

So local AI is not cheaper. That claim does not survive contact with the corrected numbers, and I am not going to defend it.

What survives is everything the electricity bill cannot price:

• The data never leaves the house. Not a marketing claim — a property of the setup. For anything that touches client code, medical or administrative context, or work under an NDA, that is the whole argument.
• The understanding. I know now what quantization actually costs in quality, why prefill and decode behave completely differently, where the KV cache budget goes, and why a model that benchmarks well can be useless inside a given harness. None of that evaporates the day I go back to an API. If anything it makes me a better API user, because I finally know which knob I am paying for.
• The hedge. A working local stack is an option on a future where API pricing, availability or terms change. That option has value even when it does not pay for itself this month.

Ollama, llama.cpp, and the Learning Curve

I started with Ollama. Simple, clean, perfect for quickly testing models. But as soon as you want to step outside the catalog, use models downloaded from Hugging Face, or fine-tune quantization settings, you hit limits.

I quickly moved to llama.cpp. More control, more flexibility, but also more parameters to understand. IQ1_M vs Q4_K_M quantization, KV cache, the number of GPU vs CPU layers, flash attention. None of these concepts exist when you just call an API.

On AMD, you add the compatibility layer: ROCm for GPU acceleration — progress is being made, but it still lags behind the CUDA ecosystem. Some tools, like vLLM, are simply unavailable. Fortunately, llama.cpp handles AMD GPUs very well.

The Uncomfortable Question: Is It Worth It?

If you’re a developer who just wants coding assistance, 12 GB of VRAM is plenty. An RX 6750 XT runs a 9B model with a reasonable context just fine.

If you want autonomous agents working across entire projects, with contexts from 100k to 260k tokens without constant compaction, 24 GB is the sweet spot.

But the real question isn’t technical, and it isn’t the electricity bill either.

Local AI is an investment in personal research and development. It isn’t “plug and play.” It’s time-consuming, frustrating at times, rewarding at others. It’s a path you take out of curiosity, conviction, or a desire for technological autonomy — not because it’s the fastest or cheapest way to move a project forward. On price, it is neither.

If your goal is to accelerate a concrete project right now, stick with an API. For a few CHF per month on Flash models (GLM, DeepSeek, Qwen), you’ll move faster and spend less.

If your goal is to understand, to build infrastructure you control, and to be independent of cloud providers — then the time and hardware investment is justified, but justify it on those grounds. Not on the ratio.

My Numbers, in Full Transparency

Because an article about the cost of local AI that doesn’t show its raw data has no value, here are the essentials — including the parts that are still broken:

• Period: September 25 – October 25, 2026 (rolling 30 days).
• Hardware: RX 7900 XTX, 24 GB VRAM, 32 GB DDR4. Dedicated inference box, powered 24/7.
• Software: llama.cpp, GGUF-format models, Grafana dashboard for tracking.
• Electricity: 30.20 kWh, standard Swiss residential rate → 12.38 CHF (~$14.05) at 1.135 USD/CHF.
• API reference: qwen3.8-flash, International, Alibaba Cloud Model Studio pricing grid, consulted October 2026 — note the mismatch with the qwen3.8-flash-next-coder model actually run.
• Known gaps: idle-vs-inference power split not yet isolated; cache-hit ratio not measured; pricing grid not aligned with the model used.

What I am fixing. The next month’s dashboard will report an idle-only series next to the inference series, so the power figure is a delta rather than a plug total. I will align the pricing grid with the model actually used, and measure the real cache-hit ratio instead of assuming one. Then I will publish a production-month comparison, which is the number that actually matters — this R&D month is not representative of daily use.

Conclusion

Local AI inference, at equivalent usage, costs roughly twice as little in marginal terms as a cloud API — not eleven times. Including the hardware, it costs more. The original 10.72× was the product of a careless API calculation, and it took a comment thread to find it.

But reducing the cost of local AI to its electricity bill is like reducing the cost of a car to the price of fuel — except this time the fuel turned out to be more expensive than I thought.

Do I regret my 1,000 CHF (~$1,135) GPU and my month of R&D? Not for a second — but not because it saved me money. It did not. It bought me understanding, a setup I control, and data that never left the house. Those were always the real reasons. I just dressed them up in a number that flattered them.

All mentioned metrics come from a shared Grafana dashboard (llama.cpp — Monthly Model Tracking). API pricing is from Alibaba Cloud Model Studio for qwen3.8-flash as of October 2026. Corrections after discussion on r/LocalLLM, October 2026.