Compare Inference Providers¶

Real-world cost and perceived performance of five provider/model combinations for agentic coding use, based on my own usage. Input/output prices are per 1M tokens (uncached list prices):

Derived cost per hit (approximate, FX roughly 1 EUR ≈ 1.08 USD, 1 CHF ≈ 1.10 USD):

The caching insight¶

The list prices are misleading; prompt caching is the real cost driver for agentic workloads.

Every "hit" resends the same large context (system prompt, file contents, conversation history), so the prefix is identical request after request. Providers that support prompt caching only charge full price for the new portion of each prompt:

• Mistral flash charges $0.007/1M cached input vs \(0.30/1M uncached — a ~43× discount. With most input tokens cached, the effective per-hit cost collapses to ~\)0.001, which is why $1.08 covers 963 hits.
• Mistral glm-5-2 shows the same pattern: $0.69 for 101 hits at \(1.40/\)4.40 list prices only works if most input was billed at a cache discount.
• Infomaniak lists no prompt cache support at all, so every request pays full input price — that's why 1541 hits cost CHF 54.214 despite a lower list price than HF.

Takeaway: for cache-friendly workloads, compare cached-input price + cache coverage, not list price. A model with a higher list price (Mistral's GLM-5.2 at $1.40 input) can end up far cheaper per hit than a "cheaper" model on a provider without caching (Infomaniak at $0.74 input).

Backlinks

🔗 Evaluation

📝 Edit on GitHub · 🔷 Open in Obsidian