Essays on AI architecture, strategy and the org problems in between. Heart what’s useful, argue with me in the comments.
Tagged “llm-cost”clear ✕
Aug 10, 2026
Facing a surprise LLM bill? Learn how enterprise teams are cutting AI inference costs by 50-80% using a practical framework of model routing, prompt caching, and efficient architecture, without sacrificing quality.
2 reads
Feb 17, 2026
Explore four essential caching layers for LLM applications—prompt-prefix, full response, retrieval, and semantic—that can cut costs by up to 70% and serve responses in milliseconds. This guide covers the mechanism, risks, and metrics for each, plus when caching is the wrong choice.
2 reads
Dec 04, 2025
A simple retry-on-failure wrapper seems harmless, but in AI agent systems, it can lead to silent cost overruns and cascading failures. This article explains why, and how to build resilient, cost-aware error handling using techniques like total budget caps and structured escalation ladders.
2 reads
Nov 07, 2025
Large context windows are not a free upgrade; they are a budget with steep costs in money, latency, and accuracy. This article explains the three hidden costs of long context and provides a framework for deciding when to use it.
1 reads