16 results for “RAG”
Related topicsragRAG evaluation
Dec 16
The 'R' in RAG Was Meant To Be Trained, Not Just Searched
The original "Retrieval-Augmented Generation" (RAG) paper proposed a deeply integrated system where the retriever and generator were trained together. This is a stark contrast to today's common "vector search + prompt" pattern, and understanding the original vision reveals the trade-offs we've made.
Feb 24
Claude's 1M Tokens & 90% Recall: What It Solves, What It Doesn't
Anthropic's Claude Opus 4.6 offers a 1M token context window, but its real value lies in its claimed 90% recall. This article explores when this massive context replaces RAG and when it's an expensive distraction.
Oct 07
Why Your RAG Pilot Won't Survive Production
A RAG pilot on 200 curated documents looks magical, but scaling to 200,000 uncurated ones in production causes it to break. This article explains why recall collapses, how to fix a failing RAG pipeline, and what you should be measuring instead.
Apr 23
Most Enterprise RAGs Have No Access Control: How to Fix Yours
Vendor research shows most enterprise RAG deployments lack basic access controls, creating major data leak risks. This article details how to fix it by implementing pre-retrieval filtering in your vector database to enforce permissions before data reaches the LLM.
Jun 11
Stop Fine-Tuning Your LLM: Why Retrieval Is Cheaper, Faster & More Reliable
Developers often try to add knowledge to a large language model by fine-tuning it, a costly and often ineffective approach. A better strategy follows a clear hierarchy: start with the prompt, then use Retrieval-Augmented Generation (RAG) for facts, and only use fine-tuning to change the model's core behavior.
Aug 10
LLM System Design Architecture: The End-to-End Guide to Building Production AI
A comprehensive guide to end-to-end LLM system design architecture, covering the core patterns like RAG, vector databases, and agentic workflows required to ship production-grade AI applications.
Oct 17
Chunking Is a Retrieval Decision: Up to 9% Recall Is on the Line
Most RAG systems fail due to a single, overlooked setting: chunk size. This article explains how rethinking chunking as a dynamic retrieval parameter, not a static preprocessing step, can boost recall by up to 9% and why the optimal size depends entirely on your users' questions.
May 07
Your AI Team Is a Bottleneck: Why LLMs Belong on the Platform
Stop creating siloed "AI teams" that become bottlenecks. To scale LLM development, treat AI capabilities as a product built by your platform team, enabling product teams to innovate faster and more safely.
Nov 07
The 1M Token Budget: Why 1 in 10 Queries Fail at Scale
Large context windows are not a free upgrade; they are a budget with steep costs in money, latency, and accuracy. This article explains the three hidden costs of long context and provides a framework for deciding when to use it.
Feb 17
Cut Your LLM Bill by 70%: A Guide to Four Caching Strategies for 2026
Explore four essential caching layers for LLM applications—prompt-prefix, full response, retrieval, and semantic—that can cut costs by up to 70% and serve responses in milliseconds. This guide covers the mechanism, risks, and metrics for each, plus when caching is the wrong choice.
Feb 10
Your LLM Judge's Secret Drift: Why 80% Agreement Isn't Enough
LLM-as-a-judge provides scalable evaluation but suffers from silent drift as models and rubrics change. This guide explains how to use a human-labeled holdout set to calibrate your judge, diagnose regressions, and prevent your evaluation system from misleading you.
Dec 23
The GenAI Divide: Why 95% of Enterprise AI Projects Delivered No Value
In 2025, enterprises spent $37 billion on generative AI, yet a stark "GenAI Divide" emerged. A landmark MIT study found 95% of projects failed to deliver any business value, revealing that success depends not on model quality, but on organizational strategy.
Jul 28
Kimi K3: a 2.8-trillion-parameter open model that bets on linear attention
Moonshot's new open-weight flagship isn't chasing a leaderboard — it's a bet that a cheap million-token context changes what we build. What actually changed under the hood, how to run it, and where it sits against the open-weight frontier.
Apr 07
Why Your First AI Invoice Is a Trap: A Cost Model for Survival
Most engineering teams learn about their AI costs from a surprise invoice. This article argues for building a cost model based on user actions, not API calls, to prevent margin erosion and build a profitable AI business.
Apr 02
The 76% Mistake: Why AI Build vs. Buy Is a Layer-by-Layer Choice
The "build vs. buy" decision for LLM applications is no longer a single choice. This article presents a framework for analyzing the AI stack layer by layer—from the model to evaluation—to avoid common pitfalls and focus engineering effort where it creates the most value.
Feb 03
How to Log Your LLM: Unlock 99% Automation With Better Data
Traditional logging can't tell you if your LLM is getting worse. This article details the essential fields to log—from costs to verdicts—to debug silent failures, track quality, and automate up to 99% of your AI-driven workflows.