The Problem With a Bigger Memory
In February 2026, Anthropic released Claude Opus 4.6 with something developers had been asking for for years: a much bigger "context window." Think of the context window as the model's short-term memory—the amount of text it can hold in view at once while it reads your question and writes an answer. Text gets measured in "tokens," which are roughly word-fragments; a token is not quite a word, but close enough that a million tokens works out to about 750,000 words, or somewhere between 2,000 and 3,000 pages. That's enough to hold an entire codebase, or a small library of legal documents, in a single request.
The old workaround, for anything too big to fit, was to chop the material into pieces and feed the model only the relevant fragments. Anthropic's pitch is that you no longer have to do that: just paste everything in and ask your question.
There's a catch, though, and it's one that's dogged large-context models for years. A model that can accept a million tokens isn't the same as a model that actually uses all of it. Researchers have a name for the common failure: the "lost in the middle" problem. Models tend to remember the very start and the very end of a long document well, but their recall of material buried in the middle drops off—sometimes badly. A model might advertise a 200,000-token window, but if it can't find a fact on page 400 of a 500-page document, its window is really much smaller than advertised.
That's why the real headline in Anthropic's announcement wasn't the million-token figure—it was a second number: 90% accuracy when retrieving facts from anywhere across that entire million-token span. If that number holds up, it changes how developers should think about building AI systems around large documents. It doesn't make careful system design unnecessary. It just moves the problem somewhere else.
The Brute Force Approach: From Kilobytes to Gigabytes
For years, the size of a model's context window has been a primary bottleneck. Early models could barely handle a few pages of text, forcing developers into a pattern known as Retrieval-Augmented Generation, or RAG. The core idea of RAG is to treat the language model as a reasoning engine, not a database. You store your documents externally (often in a specialized vector database), use a search algorithm to find the most relevant snippets for a given user query, and then "stuff" only those snippets into the model's limited context window.
This is effective, but it's an entire field of engineering unto itself, and it introduces multiple points of failure:
The Search: What if your search algorithm fails to find the right document? The model, no matter how smart, will never see the information it needs.
The Chunks: How do you split your documents? Splitting mid-sentence can lose crucial context. Making chunks too large might exclude other relevant information.
The Synthesis: The model gets a Frankenstein's monster of a document, assembled from disparate paragraphs. Can it stitch them together into a coherent answer?
A million-token context window presents an alternative: brute force. Instead of retrieving, just stuff the entire thing in. Anthropic's Claude Opus 4.6 is the first of their top-tier "Opus" models to offer this capability, which in practical terms means you can fit:
Roughly 750,000 words, or 2,000-3,000 pages of text.
An entire codebase for a moderately complex application.
Months of customer support transcripts or financial records.
The community's reaction was immediate: the announcement shot to the top of Hacker News, sparking hundreds of comments from developers who saw an entire category of problems potentially becoming obsolete. And for a certain class of problem, they're right.
Retrieval vs. Stuffing: The New Economics of Context
The choice between RAG and stuffing everything into the prompt is not just technical; it's economic. Every token sent to a model, and every token generated by it, has a price. A million-token prompt is not a cheap API call, and it certainly isn't one you want to make repeatedly. This creates a new set of trade-offs for architects to consider.
The Cost of Stuffing
Think of a chatbot built to answer questions about your company's internal documentation. With a 1M token window, you could load the entire knowledge base into the context on the first turn of the conversation. The model would have perfect, complete access.
But what happens on the second turn? The user asks a follow-up question. To maintain that perfect context, you must send the entire million-token payload again, plus the conversation history. This is financially untenable for almost any interactive application. The cost and latency of processing a million tokens for every single message would be astronomical.
Therefore, "stuffing" is best suited for one-shot tasks:
Summarizing a massive research anthology.
Performing a one-time analysis of a year's worth of financial statements.
Refactoring a codebase by providing all the source files at once.
In these cases, you pay the high cost once to get a comprehensive result that would be difficult or impossible with a smaller context.
The Cost of Retrieval (RAG)
RAG architecture inverts the cost structure. It requires a significant upfront investment in engineering. You need to set up a data pipeline to chunk and embed your documents, a vector database to store and query them, and logic to manage the retrieval and prompting process. This is complex and can take weeks or months to get right.
The payoff comes at inference time. Instead of sending 1,000,000 tokens, a well-tuned RAG system might only need to send 8,000 or 16,000 tokens—the most relevant chunks for the user's specific query. The cost per API call is dramatically lower, making interactive applications feasible. The trade-off is accepting that your retrieval system might not always be perfect, and that the model's context will be incomplete by design.
The arrival of a usable 1M token window doesn't kill RAG. It redefines it. RAG is no longer the only way to handle large documents; it is the economically preferred way for high-volume, interactive use cases. The giant context window becomes an escape hatch for tasks where retrieval is too complex or unreliable.
The 'Lost in the Middle' Problem: When Size Isn't Enough
Attention dilution, or the "lost in the middle" problem, is the biggest skeleton in the closet of large-context models: recall is strong for the very start and end of the window, but drops off for facts buried in the middle, so an advertised window can be much bigger than the effective one.
This is what makes Anthropic's claim of 90% accuracy on retrieval tasks across the full 1M token window so important. They are not just announcing a bigger box; they are claiming to have solved the problem of making the entire box usable. This suggests a fundamental improvement in the underlying Transformer architecture or training methodology, allowing the model's attention mechanism to operate effectively at a scale where previous models failed.
If this benchmark holds up in real-world production scenarios, it's a genuine breakthrough. It means that for tasks that fit the "one-shot analysis" profile, you can trust the model to find the needle in the haystack, no matter where in the million-token haystack you put it. However, practitioners should remain skeptical until they have validated this performance on their own data. The nature of the data, the phrasing of the question, and the position of the key fact can all influence recall.
A Worked Example: Debugging a Large Codebase
Let's make this concrete. Imagine you're debugging a subtle race condition in a multi-service application. The bug only appears when the UserService receives a specific gRPC call from the BillingService while simultaneously updating a user's cache in Redis. The root cause is buried in the interaction between three different parts of the codebase.
The Old Way: RAG and Hope
With a traditional 32K context window, your workflow would be a series of educated guesses:
Embed the Codebase: You'd use a tool to parse your entire codebase into abstract syntax trees (ASTs), chunk them into logical blocks (functions, classes), and embed them into a vector database.
Semantic Search: You'd start by searching for the code that handles the gRPC endpoint in the
UserService.results1 = db.search("gRPC endpoint for UpdateUser").Expand Context: You'd look at the results and realize you also need the client code from the
BillingService.results2 = db.search("BillingService client call to UserService").Guess the Interaction: You remember the Redis cache.
results3 = db.search("UserService Redis cache update logic").Stuff the Prompt: You would then construct a prompt that includes the code snippets from
results1,results2, andresults3, and ask the model to find the race condition.
If any of your searches in steps 2-4 missed the crucial function, the model would have no chance of solving the bug. Your success is entirely dependent on your own ability to play retrieval system.
The New Way: Stuffing and Analysis
With Claude Opus 4.6's 1M token window, the workflow could be radically simpler.
# Hypothetical prompt for Claude Opus 4.6
Here is the complete source code for our application, which consists of three services: UserService, BillingService, and NotificationService.
[... paste entire contents of /src/userservice ...]
[... paste entire contents of /src/billingservice ...]
[... paste entire contents of /src/notificationservice ...]
We are experiencing a race condition. Users are reporting that their profile information occasionally reverts to an older state. We have correlated this with high-volume payment processing.
Our hypothesis is that a gRPC call from the BillingService to the UserService to confirm a payment is interfering with the UserService's own asynchronous cache-warming process, which uses Redis.
Please analyze the entire codebase provided. Trace the lifecycle of a user update initiated by the gRPC endpoint in BillingService and the cache update logic in UserService. Identify the potential race condition and suggest a fix using mutexes or another locking mechanism.Here, the trade-off is clear. The second prompt is vastly simpler to construct. It delegates the "retrieval" part—finding the relevant functions—to the model itself. Given that Opus 4.6 also achieved a state-of-the-art score on the Terminal-Bench 2.0 coding benchmark, it has the reasoning capability to use this context effectively. You pay a premium in cost and latency for this single, powerful query, but it may solve in minutes a problem that would have taken days of painstaking RAG-based detective work.
Where Large Context Wins (and Where It Doesn't)
A million-token context window is a powerful new tool, not a silver bullet. Knowing when to reach for it, and when to stick with retrieval, matters more than the raw size number.
Large Context Wins:
Complex, Self-Contained Analysis: Analyzing a single, massive input like a novel, a full codebase, or a long legal discovery document. The context is self-contained, and the task is a one-shot analysis.
Holistic Code Refactoring: When you need to rename a core class that is referenced by hundreds of other files, providing the whole codebase allows the model to trace and update every dependency correctly.
Onboarding for Agentic Tasks: For an AI agent tasked with a complex goal (e.g., "produce a market research report"), you can front-load the context with all the raw data, source materials, and instructions it will ever need.
RAG Still Wins:
Interactive Q&A over Static Documents: For any chatbot-like interface where users ask multiple questions against a large but unchanging set of documents, RAG is far more cost-effective.
Knowledge Bases Larger than 1M Tokens: If your knowledge base is 10M, 100M, or billions of tokens (like the web), you have no choice but to retrieve.
Real-Time Information: When answers must incorporate information created seconds ago, a RAG system that can index new data instantly will always beat a static context window.
Anthropic itself seems to accept this split. Alongside the 1M window, it introduced a feature called compaction in the API, which lets the model summarize its own conversation history automatically during long-running tasks rather than resending everything at full length. That's a quiet admission that even with a million tokens on tap, nobody wants to pay for a million tokens on every single turn—so the model is given a way to compress what it already knows into something smaller before continuing. It's a hybrid: start big and comprehensive, then let the model boil that down as the conversation goes on, borrowing the cost-efficiency that RAG was built for.
For anyone building with this technology now, the practical takeaway is narrower than the headline number suggests: use the million-token window for one-off, deep-dive jobs—a full codebase audit, a single massive document to summarize, a one-time refactor—where you can afford to pay for a big, expensive request and want the model to see everything at once. Keep RAG for anything a user will ask more than once, anything that changes minute to minute, or any knowledge base too big to fit even in a million tokens. And treat Anthropic's 90% recall figure as a claim to test on your own documents and your own questions before you build a product around it, since recall in a benchmark and recall on your specific contracts, logs, or codebase are not guaranteed to match. The box got bigger. What you choose to put in it, and when, still has to be decided case by case.