The History of Retrieval-Augmented Generation

By Victor Hoang · Updated Oct 24, 2026

Picture a researcher in the summer of 1945. She is surrounded by journals she has no hope of reading. She files them by alphabet and number because nothing better exists.

Retrieval-augmented generation (RAG) is a way of making a language model look something up before it answers. The named method comes from a paper posted on May 22, 2020 by researchers at Facebook AI Research, University College London and New York University. The problem underneath it is much older: in 1945 Vannevar Bush was already worrying that people couldn't find what their own civilization had recorded. The throughline: every era built a better way to find the right passage. Every era's limit was what the machine could do with the passage once it found it.

For the wider field, see The History of AI. Now picture a team in 2020 holding a language model that writes fluent paragraphs but can't tell you where it learned a fact or update that fact without retraining. This article follows the line between those two scenes, and says plainly at each step what was a precursor and what was RAG itself. For the working definition, see what retrieval-augmented generation is.

Timeline at a Glance

The Finding Problem (1945-1970s)

Libraries run out of room, so machines learn to index

The oldest ancestor isn't a machine that answers. It's a complaint about finding. In July 1945, Vannevar Bush's essay As We May Think appeared in The Atlantic Monthly. He argued that research output had outgrown anyone's ability to use it, that conventional storage forces you down alphabetical or numerical paths where an item sits in one place, and that the mind works by association instead. His proposed fix was the memex, a desk-sized device holding a person's records on microfilm, with linked "trails" between items. It was never built as described, and it wasn't retrieval-augmented anything. But it states the problem every later system answers.

The first working attempts were statistical. Hans Peter Luhn, who joined IBM in 1941 and later led its information retrieval research, wrote A Business Intelligence System, published in 1958. The page describing his work credits him with automatic indexing, automatic abstracting and what became selective dissemination of information, a computer-based service that routes new documents to the people who need them. Those are precursors: they match text to a profile, and they don't compose an answer.

Gerard Salton carried the idea into a research program. SMART, the automatic text processing system, was first established at Harvard and reached full development at Cornell, where Salton moved in 1965 and co-founded the computer science department. Cornell's tribute lists the vector space model and relevance feedback among his contributions. The vector idea matters here: documents and queries as points in a shared space is the same picture behind today's embedding search, although the 1960s systems counted words rather than learning meaning.

Then came the weighting insight. In 1972, Karen Spärck Jones published "A Statistical Interpretation of Term Specificity and Its Application in Retrieval" in the Journal of Documentation (volume 28, issue 1, pages 11-21). Its abstract argues that terms should be weighted by how often they appear across the collection, so a match on a rare term counts for more than a match on a common one. Weighting by collection frequency is the idea behind inverse document frequency in TF-IDF, and decades later the Dense Passage Retrieval authors still described sparse methods such as TF-IDF or BM25 as the standard to beat.

This era solved ranking: given a query, which documents should come first? It left the reader to do the last step. You got a list, then you read.

For teams today: most RAG quality problems are still ranking problems. Before you tune the model, check whether the right passage is even in the top results.

The Answer Machines (1961-2011)

Programs stop returning lists and start returning answers

In May 1961, at the Western Joint Computer Conference in Los Angeles, four researchers at MIT's Lincoln Laboratory presented BASEBALL: An Automatic Question-Answerer. Bert Green, Alice Wolf, Carol Chomsky and Kenneth Laughery built a program that read a question in ordinary English from punched cards, looked the words up in a dictionary, analyzed the syntax, extracted the matching data and printed the answer. Its data was the month, day, place, teams and scores for every American League game in one year, and it handled questions such as where each team played on a given date. It was written in IPL-V and restricted to single-clause questions. The authors framed it as a first step toward business executives, commanders and scientists asking computers questions in plain English.

BASEBALL is a precursor in the cleanest sense. It retrieved from a structured store and answered from what it found, but the store was a small table and the language was tightly limited. Open text was a different problem.

That problem got a shared yardstick in the 1990s. The Text REtrieval Conference, run by NIST, began in 1992 as part of the TIPSTER Text program, co-sponsored by NIST and the U.S. Department of Defense, to support retrieval research with large test collections. In 1999 TREC-8 added a question answering track, and its proceedings list entries from groups including AT&T, Xerox, the University of Sheffield and the University of Maryland. The pattern those teams used is the ancestor of RAG's two halves: find candidate passages with a retrieval engine, then extract or compose the answer.

The most visible descendant came from IBM. David Ferrucci and colleagues described the DeepQA project in Building Watson, published in AI Magazine in July 2010. The paper reports about three years of work by a core team of roughly 20 researchers, aiming at expert-level accuracy, confidence estimation and speed, and presents DeepQA as a foundation for combining many question answering techniques. In February 2011 Watson played three days of televised Jeopardy! matches against Brad Rutter and Ken Jennings. It's a sibling of RAG rather than an ancestor of the method: a large pipeline engineered around many question answering techniques, built for one quiz show.

This era solved the shape of the task: question in, answer out, with evidence behind it. It left two gaps. Every component was hand-built, and none of it could write fluent prose. The rule-based wave is covered in the history of expert systems, and the language side in the history of natural language processing.

For teams today: an answer without evidence is a guess. The old question answering systems were judged on whether the answer was supported, and your internal assistant should be too.

The Dense Turn (2017-2020)

Neural readers arrive, then retrievers that learn meaning

The modern line starts in March 2017, when Danqi Chen, Adam Fisch, Jason Weston and Antoine Bordes posted Reading Wikipedia to Answer Open-Domain Questions, later published at ACL 2017 in Vancouver. Their system, often called DrQA, treated Wikipedia as the only knowledge source and split the job in two: a search component based on bigram hashing and TF-IDF matching, and a multi-layer recurrent neural network that found the answer span in the retrieved paragraphs. The retriever was still the old statistical kind. The reader was new.

Meanwhile the tooling for fast vector search arrived. In February 2017, Jeff Johnson, Matthijs Douze and Hervé Jégou posted Billion-scale similarity search with GPUs, the paper behind the open-source FAISS library. It reports a nearest-neighbor implementation 8.5 times faster than the prior GPU state of the art, and a graph connecting one billion vectors built in under 12 hours on four GPUs. Without cheap similarity search over huge sets of vectors, learned retrieval would have stayed a lab curiosity.

The learned retriever followed in 2020. On February 10, Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat and Ming-Wei Chang posted REALM: Retrieval-Augmented Language Model Pre-Training. They added a latent knowledge retriever that lets a model attend to documents from a large corpus during pre-training, fine-tuning and inference, and report gains of 4 to 16 percent absolute accuracy over earlier methods on three open-domain question answering benchmarks. Two months later, on April 10, Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen and Wen-tau Yih posted Dense Passage Retrieval for Open-Domain Question Answering, published at EMNLP 2020. A simple dual-encoder, trained on a small number of questions and passages, beat a strong BM25 system by 9 to 19 percent absolute in top-20 passage retrieval accuracy. Dense retrieval alone, with no keyword matching, was good enough.

Both are precursors to the named concept, not the concept itself. REALM retrieves while pre-training a model for extraction tasks; DPR retrieves but doesn't generate. The generation side was exploding at the same moment: on May 28, 2020 Tom Brown and 30 co-authors posted Language Models are Few-Shot Learners, describing GPT-3 with 175 billion parameters. Fluent text was now cheap. The history of that wave is in the history of large language models and the history of transformer models.

This era solved the match between a question and a passage even when they share no words. It left the generation question open: a retriever that finds the right passage doesn't write the answer.

For teams today: keyword search and embedding search fail in different ways. Test both on your own documents before choosing one.

The Lewis Paper (2020-2022)

Retriever and generator are trained together and given a name

Six days before the GPT-3 paper, the idea got its name. On May 22, 2020, Patrick Lewis and eleven co-authors posted Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. The first page lists Facebook AI Research, University College London and New York University. The paper appeared at NeurIPS 2020, volume 33. The authors describe general-purpose fine-tuning for models that combine pre-trained parametric memory (the language model's own weights) with non-parametric memory (an external index). Their parametric memory was the seq2seq model BART, and their non-parametric memory was a dense vector index of Wikipedia reached through a DPR retriever. The index held 21 million passages, each a 100-word chunk of a December 2018 Wikipedia dump, stored with FAISS.

The abstract frames the motivation plainly. Large language models store facts in their parameters, but their ability to access and precisely manipulate that knowledge is limited, and providing provenance and updating knowledge were open problems. The paper reports state-of-the-art results on three open-domain question answering tasks and more specific, diverse and factual generated text than a parametric-only baseline. It also demonstrates what teams later cared about most: because the knowledge sits in an index, the index can be hot-swapped to update the model without retraining. Two members of the DPR author list, Lewis and Karpukhin, are also on this paper, which is why the pieces fit together so cleanly.

Two more systems showed the direction. On December 8, 2021, a DeepMind team led by Sebastian Borgeaud, Arthur Mensch and Jordan Hoffmann posted Improving language models by retrieving from trillions of tokens. Their RETRO model conditions on chunks retrieved from a database of 2 trillion tokens and, the authors report, performs comparably to GPT-3 and Jurassic-1 on the Pile with about 25 times fewer parameters. Then on August 5, 2022, Gautier Izacard, Patrick Lewis and eight co-authors posted Atlas, which reaches over 42 percent accuracy on NaturalQuestions from only 64 training examples and, the authors report, beats a 540 billion parameter model by 3 percent with 50 times fewer parameters.

This era solved the update and provenance problem for facts, and showed retrieval could substitute for sheer model size. It also created the next problems: what to put in the index, how to cut documents into chunks, and how much retrieved text a model can actually use. For the business-facing version, see the RAG assistant pattern.

For teams today: the main business case in the original paper is still the main one. If your facts change, keep them in a store you can edit, and keep the model for language.

The Enterprise Wave (2023-2024)

Everyone builds a RAG pipeline, then finds out how it breaks

By 2023 the RAG pattern was widely studied and widely built, and the research turned to its weak points. On July 6, 2023, Nelson Liu and colleagues, including Fabio Petroni from the original RAG author list, posted Lost in the Middle. They found that model performance is typically best when relevant information sits at the start or end of the input and drops when it sits in the middle, even for models built for long contexts. Retrieving more passages isn't automatically better.

The field also needed a map. On December 18, 2023, Yunfan Gao and nine co-authors posted a survey of RAG for large language models that organizes the work into three paradigms: Naive RAG, Advanced RAG and Modular RAG. Then on January 11, 2024, Scott Barnett and four co-authors posted Seven Failure Points When Engineering a Retrieval Augmented Generation System, an experience report from three case studies in research, education and biomedicine. Their conclusions are the kind a working team recognizes: validation is only feasible during operation, and robustness develops over time instead of being built in at the start.

A related line looked at what plain retrieval can't do. On April 24, 2024, Darren Edge and nine co-authors posted From Local to Global: A Graph RAG Approach to Query-Focused Summarization. They argued that standard RAG handles specific questions well but fails on global questions about a whole corpus, such as its main themes. Their method has a model build an entity graph from the documents, summarize clusters of related entities, and combine partial answers at query time; on datasets of about 1 million tokens they report gains in comprehensiveness and diversity over a conventional RAG baseline.

This era solved the engineering question of how to run RAG, and replaced the demo with a list of known failure modes. It left the pipeline itself rigid: retrieve once, then generate.

For teams today: treat a RAG system as a product you operate, not a feature you ship. Keep a fixed set of real questions, measure retrieval separately from generation, and review the misses every week.

The Agentic Turn (2023-present)

Retrieval becomes a decision the model makes, not a fixed step

The newest chapter overlaps the last one, since both began in 2023. Standard RAG retrieves a fixed number of passages whether or not it needs them. On October 17, 2023, Akari Asai and four co-authors posted Self-RAG, which trains a single model to retrieve on demand and to critique both the retrieved passages and its own output using special "reflection tokens". The authors report that their 7 billion and 13 billion parameter versions outperform ChatGPT and retrieval-augmented Llama2-chat on several tasks, with gains in factuality and citation accuracy for long-form generation.

On January 15, 2025, Aditi Singh and four co-authors posted Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG. It describes agentic RAG as embedding autonomous agents into the RAG pipeline, using reflection, planning, tool use and multi-agent collaboration to get past the static workflows of traditional RAG, and it lists open challenges in evaluation, coordination, memory management, efficiency and governance. The agent side of that merger is told in the history of AI agents, and the practical versions are in what agentic RAG is and RAG for AI agents. Assistants built on the same retrieval layer are the subject of the history of AI copilots, and the broader generative wave is in the history of generative AI.

The history is still open. The live problems are the old ones, and Bush would recognize them: what to find, how to judge it, and who decides. For where the field may go, see the future of AI history.

For teams today: add judgment to retrieval only where a fixed pipeline fails. A decision step costs time and money, and it adds one more place to audit.

Key People

Key Facts

• The RAG paper was posted on May 22, 2020 and appeared at NeurIPS 2020.
• Its index held 21 million 100-word Wikipedia passages, reached through a DPR retriever and read by BART.
• Dense Passage Retrieval beat a strong BM25 system by 9 to 19 percent absolute in top-20 retrieval accuracy.
• RETRO retrieved from a database of 2 trillion tokens and matched GPT-3 on the Pile with about 25 times fewer parameters.
• BASEBALL, presented in May 1961, answered English questions about one year of American League games.
• TREC began in 1992, and its 1999 edition added a question answering track.

The Pattern Behind the History

Across six eras, retrieval and generation kept trading places as the bottleneck. We call it the Find-Then-Say Pattern. It's our synthesis of the history above, not a research finding.

1. Finding comes first. Bush, Luhn, Salton and Spärck Jones all worked on finding the right record before anyone tried to say anything about it.
2. Answers need evidence. BASEBALL, TREC and Watson were judged on whether the answer was supported by data.
3. Language models fixed the saying. GPT-3 made fluent output cheap, and exposed that the model couldn't show where its facts came from.
4. Joining the two made it updatable. RAG, RETRO and Atlas moved facts into an index that can be changed without retraining.
5. The joint becomes the weak point. Context position, chunking and failure points, and then agents deciding when to retrieve, are all about the handoff between the two halves.

What It Teaches Teams Today

• Start with the finding problem. Most of a RAG system's quality is retrieval quality, as it was in 1972.
• Keep facts outside the model. The 2020 paper's biggest practical point is that an index can be swapped, and a model's weights can't.
• Demand evidence. Treat any answer without a source passage as unverified.
• Test the whole handoff. More retrieved text isn't better, and failures often come from how passages are cut, ranked and ordered.
• Add agent behavior last. Fixed pipelines are easier to audit. Let a model decide when to retrieve only when the fixed version stops being enough.

RAG shows that a very old question, how do you find the right record, keeps returning in new clothes as the machine on the other end gets more capable. Ask of any new system: which half is new here, the finding or the saying? For the wider timeline, see the AI timeline.