The History of Search Engines: From the Pinakes to AI Answers

By Victor Hoang · Updated Oct 24, 2026

In March and April 1994, one of the first web search engines, the World Wide Web Worm, answered queries over an index of pages and documents. Sergey Brin and Larry Page later recounted it in their Google paper. The web had stopped being something to browse. It had become something to search.

Web search began in 1990 with Archie, a tool from McGill University in Montreal. It indexed file names on public File Transfer Protocol servers. By November 1997 the same paper says AltaVista claimed roughly 20 million queries a day. Search took its modern shape in 1998, when Brin and Page of Stanford described Google. It ranked pages by the links pointing at them. The throughline is a loop: each time the web outgrew the last way of finding things, someone built a better index. The best index soon found a business model that tested its own honesty.

That's the problem this whole history keeps answering.

Timeline at a Glance

The Catalog Idea (3rd century BCE-1960s)

People built lookup tools long before machines could search

Picture the shelves of Alexandria in the 3rd century BCE. The poet and scholar Callimachus of Cyrene compiled the Pinakes ("Tablets"), which the World History Encyclopedia describes as a bibliographic inventory of all Greek literature. It divided writing into poetry and prose and listed each author, the author's other works and biographical details, and it ran to about 120 books. The work is lost today, so we know it only from later writers. Its core question, how do you find the one text you need in a collection too big to browse, is the question every search engine has asked since.

None of what follows in this era was a search engine. They are precursors, and the honest way to read them is as the same problem solved with whatever tools each age had.

Around 1230, Dominican friars finished the first verbal concordance of the Bible. The Catholic Encyclopedia says Hugh of Saint-Cher guided the work and is said to have been helped by about 500 fellow Dominicans. It held no quotations, only the book and chapter where each word appeared. Verses weren't numbered yet, so the friars split every chapter into seven nearly equal parts marked with the letters a, b, c and so on. That is a keyword index, built by hand.

The nineteenth century industrialized the shelf. OCLC's timeline records that the first edition of the Dewey Decimal Classification was published anonymously in Amherst, Massachusetts, in 1876. In 1895, the International Office of Bibliography was created after the first international bibliography conference in Brussels, and its goal, according to the Mundaneum, was the Universal Bibliographic Repertory, "a tool that allows for the retrieval of any book in the world." The Mundaneum says Paul Otlet and Henri La Fontaine met in 1890 and conceived the project together.

Then came the machine age of ideas. In July 1945, Vannevar Bush published "As We May Think" in The Atlantic Monthly, and the W3C's copy of the essay defines a memex as "a device in which an individual stores all his books." Bush argued that the human mind "operates by association" and that selection by association could one day be mechanized. In November 1958, at a six-day international conference on scientific information in Washington, D.C., IBM's Hans Peter Luhn demonstrated a machine that built an alphabetical index of keywords shown in context, called KWIC. IEEE Spectrum notes that by the early 1960s it sat at the center of hundreds of computerized indexing systems and calls it the era's equivalent of a search engine.

The field itself took shape around Gerard Salton. Cornell's memorial says his SMART system was established at Harvard and came into full bloom at Cornell, where he moved in 1965 and helped found the computer science department. It lists the vector space model, term weighting and relevance feedback among his topics. Hold on to the vector space model, because it turns up again in 1994.

What this means for teams today: The oldest lesson in search is that people don't need more content, they need a reliable way into it. If your own knowledge base has no index, you're back at Alexandria.

The Index Rush (1990-1997)

Files first, then pages, then a crawl of everything

Think of a university system administrator in Montreal, asked to track down free software scattered across the early Internet. McGill's history site says Alan Emtage wrote an open-source program to automate that hunt and named it Archie. The site tags the work 1989 and says Archie drew half of Canada's web traffic until the mid-1990s. Poynter dates the release to 10 September 1990, so treat 1989 as the build and 1990 as the public launch. Archie indexed files on anonymous FTP sites. It found file names, not the words inside pages, because the web barely existed yet.

Menus came next. RFC 1436, published in March 1993 by a University of Minnesota team, describes the Gopher protocol as "designed for distributed document search and retrieval." The web, whose own origin story is told in The History of the World Wide Web, then supplied something Archie and Gopher lacked: pages linked to pages. That made a new kind of program possible, one that followed the links.

By 1994 the crawlers had arrived. Brian Pinkerton's WebCrawler paper, presented at the Second International WWW Conference, opens with a comparison any librarian would recognize: "Imagine trying to find a book in a library without a card catalog." WebCrawler indexed both document titles and content using a vector space model, Salton's idea in a new setting. Pinkerton reported an index of nearly 50,000 documents on more than 9,000 servers answering over 6,000 queries a day, and about a quarter of a million queries from over 23,000 users in its first six months.

A different idea grew in a campus trailer. The Centre for Computing History says Stanford doctoral candidates Jerry Yang and David Filo started "Jerry and David's Guide to the World Wide Web" in January 1994 and incorporated Yahoo in March 1995. It was a human-edited directory. Brin and Page later judged such lists "subjective, expensive to build and maintain, slow to improve," yet users loved them: a federal court later recorded that Yahoo already had hundreds of millions of users in 1998. Lycos, built at Carnegie Mellon by Michael Mauldin, took the crawler route, and the university's news office later reported that $48 million of its more than $60 million in royalties and capital gains came from selling Lycos stock.

AltaVista raised the bar. A Red Hat podcast transcript describes Louis Monier, a Digital Equipment Corporation computer scientist, building a crawler called Scooter, and says AltaVista launched in December 1995 and took 300,000 queries on its first day. According to the same transcript, DEC didn't know how to profit from search.

The new problem was quality. Brin and Page wrote that automated engines relying on keyword matching "usually return too many low quality matches," and that some advertisers tried to mislead them. Counting words was no longer enough. The dot-com boom was about to hand every company a reason to game the count.

What this means for teams today: Curation scales badly and automation scales noisily. Most internal search, from a wiki to a CRM, still has to choose between a tidy directory and a messy full-text index, and the history says you eventually need both.

The PageRank Turn (1996-2000)

Links become the votes that rank the web

This era overlaps the last one, because Google was being built while AltaVista was winning. Google's own account says Larry Page was visiting Stanford in 1995 and Sergey Brin was assigned to show him around. By 1996 they had formed a partnership and built a search engine that ranked pages by their links, which they called Backrub. They renamed it Google, and the paper explains the choice: it's a common spelling of googol, or 10 to the power of 100, and fit their goal of very large-scale search.

The idea fits in a sentence. In the paper, PageRank is the probability that a "random surfer" who keeps clicking links lands on a page, with a damping factor the authors usually set to 0.85. Their prototype held a full-text and hyperlink database of at least 24 million pages, and they had built maps of as many as 518 million links. Page filed his patent application, "Method for node ranking in a linked database," on 9 January 1998, with Stanford as the original assignee, and it was issued on 4 September 2001.

The money arrived in August 1998, when Sun co-founder Andy Bechtolsheim wrote a $100,000 check, and the company worked first from a garage in Menlo Park. Google's story page calls that the moment Google Inc. "was officially born." Google's 2004 IPO filing says it was incorporated in California in September 1998, so the check and the incorporation sit a month apart.

The most telling passage in the paper is a warning. Appendix A says "advertising funded search engines will be inherently biased towards the advertisers," and it cites OpenText, which was reported to be selling the right to appear at the top of results. Two years later, Google would build an ad business on its own results page.

What this means for teams today: The winning move was a new signal, not a bigger index. When every competitor measures the same thing, look for the evidence nobody is counting yet, as links were in 1996.

The Auction Years (1998-2008)

Clicks become the product, and search starts to pay for itself

Return to 1999. A patent filed on 28 May 1999 by inventors who assigned their rights to GoTo.com, US 6,269,361, describes a search results page where advertisers bid for position. Its abstract says "A higher bid will result in a higher rank value," advertisers pay each time a searcher clicks, and paid listings are marked as such. The patent issued on 31 July 2001. The wider story of paid placement, from banners to programmatic auctions, is in the history of online advertising.

On 23 October 2000, Google announced AdWords, a program that let any advertiser buy keyword advertising that appeared instantly on the google.com results page, through a self-service system. The release says the beta had drawn about 350 businesses and agencies, and that AdWords extended a premium sponsorship program announced in August. Four years later the filing quoted above shows what it had become: advertising supplied about 95% of Google's 2003 net revenues through AdWords and AdSense.

Search had solved its funding problem, and created a new discipline. If a ranked list decides who gets found, companies will pay people to influence it. Paid search and search engine optimization both live inside that tension, and they are the two skills behind modern inbound lead generation and, today, AI-assisted paid advertising.

What this means for teams today: Whoever owns the ranking owns the market's attention, and the rules change when the engine's incentives do. Budget for the channel you pay for and the one you earn, and don't confuse them.

The Answer Engine Years (2009-2021)

Search stops listing links and starts answering

On 28 May 2009, Microsoft's press release unveiled Bing as a "Decision Engine," aimed first at four jobs: making a purchase decision, planning a trip, researching a health condition and finding a local business. The release cited a comScore study finding that as many as 30 percent of searches were abandoned without a satisfactory result, and said Bing would be fully deployed worldwide on 3 June.

It didn't change the standings. The 2024 opinion in United States v. Google records that by 2009 Yahoo had stopped crawling the web and would deliver Bing's results, and that Google's share of general search queries rose from 80% in 2009 to 89.2% by 2020, or 94.9% on mobile. The court found that Bing is the only rival that crawls the web and builds its own index, with Yahoo and DuckDuckGo syndicating from it. It also found that Google paid more than $26 billion in revenue share in 2021 to secure default placement on browsers and phones, the same default-setting fight told in The History of Web Browsers. The fight was over distribution as much as quality.

Meanwhile the engines learned to understand. On 16 May 2012, Amit Singhal announced the Knowledge Graph, a model with more than 500 million objects and 3.5 billion facts, and summed up the shift as "things, not strings." On 25 October 2019, Pandu Nayak said Google was applying the language model BERT to ranking, calling it "one of the biggest leaps forward in the history of Search," and said it helped with one in ten English searches in the US.

The two sources measure different things, a court exhibit on queries and a traffic tracker, so compare the direction, not the decimals. The new question was whether an answer shown on the results page still sends the reader on to the site that supplied it. That question sets up the last era.

What this means for teams today: Structured, unambiguous facts about your company, products and people are now an input to search, not decoration. Treat them with the care you give a price list. AI SEO covers how this work changes day to day.

The AI Answer Age (2022-present)

Chat models read the web and write the reply

On 7 February 2023, Microsoft's Yusuf Mehdi announced a new AI-powered Bing and Edge, available in preview, running on a next-generation OpenAI model customized for search and combining search, browsing and chat. On 14 May 2024, Google's Elizabeth Reid wrote that AI Overviews were reaching all US users, with the goal of more than a billion people by the end of the year. On 31 October 2024, OpenAI introduced ChatGPT search, pitched as "fast, timely answers with links to relevant web sources."

The legal record caught up with the technology. In its December 2025 opinion on the final judgment, the court wrote that generative AI "burst on to the scene as a formidable nascent threat" to general search engines, and that the remedies phase became as much about stopping Google's dominance in search from carrying over into generative AI as about competition among search engines.

For now the incumbent is still far ahead. StatCounter's worldwide ranking for September 2026 shows Google at 89.94%, Bing at 5.29%, Yahoo at 1.36%, Yandex at 1.3%, DuckDuckGo at 0.74% and Baidu at 0.44%. Whether answer products erode that number is the open question of this era. The history suggests the answer will come from distribution and trust as much as from model quality. For the longer story of the models themselves, read the history of AI, and for the practical shift in visibility work, see AEO vs SEO.

What this means for teams today: Being found now includes being quoted. Write pages that answer a real question in the first two sentences, and check how your brand is described by the assistants your buyers use.

Key People

Key Facts

Key Facts: Search Engine History

• The Pinakes of Callimachus ran to about 120 books, according to the World History Encyclopedia.
• Archie was released on 10 September 1990, per Poynter.
• Google's 1998 prototype held a database of at least 24 million pages, per the Brin and Page paper.
• AdWords launched on 23 October 2000 after a beta of about 350 advertisers, per Google's release.
• Google held 89.2% of general search queries by 2020, according to the federal court's 2024 opinion.
• Worldwide, Google's search share was 89.94% in September 2026, per StatCounter.

The Pattern Behind the History

Across more than two thousand years, the history of search follows four repeating moves. We call it the Findability Loop. It's our own synthesis of the sources above, not a published study.

1. Overflow. A collection grows past what anyone can browse: Alexandria's scrolls, the Bible's text, a web of 110,000 pages.
2. Index. Someone builds a lookup tool, whether a Pinakes, a concordance, a card system or a crawler.
3. New signal. The index only improves when it uses evidence the last one ignored: author and subject, keywords in context, links.
4. New money. Whoever controls the index finds a way to be paid, and the payment bends the index, which creates the next overflow of noise and the next search for a better signal.

What It Teaches Teams Today

• Findability is a product feature. Customers, employees and AI assistants all reach you through some index. If you can't say which, start there.
• Look for the uncounted signal. PageRank beat keyword counting by measuring links. Ask what your own reports ignore.
• Distribution beats quality for long periods. The court record shows defaults and revenue share shaping who wins. Don't rely on one channel, whether it's a search engine or an assistant.
• Write for questions. Answer-first pages serve readers, snippets and AI answers alike. Our guide to inbound lead generation shows how to turn that into pipeline.
• Keep reading the foundations. Search sits on top of DNS, markup standards such as those in the history of HTML and web standards, and the wider arc told in the history of the Internet.

Keep reading

• Internet timeline: the dated milestones from telegraph lines to AI agents.
• The Dot-Com Bubble: What Happened and Why: the boom that funded the early engines and the crash that followed.
• The History of DNS: how names became addresses.
• AI SEO Explained: how AI changes keyword research, on-page and technical SEO.

Search began as the librarian's answer to a room too full to browse, and every era since has replaced the librarian's tool with a bigger one. The tools changed. The question, and the temptation to bend the answer for money, stayed the same.