AllTheWeb News search

AlltheWeb's news search engine was launched in November 2001. At the time, AlltheWeb had an extremely fast re-indexing technology that made news searches very powerful. Almost immediately, it became the most comprehensive search engine available. It indexed more non-English content than any other news search engine at the time and offered many powerful search capabilities. It searched news sites on the web, presumably focused crawling. Clustering in the advanced news search page allows news sources to be limited to specific categories such as sports or technology. After Yahoo purchased AlltheWeb in 2004, AlltheWeb's news search technology became an integral part of Yahoo! News. AlltheWeb's news search was discontinued in 2008.

Language Multilingual

Launched 2001
Closed 2010

Developer Fast Search & Transfer (FAST)

Country of Origin Norway

2001 - 2003 Fast Search & Transfer (FAST)
2003 - 2004 Overture
2004 - 2010 Yahoo Inc

Topic News SeEn

Region No Limitation

Technical functionalities
Robot/Crawler based, algorithmic search
Focused crawling

Used SeEn Alltheweb (FAST) News

Robot: AllTheWeb News spider (Source)

Older Version Internet Archive / WebCite

»This Overture-owned service lets you keyword search against content gathered from news sites across the web. Advanced news search page allows narrowing news sources to particular categories, such as sports or technology. Honorable mention winner for Best News Search Engine from Search Engine Watch in 2002.« Source

Lars Vage, Lars Iselid (2010): »In the aftermath of the events of September 2001, both Google and its main competitor in the fight for the world’s largest index of web pages, AlltheWb, started thinking about creating their own news search. AlltheWeb launched its news search engine in November 2001, whereas the first beta version of Google News didn’t appear until the year after. The AlltheWeb news search engine was extremely powerful and almost immediately became the most comprehensive one available. Unfortunately, it was little known outside of Europe, and Scandinavia in particular. AlltheWeb had access to extremely fast reindexing technology and this became apparent in its news search. It also indexed much more non-English-language content than any of the other news search engines and offered many powerful search features.

[2004 Yahoo bought AlltheWeb]
The news search technology created by FAST for AlltheWeb News was inherited by Yahoo! And became an integral part of Yahoo! News.« Source

FAST (2004): »Real-time, incremental indexing ensures the index is always fresh.This enables query off-load from live databases and news search, lowering TCO of existing applications. Even sub-second freshness can be provided for time-critical content, and this can be combined with unlimited query load.« Source

FAST (2003): »Our News search provides up to the minute news from thousands of news sources all across the globe. We index hundreds of stories every minute.« Source

Searcher (2002): »Limits include: Domain (e.g., CNN.Com), Time (e.g., only articles indexed in the past 12 hours), and Type of news source (U.S., International, Sports, etc.) Comment: The AllTheWeb News spider crawls 3,000 news site from around the globe (English and Non-English language content) continuously. The database is built from a separate "news only" of the Web. URLs remain in the database for 5 days.« Source


Critical points

Features & Functionality


References & further Publications

Wikipedia (EN):
Wikipedia (Others):

Other Sources

Sullivan, Danny (2003): News Search Engines. Online available at: Search Engine Land. URL:
Lars Vage, Lars Iselid (2010): News Search, Blogs and Feeds: A Toolkit URL:

Created: 2013-02-27