Thunderstone Web Site Catalog

✓ *

The Thunderstone Web Site Catalog is a Thunderstone product that has been active in database search and retrieval since 1980. In 1995, the Webinator released a Web Walking and Indexing package, which also provides the content for the Web Site Catalog. At the beginning the catalogue served as a showcase for the Webinator. Thunderstone focused especially on providing content and search technology for other web site and search engine providers. Well-known providers included,,, and Wall St. Journal Web (for realtime news). The Thunderstone Web Site Catalog is still active today with its own crawler and index. The results page displays a link, title, text excerpt and a hit count in percent. In addition to the web search, a news search is offered right from the start.

Language English

Launched 1998
Closed No

Developer Pincus, Kathy and Michael

Country of Origin US America

1998 - [...] Thunderstone Software LLC,

Topic Universal

Region No Limitation

Technical functionalities
Robot/Crawler based, algorithmic search
Search Portal

Used SeEn Thunderstone

Robot: Webinator (Source)

Older Version Internet Archive / WebCite

About: »The 'Distillation' process
Our Webinator web robot is dispatched to each site to obtain their web pages. Then, each site's pages are examined as a whole to determine the principal subject matter areas that would best characterize the entire site. Additions and updates are performed at the rate of about 100 sites per minute. In October 1998 more than 350 gigabytes of content was represented in this database. The growth rate is about 200,000 newly discovered sites per week. Webmasters may use the User-Agent "thunderstone" in their robots.txt file to control access.

The Categorization process
After a site's content has been acquired they are passed to Thunderstone's Automated Categorization Engine. This process seeks to identify the general classifications under which a site belongs. The % figure that follows a site's category indicates the degree of confidence that the categorization engine had in its answer.

While this technology is not always completely accurate, it does perform a task that would otherwise require 75-80 people to accomplish. Each site is assigned up to 4 categories, but our search interface currently only displays the best one.« Source


Stephen E. Arnold / John Turnbull (2008): »Where did the name "Thunderstone" come from? Most search vendors try for tech sounding names or use Greek or Latin to give their systems some cachet. Is your company named after the rock band?

No, Thunderstone is the name given to rocks that have been weathered into a donut shape. The Pacific Islanders believed they were created by a god blasting the hole with lightning. We've always been more focused on solving the customer's problem rather than the academic or theoretical possibilities. Thunderstone connotes our pragmatic approach to search.« Source

Critical points

Features & Functionality

About The Thunderstone Web Site Catalog (2000): »This is one of several experimental search engines produced by Thunderstone's R&D group whose mission is to advance our overall technology leadership. We are very pleased with the results; their little experiment now receives considerably more than 2,000,000 hits per day.

We continuously survey all primary COM, NET, ORG, EDU, and GOV web-servers and distill their contents to produce this database. This is an index of sites not pages. It is very good at finding companies and organizations by purpose, product, subject matter, or location. If you're trying to finding things like "BillyBob's personal beer can page on AOL", try Inktomi or Altavista. This engine attempts to focus on the quality of answers, not the quantity.« Source


Example results page for "sand": Source

References & further Publications

Wikipedia (EN):
Wikipedia (Others): n.a.

Other Sources

Stephen E. Arnold (2008): Thunderstone Software LLC : An Interview with John Turnbull URL:
Hess, Kenneth (2011): 7 Enterprise Search Appliances That Can Save the Day URL:
Company Profile - 1980 - 2000 URL:

Created: 2017-04-23