NWI- Nordic WebIndex
|Country of Origin|
|Robot:||Combine System (Source)|
|Older Version||Internet Archive / WebCite|
|About the robot: »Combine is an open system for crawling [harvesting and threshing (indexing)] Internet resources. It can be used both as a general and focused crawler. The name is derived from the combine-harvester since the two perform their jobs in a similar way.
Its is implmented in Perl and is easily configured for most types of Web-crawling.
The Combine Robot was first released (spring '98), and was primarily used whithin the nordic NWI services (out of commision). A modified version of Combine was also used in the Kulturarw Web archival project.
We envisage it as being useful for regional web indexing in general.
SOSIG use Combine as harvester for their Social Science Search Engine, based on urls from the main catalogue.
Combine is beeing used as a focused crawler by the Materials.dk, an automatic subject portal in the topical area of materials science.
In the EU project ALVIS it is used (and further developed) as a focused crawler in order to produce data collections for the topic-specific semantic-based search engine that ALVIS develops. There are some ALVIS demos are available.
The EU project DELOS is using Combine to create a database of all Web-pages from all partners. The database is searchable either from the DELOS home-page or from a native interface.« Source|
Features & Functionality
References & further Publications