A Distributed Web Querying System

based on Web Automata

We present an experimental distributed Web querying system based on simple, cooperative information agents, called Web automata. The system implements distributed runs of Web automata as defined in [STVdB02]. We describe how to install the system and provide some initial examples.
 
 

Introduction

Current Web querying systems follow a central-processing strategy: Web queries are computed by downloading raw data from the Web and processing the data in a centralized fashion, e.g., by means of a single search-and-evaluation engine running at the user's site. Since not all the data downloaded for evaluation is really required for computing the answer to a given query, this strategy leads to unnecessary data transmissions, thereby increasing the danger of network congestion. A further disadvantage is that the computational load is entirely on the user's site, while the resources of the Web (e.g., the participating Web servers) are used merely to retrieve and forward data, rather than for data evaluation and processing.

Here, we introduce an experimental Web querying system which uses the computational resources of the Web. The system is based on Web automata [STVdB02], which can be viewed as simple, cooperative information agents. Instead of downloading and centrally processing data, the system sends Web automata to the Web servers holding the data. The data is then evaluated and processed by the transmitted Web automata, which run on the participating servers and thus use the computational resources of those servers. From the viewpoint of the user, Web automata compute Web queries as follows. When a Web automaton is  started at some Web page (e.g., the user's homepage), it first distributes copies of itself in a finite portion of the Web by means of a straightforward recursive procedure: upon creation at a page p, the automaton creates a copy of itself at every page which p links to, except if the page is already equipped with a copy. The procedure stops when a certain depth (to be specified by the user) is reached. Once every page in the vicinity of the starting page is equipped with a copy, all automata begin their computations concurrently. Each automaton sends messages to the automaton which created it, following a simple protocol (for details the reader is referred to [STVdB02]). The result of the distributed computation (i.e., an answer to the Web query) is output by the automaton at the starting page.

Outline. The system can be download on the next page. Page 3 explains how to set up a Web server supporting Web automata, and how Web automata are started on such a server. Page 4 provides supplementary information concerning the current implementation of Web automata and their distributed runs.

Contact. Please send bug reports, questions, suggestions, etc. to marc.spielmann@luc.ac.be.
 

Apr 30, 2002