Re: searchable index of the web

Ray Stell (
Wed, 30 Jun 93 15:59:37 EDT

> I have written a perl script that wanders the WWW collecting URLs, keeping
> tracking of where it's been and new hosts that it finds. Eventually,
> after hacking up the code to return some slightly more useful information
> (currently it just returns URLs), I will produce a searchabe index of this.
> There is a complete list of all the sites it has found at
> <a href="">
> A complete list of sites found by the W4 (World Wide Web Wanderer)
> </a>
> I'll announce here when we get this index properly running, however it probably
> won't be until sometime in August, as I am going on vacation. Until then...
> Matthew Gray
> Visit the SIPB WWW Plexus Server. URL:

This sounds good, better than depending on my memory. Will w4
crack open an log the <Title> or is this directory tree oriented?
If I'm running a server, how will you know to wander to it?
Should there be a web of wanderers (w5?) so w4e in Europe can
feed from w4u in the US, etc? Why is this topic such a quiet one?
Ray Stell (703) 231-4109