Searchable Indexes: LISTEN NOW!
Wed, 30 Jun 93 22:52:53 -0400

>1) Sites should be able to just say ``No''. If I don't want my ftp
>site to appear on archie, I can say so -- the web should be the same.

The way I've dealt with the problem is to list only sites that seem to
be public. The list of sites that I have are gotten from public
sources and I try and cull sites with any sort of restriction on them.

>2) Sites should be able to say ``Go away -- I've done it all for you''
>and have the equivalent of ftp:/ls-lR.Z in /

The trouble with ls-lR, is that it doesn't contain keyword and breadth
information. Without such information, an index is very difficult to
deal with.