Re: Byte ranges (actually robots)

Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Next message: Gavin Nicol: "Re: Byte ranges (actually robots)"
Previous message: Dave Hollander: "Browser Displayed URL"

Darren New (dnew@sgf.fv.com)
Thu, 25 May 1995 15:38:44 +0500

> and I can't just flag anyone who fetches robot.txt as a robot, since of
> course a person may fetch it too.

And of course not all robots will fetch robot.txt, either. Welcome to
the internet! :-)