At 05:11 1/18/01, Garth Wood wrote:
Yeah, I debated whether or not to put a "robots.txt" file on the Gallery
when I first put it up, and decided against it -- I wanted the Gallery to
be found in search engines.
Just not like this. :-/
Garth
I have not used one either and have no way of testing the efficacy of
one. It's my understanding that one can list what the spiders can and
cannot crawl through by directory and sub-directory. Not certain that it
can still keep from linking to just a jpeg if all the jpegs are moved
(along with everything else to be protected from it) into their own
directory listed in the robots.txt file.
-- John
< This message was delivered via the Olympus Mailing List >
< For questions, mailto:owner-olympus@xxxxxxxxxxxxxxx >
< Web Page: http://Zuiko.sls.bc.ca/swright/olympuslist.html >
|