robots.txt

[I, Brian]

Hi -

May sound alike a strange question - but if I were to set up a simple "robots.txt" file on my site -

User-agent: *
Disallow:


or similar, then should I set up any restrictions to the YaBB folders?

[David]

If you are allowing everything, why have one in the first place?

[I, Brian]

Because it's jamming up the error logs.

But I figured there's a security issue when allowing crawlers to download all folders.

I wondered if anyone here at yaBB was aware of the issue, and whether there were specific concerns that had to be made.

[Spaceman-Spiff]

u can hide the new topic, reply, IM and email links from guest
i think there's a mod for that somewhere...