David Naylor from Bronco, who was one of the speakers at the Organic Listings Forum session at the Search Engine Strategies conference, advised site owners to have a robots.txt file, even if it’s just an empty file, because Ask Jeeves’ spider seems to favor web sites that have one.

Anyone noticed an improvement with your presence in Ask Jeeves after creating a robots.txt file?

Of course there’s also the side benefit that you’ll eliminate all those “File Not Found” error messages for robots.txt in your server error log, which tend to overwhelm the error log, making it harder to spot more concerning error messages. That assumes of course that you actually examine your error log on occasion. 😉