Knocking off some of the dust here... wanted to share two quickies:
I finally figured this out, and it may be beneath you, but let's say you have a web document root that's shared between ports 80 and 443 (iow, http and https go to the same place). Then your site gets spidered by the search engines and they put a bunch of your stuff in a supplemental index because it's redundant. Since there's only one robots.txt file, you can't easily say IF https, then go away w/o saying the same thing to the http version. So what do you do? Create a robots-ssl.txt file and then in your ssl apache configuration, use Alias:
Then http://www.example.com/robots.txt and https://www.example.com/robots.txt have two different contents, while sharing the same web root directory!
You probably already knew this, didn't you?
Ok, next up
PS: How ironic (ok, English nerds, coincidental