Use nofollow

We recently were looking through log files and found that Google bots causing errors when going places like /sites/all/modules/civicrm/bin/deprecated/action.cronjob.php

So after tightening up access to the site I'm wondering if in addition it might be a good idea to put in a robot.txt file in civicrm's root with a nofollow, or maybe add it into the page head of some/all pages using a meta tag (http://www.robotstxt.org/meta.html). I don't know the details of SEO and bots, but I imagine we would be comfortable having bots look at contribution, event, etc pages, but not the admin pages or bin directories.