The Google Webmaster Blog has some update info on how the googlebot interacts with your web site and some tips on how you can save some bandwidth with the Googlebot cacheing your web site.
Link: http://googlewebmastercentral.blogspot.com/2006/09/better-details-about-when-googlebot.html |
Googlebot seems to be ill theses days : few days ago it crawled one of my pages which I have deleted (not really, I just return a 404 error with the "header()" PHP function) and what a surprise to see that it reindexed the page... with the error message :-S Really strange
PS : the HTTP header 404 was and is yet good... |