Increase Google AdSense Ad Crawler Success

February 11, 2012 · Posted in Site Info 

Web Ad CrawlerGoogle AdSense has recently sent me a reminder to advise me on how to improve the success of Ad Crawler on one of my websites that I hosted. It arises when Google detected a couple of failed crawl requests. I guess the reason is because Google is engaging a third party partner to help them work on crawling.

The following advise was given by Google:

Edit the robots.txt file to allow AdSense crawler by adding these two lines to the very top:

User-agent: Mediapartners-Google

Note, that does not affect or impact the SEO or ranking of the site; but it does provide user with more relevant targeted ads.

SociBook Digg Facebook Google Yahoo Buzz StumbleUpon


One Response to “Increase Google AdSense Ad Crawler Success”

  1. […] Increase Google AdSense Ad Crawler Success […]