for someone (like me) using Yahoo's web search api (which is far better than Google's imho), it would be ideal if you left Yahoo's crawler to do it's job here.