My latest Search Engine Land column just went live. It's about Crawl Budget and how it may work. Having re-read it, though, I want to make sure I add even more qualification to what I wrote:
I talk about crawl budget a lot, but the most important part of this article, really, is 'making the most of your crawl budget'. That's what Vanessa Fox refers to as 'crawl efficiency'. If you have a huge crawl budget but 90% of it is sucked up by duplicate content and such, then you won't see much benefit.Search engines apply PageRank to individual pages, not to entire sites. However, I definitely see evidence that they look at some version of average PageRank, or domain authority, or whatever you want to call it, site-wide. That appears to impact crawl budget. There may not be a causal link - it may simply be that sites with lots of pages that all have higher PageRank get higher crawl budgets as a side effect of deep links to individual site pages.This is a very deep, geeky topic - I hope I don't ruffle any feathers too badly, but I also hope that a lot of people smarter than I will pick up the discussion, so we can all learn something.
No comments:
Post a Comment