Crawl Budget

What is the Crawl budget? – Definition

Crawl budget determines how much resources – time and computing power – Google robots spend on indexing a given site. This reflects the number of pages added to the search engine index and the indexing frequency. The crawl budget is influenced, among others, by:

  • crawl rate limit – the number of pages crawled by search engine robots in a short time,
  • crawl health – low server response time, correct response codes, loading speed,
  • crawl demand – the demand for (re)crawling, which depends, among others, on the frequency of content updates, web traffic,
  • page size, use of JavaScript code – pages with JS take more indexing budget.

Where Can I Check the Crawl Budget of My Website?

In Google Search Console, go to the “Legacy tools and reports” section and choose “Crawl stats”. You will find there 3 charts:

  1. Pages crawled per day,
  2. Kilobytes downloaded per day,
  3. Time spent downloading a page (in milliseconds).

Crawl budget - definition

Crawl statistics

These are data from the last 90 days. Based on them, you can specify how the crawl budget has changed during this period.

Read related posts

Get a free quote

SEO SEM Agency based in Europe