Crawl Budget

What is the Crawl budget? – Definition

Crawl budget determines how much resources – time and computing power – Google robots spend on indexing a given site. This reflects the number of pages added to the search engine index and the indexing frequency. The crawl budget is influenced, among others, by:

  • crawl rate limit – the number of pages crawled by search engine robots in a short time,
  • crawl health – low server response time, correct response codes, loading speed,
  • crawl demand – the demand for (re)crawling, which depends, among others, on the frequency of content updates, web traffic,
  • page size, use of JavaScript code – pages with JS take more indexing budget.

Where Can I Check the Crawl Budget of My Website?

In Google Search Console, go to the “Legacy tools and reports” section and choose “Crawl stats”. You will find there 3 charts:

  1. Pages crawled per day,
  2. Kilobytes downloaded per day,
  3. Time spent downloading a page (in milliseconds).

Crawl budget - definition

Crawl statistics

These are data from the last 90 days. Based on them, you can specify how the crawl budget has changed during this period.

Was this definition helpful?
(0)
Author
Delante is an online marketing agency specialized in generating website traffic from search engines and paid ads (SEO / SEM). Over 80% of our clients are from rapidly growing e-commerce industry.

RECENTLY ON OUR BLOG