Crawl Budget

What is the Crawl budget? – Definition

Crawl budget determines how much resources – time and computing power – Google robots spend on indexing a given site. This reflects the number of pages added to the search engine index and the indexing frequency. The crawl budget is influenced, among others, by:

  • crawl rate limit – the number of pages crawled by search engine robots in a short time,
  • crawl health – low server response time, correct response codes, loading speed,
  • crawl demand – the demand for (re)crawling, which depends, among others, on the frequency of content updates, web traffic,
  • page size, use of JavaScript code – pages with JS take more indexing budget.

Where Can I Check the Crawl Budget of My Website?

In Google Search Console, go to the “Legacy tools and reports” section and choose “Crawl stats”. You will find there 3 charts:

  1. Pages crawled per day,
  2. Kilobytes downloaded per day,
  3. Time spent downloading a page (in milliseconds).

Crawl budget - definition

Crawl statistics

These are data from the last 90 days. Based on them, you can specify how the crawl budget has changed during this period.

Was this definition helpful?
(0)
Author
Delante is an online marketing agency specialized in generating website traffic from search engines and paid ads (SEO / SEM). Over 80% of our clients are from rapidly growing e-commerce industry.

RECENTLY ON OUR BLOG

What’s EEAT & Why Is It Important for SEO?

What’s EEAT & Why Is It Important for SEO?

Some time ago SEO specialists thought that EEAT was only about creating elements such as author’s bios or links to resources to make the website more authoritative for crawlers. Today we know that Google perceives authoritativeness differently. It expects both the brand itself, and the people standing behind it to be industry experts. What’s Google EEAT and how to convince the search engine robots that your content is trustworthy and should be displayed high in the search results?

Read more