Crawl Budget

What is the Crawl budget? - Definition

Crawl budget determines how much resources - time and computing power - Google robots spend on indexing a given site. This reflects the number of pages added to the search engine index and the indexing frequency. The crawl budget is influenced, among others, by:

  • crawl rate limit - the number of pages crawled by search engine robots in a short time,
  • crawl health - low server response time, correct response codes, loading speed,
  • crawl demand - the demand for (re)crawling, which depends, among others, on the frequency of content updates, web traffic,
  • page size, use of JavaScript code - pages with JS take more indexing budget.

Where Can I Check the Crawl Budget of My Website?

In Google Search Console, go to the "Legacy tools and reports" section and choose “Crawl stats”. You will find there 3 charts:

  1. Pages crawled per day,
  2. Kilobytes downloaded per day,
  3. Time spent downloading a page (in milliseconds).

Crawl budget - definition

Crawl statistics

These are data from the last 90 days. Based on them, you can specify how the crawl budget has changed during this period.

Was this definition helpful?
Delante is an online marketing agency specialized in generating website traffic from search engines and paid ads (SEO / SEM). Over 80% of our clients are from rapidly growing e-commerce industry.


Ahrefs vs SEMrush - Which SEO Tool to Choose?

Ahrefs vs SEMrush - Which SEO Tool to Choose?

If you are working in SEO, whether in an agency or as a freelancer, you have probably heard more than once of Ahrefs and SEMrush. These two tools are, without a doubt, the biggest players in SEO right now. What features do they offer and which of them is a better solution for you? In this article, I will compare the two most popular SEO tools - Ahrefs and SEMrush.

Read more