Crawl Budget

What is the Crawl budget? – Definition

Crawl budget determines how much resources – time and computing power – Google robots spend on indexing a given site. This reflects the number of pages added to the search engine index and the indexing frequency. The crawl budget is influenced, among others, by:

  • crawl rate limit – the number of pages crawled by search engine robots in a short time,
  • crawl health – low server response time, correct response codes, loading speed,
  • crawl demand – the demand for (re)crawling, which depends, among others, on the frequency of content updates, web traffic,
  • page size, use of JavaScript code – pages with JS take more indexing budget.

Where Can I Check the Crawl Budget of My Website?

In Google Search Console, go to the “Legacy tools and reports” section and choose “Crawl stats”. You will find there 3 charts:

  1. Pages crawled per day,
  2. Kilobytes downloaded per day,
  3. Time spent downloading a page (in milliseconds).

Crawl budget - definition

Crawl statistics

These are data from the last 90 days. Based on them, you can specify how the crawl budget has changed during this period.

Was this definition helpful?
(0)
Author
Delante is an online marketing agency specialized in generating website traffic from search engines and paid ads (SEO / SEM). Over 80% of our clients are from rapidly growing e-commerce industry.

RECENTLY ON OUR BLOG

Google Analytics Account Structure - the Ultimate Guide

Google Analytics Account Structure - the Ultimate Guide

Google Analytics is the most popular online tool used to analyze statistics of websites, mobile apps and devices. It doesn’t really matter whether you’ve already encountered this tool or whether you’re just beginning your analytical adventure. It’s worth taking a closer look at its account structure. Thanks to it, benefiting from the possibilities offered by Google Analytics will be much easier and safer.

Read more
How to Do SEO on a Single-Page Website? 9 Steps

How to Do SEO on a Single-Page Website? 9 Steps

Single page websites have gained immense popularity in recent years and there is absolutely nothing strange about it as this solution offers many great benefits. You must know, however, that one-page website causes many difficulties and that their SEO can be much more challenging - but is SEO for a single page website at all possible? And if so, how is it different from regular website SEO? In this article, I'll tell you what SEO strategies you should adopt to make your single-page website rank high in the search results and have massive web traffic. Let’s dive right in!

Read more