What is the llms.txt file and do you really need it?

*Shoutout to our CEO, Matt Calik, for this comparison. 😉

What is the llms.txt file?

The llms.txt is a markdown text file that summarizes the key content of your website. You can use it to highlight the most important pages for your business. Think of llms.txt as a hybrid between robots.txt and sitemap.xml – but specifically designed for LLM bots and AI systems like ChatGPT, Google Gemini, Claude, or Perplexity.

According to its creator, Jeremy Howard, the file’s purpose is to help these AI models read and understand websites as a whole. This, in turn, would enable more effective indexing of your content and improve its visibility in AI-powered systems.

There are two versions of the file:

• The basic version: yourdomain.com/llms.txt
• The extended version: yourdomain.com/llms-full.txt (which includes more directives and content)

The basic version typically looks like this:

# Title

> Optional description goes here

Optional details go here

## Section name

- [Link title](https://link_url): Optional link details

## Optional

- [Link title](https://link_url)

The file should be placed in the root directory of your domain. Several online generators already exist, such as https://wordlift.io/generate-llms-txt/ or https://writesonic.com/free-tools/llms-txt-generator. WordPress also offers plugins that help generate this file.

Sounds great, right? But does it really work? Do we actually need it?

Does llms.txt file really work?

At the moment, there’s no direct proof that the llms.txt file works as intended. None of the major LLM providers (OpenAI, Google, Anthropic) have officially confirmed its use.

That said, supporters argue: Why not try adding it to your site? If it ever becomes a standard like sitemaps or robots.txt, you’ll be ahead of the competition by implementing it early. There’s some logic in that. 😉

One notable advocate of this logic is Alex Moss from Yoast, who spoke about llms.txt during his presentation at BrightonSEO this April. He predicts it could become a new standard, comparing it to structured data markup. Since then, Yoast has published their own llms.txt file as an example, which you can check out here: https://yoast.com/llms.txt

On the other hand, the extended version is already used by companies like Zapier. You can see their example here: https://docs.zapier.com/llms-full.txt

Our SEO Team Lead, Paulina Konopka, agrees with this positive view:

For the skeptics

Some experts remain skeptical, including our Senior SEO & R&D Specialist, Wojciech Urban. Here are some of the drawbacks he points out:

• The llms.txt file is only a proposal—there’s no evidence AI bots actually use it. Wojciech, a data-driven professional, remains a bit of a skeptic here.
• AI crawlers are designed to collect content without needing such a file (for example, check this out: https://r.jina.ai/https://delante.co)
• There’s a risk of manipulation with large language models (LLMs). Think about why Google stopped supporting the keywords meta tag… 😉
• If AI bots do start relying on the file, they might link to the llms.txt text file instead of the actual webpage, which could harm user experience (UX).

On this last point, Wojciech’s comparison with meta keywords is echoed by… John Mueller of Google. Whether John is the definitive SEO authority is up to you, but here’s what he wrote on Reddit about llms.txt:

“AFAIK none of the AI services have said they’re using LLMs.TXT (and you can tell when you look at your server logs that they don’t even check for it). To me, it’s comparable to the keywords meta tag – this is what a site-owner claims their site is about … (Is the site really like that? well, you can check it. At that point, why not just check the site directly?)”

Source: Search Engine Journal

In summary

The llms.txt file sparks plenty of debate and has both advocates and critics. It’s definitely worth testing and waiting for official confirmation of its effectiveness. SEO is a field that constantly evolves – change is the only constant! 😉

What do you think? Would you add llms.txt to your website? Or maybe you already did?