1. Technical barriers
For AI bots to access your website, you must first open the door for them. How can you make sure it is not closed? There are two possibilities. Start by checking your website’s robots.txt file. From there, you can block bots from accessing individual pages or the entire website. Below is an example of what a block for ChatGPT and other models might look like:
How can you fix this if you find such an entry? Simply delete it from the file.
The second option is to verify that the block is not already at the server level. It is best to ask the webmaster who manages your website and server to check whether access for AI crawlers is open.
d-tags


Source: Wikipedia




