tribedude
Guest
offline
I am using various robots.txt lines to block specific bots but I think some are ignoring that.
Google has just informed me that our domain won't get indexed unless we allow their AI training bots to roam free in the domain. Other Canadian publishers have also received this notification. We have no access to Google AI products in Canada, so this is doubly sketchy, I think.
I am thinking of taking the board completely private.
What are the best ways to stop AI bots from scraping your...
Read more
Continue reading...
Google has just informed me that our domain won't get indexed unless we allow their AI training bots to roam free in the domain. Other Canadian publishers have also received this notification. We have no access to Google AI products in Canada, so this is doubly sketchy, I think.
I am thinking of taking the board completely private.
What are the best ways to stop AI bots from scraping your...
Read more
Continue reading...