Preeti
Cloudflare has introduced a new feature that allows web hosting customers to block AI bots from scraping their website content without permission. This move comes in response to growing concerns among website owners and publishers about AI bots harvesting data. The tool, available to all Cloudflare customers, including those on the free tier, aims to protect content creators on the internet by using machine learning to identify and block bots. Cloudflare’s system relies on digital fingerprinting to differentiate bots from legitimate users.
Previously, website owners could use the robots.txt file to prevent bots from crawling their sites, but the new tool offers a more robust defense against content scraping. Cloudflare’s announcement stated that customers do not want AI bots visiting their websites, especially those that do so dishonestly, and the new one-click feature to block all AI bots is designed to address this concern.
The company reported a significant amount of AI bot activity across its network, with popular bots like Bytespider, GPTBot, and ClaudeBot accessing more than one-third of the top one million sites served by Cloudflare in June. By navigating to the Security > Bots menu and blocking “AI Scrapers and Crawlers,” users can now protect their websites from unwanted AI bot activity.
Source link
Source link: https://www.techradar.com/pro/cloudflare-wants-to-help-stomp-out-ai-scraper-bots-for-good
GIPHY App Key not set. Please check settings