=== AI Scrape Protect === Contributors: daanverbaan, uisce Donate link: https://paypal.me/AiScrapeProtect Tags: robots.txt, scraping, AI, security, AI scraping protection Requires at least: 5.0 Tested up to: 6.7.1 Requires PHP: 7.4 Stable tag: 3.0 License: GPLv2 or later License URI: https://www.gnu.org/licenses/gpl-2.0.html Protect your website from AI scraping by adding opt-out instructions to your `robots.txt` file and including meta tags in the HTML `` with this easy-to-use WordPress plugin. == Description == **AI Scrape Protect** is a WordPress plugin designed to protect your website from scraping for AI training purposes. It achieves this by adding opt-out instructions to the `robots.txt` file for the most common AI scraping bots and including meta tags to control how your content is used. **Note:** These instructions are not always respected by all bots. == Features == - Adds specific `User-agent` and `Disallow` rules to your `robots.txt` file to block a comprehensive list of AI scraping bots. - Introduces meta tags in the HTML `` to provide additional instructions to AI bots, including new tags for Bingbot and general AI compliance. - Prepares for future standards with support for `DisallowAITraining` and `noimageai` meta tags. - Dedicated handling of specific bots like CCBot and Bingbot for better protection and compatibility. == Installation == 1. Upload the `ai-scrape-protect` folder to the `/wp-content/plugins/` directory. 2. Activate the plugin through the 'Plugins' menu in WordPress. == Changelog == = 3.0 = * Added admin bar icon functionality to indicate plugin activity. * Updated meta tags for compliance with official documentation and improved AI scraping protection: - Adjusted Bingbot tag to use `nocache` for better compatibility with Bing AI Chat. - Removed "noindex" from the `ai-bot` meta tag to allow search engine indexing. - Added `DisallowAITraining` and `noimageai` to the `robots` meta tag. - Introduced dedicated meta tags for CCBot. * Removed OpenAI SearchBot from the `robots.txt` blocklist based on OpenAI's recommendations. = 2.4 = * Updated meta tags for improved AI scraping protection. - Prevent Bingbot and general AI bots from using the content for AI purposes. - Added clear comments in the code to describe the functionality of each meta tag. = 2.3 = * Added the following new AI bots to the block list: Ai2Bot-Dolma, cohere-training-data-crawler, DuckAssistBot, img2dataset, magpie-crawler, PanguBot = 2.2 = * Optimized `robots.txt` output by consolidating multiple `Disallow` rules into a single `Disallow: /` statement. * Added new AI bots to the block list: StableDiffusionBot, sentibot, Grok, GrokAI, XAI, XBot. * Enhanced code structure for better readability and maintainability. = 2.1 = * Updated Features section in `readme.txt` to include new functionality for meta tags in HTML ``. * Updated Frequently Asked Questions section in `readme.txt` to include new functionality for meta tags in HTML ``. = 2.0 = * Added AI bots: AI2Bot, AmazonBot, Applebot-Extended, ClaudeBot, OAI-SearchBot, PetalBot, webzio. * Introduced meta tags for AI bot protection in HTML ``. == Frequently Asked Questions == = How does this plugin protect my site from AI scraping? = The plugin adds specific `User-agent` entries to your `robots.txt` file to instruct common AI scraping bots not to crawl or scrape your site. It also introduces meta tags in the HTML `` to provide additional instructions to AI bots. = Will this completely stop AI scraping of my site? = While this plugin adds recommendations to the `robots.txt` file and includes meta tags, not all bots follow these rules. This is a measure to discourage scraping rather than a foolproof solution. = Can I add or remove bots from the list? = Currently, the plugin includes a predefined list of bots. If you need to add or remove specific bots, you would need to modify the plugin code or contact the plugin author for customization. = What happens if I deactivate the plugin? = The `robots.txt` file will revert to its previous state before the plugin was activated, and the meta tags added to the HTML `` will be removed. == Screenshots == 1. **robots.txt File Example:** Shows how the plugin updates the `robots.txt` file. 2. **meta tags Example:** Shows an example of the Meta Tags added to the head section. == Upgrade Notice == = 3.0 = New functionality added to meta tags and `robots.txt` updates for better compatibility with AI and search engine standards. Removed OpenAI SearchBot from the blocklist as recommended. Added admin bar icon functionality to indicate plugin activity. = 2.4 = New functionality added to the meta tags for enhanced AI scraping protection. The meta tags now provide clearer instructions to prevent AI bots from using your content. == License == This plugin is licensed under the GNU General Public License v2 or later. See the [GPL v2 license](https://www.gnu.org/licenses/gpl-2.0.html) for details. == Author Information == This plugin is developed by [Uisce Web Development](https://uisce.eu), specifically by Daan Verbaan. For support or inquiries, please visit our website or contact us directly.