RSL 1.0: AI Scrapers Must Pay

RSL 1.0 Makes AI Companies Pay for Web Content

A new open licensing standard is changing how AI companies access online content. Really Simple Licensing 1.0, or RSL 1.0, now allows publishers to set clear rules for licensing and compensation when AI scrapers collect data from their websites. This move answers a growing question among publishers: “How can we get paid when AI uses our content?” With support from major players like Yahoo, Ziff Davis, and O’Reilly Media, RSL is poised to reshape AI’s interaction with web content.

How RSL Works

RSL 1.0 builds on the familiar robots.txt system, which traditionally controls which parts of a website crawlers can access. The key difference: RSL can specify whether AI companies must pay or request a license before using content. While RSL alone cannot physically block non-compliant AI scrapers, web infrastructure providers like Cloudflare, Akamai, and Fastly can enforce these rules. This gives publishers more power to monetize their content in the age of AI.

Protecting Content Without Sacrificing Search Visibility

One major feature of RSL is its flexibility. Publishers can block AI-powered search features, such as Google’s AI Mode, without removing their pages from standard search results. Previously, opting out of AI features often meant losing visibility in traditional search, creating a tough choice for content creators. With RSL, that compromise is no longer necessary. Publishers can maintain their search presence while controlling how AI interacts with their material.

Industry Backing and Adoption

The RSL Collective, the organization behind the standard, reports over 1,500 media organizations and brands supporting RSL. Co-founders Doug Leeds and Eckart Walther emphasized that RSL provides “the missing layer” for AI content control. By giving AI companies a clear licensing framework, publishers can demand compensation and set rules for AI use while remaining accessible to human readers.

Implications for Google and AI Search

RSL’s release comes as Google faces scrutiny from the European Commission over potential antitrust violations. The commission is investigating whether Google used web content for AI search features without giving publishers a way to refuse. RSL provides a structured solution: respecting publisher preferences at a granular level, ensuring AI training and generative answers only use content with consent.

Publishers Gain Leverage

The introduction of RSL 1.0 shifts some power back to content creators. Publishers can now enforce licensing agreements and potentially monetize AI usage more effectively. While AI companies will need to adapt, the standard promises a clearer path to fair compensation for scraped content. This could become a model for future internet governance and AI ethics.

RSL is still early in adoption, but momentum is growing. With support from major tech and media companies, the standard may soon become an essential part of web publishing strategy. For AI developers, this means rethinking how algorithms gather data and ensuring compliance with licensing rules—potentially transforming how content is accessed across the web.

RSL 1.0 represents a critical step in balancing the interests of AI companies and publishers. By enforcing licensing and giving creators control over their content, this standard could reshape AI’s role on the internet, ensuring both innovation and fair compensation coexist.