A coalition including Cloudflare and major publishers proposed a pay-per-crawl model to monetize AI training data, with Cloudflare unveiling a tool to block unauthorized AI access.
A group of publishers want to put up a new kind of paywall, with the help of networking technology, cybersecurity and CDN operator Cloudflare. Cloudflare, which manages content networks for website operators, has unveiled a new tool that seeks to block AI systems from accessing content without permission.
The idea is that this will empower publishers that are sufficiently authoritative to charge AI companies for the use of their content.
Such a “pay per crawl” model can help content creators monetise the data being used to train large language models, according to Cloudflare.
It's proposed that this new revenue may help offset the lost monetizable web traffic caused by so-called 'zero click' AI-based search interfaces, which answer directly many user queries, without going to the underlying websites.
Cloudflare’s chief strategy officer Stephanie Cohen described it as "the begining of a new model for the internet".
Prominent media outlets including Condé Nast, Reddit, Pinterest, and the Associated Press are backing the venture.
