Independent newsletter publishers can now manage which AI crawlers access their content. Beehiiv’s new integration with Cloudflare brings advanced crawl controls to creators. This move levels the playing field with larger news organizations.
Independent journalists and newsletter creators using Beehiiv now have new tools to manage how AI crawlers interact with their content, following a partnership with Cloudflare announced Tuesday. This integration gives Beehiiv users beta access to Cloudflare’s AI Crawl Control services directly from their dashboards, allowing them to opt in or out of specific AI agents crawling their newsletters.
With these controls, Beehiiv creators can see detailed data on which AI agents are accessing their work, which are being blocked, and how much referral traffic is coming from AI products. The dashboard also lets users allow or block individual AI agents, with Cloudflare automatically updating settings as new agents from the same companies appear.
Compared to other newsletter platforms, Beehiiv now offers some of the most granular on-platform control over AI crawler access. While Substack and Ghost provide robots.txt-based options to signal AI crawlers, these methods rely on voluntary compliance and do not actively block access. Beehiiv’s approach, powered by Cloudflare, gives creators more direct enforcement and visibility.
This partnership builds on Cloudflare’s broader efforts to help publishers control how their content is used by AI models. Last year, Cloudflare made blocking AI crawlers the default for all customers and introduced a pay-per-crawl marketplace, allowing publishers to charge AI crawlers per access. Cloudflare reportedly takes a 30% share of publisher earnings from this marketplace, though Beehiiv creators do not currently have direct access to this feature through the new integration.
Beehiiv’s co-founder and CEO, Tyler Denk, said the partnership aims to give creators the data and leverage needed to either maximize distribution or protect their writing on their own terms. The move comes as more publishers seek to limit unauthorized AI scraping and assert control over how their work is used by emerging technologies. For additional context on how publishers are responding to AI scraping, a recent report covers UK publishers backing new legislation targeting deceptive AI bots.
