Bot traffic now surpasses human visits online. New US and UK bills target stealth bots that scrape content and disrupt publishers. Industry leaders urge global action to restore transparency.
News publishers are facing mounting costs and operational risks as bots now generate more than half of all internet traffic, according to researchers. The surge in automated crawlers-especially stealth bots that disguise their identity and intent-has made it increasingly difficult for publishers to protect their content, manage bandwidth, and maintain site performance.
Stealth bots, which intentionally hide or falsify their identity, are being used by bad actors to scrape news content, including material behind paywalls. This stolen journalism is then resold on a billion-dollar reseller market, where AI developers purchase data from bot operators rather than from the original publishers. The result is a growing ecosystem where AI products compete directly with the very sources they extract from, undermining the business models of newsrooms.
Industry executives have described the situation as unsustainable. News Corp's Robert Thomson has called stealth bots "the silent scavengers of the internet," while Condé Nast CEO Roger Lynch has warned that these bots operate with "zero accountability." The impact is not limited to large organizations; local publishers, already operating on thin margins, are especially vulnerable to the technical and financial strain caused by relentless bot activity.
Recent incidents highlight the scale of the problem. In August 2025, a UK technology site was forced offline after being scraped 1.6 million times in a single day. The Wikimedia Foundation, which operates Wikipedia, now blocks or throttles about a quarter of all automated requests-billions each day-from crawlers that ignore its access policies. Publishers have deployed advanced blockers, but stealth bots continue to evade detection and enforcement.
Legislative efforts are now underway to address the crisis. In the United States, the bipartisan Stealth Bot Prohibition Act would require AI bots to disclose their identity and purpose when accessing websites. This measure is designed to give publishers the information needed to defend their content and infrastructure. The proposal has received strong backing from WAN-IFRA and other industry groups, who argue that transparency is essential for a functioning digital marketplace.
The United Kingdom is pursuing a similar path. The Automated Online Software (Access and Transparency) Bill, introduced by MP Damian Hinds and supported by the News Media Association, would require anyone operating a bot that systematically copies content from UK websites to reveal their identity, ownership, and intended use of the material. The convergence of US and UK legislative approaches signals a growing consensus that bot transparency is foundational for digital markets.
Transparency is also a prerequisite for effective licensing and rights management. The EU’s Copyright in the Digital Single Market Directive allows rightsholders to reserve their content from text and data mining in a machine-readable way. From August 2026, AI developers who ignore these reservations can face fines of up to €15 million or 3% of global turnover. However, these protections are only meaningful if bots honestly identify themselves when accessing content.
Publishers continue to bear the costs of producing original reporting and maintaining digital infrastructure, while bad actors extract and monetize their work. The current lack of enforceable transparency allows the strip-mining of journalism to persist, distorting the market and threatening the sustainability of news organizations. As highlighted in coverage of legislative efforts in New York and Congress, the push for bot accountability is gaining momentum on both sides of the Atlantic.
WAN-IFRA’s CEO, Stig Ørskov, has called for other countries to follow the US and UK examples by introducing bad bot legislation. Industry leaders argue that enforceable transparency is the first step toward restoring a fair digital marketplace-one where publishers, AI developers, and readers can all benefit from clear rules and accountability.
