A bipartisan House bill would force AI bots to reveal themselves when scraping websites. Publishers could see fines imposed for violations. The move aims to protect original content and revenue.
Soon we will see new protections against unauthorized AI bots that scrape their content. In July, three US House representatives introduced the Stealth Bot Prohibition Act, a bipartisan bill designed to require AI stealth crawlers to identify themselves and state their purpose when accessing websites. The legislation, authored by the News/Media Alliance, targets bots that collect and repurpose content without permission, a practice that has eroded publisher revenue and undermined content licensing models.
If enacted, the bill would empower the Federal Trade Commission and state attorneys general to pursue legal action against operators of undisclosed crawlers. Each violation could result in a $53,000 fine. Danielle Coffey, president and CEO of the News/Media Alliance, said these so-called "bad bots" have created a reseller market by scraping publisher sites and distributing the content elsewhere, often for financial gain. The bill aims to deter such activity through steep penalties and increased monitoring by both website owners and enforcement agencies.
Amelia Binder, SVP of global government affairs at Axel Springer, noted that most stealth bots on the company's sites violate terms of service and use the content for their own benefit, either by reselling it or passing it off as original. This has led to lost revenue for publishers and increased risk of inaccuracies as content is reused out of context. Binder described the bill as an important first step toward greater transparency and the development of licensing agreements.
Currently, there is no federal law requiring bots to disclose their identity or purpose. Lark-Marie Anton, chief communications and brand officer at USA TODAY Co., attributed this gap to the rapid evolution of AI, which has outpaced existing regulations. Traditionally, web crawling relied on robots.txt files, which serve as voluntary guidelines for bots. However, many modern bots now ignore or manipulate these files, disguising themselves as human users to bypass restrictions and harvest content for resale or AI training.
According to Digiday, dozens of companies, including Perplexity, have profited from selling data scraped without authorization. The practice raises additional concerns, including national security risks, as some bots originate from Russia and China. Coffey warned that large-scale harvesting of news and information could be used to train foreign AI systems and undermine the business models that support independent journalism. Conan Gallaty, CEO and chairman of the Tampa Bay Times, emphasized the need for intellectual property protections that extend beyond US borders, noting that bots often use scraped content in ways that further threaten the content marketplace.
Despite the risks, the rise of AI-generated content has also increased demand for trusted, high-quality journalism, according to Anton. This shift could encourage the creation of licensing and attribution frameworks that compensate publishers. National security concerns and the imbalance in the content marketplace have generated bipartisan interest in the bill, Gallaty said. While the current administration has generally resisted stricter AI regulation to maintain competitiveness, it is now finalizing an AI oversight framework in response to concerns about energy use and cyberattacks. Still, the bill faces an uncertain path through Congress.
Publishers are already feeling the impact of stealth bot activity. Some report that unauthorized bots have slowed website performance, making it harder for readers to access articles. Gallaty said this activity strains resources and affects customer service. At Axel Springer, Binder reported that 25% of Politico's hosting costs now go toward managing bot traffic, an expense the company had not anticipated. She described the volume of stealth bots as overwhelming. The consequences extend to consumers, who may encounter less reliable information as unmoderated chatbots republish content with potential inaccuracies or bias. Binder argued that a healthy media ecosystem depends on plurality and fair licensing, while Anton stressed that quality journalism requires investment, expertise, and accountability. For AI to play a constructive role, it must support the human work that underpins valuable content.
For more on the legislative push to address stealth crawlers and the growing impact of bot traffic, see this related coverage: how lawmakers are responding to the surge in unidentified bots scraping publisher content.
