Short user comments on Reddit and similar sites can manipulate AI-powered search tools, a new Cornell study finds. Even 13-word snippets may sway ChatGPT or Google AI to promote spam or scams. The risk to search reliability is growing.
New research from Cornell University reveals that AI-powered search tools like ChatGPT and Google’s AI search can be manipulated with minimal effort by inserting short, user-generated snippets on platforms such as Reddit, Quora, and Wikipedia. The study, titled “Deep-research agents can be poisoned via user-generated content,” demonstrates that as few as 13 words embedded in a comment can consistently prompt AI agents to generate spam or scam content.
The researchers-Hal Triedman, Tingwei Zhang, and Vitaly Shmatikov-found that deep research agents, which scrape the web in real time to answer user queries, rely heavily on user-generated content. According to their findings, about half of all AI citations come from sites like Reddit or Wikipedia, and nearly a quarter of all citations are from user-generated platforms.
This vulnerability has fueled a growing industry known as AI-engine optimization (AEO), where brands deliberately seed forums and wikis with promotional or misleading content to influence AI outputs. Moderators on Reddit and Wikipedia have reported a surge in such activity, as companies attempt to shape the information that AI tools retrieve and display.
The Cornell team showed that even a single manipulated Reddit comment could affect the AI’s responses to a cluster of related queries. For example, appending a promotional sentence about a restaurant to a Reddit post led ChatGPT to recommend that restaurant in its answers, citing the manipulated thread. Similarly, a fabricated comment about a dating app for divorced men over 50 resulted in the AI promoting the fake app in its responses.
These attacks are simple to execute: brands or individuals identify popular queries, craft comments that closely match those queries, and post them in relevant subreddits or forums. The AI’s reliance on lexical similarity-returning content that closely matches the user’s question-makes it especially susceptible to this tactic.
The researchers conducted their experiments in a controlled environment, modifying content at the retrieval level rather than posting live on Reddit, to avoid polluting the public information ecosystem. They found that even subtle additions to the end of a comment could reliably alter AI-generated answers and citations.
Moderators face significant challenges in combating this manipulation. Short, seemingly innocuous snippets are difficult to distinguish from genuine user contributions, making it hard to detect and remove poisoned content. Attempts to moderate or verify authenticity, such as requiring biometric verification or restricting copy-pasted comments, could disrupt the open nature of these platforms.
Reddit’s spokesperson told 404 Media that the platform has long-standing systems to detect and remove inauthentic content, bots, and coordinated manipulation. However, the study suggests that as AI search becomes more prevalent, the burden on volunteer moderators and editors will only increase, while the incentives for brands to game the system continue to grow.
Both Zhang and Triedman argue that the problem extends beyond individual platforms and represents a broader societal challenge. As AI search tools treat all sources with similar credibility, distinguishing between a random Reddit comment and an official government website becomes increasingly difficult for both users and the AI itself.
