Last month, news resonated across the comms community that Reddit had suddenly dropped out of ChatGPT’s citations. “Reddit Nearly Vanishes From ChatGPT” was one headline. What’s going on? After all, only two years ago OpenAI struck a partnership with Reddit to license its data, reportedly worth around $70 million a year. Did the relationship sour? Has Reddit been taken over by bots? Or worse – brands?
The truth is probably simpler. But it’s a reminder that we’re quick to jump to conclusions, and that maybe we need to take a breath, understand how these platforms work, and stop thinking about them in SEO terms.
Everyone is now on board with ChatGPT having two corpora of data at its virtual fingertips. The first is its training knowledge: everything it ingested and used to define the vast weights and measures that form the large language model. The second is web search. When you prompt the LLM, it usually fans out tens of web searches to find evidence to refine its knowledge. It used to use Bing; now it mostly uses its own system.
But we forget two other important sides of the equation.
First, we need to consider how the LLMs were trained. Back in the days of GPT-3, before ChatGPT, the answers from LLMs were perfunctory, unreliable and rambling, riddled with hallucinations. The model providers spent hundreds of millions employing people to judge the answers, a process called reinforcement learning from human feedback (RLHF). It’s RLHF that turned robotic AI responses into the friendly, helpful LLMs we know today.
The second is where Reddit comes in. We might think of ChatGPT as an algorithm, but really it’s a product built on an algorithm. Just as Google spent years tuning search (and prompted the arms race we now know as SEO), OpenAI and its peers are carefully tuning ChatGPT to get you the answer you want. This doesn’t mean it is ignoring Reddit. In fact, our research found the underlying API searches Reddit just as much as it did before. But the product that is ChatGPT is not citing Reddit as a source.
The move away from Reddit could be for lots of reasons. A cynic might suggest that Reddit is a less commercial, and therefore less desirable, destination for a product that is increasingly pushing advertising. A simpler answer might be that ChatGPT is detecting a high bounce rate on Reddit.
In comms, we’re increasingly asked to help our clients improve their AI visibility. The temptation is to reverse-engineer every platform change and adjust our advice accordingly. But that’s a shaky foundation for a communications strategy. We should build around what lasts: clear positioning, evidence to support our claims, and credible voices talking about our clients across several channels. Of course, we need to track how these platforms evolve, test our assumptions and adapt. But our value lies in helping clients make sound decisions, and that includes being honest about what we know and what we’re still guessing.