Of the myriad controversies plaguing OpenAI, the issue of training data has emerged as the most polarizing. For publishers, this polarization comes in the form of a choice between getting as far away as possible, or cozying up and making a deal.

OpenAI has kept a lid on information about what models like GPT4o were trained on, meaning ChatGPT's recipe is a secret. Similar LLMs, however, are fed on social media posts, blogs, digitized books, online reviews, Wikipedia pages, and pretty much any piece of information on the web that you can think of. In fact, at least one scholar, Berkley computer scientist Stuart Russell, thinks most of the known internet was gobbled up by LLMs in order to replicate human intelligence and mirror it back to us in automated form.