Where the Data Comes From

Training a language model requires an almost incomprehensibly large amount of text. GPT-3 was trained on roughly 300 billion tokens — around 225 billion words, or roughly 300,000 novels worth of text. Estimates for GPT-4 and its contemporaries run into the tens of trillions of tokens. Somewhere, someone had to obtain all of that.

This is where the story gets complicated.

The sources

The backbone of most large model training datasets is the public web, captured through large-scale scraping operations. A few sources appear in virtually every major model's training data:

Common Crawl is a nonprofit that has been systematically crawling and archiving the web since 2008. Its archive runs to hundreds of petabytes. It's publicly available, frequently downloaded, and processed versions of it (like C4 and The Pile) have become standard training ingredients. It captures a wide cross-section of the internet — news sites, blogs, forums, e-commerce, personal websites, and everything in between.

Books have featured prominently in most flagship models. GPT models used corpora referred to internally as "Books1" and "Books2." Court filings in the New York Times lawsuit revealed that Books2 was likely sourced from shadow library sites — repositories of pirated ebooks — containing millions of copyrighted titles, obtained without payment or permission to the authors or publishers.

GitHub provided the code training data for models with strong programming capabilities. Microsoft-owned GitHub gave OpenAI access to its repository data; this became the foundation for GitHub Copilot. Individual developers whose public repositories were included had no knowledge this was happening at the time.

Wikipedia, academic papers, and government documents round out the mix. These are either freely licensed or in the public domain, and their inclusion is largely uncontroversial.

News and journalism — the full archives of thousands of publications — were scraped and included, again without payment or consent.

The consent problem

Here is the uncomfortable fact at the center of this: almost all of this content was created by people who did not agree to have it used to train AI systems.

Copyright law in most countries grants the creator of a work the right to control how it's used. Under the Berne Convention, which governs most of the world, copyright attaches automatically at the moment of creation. A blog post, a tweet, a news article, a novel — all are copyrighted by their authors from the moment they're written, regardless of whether they display a copyright symbol.

The legal theory labs use to justify this practice is that training on copyrighted works constitutes "fair use" under US law (and analogous exceptions elsewhere) — a transformative use that doesn't substitute for the original work. Critics argue that the resulting models can reproduce original content near-verbatim, compete directly with the creators whose work they trained on, and extract commercial value at massive scale without compensation.

The lawsuits

This is no longer a theoretical debate.

The New York Times v. OpenAI and Microsoft (filed December 2023) is the highest-profile case. The Times alleges that OpenAI trained its models on millions of Times articles without permission, that the models can reproduce those articles verbatim, and that this constitutes copyright infringement on a massive scale. The case is ongoing and is being watched closely as a potential landmark.

A group of prominent authors — including George R.R. Martin, John Grisham, Jodi Picoult, and others represented by the Authors Guild — filed a class action against OpenAI in 2023 alleging their books were used without consent.

Getty Images filed suit against Stability AI (maker of Stable Diffusion) in both the US and UK, alleging that millions of images from Getty's library were scraped and used to train image generation models without licensing agreements.

These cases will take years to resolve. Their outcomes could reshape what training data AI labs are permitted to use going forward.

What's changing

The industry has started to move in response to legal pressure and public criticism — though critics argue not fast enough.

Some publishers have struck licensing deals: the Associated Press, Reuters, the Financial Times, and The Atlantic have all reached agreements with AI companies to license their archives. These deals typically provide revenue to the publisher and explicit permission for training use.

Several AI labs have published tools allowing creators to opt out of training data collection. The effectiveness of these systems is disputed — opting out of future training is difficult to verify, and it does nothing about work already incorporated into existing models.

Synthetic data — models generating training data for the next generation of models — is becoming a larger component of new training runs. This reduces dependence on scraped human content but raises its own questions about feedback loops and quality degradation.

What this means in practice

The models you're using today were almost certainly trained on content created by people who never consented to it. That doesn't make them useless or the companies behind them uniquely villainous — the legal and normative frameworks simply didn't exist yet when this data was collected. But it does mean you're interacting with a technology that was built, in part, by taking something without asking.

The shape of this debate will determine a great deal about how AI development proceeds over the next decade — who gets compensated, who retains control over their creative work, and whether the next generation of models is built on a more equitable foundation.