All the Books in the World

A book, cut apart

Start with a single book. A paperback, bought used. First its binding is stripped off. Then its pages are trimmed so they come apart cleanly. Each page is scanned, covers included, until the book exists as a PDF: pictures of every page, with a layer of text underneath that a computer can read. Then the paper is thrown away.¹

This happened millions of times. The company paying for it was Anthropic, the maker of the AI model Claude, and the scans went into what its employees called a research library. Among the books that went through this process, according to a federal court, were books by the three authors who would later sue the company: Andrea Bartz, Charles Graeber and Kirk Wallace Johnson.²

And this wasn’t even the first way Anthropic got books. Years before it started buying them, it had downloaded more than seven million pirated copies from the internet, for free. That shortcut would end up costing it $1.5 billion.

So there is an obvious question here. Why would a company that already had millions of free, pirated books go out and buy them, only to destroy them?

The answer turned up in a court ruling, and it has less to do with books than with counting. But to understand it, we first need to know where the books came from. And that story starts at another company.

Where the books came from

In 2020, OpenAI, the company that would later make ChatGPT, published a paper introducing GPT-3, an early model in the line that led to ChatGPT. Among the things it was trained on, the paper listed “two internet-based books corpora,” called Books1 and Books2. Together they made up about 16 percent of the training mix. The paper said nothing else about where the books came from.³

Five years later, a federal judge filled in the blank. It is “undisputed,” she wrote, that in 2018 an OpenAI employee downloaded pirated books from Library Genesis, known as LibGen. LibGen is the biggest of the so-called shadow libraries: websites that share pirated books and academic papers for free. According to the authors now suing OpenAI, the collections were originally named LibGen1 and LibGen2, and only later renamed Books1 and Books2. According to testimony in the case, they were used to train GPT-3 and GPT-3.5.⁴

Look at who signed that 2020 paper. Among its authors were Dario Amodei and Ben Mann. By 2021, both had left OpenAI and co-founded a new company. It was called Anthropic.⁵

People inside OpenAI knew the source was a problem. In internal messages quoted by the authors’ lawyers in a filing made public this month, Amodei, then still at OpenAI, wrote that “as a training set [LibGen is] a bit sketchier.” Another researcher, Sam McCandlish, who would also go on to Anthropic, said he was “worried about optics,” picturing the headline “openai uses copyrighted data from sketchy russian website.”⁶

At Anthropic, the same shortcut came back. According to the judge’s summary of the evidence in the Anthropic case, in early 2021 Ben Mann downloaded Books3, a collection of 196,640 books that he knew had been put together from pirated copies. That June he downloaded at least five million more from LibGen. In July 2022 the company added at least two million from another shadow library, the Pirate Library Mirror. In total, more than seven million pirated copies, held in a central library of what internal documents called “all the books in the world,” to be kept “forever.”⁷

It didn’t have to be that way. Anthropic “ha[d] many places from which” it could have bought books, the judge wrote, but it preferred to avoid what Amodei, now its chief executive, called the “legal/practice/business slog.”⁸

Meanwhile, at OpenAI, the books were being quietly removed. In June 2022 an executive, Bob McGrew, wrote in a Slack channel: “Given how much OpenAI is in the news, now is the right time to excise Libgen from our systems.” The channel had originally been called “excise-libgen.” At some point it was renamed “project-clear.” Around the middle of 2022, about a year before authors started suing, OpenAI deleted Books1 and Books2. It first told the court the reason was that they were no longer being used. Later it withdrew that explanation, and then said the reasons were part of confidential conversations with its lawyers. In February 2026 a judge agreed those conversations could stay private, so we may never know exactly why.⁹

Why destroy a book you bought?

By 2024, Anthropic wanted a cleaner way to get books. In February it hired Tom Turvey, who had been head of partnerships for Google Books, Google’s project to scan the world’s books. His job was to get “all the books in the world” with as little of that slog as possible. Turvey sent an email or two to big publishers about paying for permission to use their books for AI training. Then, in the judge’s words, he “let those conversations wither.” Instead, his team started buying printed books in bulk from distributors and stores, many of them used.¹⁰

Inside the company this became Project Panama. A planning document that came out during the lawsuit, first reported by The Washington Post in January 2026, called it “our effort to destructively scan all the books in the world.” It added: “We don’t want it to be known that we are working on this.”¹¹

In June 2025, Judge William Alsup, a federal judge in San Francisco, ruled on all of this. His decision matters because it separates three things that headlines kept mixing together.

First, some background. American copyright law, the law that decides who may copy someone else’s work, has an escape valve called fair use. It lets you use someone’s work without asking, as long as you turn it into something new: a review that quotes a novel, a parody, a search engine that indexes every book. Courts call that kind of use “transformative.”

The first question was training, feeding the books to an AI model. The judge decided that was fair use. The model doesn’t sell you the book, he reasoned; it learns from it, “like any reader aspiring to be a writer,” not “to race ahead and replicate or supplant” the originals “but to turn a hard corner and create something different.” He called it transformative, “spectacularly so.”

The second was the pirated library. Downloading seven million pirated books to keep, the judge ruled, was not fair use, and he sent that part to trial. Pirating books you could have bought, he wrote, “is inherently, irredeemably infringing.”

The third was the scanned books, and this is where the answer to our opening question lives. Anthropic had paid for every copy. Once you buy a book, the law lets you do almost anything with it except make more copies, and a scan is a copy. But Anthropic destroyed each paper book as it scanned it, and kept the digital version inside the company. “The print original was destroyed,” Alsup wrote. “One replaced the other. And, there is no evidence that the new, digital copy was shown, shared, or sold outside the company.” In other words, the number of copies in the world didn’t go up. The book just changed format, from paper to file. That, he ruled, was fair use too.¹²

Nothing in the record says Anthropic cut up the books in order to win this argument. The likelier reason, and this is my guess, not the record’s, is that a book without a spine is much faster to scan. But in court, the destruction did the work. Because the paper was gone, the scan counted as a change of format rather than a new copy of the book.

In September 2025, Anthropic settled the piracy part of the case, which means it agreed to pay rather than go to trial. The price was $1.5 billion, about $3,000 for each of roughly half a million books, and it promised to delete the pirated files. Notice what it paid for: how it had gotten the books, not what it did with them. The court had already said the second part was fine. The settlement received final approval in July 2026.¹³

One line from a hearing in the case deserves more attention than it got. Speaking to the judge, Anthropic’s own lawyer said: “You can’t just bless yourself by saying I have a research purpose and, therefore, go and take any textbook you want. That would destroy the academic publishing market if that were the case.” The judge agreed that this summed up the problem, and used it against the company.¹⁴

The other library

There was another library of scanned books in this story, and it lost.

In 2007, a young programmer named Aaron Swartz helped launch Open Library, a project of the Internet Archive, the nonprofit best known for the Wayback Machine. Its goal was “one web page for every book ever published.” In 2011 the Archive started lending scanned books through it, and it did so carefully. The Archive and a nonprofit partner acquired printed books, bought or donated, mostly from Better World Books, a used-book seller. They scanned them and lent out one digital copy at a time for each physical copy they owned, the way a library lends a book. The paper books were not destroyed. According to the court, they were “placed in shipping containers and kept out of circulation after scanning.”¹⁵

Publishers sued in 2020. In September 2024, nine months before Judge Alsup’s ruling, a federal appeals court in New York decided that the lending was not fair use. Look at what it identified as the problem: “[The Archive’s] digital books serve the same exact purpose as the originals: making authors’ works available to read.”¹⁶ In the court’s eyes, the trouble was that the copies did exactly what books do. They let people read.

The two libraries were built in similar ways. Both bought printed books in bulk, and, if press reports are right, from at least one of the same sellers. Both scanned them. The one that kept its paper and let the public read the scans lost. The one that threw its paper away and kept the scans inside the company, where its AI models read them, won. These were different courts with different facts, and neither ruling mentions the other. Read together, though, they draw a line that neither draws on its own. A copy that people can read stands in for the book. A copy read privately, by a machine, counts as something new.

Where that line came from

The line didn’t appear out of nowhere. Its roots go back to Google.

In the 2000s, Google scanned millions of books from university libraries for Google Books, and authors sued. In 2015, the same appeals court in New York decided that Google Books, a search engine for books that shows only short snippets, was fair use. Judge Alsup relied on that decision when he ruled that turning Anthropic’s paper books into files was fine. And remember who Anthropic hired to get all the books in the world: the former head of partnerships for Google’s scanning project.¹⁷

Someone had seen this coming. In July 2008, while Google was still scanning, a short text signed by Aaron Swartz, the same programmer who had helped start Open Library, appeared online under the title “Guerilla Open Access Manifesto.” In a list of complaints about who controls knowledge, it asked:

Scanning entire libraries but only allowing the folks at Google to read them?

Eighteen years later, the arrangement he was complaining about, entire libraries scanned and readable by the company that owns the scans, is the arrangement the law has approved. Anthropic just added a step Google never needed: when you’re done, throw the paper away.¹⁸

Aaron’s own story, and why the U.S. government prosecuted him for downloading academic articles, is too big to tell in a paragraph. It deserves its own essay. It will be the next one.

What is still undecided

Anthropic’s case is closed. OpenAI’s is not.

This month, both sides in the authors’ lawsuit against OpenAI and Microsoft asked Judge Sidney Stein in New York to rule in their favor without a trial. The authors argue that OpenAI “built the foundations of its business on mass piracy,” and that its models “pose an existential threat to those who write and publish books.” OpenAI argues that training on books is fair use, and that its models rarely reproduce the books they learned from. A hearing is expected early next year.¹⁹

The U.S. government has already picked a side. On September 1, the Department of Justice asked the court to rule that training AI models on copyrighted text is fair use, quoting Judge Alsup: “transformative — spectacularly so.” It grounded its position partly in national security and in competition with “foreign adversaries.” But the filing is careful. It only covers training, and says nothing about how the books were obtained.²⁰

That gap is where the real fight is now. Everyone seems ready to accept that an AI can learn from books. The open question is whether a company can build that AI on books it never paid for, and then keep the benefit after deleting the copies. Anthropic’s judge said no, and Anthropic paid. OpenAI’s judge hasn’t said yet.

The best argument against this essay

The strongest critic of this essay would say I’m making something ordinary sound sinister. The line courts draw has nothing to do with inside and outside, or companies and libraries. It is the oldest question in fair use: does your copy replace the original? Google’s snippets don’t replace a book. A trained AI model, working as intended, doesn’t hand you one. The Internet Archive’s lending did, one loan at a time. The same rule protects researchers who analyze huge collections of text, and it would protect a public library that used scans for search or research instead of lending. And the one thing the AI companies did that looks most like theft, taking copies they had no right to, is exactly what the court refused to excuse, at a cost of $1.5 billion. The system, in other words, worked.

Most of that is right. The test is whether a copy replaces the original, and there is nothing sinister about it. Anthropic did pay.

But a fair rule can still have unequal effects. A company that turns books into a product can profit from them without ever handing out a copy, so it passes the test by design. A public library can only give people the value of a book by letting them read it, so it fails the test by design. The rule never asks who can afford not to share. It just rewards them.

The AI model does give something back to the public, and that should be said plainly: what it learned from the books, sold by subscription. Whether that counts as sharing knowledge, or is the most efficient way anyone has found to avoid sharing it, is, I think, the real argument. Copyright law was not built to settle it.

What is left

The pirated copies are disappearing. Anthropic agreed to delete its shadow libraries. OpenAI deleted Books1 and Books2 before anyone asked it to. From those books, models were built, and the models are still here.

And then there is the paper. The Internet Archive’s printed books sit in shipping containers, out of circulation, so that each scan can stand in for exactly one book, a system a court has now said was not enough. Anthropic’s printed books are gone, thrown away page by page as they were scanned. The paper that survived is the paper nobody can read. The paper that was destroyed gets read, in the only way it still can, every time someone asks a question of a model trained on it.

Who gets to copy knowledge? Almost anyone, it turns out, who can buy it first and keep it to themselves. What’s left is the question from 2008, the one that ends with a question mark: scanning entire libraries, and only allowing the folks at the company to read them. Back then it was written as an outrage. Today it reads like a fair summary of the law.

Footnotes

1. Bartz v. Anthropic PBC, No. C 24-05417 WHA, Order on Fair Use (N.D. Cal. June 23, 2025), at 4, 15. ↩

2. Bartz, at 4 n.2. ↩

3. Brown et al., “Language Models are Few-Shot Learners” (OpenAI, 2020), section 2.2 and Table 2.2. ↩

4. In re OpenAI, Inc. Copyright Infringement Litigation, No. 25-md-3143, Opinion and Order (S.D.N.Y. Nov. 24, 2025) (Wang, M.J.). ↩

5. The paper’s author list includes Dario Amodei and Benjamin Mann (arXiv:2005.14165). The Bartz order, at 2, identifies Amodei as Anthropic’s “cofounder and chief executive officer” and Mann as “another Anthropic cofounder.” ↩

6. Class Plaintiffs’ Memorandum in Support of Partial Summary Judgment, In re OpenAI, No. 25-md-3143, Doc. 1982 (unsealed Sept. 17, 2026), as quoted by the Authors Guild. These are the plaintiffs’ exhibits; OpenAI disputes the plaintiffs’ reading of them. ↩

7. Bartz, at 1–3, 19. ↩

8. Bartz, at 2. ↩

9. Authors Guild summary of Doc. 1982, above; Wang order, above; on Judge Stein’s order of Feb. 6, 2026, see Sterne Kessler, “Privilege Preserved”. ↩

10. Bartz, at 3–4. ↩

11. “Anthropic ‘destructively’ scanned millions of books to build Claude,” The Washington Post, Jan. 27, 2026. ↩

12. Bartz, at 13–19. ↩

13. Authors Guild, “What Authors Need to Know About the Anthropic Settlement”; Authors Guild, “Court Grants Final Approval” (July 2026). ↩

14. Bartz, at 18 (quoting the hearing transcript at 53). ↩

15. Open Library, “About”; Hachette Book Group, Inc. v. Internet Archive, No. 23-1260 (2d Cir. Sept. 4, 2024), at 11. ↩

16. Hachette, at 24. ↩

17. Authors Guild v. Google, Inc., 804 F.3d 202 (2d Cir. 2015); Bartz, at 15. ↩

18. “Guerilla Open Access Manifesto” (July 2008). MIT’s 2013 review of the Swartz case notes that he was not its only author (MIT report, Part II.A.2 n.13). ↩

19. “Authors Guild, Co-Plaintiffs Seek Summary Judgment in OpenAI Case,” Publishers Weekly, Sept. 2026; “OpenAI’s ChatGPT Was Built on Concealed ‘Mass Piracy’, Authors Tell Court,” TorrentFreak, Sept. 2026. ↩

20. Statement of Interest of the United States, In re OpenAI, No. 25-md-3143 (S.D.N.Y. Sept. 1, 2026). ↩