Last summer, Peter Degen’s postdoctoral supervisor came to him with an unusual problem: One of his papers was being cited too much. Citations are the currency of academia, but there was something unusual about these. Published in 2017, the paper had assessed the accuracy of a particular type of statistical analysis on epidemiological data and had received a respectable few dozen citations in other research papers over the years, but now it was being referenced every few days, hundreds of times, placing it among the most cited papers of his career. Another professor might be thrilled. Degen’s adviser asked him to investigate. 

Degen, a postdoctoral researcher at the University of Zurich Center for Reproducible Science and Research Synthesis, found that the citing papers all followed a similar pattern. Like the original, they were analyzing the Global Burden of Disease study, a publicly available dataset compiled by the Institute for Health Metrics and Evaluation at the University of Washington. But they were using the dataset to churn out a seemingly endless supply of predictions: about the future likelihood of stroke among adults over 20 years old, of testicular cancer among young adults, of falls among elderly people in China, of colorectal cancer among people who eat minimal whole grains, of disease X among population Y, and so on. 

Searching on GitHub for code that would be used to do this sort of analysis, Degen followed some links and wound up on the Chinese social media site Bilibili, where he discovered a Guangzhou-based company touting tutorials on how to produce publishable research in under two hours using its software tools and AI writing assistance. These studies were not very good. Researchers who analyzed a subset of studies about headaches found they were rife with errors and misrepresentations. But they were also not as flagrantly wrong as AI-generated papers of the recent past, making them more difficult to filter out. 

“It’s a huge burden on the peer-review system, which is already at the limit,” Degen said. “There’s just too many papers being published and there’s not enough peer reviewers, and if the LLMs make it so much easier to mass produce papers, then this will reach a breaking point.”

Optimists about generative AI have high hopes for its ability to produce future scientific breakthroughs — accelerating discovery, eliminating most types of cancer — but the technology is currently undermining one of the pillars of scientific research, inundating editors and reviewers with an endless stream of papers. Paradoxically, the better the technology gets at producing competent papers, the worse the crisis becomes. 

For the past decade, academic publishing has been contending with so-called “paper mills,” black-market companies that mass-produce papers and sell authorship slots to academics, doctors, or others who hope to gain a competitive edge by having published research on their resumes. It has been a game of cat and mouse, with publishers — often pressed by so-called science sleuths, researchers who specialize in ferreting out fraudulent research — closing one vulnerability only to have the mills find a new one. Generative AI was a boon to the mills, helping them to skirt plagiarism detectors by creating wholly new images and text. Still, the technology’s telltale hallucinations meant that publishers could at least theoretically screen out much of their work. In practice, papers still got through, only to get retracted when sleuths encountered a diagram of a rat with inexplicably gargantuan genitals labeled “testtomcels” or prose sprinkled with “as an AI assistant”s that someone forgot to delete. 

But now AI has improved to the point where it can produce convincing papers almost wholesale, allowing desperate academics in need of a publication to mill papers of their own. The result is a deluge of scientific slop that threatens to swamp publishing, peer review, grant making, and the research system as it exists today. 

Matt Spick, a lecturer in health and biomedical data analytics at the University of Surrey and an associate editor at Scientific Reports, first noticed the phenomenon when he received three strikingly similar papers analyzing the US National Health and Nutrition Examination Survey (NHANES), another public dataset. He checked Google Scholar and realized that it wasn’t a coincidence: There had been a sudden explosion in papers citing NHANES that all followed a similar formula, each purporting to discover an association between, for example, eating walnuts and cognitive function or drinking skim milk and depression. 

“If you’ve got enough computing power, you go through and you measure every single pairwise association, and eventually you find some that haven’t been written on before and you just publish: There is a correlation between this and that,” Spick said. These correlations are often misleading simplifications of phenomena with multiple causes or random statistical flukes. “One was that how many years you spend in education will cause postoperative hernia complications. That is just a random correlation. What am I supposed to do with that? Leave school early so that I won’t get a postoperative hernia complication later?”

Over the years, sleuths have developed a variety of methods for detecting inauthentic papers. Some search for “tortured phrases,” instances where someone was trying to skirt plagiarism detectors by feeding an existing paper through a synonym generator, which often has the effect of turning technical terms like “reinforcement learning” into nonsense like “reinforcement getting to know,” to cite one recent example. Other sleuths track duplicated images, perform network analysis of authors, or check citations for hallucinated publications, a classic sign of LLM use. Spick searches for masses of papers following the same template as they analyze public datasets. 

These papers may not necessarily be wrong, though they are often misleading. Nor are they strictly speaking fraudulent. They’re just useless, and suddenly very easy to make. Last year, several journals began restricting submissions of papers analyzing public datasets, citing a flood of redundant research. 

Spick fears these measures may be fighting the last battle. In recent months, AI companies have released a range of “agentic” science assistants capable of analyzing data, generating hypotheses, and writing research papers with a high degree of autonomy. While a possible step toward the goal of AI-accelerated science, these systems also come with novel risks. When Carnegie Mellon researchers tested several agentic tools, they found that they sometimes invented data or used misleading techniques, but that these errors were only apparent upon close analysis of the full workflow; the final papers looked polished. 

Announcing an AI paper writing assistant earlier this year, OpenAI’s then-vice president for science, Kevin Weil, predicted, “I think 2026 will be for AI and science what 2025 was for AI and software engineering.” Spick and some colleagues, curious what it could do, gave the tool, called Prism, some data from an already published paper documenting ripening times of eggplants and peppers. Prism analyzed the data, proposed a new statistical method that could be applied to it, and wrote an entire paper complete with charts and correct citations. 

“We were all looking at each other like, ‘What the [expletive], this is actually a decent piece of work!’” Spick recalled. Unlike the generated papers he’d encountered previously, this one didn’t follow a template, nor was it using a single well-known database. It took 25 minutes and 50 seconds to produce. 

“I’m genuinely not sure at what point we will suddenly realize that more are getting through than we realize because we can’t easily tell the difference anymore,” Spick said. 

This raises some philosophical questions, Spick said, like: Does it matter who or what writes the paper if the information is accurate? And should science be in the business of publishing every possible fact?

“Part of science is supposed to be the filter. We’re supposed to publish the stuff that we think is interesting, not publish literally everything that we can possibly find,” Spick said. “Because if we do that, science is just spamming the world with all the data, irrespective of whether it constitutes actual new knowledge or not, and in any kind of medium-term time frame, it’s almost impossible to work out what’s meaningful and what isn’t.”

This is the immediate practical challenge posed by AI agents. They threaten to overwhelm the human systems that create and organize knowledge. Research funders are contending with onslaughts of proposals perfectly tailored to their particular grant, unable to parse which projects represent the next step in years of work and which were generated in minutes. Conference organizers, journal editors, and peer reviewers are all struggling to sort through a flood of material that all seems good enough at first glance to warrant a close read. There is an enormous and growing asymmetry between the time it takes to produce new work and the time it takes a subject-matter expert to vet it. 

For Marit Moe-Pryce, the managing editor of the international relations journal Security Dialogue, submissions are up 100 percent over where they were a year before. Just as problematic: All the submissions have become pretty good. Gone are the blatant hallucinations and leftover prompts; everything has suddenly become coherent, well structured, and stylistically similar, difficult to say whether it is a wholly generated paper, an experienced academic, or a young scholar using AI as an editor. 

“The main problem that we see currently from the desk is that the fraudulent side and the academic side are conflating, which ends up with a big gray mass of articles that we as editors need to sit and try to figure out, ‘What is this? Is this something that we need to engage with? Is it not?’” Moe-Pryce said. 

One paper made it past at least 10 editors and two rounds of peer review before she noticed a fake citation — a very plausible one, involving several former editors of the journal on a topic they could have written about but never did. She then found several more. She doesn’t know at what stage of revision the hallucinations were introduced, but the close call underscored the level of care required to ensure nothing false gets published. Now that models increasingly cite real papers, she has to read for whether the works cited are the ones an expert would actually use, AI not yet having mastered the difference between canonical literature and more peripheral work.

“It’s incredibly detailed, and this is a normal part of the editorial work. The difference is that now you have to do that for all the rubbish that comes through the door,” Moe-Pryce said. “That’s why our workload becomes so unmanageable.”

Academic papers go through a multi-stage review process before publication. First, manuscripts are triaged for obvious problems, then sent to a journal’s editor, who decides whether it might be worth publishing. The editor then sends it to an associate editor with experience in the field, who again vets it before recruiting two or three subject-matter specialists — the “peers” in peer review — to read the paper and write responses. The editors and reviewers are typically working for free, volunteering their time in addition to their primary academic job. 

The review system was already struggling under increasing volumes of submissions, and now AI is increasing those volumes while also making the bad ones more difficult to filter out. Moe-Pryce now spends more time sorting papers before deciding what to send out for review, and prospective reviewers, swamped themselves, are less and less likely to respond. Where she previously could send four queries out and get three replies, it now takes her a dozen tries to get two people. Increasingly, she reaches out to 20 reviewers and hears nothing. 

“It’s fatigue. Academic journals have mushroomed, and then you have AI helping everyone fraudulent or not generate more, faster, so you have a massive increase in volume,” she said. “AI currently holds the potential to bring down the publishing system as we know it.”

The journal Accountability in Research has seen a 60 percent surge in submissions this year, according to David Resnik, an associate editor at the journal. Ironically, he has been besieged by likely AI-generated papers about fraudulent academic papers that have mined public data compiled by the organization Retraction Watch. 

He, too, is struggling to find reviewers. At times, he’s had to send out 20 requests just to get two responses — and he’s suspected that some of the responses he’s received are AI-generated themselves. He has reason to be suspicious. A survey conducted by the publishing company Frontiers last year found that more than half of researchers have used AI assistance in their peer review. 

“I’m very worried about this straining, breaking the back of the peer-review system,” said Resnik. 

AI agents arrive at a time when the quality filters of academia are already struggling to cope with a superabundance of papers. The number of scientific papers published has grown exponentially in recent years, according to an analysis of data published in Quantitative Science Studies, while the number of PhDs who might review them has not. Unfortunately, the authors attribute this explosion in productivity not to rapid progress in science but to the fact that commercial and professional incentives align to publish the maximum quantity of papers. 

Many journals have shifted to an “open access” model where they earn revenue by charging authors processing fees to have their papers published, as opposed to charging for subscriptions. In earnings calls, publishing companies tout the recent 20 percent or more increase in submissions as a positive growth story. Universities and funding agencies, meanwhile, look at researchers’ publication metrics when deciding whom to fund or promote, which means researchers are under pressure to “publish or perish.” Nor is it only traditional academics who are under this pressure to publish. Overseas medical students can improve their chance at a US residency program by having a few peer-reviewed papers on their resume. In China, medical doctors have strong incentives to publish despite neither having the time nor resources to conduct research, making quick paper generation an attractive option.

If you introduce an infinite paper-writing machine to a system that defines productivity by the number of papers written, people will use it to write a lot of papers. A study published in Nature this year found that scientists who adopted AI published three times more papers and received nearly five times more citations than those who didn’t. They also became research project leaders 1.37 years earlier than those who did not use AI. While individually beneficial, the embrace of AI to mass-produce papers may be detrimental to science as a collective endeavor, beyond exhausting journal editors and peer reviewers. The same study found a collective narrowing of focus as these newly productive scientists gravitated toward well-studied fields with abundant existing data for AI to synthesize.

There are no easy solutions to this problem. In 2022, the scientific organization STM launched an initiative called Integrity Hub to contend with paper mills. Since then, it has been engaged in an “arms race” with AI, according to Joris van Rossum, the project’s program director — assembling automated tools that check for plagiarism, then tortured phrases, then fake citations — but the group must now consider more sweeping remedies. 

“We anticipate a future where it’s going to be more realistic to enable submitters to demonstrate authenticity rather than trying to detect fabrication,” he said. That is, once fraudulent manuscripts are impossible to detect, publishers will have to find a way for researchers to prove their work is real — perhaps by working with instrument manufacturers to develop ways of watermarking their images, he said, or having researchers submit more of the data behind their work so it can be analyzed for suspicious signals. 

This would entail changing the way research is done on a massive scale, and while it might stem outright fraud, it would do little to reduce the volume problem. Using AI to assist with peer review, as some have proposed — and some reviewers are already doing, permitted or not — raises a nest of other possible risks. Studies have found that models often continue to cite retracted studies as valid and write superficially good reviews while overlooking methodological problems. AI reviewers also appear to prefer AI-generated writing.

“It’s not really a tractable problem,” said Reese Richardson, a postdoctoral fellow at Northwestern University who studies mass-produced papers. “I think that the only way out of this situation is to actually change the way that the scientific enterprise awards prestige and awards resources. As long as we have this hyper-competitive, hyper-unequal rat race where people’s productivity and their worth as scientists is being measured by how many publications they put out and how many times they get cited, it’s just going to incentivize this behavior.”

Vincent Larivière, the editor-in-chief of Quantitative Science Studies, had a similar diagnosis. His journal has seen a 40 percent increase in submissions this year. 

“We need a reform of what matters in science,” Larivière said. The conflation of scientific productivity with publication counts has had a distorting effect on science, causing research to gravitate toward small, tractable problems that are guaranteed to result in something publishable. AI could do great things, he said — help cure cancer, develop fusion energy — but right now it is being used to generate papers to “pad CVs.” 

“Of course we need more science,” he said, “but do we need more papers?”

The vast majority of the world’s data — emails, financial transactions, the internet — is carried by fiber optic cables that run along the ocean floor and converge at a few narrow choke points. Periodically, policymakers will release reports noting that this arrangement seems risky, but these routes are the shortest, often in use since the telegraph era, and the system has managed remarkably well. Cables break regularly, and traffic gets rerouted until a repair ship can come and fix the cut. But the war in Iran, coming after several years of disruptions from conflict in Yemen, is spurring governments and companies to consider alternate routes, including one going across the North Pole. 

The current problems began in 2024, when a Houthi missile struck a cargo ship in the Bab-el-Mandeb Strait off the coast of Yemen, causing the vessel to drift for days and drag its anchor across three of the more than a dozen submarine cables crammed into the narrow Red Sea passage. 

Cable repair is carried out by specialized ships that fish up the broken ends and splice them back together. It’s delicate work that involves slowly dragging grapnels along the seafloor and floating very still for hours while fiber strands are spliced together, none of which can be safely done in a war zone. Consequently, it took more than four months to broker the agreements necessary to bring in a ship. Last September, another four cables were severed, likely by a commercial vessel dragging its anchor, again disrupting internet traffic in Africa, Asia, and the Middle East. Again, months of negotiations before a repair could be done. 

The Red Sea cuts spurred companies and governments to look for alternate routes, and the Strait of Hormuz seemed promising. Then the US and Israel attacked Iran, cable projects were halted, and now the world is looking elsewhere once again. 

“When the Red Sea shut everything down, everyone swung over to the Persian Gulf, and now you can’t do that either,” said Roderick Beck, a cable industry veteran who sources telecom capacity for ISPs. “The Persian Gulf will never go back to what it was before, when the Iranians wouldn’t dare assert control.” 

The Gulf states, which have been aggressively building data centers in an attempt to shift their economies from oil to AI, are looking to avoid the Red Sea by going overland, building routes to Europe via Syria, Iraq, and Oman. But the most ambitious proposal is in Europe, where the repeated cable cuts have the continent looking to the Arctic. 

Earlier this year, a European Union panel on cable resilience recommended building two Arctic cables in order to find a route to Asia without traveling through the Red Sea, where 90% of Europe’s traffic currently passes. One cable would go through Canada’s Northwest Passage. The other would link Scandinavia to Asia by going straight across the North Pole. 

The second of these routes is already in the early planning stages. Called Polar Connect, it’s being led by Nordic academic-network operators, Sweden’s polar research agency, and the telecom firm GlobalConnect Carrier. This year, the EU designated it a “Cable Project of European Interest” and has put approximately 9 million euros toward preparatory work. (The EU report estimated the full cost would be approximately 2 billion.) A route survey is planned for this summer. 

“It started before the unrest, but the geopolitical situation has resulted in an increased interest in finding alternate routes,” said Pär Jansson, Senior Vice President (Carrier) at GlobalConnect, the telecom company working on the Polar project. The group’s white paper notes that Europe’s data currently has three routes to Asia, none of them ideal: through the Red Sea, through Russia, or through the US, a “long route controlled by non-European entities.” The cable would make Europe’s data infrastructure more resilient, lower latency between the EU and Asia, and “strengthen Europe’s autonomy,” Jansson said, adding that it could also allow for better environmental monitoring of the Arctic. 

Others have attempted an Arctic cable, never successfully. “People have discussed this for at least 20 years,” said Alan Mauldin, a research director at TeleGeography, the cable industry research firm. Installation would be challenging and expensive, requiring retrofitting a cable ship for Arctic conditions and procuring icebreakers to escort it across the North Pole. But the real obstacle is maintenance. 

“What if there is damage to the cable from, it’s called ice scour, when ice scrapes against her cable and damages it. Then you can’t repair it until summer,” Mauldin said. “We’ve seen so many projects come and go. There’s a reason for that, right? It’s very challenging.”

Beck raised the same repair issue. “The problem is icebergs,” said Beck. They can drag along the bottom of the ocean floor, digging long grooves deeper than a cable can be buried. “That’s what happened to Quintillion. Twice.”

Quintillion was the last attempt at an Arctic cable. In 2016 it acquired the assets of Arctic Fibre, the previous attempt to build an Arctic cable between Europe and Asia. Quintillion activated a portion that ran from Nome along the northern coast of Alaska to Prudhoe Bay, but in June 2023, sea ice broke it. Because there are no icebreaker cable ships, Quintillion had to wait for the summer ice to melt before it could fix the cable. Then in January of last year, an iceberg struck again. This time in deep winter, no one could repair the cable for eight months. The rest of the route was never laid.

The expensive repair costs and potential for lengthy downtimes makes an Arctic cable financially unattractive, Mauldin and Beck said. The question is whether governments now see the cable as strategically important enough to outweigh that. “I think the EU is really big on this thing because they think it’s data sovereignty, but it would be enormously expensive. It’s never been done before,” said Beck. 

Jansson is aware of the challenges, but he believes the new geopolitical situation and new technologies will make it feasible. Tech companies are building data centers in the Nordic countries, he said, and will want fast and resilient connectivity, but ultimately it will require public investment. He places the cost estimate for the Norway-Japan leg at “below 1 billion euros.” 

The goal is for it to go live in 2030. That may be the easy part.

The LinkedIn post seemed like yet another scam job offer, but Katya was desperate enough to click. After college, she’d struggled to make a living as a freelance journalist, gone to grad school, then pivoted to what she hoped would be a more stable career in content marketing — only to find AI had automated much of the work. This company was called Crossing Hurdles, and it promised copywriting jobs starting at $45 per hour.

Katya clicked and was taken to a page for another company, called Mercor, where she was instructed to interview on-camera with an AI named Melvin. “It just seemed like the sketchiest thing in the world,” Katya says. She closed the tab. But a few weeks later, still unemployed, she got a message inviting her to apply to Mercor. This time, she looked up the company. Mercor, it seemed, sold data to train AI, and she was being recruited to create that data. “My job is gone because of ChatGPT, and I was being invited to train the model to do the worst version of it imaginable,” she says. The idea depressed her. But her financial situation was increasingly dire, and she had to find a new place to live in a hurry, so she turned on her webcam and said “hello” to Melvin.

It was a strange, if largely pleasant, experience. Manifesting on Katya’s laptop as a disembodied male voice, Melvin seemed to have actually read her résumé and asked specific questions about it. A few weeks later, Katya, who like most workers in this story asked to use a pseudonym out of fear of retaliation, received an email from Mercor offering her a job. If she accepted, she should sign the contract, submit to a background check, and install monitoring software onto her computer. She signed immediately.

She was added to a Slack channel, where it was clear she was entering a project already underway. Hundreds of people were busy writing examples of prompts someone might ask a chatbot, writing the chatbot’s ideal response to those prompts, then creating a detailed checklist of criteria that defined that ideal response. Each task took several hours to complete before the data was sent to workers stationed somewhere down the digital assembly line for further review. Katya wasn’t told whose AI she was training — managers referred to it only as “the client” — or what purpose the project served. But she enjoyed the work. She was having fun playing with the models, and the pay was very good. “It was like having a real job,” she says.

Two days after Katya started, the project was abruptly paused. A few days after that, a supervisor popped into the room to let everyone know it had been canceled. “I’m working assuming that I can plan around this. I’m saving up for first and last month’s rent for an apartment,” Katya says, “and then I’m back on my ass. No warning, no security, nothing.” Several days later, she got an email from Mercor with another offer, this one for a job evaluating what seemed to be conversations between chatbots and real users — many appeared to be from people in Malaysia and Vietnam practicing English — according to various criteria, like how well the chatbot followed instructions and the appropriateness of its tone. Sign the contract, the email said, and you’ll have a Zoom onboarding call in 45 minutes. It was 6:30PM on a Sunday night. Scarred from the abrupt disappearance of the previous gig, she accepted the offer and worked until she couldn’t stay awake.

Machine-learning systems learn by finding patterns in enormous quantities of data, but first that data has to be sorted, labeled, and produced by people. ChatGPT got its startling fluency from thousands of humans hired by companies such as Scale AI and Surge AI to write examples of things a helpful chatbot assistant would say and to grade its best responses. A little over a year ago, concerns began to mount in the industry about a plateau in the technology’s progress. Training models based on this type of grading yielded chatbots that were very good at sounding smart but still too unreliable to be useful. The exception was software engineering, where the ability of models to automatically check whether bits of code worked — did the code compile, did it print HELLO WORLD — allowed them to trial-and-error their way to genuine competence.

The problem was that few other human activities offer such unambiguous feedback. There are no objective tests for whether financial analysis or advertising copy is “good.” Undeterred, AI companies set out to make such tests, collectively paying billions of dollars to professionals of all types to write exacting and comprehensive criteria for a job well done. Mercor, the company Katya stumbled upon, was founded in 2023 by three then-19-year-olds from the Bay Area, Brendan Foody, Adarsh Hiremath, and Surya Midha, as a jobs platform that used AI interviews to match overseas engineers with tech companies. The company received so many inquiries from AI developers seeking professionals to produce training data that it decided to adapt. Last year, Mercor was valued at $10 billion, making its trio of founders the world’s youngest self-made billionaires. OpenAI has been a client; so has Anthropic.

Each of these data companies touts its stable of pedigreed experts. Mercor says around 30,000 professionals work on its platform each week, while Scale AI claims to have more than 700,000 “M.A.’s, Ph.D.’s, and college graduates.” Surge AI advertises its Supreme Court litigators, McKinsey principals, and platinum recording artists. These companies are hiring people with experience in law, finance, and coding, all areas where AI is making rapid inroads. But they’re also hiring people to produce data for practically any job you can imagine. Job listings seek chefs, management consultants, wildlife-conservation scientists, archivists, private investigators, police sergeants, reporters, teachers, and rental-counter clerks. One recent job ad called for experts in “North American early to mid-teen humor” who can, among other requirements, “explain humor using clear, logical language, including references to North American slang, trends, and social norms.” It is, as one industry veteran put it, the largest harvesting of human expertise ever attempted.

These companies have found rich recruiting ground among the growing ranks of the highly educated and underemployed. Aside from the 2008 financial crash and the pandemic, hiring is at its lowest point in decades. This past August, the early-career job-search platform Handshake found that job postings on the site had declined more than 16 percent compared with the year before and that listings were receiving 26 percent more applications. Meanwhile, Handshake launched an initiative last year connecting job seekers with roles producing AI training data. “As AI reshapes the future of work,” the company wrote, announcing the program, “we have the responsibility to rethink, educate, and prepare our network to navigate careers and participate in the AI economy.”

There is an underlying tension between the predictions of generally intelligent systems that can replace much of human cognitive labor and the money AI labs are actually spending on data to automate one task at a time. It is the difference between a future of abrupt mass unemployment and something more subtle but potentially just as disruptive: a future in which a growing number of people find work teaching AI to do the work they once did. The first wave of these workers consists of software engineers, graphic designers, writers, and other professionals in fields where the new training techniques are proving effective. They find themselves in a surreal situation, competing for precarious gigs pantomiming the careers they’d hoped to have.

Each of the more than 30 workers I spoke with occupied a position along a vast and growing data-supply chain. There are people crafting checklists that define a good chatbot response, typically called “rubrics,” and other people grading those rubrics. Others grade chatbot answers according to those rubrics, and still others take the rubrics and write out what’s often described as a “golden output,” or the ideal chatbot answer. Others are asked to explain every step they took to arrive at this golden output in the voice of a chatbot thinking to itself, producing what’s called a “reasoning trace” for AI to follow later when it encounters a similar task out in the real world.

Sometimes the labs want only rubrics for prompts their AI can’t already do, which means companies like Mercor ask workers to produce “stumpers,” or requests that will make the model fail. “It sounds easy, but it’s really hard,” says a worker who was trying to stump models by asking them to make inventory-management dashboards. Models fail in counterintuitive ways. They may be able to solve advanced-physics exam questions, but ask them for transit directions and they’ll recommend transferring on nonconnecting train lines. Finding these weak spots takes time and creativity.

One type of project gathers groups of lawyers, human-resources managers, teachers, consultants, or bankers for something Mercor calls world-building. “You and your team will role-play a real-life team within your profession,” the training materials read. The teams are given dedicated emails, calendars, and chat apps and asked to create a hundred or more documents that would be associated with some corporate undertaking, like a fictional mining company analyzing whether to enter the data-center business.

After several 16-hour days of fantasy document production, one worker recounts, the resulting slide decks, meeting notes, and financial forecasts are sent to another team, which uses them as grist in their attempts to stump a model operating in this simulated corporate environment. Then, having stumped the model, that team writes new, more nuanced rubrics, golden answers, and so on. Workers can only guess who the customer is or how many others are working on the project — based on references to teams like Management Consulting World No. 133, there could be hundreds, maybe thousands.

There are people hired to evaluate the ability of image models to follow their prompts and others who summarize video clips in extraordinary detail, presumably to train video models. Efforts to improve AI’s ability to have spoken conversations have resulted in a surging demand for voice actors, who might find themselves recording “authentic, emotionally resonant” speeches, according to one listing. “I just tell people I’m an AI trainer, then it sounds more professional than what I’m doing,” says an aspiring screenwriter who was instructed to record himself pretending to ask a chatbot for a fitness plan while pots and pans clanged in the kitchen. Another time, he was told to record himself dispensing financial advice over the phone to a parade of people he assumed were other workers.

This audio might then be broken down and sent to someone like Ernest, who used to make a living as an online tutor until the company he worked for replaced him with a chatbot. When we spoke, he was listening to minutelong clips of random dialogue slowed to 0.1x speed and marking when someone started and stopped speaking down to the millisecond. Many of the clips included a person talking with a chatbot and interjecting “huh” or “I see,” so he assumes he was improving AI’s ability to have naturally flowing conversation, but he has no actual idea.

As is standard practice in the field, the project was referred to by a codename and the client only ever as “the client.” The entire system is designed so that workers have minimal insight into the supply chain they are part of. If they find out who the customer is, they are contractually forbidden from telling anyone, even their own colleagues. Nor are they allowed to describe the details of their work beyond broad generalities like “providing expertise in XYZ domain to improve models for a top AI lab,” according to one Mercor agreement. So afraid are workers of inadvertently violating their confidentiality agreements and getting fired that when they discuss their work in public forums, they mask their already codenamed projects with additional codenames, for example by referring to a project called “Raven” as “Poe.”

Katya’s second project with Mercor was far more stressful. There was less work to go around, and it came in fits and starts. Managers would drop a message in the Slack channel saying new tasks were incoming in half an hour, and, she says, “everyone in Slack would drop what they were doing and jump on them like piranhas,” working as fast as they could while the bar showing how many tasks remained slid toward zero. Then they were back in Slack again, politely begging supervisors for more work and more hours, talking about their kids’ birthdays or their need to pay rent, or telling anyone who might be listening that their availability was wide open in case there was more work to be done. Soon, Katya was dropping everything at the sound of a Slack ding too. “Sometimes I’m on the toilet or at dinner and I get the Slack notification. I’m like, ‘Oh, sorry, I gotta work now.’”

That project soon ended and then came another. It was nearly identical to the first, which she had enjoyed, but now, on top of writing rubrics, she had to stump the model and complete the more difficult task in the same amount of time. She was also getting paid $8 an hour less. This is common at Mercor. Nearly every worker I spoke with reported that demands increased, time requirements shrank, and pay decreased as projects continued. Those who couldn’t meet the new demands got “offboarded” and replaced by new recruits.

Chris joined Mercor last year, after a difficult few months struggling to find film work. Unlike many people who suspect they’re casualties of automation, he knew for certain that this was the case. He’d had a recurring job drafting episodes for an unscripted television show — doing preinterviews, sketching scenes, writing the reality TV equivalent of a screenplay. But in late 2024, he was told the show would be running on a “skeleton crew” and his work was no longer needed. He found out later the company was using ChatGPT to draft new episodes. So that October, when Chris received an offer to write an entire sci-fi screenplay for a major AI company, he said “yes,” grim as the prospect was. Since then, he has gone from gig to gig. “This is my only source of income right now,” he says. “I know people who are award-winning producers and directors, and they’re not advertising that they’re doing this work, but that’s how they’re putting food on the table.”

His first jobs with Mercor were, like Katya’s, relatively pleasant and well paid, but soon came the 6PM fist-bump-emoji Slack exhortations to “come on team, let’s push through this,” followed by sudden halts and months of silence. “You were just constantly waiting for the crack of the starting gun at any hour of the day,” Chris says. Then it was crunch time again and managers, increasingly panicked as deadlines neared, started threatening workers with offboarding if they didn’t complete tasks quickly enough.

The time he spent working was tracked to the second by software called Insightful, which monitored everything he did on his computer. Time that the software deems “unproductive” could be deducted from his pay, and if a few minutes passed without him typing, the system pinged him to ask whether he had been working. Sometimes Chris saw people post in Slack that they’d gone over the target time on a particularly tricky task and that they hoped it would be okay; the next day, they would be gone.

Increasingly worried he would be offboarded too, he started working off the clock, deactivating Insightful while reading instructions so he could move faster. If he went over the target time, he turned the clock off and kept working for free.

Companies say this software is necessary to accurately track hours and prevent workers from cheating, which, in this case, means using AI, something all data companies strictly forbid. The ground truth of verified human expertise is what they’re selling, and when AI trains on AI-generated data, it gradually degrades, a phenomenon researchers call “model collapse.” Employees of data companies say it is a constant battle to screen out AI slop. For workers, AI is a particular temptation as pressure increases. When the retail expert trying to stump models with analytics dashboards had her target time dropped from eight hours per task to five to three and a half, she turned off Insightful and sought outside help. “To be honest, I went into Copilot and ChatGPT and put my prompt in there and said, ‘How can I work this so you guys can’t answer it?’” Then she went to another chatbot and asked if the prompt sounded AI generated and, if so, to make it sound more human.

“It’s just so horrible, the mental effect of it,” says Mimi, a screenwriter who has worked on multiple streaming shows and has been training AI for Mercor for several months. She found out about Mercor from a fellow screenwriter who dropped one of its job links in a Writers Guild of America Facebook group.

Like a lot of people in this line of work, Mimi is conflicted. “One documentary-maker who’s won Emmys, he messaged me and he was like, ‘I’m being handed a shovel and told to dig my own grave,’ and that’s exactly how everyone thinks about it,” she says. Still, as a single mom, she needed the money. She was thankful for the work at first, then the project was paused, unpaused, and paused again. For five weeks, she was told a project would be starting imminently. When it finally did, requirements were added, while the expected time shortened, and she raced to keep up under the watchful eye of Insightful. She felt that someone put it well on Slack when they said it was like they were living in a fishbowl waiting for their human masters to drop in food, and only the ones who were fast enough to swim to the top could eat.

“Last night, I got so fucking stressed because my kid came home and it was 7PM, and I get this message, ‘The tasks are out!’ and I’m just working, just trying to get as many hours in before I can go to bed,” Mimi says, choking up. “I spend no time with my kid, and at one point, he can’t find something for school and I just start screaming at him. This work is turning me into a fucking demon.” She’s especially disturbed by the surveillance: “The idea that somebody can measure your time and that all the little bits that go into being a human are taken away because they’re not profitable, that you can’t charge for going to the toilet because that’s not time you’re working, you can’t charge for making a cup of coffee because that’s not time you’re working, you can’t charge for having a stretch because your back hurts. This is why unions were formed, so people could have guaranteed hours and guaranteed lunch breaks and guaranteed holidays and sick pay. This is the gig economy to the very extreme.”

This is what concerns her more than the AI itself: that it’s bringing to knowledge work the sort of precarious platform labor that has transformed taxi driving and food delivery. Meanwhile, she watches in horror the desperate gratitude of her colleagues as they rejoice at the 7PM announcement of incoming work.

“How long are these tasks expected to last?” one worker asked in Slack.

“I’m wondering too, I’d like to know whether I can sleep or not.”

With no answer forthcoming, they swapped tips on how to stave off sleep.

When Mercor began recruiting aggressively last year, it framed itself as a more worker-friendly version of the platforms that had come before it. Criticizing his rival Scale AI on a podcast, Foody, Mercor’s CEO, said, “Having phenomenal people that you treat incredibly well is the most important thing in this market.” Workers who joined during this time do report being treated well; the pay was better than elsewhere, and instead of being managed by opaque algorithms, as is common, there were actual human supervisors they could go to with questions.

But people who have worked in management at data companies say they often start out this way, wooing workers off incumbent platforms with promises of better treatment, only for conditions to degrade as they compete to win eight-figure contracts doled out by the half-dozen AI companies who are interested in buying this data in bulk. At Mercor, there was the additional complication of management largely consisting of people in their 20s with minimal work experience who had been given hundreds of millions of investor dollars to pursue rapid growth.

“I don’t care if somebody’s 21 and they’re my manager,” says Chris, the reality TV producer. “But they’ve never worked at this scale. When you try to find some kind of guidance in Slack, very maturely and clearly explaining what the situation is, you get a meme back with a corgi rolling its eyes and it says, ‘Use your judgment.’ But it’s like, ‘Use your judgment and fuck it up, and you get fired.’ You went to Harvard, you graduated last year, and your guidance for a group of people, many of whom are experienced professionals, is a meme?”

Lawyers, designers, producers, writers, scientists — all complained of inexperienced managers giving contradictory instructions, demanding long hours or mandatory Zoom meetings for ostensibly flexible work, and threatening people with offboarding for moving too slowly, threats that were particularly galling for mid-career professionals who felt their 20-year-old bosses barely understood the fields they were trying to automate.

“The founders pride themselves on ‘9-9-6,’” says a lawyer, referring to a term that originated in China to describe 72-hour workweeks associated with burnout and suicide but has been appropriated by Silicon Valley as aspirational. “You need to be accessible at all hours, and they’re going to pump out messages at 6AM, and you better jump because the perception is you will be offboarded and another person will replace you.”

“It’s not just that team leads are young, project managers are young, senior project managers are young. It’s that the senior-senior project managers, the ones responsible for the project in its entirety, are young. I guess that comes from the top because they’re young, right?” says Lindsay, a graphic designer and illustrator in her 50s who came to Mercor after 85 percent of her work evaporated over the past year, owing, she believes, to improvements in generative AI.

Increasingly desperate for work, she scoured job boards; it seemed the only listings matching her expertise were offers to help build the technology she blamed for demolishing her career. “I swallowed my hatred and signed up,” she says. After some initial work producing graphic-design data, she was invited to join a job for Meta grabbing videos from Instagram Reels and tagging whatever was in them. It was boring, and at $21 per hour, the pay was middling, but Lindsay needed the money. So, she discovered when she was brought into the project’s Slack, did approximately 5,000 others.

In early November, a Mercor representative announced that Lindsay’s project would be ending owing to “scope changes,” though workers had previously been told the project would run through the end of the year. Lindsay and thousands of others found themselves removed from the company’s Slack.

Soon, an email arrived in their inbox, inviting them to a new project called Nova paying $16 per hour.

Thousands of workers poured into the new Slack only to discover it was the exact same job, now paying 24 percent less. All but two of the Slack channels had been deleted, including the watercooler, support, and help rooms. The ability to direct-message one another had also been cut off. There were no team leads to be found. With no one to ask for assistance, workers flooded the main rooms with pleas and indignation.

“Nobody knows what’s going on. Everybody’s really confused,” says Lindsay. “The messages are coming so fast in that channel. It’s just absolute chaos. ‘Help, please. What do I do? What am I supposed to do? Where do I go? Can I get started tasking? Am I supposed to redo all the assessments that I’ve done before?’”

Someone emailed support asking for help, and for some reason that email was sent to every one of the thousand-some people on the project, who seized on it and began to reply-all with their bafflement and outrage. “It was absolute carnage,” says Lindsay. “There’s no other word for it.”

Workers began posting complaints on Mercor’s subreddit, only to have their posts quickly deleted by the Mercor representatives who moderate it. In response, two unsanctioned Mercor subreddits were created, where workers could freely express such sentiments as “CHILDREN RUN THIS COMPANY, THEY WILL SOON HAVE THEIR DAY OF RECKONING.”

“It’s just really sad,” says Lindsay. “There are some people in there where it’s genuinely the difference between them being able to feed their families and not feed their families.”

“I hate gen AI,” she adds. “I think AI should be used for curing cancer. I think it should be used for space exploration, not in the creative industries. But I need to be able to pay my rent. And then when people like Mercor pull this stuff where they treat you like nothing more than a lab rat — I’ve been working for a very long time. I have never, ever been treated as badly as this.”

Intermittent work, extreme secrecy, and abrupt firings are the norm across the data industry. On Surge AI’s work platform, called Data Annotation Tech, workers are not only regularly terminated without explanation; they are often not even told they’ve been fired. They just log in one day and find the dashboard empty of tasks. The phenomenon is so ubiquitous they call it simply “the dash of death.”

Last year, a Texan with a master’s degree in divinity who was teaching voice models to respond to queries with appropriate levels of feeling — different tones for a user telling them their dog died versus asking for a trip itinerary — logged in to work one morning and found his dashboard empty. Scrolling to the bottom of the page for the support button, he discovered it no longer worked. That’s when he knew he had been terminated. His mind raced through possible reasons: Had he worked too much? Had his quality slipped? He knew he would never find out. “I felt cut adrift,” he says. Anxious about how he would pay his bills and care for his ailing dog, he grew depressed, then horrified. He thought about his teacher friends who couldn’t get their students to write and all the people graduating with now-worthless computer-science degrees. “The technology makes us see everything as a utility, something to be used,” he says, a category that he feels includes discarded data workers like himself. He resolved to become a chaplain, figuring that no matter what the AI future holds, people will need a fellow human to be there for them.

The on-again, off-again nature of the work is not just the result of company culture; it stems from the cadence of AI development itself. People across the industry described the pattern. A model builder, like OpenAI or Anthropic, discovers that its model is weak on chemistry, so it pays a data vendor like Mercor or Scale AI to find chemists to make data. The chemists do tasks until there is a sufficient quantity for a batch to go back to the lab, and the job is paused until the lab sees how the data affects the model. Maybe the lab moves forward, but this time, it’s asking for a slightly different type of data. When the job resumes, the vendor discovers the new instructions make the tasks take longer, which means the cost estimate the vendor gave the lab is now wrong, which means the vendor cuts pay or tries to get workers to move faster. The new batch of data is delivered, and the job is paused once more. Maybe the lab changes its data requirements again, discovers it has enough data, and ends the project or decides to go with another vendor entirely. Maybe now the lab wants only organic chemists and everyone without the relevant background gets taken off the project. Next, it’s biology data that’s in demand, or architectural sketches, or K–12 syllabus design.

To compete, data companies arrange things so that they will always have workers on call while preserving their freedom to drop them at a moment’s notice. “Every vendor is going to have some kind of setup whereby they don’t really make promises to people,” says a senior employee of a major data company. The companies rarely have much notice of these shifts themselves, sometimes because the AI developers aren’t sure exactly what data they need in the first place, other times because they are shopping around for the best deal. “They want to keep us in the dark,” the employee continues, “so we inevitably keep the contributors in the dark, then a purchase falls through and you have a thousand people you’ve trained and formed a relationship with just saying, like, ‘What the fuck? Why isn’t there work?’ It’s a horrible feeling from an operator’s perspective, too, but obviously it’s way worse for them.”

The workers at the bottom of this supply chain exist in a state of extreme precarity and maximum competitive frenzy — especially because their strict confidentiality agreements make it impossible for them to establish any kind of seniority or relationship that might outlast a particular project. “The power is all on one side because they can’t talk about it,” says Matthew McMullen, a strategy and operations executive who has worked in the industry since the self-driving-car boom in the mid-2010s. “The labs benefit from you not being able to leverage your experience in the market, and this silence is like their pricing power. The silence is their ability to extract mass information from people without giving them the power to object or to unionize or to make companies themselves. As long as they can’t prove what they’ve done, these raters can’t demand what they’re worth. The only way that people can demand things is by showing their ability to step up, to take on more work. The only power that they have is to keep going, to get back in line.”

Which is what they do. When a project for Mercor ends, managers often post a link to other projects on the platform and encourage people to apply. “But again, there are thousands of people applying, so you throw your application into a hole and hope to hear back at some undefined point,” says Katya. While they wait, workers sign up for Handshake, Micro1, Alignerr, or another of the ever-growing number of data providers.

These companies are always recruiting. Like Mercor, many use AI interviewers and automated evaluations, meaning they have no incentive to limit the number of interviews they do. Mercor offers referral bonuses of several hundred dollars, leading some to promote the company so aggressively that mentions of it have been banned from several subreddits. Katya has applied for dozens of jobs and gotten three, not an unusual ratio.

Nor do companies bear any cost for overhiring. Because workers are ostensibly independent contractors, they are not owed paid time off, breaks, healthcare, overtime pay, or unemployment benefits. It’s free to keep them hanging around, and a surplus of vetted workers ensures they will jump quickly to finish tasks before someone else does. It all combines to create an arrangement in which employers can turn labor on and off like a tap. (Reached for comment, Mercor spokesperson Heidi Hagberg said that “the nature of this is project based contract work, meaning it can extend, pause, or end at any time, especially as the client’s scopes and needs evolve,” and that many of the worker complaints “were centered around the misalignment of expectations of a full-time job versus -project-based work.”)

If you move fast and get lucky and have the right combination of expertise and stay on the right side of each platform’s unique and mysterious recipe of productivity metrics, you can make decent money. I spoke to a playwright making $10,000 a month, a multitalented chemist who at various points found gigs demonstrating poker and singing for AI. But even then, there is an inescapable awareness of ephemerality because producing training data means working toward your own obsolescence. While the number of people doing data work may continue to rise, any particular gig will last only as long as it takes for the machines to successfully mimic it. It takes years for a human to develop expertise, and sooner or later, they’re going to run out of skills to sell.

A worker with a master’s in linguistics had found steady rubric work for a year, but late in 2025, he noticed it was becoming more difficult to stump the models. Any obscure theory or Indigenous language he asked about, the model would find the correct papers. Instead of submitting three or four rubrics per week, he was lucky to get one. Everyone else on the project was following the same trajectory, so he wasn’t surprised when it came to an end. Their know-how had been extracted. In the past, he’d always been able to find a new gig, but now when he looked around, he saw only requests for medical experts, human-resources managers, and teachers. He has now been without work for five months and isn’t sure what to do next.

To the extent that policy responses to AI automation are discussed at all, they mostly concern what to do when AI renders large categories of workers obsolete. Maybe this will happen, but another possibility is that particular tasks will get automated and humans redistributed to other parts of the production process, some revising so-so AI output, others crafting rubrics to improve it. Much of this work will be inherently intermittent, which means it will be done by independent contractors, workers whom current regulations leave almost wholly unprotected. Daron Acemoglu, a professor of economics at MIT who studies automation, compares the situation to that of weavers, who before the industrial revolution were “like the labor aristocracy,” self-employed artisans in control of their own time. Then came weaving machines, and in order to survive, they were forced to take new jobs in factories, where they worked longer hours for less money under the close supervision of management. The problem wasn’t simply that technology took their jobs; it enabled a new organization of work that gave all power to the owners of capital, who made work a nightmare until labor organizing and regulation set limits.

Early labor skirmishes are already happening, mostly in California, which has some of the most aggressive rules around classifying platform workers. Three class-action lawsuits have been filed against Mercor in the past six months. (Similar suits were previously filed against Surge AI and Scale AI, which is settling.) The lawsuits all accuse the companies of misclassifying workers as independent contractors given the “extraordinary control” they exert over them. This is “an entirely new kind of work,” one that the company trains people to do and that cannot be done except on the company’s platform. Workers have so little visibility into what they’re working on that one person, alleges a suit filed in December, accepted a Mercor project only to be tasked with recording himself reading sexually explicit scripts. Once he discovered this, the worker risked deactivation if he abandoned the project, forcing him to “choose between being paid and being humiliated.”

These companies are reminiscent of Uber and Lyft a decade ago, says Glenn Danas, a partner at the law firm Clarkson, which is suing Mercor and several other data platforms. Yet in some ways these workers are in a worse position, more replaceable despite their advanced degrees. Uber drivers have to be physically present in a city to work, and they can organize and push for regulation there. If the same were to happen with data workers, companies could just recruit from somewhere else where people will work for less. When Mercor cut pay for its Meta project to $16 per hour, it dropped below the minimum wage in California and other states, yet people there kept working because they needed the money. This was something at least one supervisor acknowledged, writing in Slack, “While we won’t actively hire from any states where the minimum wage is above the project’s rate, if you are already active on the project and would like to work at the $16/hr rate, we want to enable you to do so.”

Entire professions risk a similar race to the bottom, says Acemoglu, if companies are able to pit workers against one another, each selling their data before someone else can underbid them. “We may also need unionlike organizations that exercise some sort of collective ownership and prevent any kind of simple divide-and-rule strategies by large companies to drive down data prices,” he says. “If there isn’t the legal infrastructure for a data economy of this sort, many of the people who produce the data will be underpaid or, to use a more loaded term, exploited.”

Katya was among the thousands of people invited to join the $16-an-hour Project Nova and was appalled by the low pay. “I think that was Mercor’s experiment in how close to the bottom they can scrape without jeopardizing the data that they’re getting,” she says. Her main project had been paused for weeks and might resume the next day or never.

In the end, she decided the money wasn’t worth it. She applied to work at a local coffee shop. It wasn’t the career pivot she’d imagined when she went to grad school; she just hoped working as a barista would be more stable. “At least when you work at a coffee shop for minimum wage, you have some friends to talk to and a boss who pretends to care about you. You have some kind of security; you know what your hours are going to be week to week,” she says.

But then she heard her phone ding. One of her projects was back on.

Last November, the House Oversight Committee had just released 20,000 pages of documents from the estate of Jeffrey Epstein, and Luke Igel and some friends were clicking around, trying to follow the threads of conversation through garbled email threads and a PDF viewer that was, frankly, “gross.” In the coming months, the Department of Justice would release its own batches of files, more than three million of them — again, all PDFs.

This was a problem. While the Department of Justice had run optical character recognition over the text, it was not very good, Igel said, rendering the files more or less unsearchable. 

“There was no interface the government put out that allowed you to actually see any sort of summary of things like flights, things like calendar events, things like text messages. There was no real index. You just had to get lucky and hope that the document ID that you were looking at contains what you’re looking for,” said Igel, cofounder of the AI video editing startup Kino. What if, Igel thought, they built a Gmail clone to view and search all this correspondence in a more intuitive way? 

To do this, they would need to extract the information contained in PDFs, which is far less straightforward than it might sound. Despite rapid progress in AI’s ability to build complex software and solve advanced physics problems, the ubiquitous format of PDF remains something of a grand challenge. Edwin Chen, the CEO of the data company Surge, includes it among AI’s “unsexy failures” limiting real-world usefulness. Last year, he found that even state-of-the-art models asked to extract information from a PDF will instead summarize it, confuse footnotes with body text, or outright hallucinate contents. In a half-joking timeline of AI development, the researcher Pierre-Carl Langlais placed “PDF parsing is solved!” shortly before AGI. 

First, Igel’s friend, the “tech jester” Riley Walz, used his remaining credits on Google’s Gemini. It only worked reliably for some of the cleanest scans, and would be prohibitively expensive to run on millions of documents anyway, so Igel reached out to his former MIT classmate Adit Abraham, who happened to work in the office above his, where he ran a PDF-parsing AI company called Reducto. 

Reducto, one of several companies trying to solve PDFs, was able to extract information from email threads with cryptic decoding errors, heavily redacted call logs, and low-quality scans of handwritten flight manifests. After the data was exported in a usable format, Igel and Walz went on a building spree, creating essentially a full Epstein-themed app ecosystem: Jmail, an unsettling, searchable prototype of Epstein’s inbox; Jflights, an interactive globe crisscrossed with flight paths, each one clickable to view underlying PDFs of flight data, passenger manifests, and scanned email invitations; Jamazon, to search Epstein’s Amazon purchases; and Jikipedia, to search businesses and people who turn up in the files, citing, naturally, more PDFs.  

“That’s where the magic of extracting information of PDFs became real for me,” Igel said. “It’s going to completely change the way a lot of jobs happen.” 

PDFs are notoriously difficult for machines to parse, in part, because they were never meant to be read by them. The format was developed by Adobe in the early 1990s as a way to reproduce documents while preserving their precise visual appearance, first when printing them on paper, then later when depicting them on a screen. Where formats like HTML represent text in logical order, PDF consists of character codes, coordinates, and other instructions for painting an image of a page. 

Optical character recognition (OCR) can turn those pictures of words back into text computers can use, but if it comes across a PDF where text is displayed in multiple columns — as many academic papers are — it will plow ahead left to right and create an unintelligible jumble. OCR tools are designed to detect and correct for these sorts of formatting variations, but tables, images, diagrams, captions, footnotes, and headers all present further obstacles. If you give an AI assistant like ChatGPT a PDF, it will cycle through a variety of these tools, sometimes fail, sometimes pass the PDF to a large vision model to perform OCR, sometimes hallucinate, and generally take a very long time and use a lot of computing power for uneven results. 

“The key issue is that they cannot recognize editorial structure,” said Langlais. “It’s all fine while it’s relatively simple text, but then you’ve got all these tables, you’ve got forms. A PDF is part of some kind of textual culture with norms that it needs to understand.”

A further problem that arises from and compounds PDF’s inherent difficulty is that models rarely train on them. This has begun to change, partly because AI developers are increasingly desperate for high-quality data, and PDFs contain a disproportionate amount of it. Government reports, textbooks, academic papers — all PDFs. “PDF documents have the potential to provide trillions of novel, high-quality tokens for training language models,” wrote researchers at the Allen Institute for AI last year in a paper announcing a new specialized PDF-reading model. 

“The lore has it that the very first PDF ever was an IRS 1040,” said Duff Johnson, CEO of the PDF Association, the industry organization that helps develop the PDF global standard, ISO 32000-2:2020, itself a PDF nearly a thousand pages long. In 1994, the IRS wanted a way to share forms that were absolutely consistent without printing and mailing every possible document, so it mailed CDs full of PDFs instead. From there, PDF spread with email to become a fundamental component of digital work. Book publishers sending manuscripts to the printer, patent applicants submitting diagrams of new devices, anyone who needed to share a document that would look the same to whomever received it turned to PDF. 

“There’s no other technology solving the problem the PDF solves,” said Duff. Websites are temporary, appearing differently depending on the browser, mediated by CSS. Links rot. Word docs change depending on your machine and can be edited and overwritten. A PDF is the same no matter who opens it, when, or how.

“That’s what engineering companies need. That’s what lawyers need. That’s what governments need. That’s what anybody who’s doing anything in the world, who has records to maintain, they need that,” Duff said. “Earlier today I opened up a PDF from 1995. I didn’t worry about it. I just opened it. It worked fine. It worked perfectly. I would expect no less.” (It was a PDF about PDFs.)

There has been a shift over the last year or so toward specialized PDF-parsing models, said Luca Soldaini, a researcher at the Allen Institute for AI who worked on their PDF model, olmOCR. They trained a vision language model — like a large language model, but with pixels instead of word tokens — on about 100,000 PDFs: public domain books, academic papers, brochures, documents from the Library of Congress with human-written transcriptions. The model was further trained to optimize specific problem areas, like parsing tables without mixing up the rows and columns. 

“If text is large on the page, the model will learn to say, ‘Oh, that’s probably a header,’” said Soldaini. The model was the most popular one the institute released last year, Soldaini said, rivaling the institute’s generalist models. A PDF reading AI doesn’t capture the spotlight like those models, Soldaini said, but people are actually using it.

A few months later, researchers at Hugging Face, the company that runs a popular open-source AI platform, had just published a 5 billion-document dataset for training multilingual models and were thinking about what to do next. They had already processed the whole of Common Crawl, the enormous archive of mostly HTML text scraped from the web that forms the foundation of many large language models. Like many AI researchers, Hugging Face’s Hynek Kydlíček recalled, they were wondering whether they had run out of easily available data.

“We thought, let’s look at the Common Crawl and, like, maybe there is more stuff we just haven’t seen,” said Kydlíček. Indeed, there was: roughly 1.3 billion PDFs. “That’s how we figured out that PDFs could be actually a super big and super high-quality source we can still train on,” Kydlíček said. “But the format of PDFs is, like, super super hard to extract text from.”

Kydlíček and his collaborators rigged up a system that separated PDFs into easy to parse — mostly text — and difficult to parse, full of images and charts. The hard PDFs were sent to a version of olmOCR that had been modified by Reducto, called RolmOCR. After they stripped out the PDFs of horse racing results that made up an inexplicably large quantity of the corpus, the team declared they had “liberated three trillion of the finest tokens,” now available for model training. 

Yet parsing PDFs well enough for model training is one thing. Extracting them with the degree of accuracy demanded by lawyers and engineers is another. When the Hugging Face team did their first tests, they found their model would invent text when there wasn’t any, filling blank pages with nonsense and describing images and art. They trained it to correct these errors, but it’s impossible to anticipate every formatting oddity or off-kilter scan. 

“It’s solved in like 98 percent of cases, and like in many areas you always have this problem of getting these last 2 percent,” Kydlíček said. “I would say OCR is one of the best economic use cases for visual language models, so there are a lot of eyes on it right now, a lot of people throwing a lot of resources onto this. So I’m very certain that we will improve fairly fast, but because all these language models are probabilistic, there is just no way to guarantee it will be correct.”

One of the teams doing the best work, Kydlíček said, is Reducto, the company Igel is using to parse the Epstein files. Abraham cofounded the company as a service that managed customers’ long-term histories with language models, similar to the “memory” feature that is now standard in chatbots. Abraham kept getting requests to manage people’s files as well, which naturally were in the form of PDFs. He found working with them to be “shockingly hard.” 

“One of our core intuitions was all these documents were made for humans like you and I to interpret, and there’s a lot of visual information here that we take for granted, like that every gap between two paragraphs is me telling you, ‘Hey, this is a new idea.’ Every indentation is me telling you, ‘Hey, this is a sub idea of the parent idea.’ The question was like, how do you encode all of that context?”

Much of the team had a background in self-driving vehicles, where computer vision models “segment” data into entities like car, pedestrian, dumpster. They took a similar approach to PDFs, using a model to first divide the page into headers, tables, footnotes, and so on, before passing them to other specialized models for parsing. When they posted about their approach in early 2024, the response was immediate. 

“This wasn’t supposed to be a pivot,” said Abraham. Other developers reached out to say that their progress had been stymied by PDFs. “It kind of spiraled from there.” 

Reducto now uses a growing assortment of small, specialized models taking multiple passes to parse a PDF. When the segmenting model detects a table, it goes to a table-parsing model. If a chart is detected, different elements get sent to different models: one trained to extract axes, another to read legends, and so on. A vision language model then takes a pass on the output to correct errors. Using this approach, Reducto is able to turn charts into spreadsheets with a high degree of accuracy, something Abraham says the company’s financial clients have long requested and that stymies far larger frontier models. 

Still, like self-driving cars, PDFs have a long tail of unusual challenges. 

“There’s a big difference between getting a car to stay in a lane versus getting a car to handle whatever would show up on the street, and we see with PDFs a similar thing. I’ve seen the most insane documents you could imagine,” said Abraham. PDF files that contain other PDFs, legal documents with passages sometimes underlined and sometimes crossed out, faxes of medical forms that doctors have scrawled over and drawn lines connecting ideas on different edges of the page. “I don’t think PDFs are a fully solved problem. I wish that were the case. We’re close, but there’s still plenty to do.” 

There will be no shortage of PDFs to parse. The format does not appear to be going anywhere. Why would it, asked Duff of the PDF Association, with some incredulity at the very thought. Companies once tried to unseat PDF, Duff said, but their products are “now a footnote in history,” while PDFs continue to proliferate. 

“Look at the Google Trends for PDF,” Duff said. It shows a steadily rising curve (with dips in August) year after year. “No other technology looks like that. More and more people over time are including PDF in their searches, because that tends to be where the high-quality content is.

“What’s going to happen is that all the world’s systems will instead understand and use PDF better and better,” Duff said. “The AI companies didn’t focus on PDF, because PDF is very hard, until they realized that, well, it turns out a lot of the really high-quality stuff is in fact in PDF, and so now we have to deal with it.”

Wikipedia will be 25 years old in January. During that time, the encyclopedia has gone from a punchline about the unreliability of online information to the factual foundation of the web. The project’s status as a trusted source of facts has made it a target of authoritarian governments and powerful individuals, who are attempting to undermine the site and threaten the volunteer editors who maintain it. (For more on this conflict and how Wikipedia is responding, you can read my feature from September.) 

Now Wikipedia’s cofounder Jimmy Wales has written a new book, The Seven Rules of Trust: A Blueprint for Building Things That Last. In it, Wales describes a global decline in people’s trust in government, media, and each other, instead looking to Wikipedia and other organizations for lessons about how trust can be maintained or recovered. Trust, he writes, is at its core an interpersonal assessment of someone’s reliability and is best thought of in personal terms, even at the scale of organizations. Transparency, reciprocity — you have to give trust to get trust — and a common purpose are other ingredients that he attributes to Wikipedia’s success. 

We spoke over video call about his book, how Wikipedia handles contentious topics, and the threats facing the project and other fact-based institutions.

The interview has been condensed and edited for clarity. 

The Verge: You wrote a book about trust, and a global crisis in trust. Can you tell me what that crisis is and how we got there?

Jimmy Wales: If you look at the Edelman Trust Barometer survey, which has been going since 2000, you’ve seen this steady erosion of trust in journalism and media and business and to some degree in each other. I think it gives rise in a business context to a lot of increased cost and complexity, and politically, I think it’s tied up with the rise of populism. So I think it’s important that we focus on this issue and think about, What’s gone wrong? How do we get back to a culture of trust?

What do you think has gone wrong?

I think there’s a number of things that have gone wrong. The trend actually goes back to before the Edelman data. Some of the things I would point to are the decline of the business model for local journalism. To the extent that the business model for journalism has been very difficult, full stop, you see the rise of low-quality outlets, clickbait headlines, all of that. But also that local piece means people aren’t necessarily getting information that they can verify with their own eyes, and I think that tends to undermine trust. In more recent times, obviously the toxicity of social media hasn’t been helpful.

Why has Wikipedia so far bucked that trend and continued to be fairly widely trusted?

Part of the rationale for writing the book is to say, “Look, Wikipedia has gone from being kind of a joke to one of the few things people trust, even though we’re far from perfect.” I think transparency is hugely important. The idea that Wikipedia is an open, collaborative system and you can come and see how decisions are made, you can join and participate in those decisions — that’s been very helpful. I think neutrality is really important. The idea that we shouldn’t take sides on controversial topics is one that resonates with a lot of people. I don’t want to come to an encyclopedia or frankly a newspaper and be told only one side of the story. I want to get the full picture so I can understand the situation for myself. 

You brought up the Edelman survey and decline in trust in media, government, and to a lesser extent individuals. Are we seeing a decline in trust or a transfer of trust from institutions to individuals? In the book, you say we are hardwired to trust at an interpersonal level by gauging other people’s authenticity, which is a trait that plays very well on social platforms, where some very trusted figures also gain extra trust by telling their followers not to trust in the media, the FDA, the universities. Do you see this dynamic playing a role, and if you do, how has Wikipedia, which is an institution, continued to be trusted? 

I think there’s some truth to that. But I also think it’s incomplete because I think a lot of people who support Donald Trump will also say they don’t really trust him. They just think it’s not relevant. They’ve sort of lost faith in the idea of people being honest. So they’re more likely to say, “All politicians lie, so why is that a big deal?” I obviously think it is a big deal. I think that’s very problematic. 

Similarly, I think a lot of the people who are jumping on a bandwagon undermining trust in science, for example, basically see a way to get successful doing it. I mean, that’s a pretty cynical view of those particular people, and I’m not a very cynical person, but it’s hard to come to any other conclusion sometimes, that there’s a lot of grifting going on.

I interviewed Francis Fry for the book, and she’s a Harvard academic who also has business experience. One of the things she said to me was, people often say that once you’ve lost trust — that’s it, you’ll never get it back. And she says that’s not true. You can rebuild trust. There are certain definable things that organizations and people can do to rebuild trust. So when we think about institutions being attacked, they probably should reflect on what made them vulnerable. 

You have some examples in the book, like the back-and-forth about masking and covid, and obviously journalists do make errors. But I tend to think that most publications are fairly transparent about issuing corrections, though maybe not to the level of Wikipedia. How much of the decline in trust has to do with actual mistakes made by those institutions, versus people or groups that want to be able to define their own reality undermining what they see as rival centers of facts, whether that’s academia or science or journalism?

I absolutely think it’s both. In many cases, we have seen media with a real blind spot, and I typically would view it more often as a blind spot problem, rather than deliberately being biased. I live in London. All three of the major political parties were all opposed to Brexit, and in London you could not really find anybody who was openly supporting Brexit, not among my social group. Everybody thought it was a completely ridiculous idea. And yet the public voted for it. 

I think a big part of that was that London wasn’t listening and the media tended too often to portray Brexit support as having to do with racism and so on. Which, of course, if that’s how you come at people, they tend to not go, “Oh, you’re right, I’m sorry. I’m going to stop being racist now and change my political views.” They’re more likely to say, “Hold on a minute, you’re not listening to me. I’m not being racist. There are these problems, functional problems, and I don’t think I’m being listened to.” To the extent the media isn’t representative of broader segments of society and isn’t listening to problems that people are having, that’s a problem. And then we also have people who are taking advantage of it and who see that opportunity to campaign and build trust by pointing the finger at the other guy.

Debates on Wikipedia talk pages can get heated. People rebut other people’s proposals without a lot of pleasantry. There is real conflict, but they are generally productive conflicts. People keep engaging with each other and usually reach a compromise, which I feel is very unique in online discourse. What do you think the mechanism or mechanisms are that make this possible? 

We have a purpose to build an encyclopedia, to be high-quality and neutral, and we have a commitment to civility as a virtue in the community. We’re human beings, so of course sometimes those conversations are, I might say, a bit brusque but hopefully not stretching quite into personal attacks. There’s also this view that you really shouldn’t attack people personally. And if it gets overheated, you should probably apologize, and things like that, which is not that unusual except in online contexts. I mean, normally I think most people in real life, if you get into a proper nasty quarrel with someone, there is a sort of feeling like, Yeah, that wasn’t productive and maybe we need to apologize to each other and find a better way to deal with each other. In terms of how do we foster more of that? I think in online spaces, it has to do with changing culture. And in many cases, I think it’s the design of algorithms. 

I don’t go on Facebook very much anymore, but if one day I logged in and Facebook had an option that said, “We’d like to show you things we think you will disagree with, but that we have some signals in our algorithm that are of quality. Would you like to see that?” I’d be like, yes, sign me up for that. As opposed to: “Our research has shown that you tend to get agitated about trolls, so we’re going to send more trolls your way because you stay on the site longer.” Or “we’re only going to send you stuff we think you’re going to agree with,” which is also not really healthy intellectually.

One of your other examples of a functional online space was the subreddit /changemyview, which feels similar to Wikipedia in some ways. It’s text-based. There are rules. You’re there for a specific purpose. Is it possible for a big platform like Facebook or X or whatever to become a healthy space, or do you need to be kind of constrained and purpose-built?

I think it’s hard for sure. And I think that’s a great question because I don’t think anybody knows right now. On Facebook, you’ll find pockets of groups that have good, well-run community members who are keeping the peace and insisting on certain standards. And you find horrible places as well. I think Reddit it’s the same. And another thing that I do think is interesting is looking back, because I’m now old, and I remember before the World Wide Web and I remember Usenet, which was a giant, enormous, largely unmoderated message board. That was super toxic. It had endless flame wars and horribleness and spam and all kinds of nonsense. So I always try to mention that when people have this view of the lovely, sweet days of the early internet — it was such a utopia. I’m like, it was kind of horrible then too. It turns out we don’t need algorithms to be horrible to each other. That’s actually something humans can do, and humans can be great to each other at the same time. But I do think, as consumers of internet spaces, I think we should say, “Actually, I really would much rather be in places that are good for me.”

You recently weighed in on one of the most contentious topics on Wikipedia or anywhere, the Israel-Gaza conflict. You wrote that you thought that it shouldn’t be called a genocide in wiki voice. You normally stay out of content debates on Wikipedia. Why did you decide to weigh in on that one?

I think it’s really important that Wikipedia remain neutral and that we refrain from saying things that are controversial in wiki voice. I think that’s not healthy for us and not healthy for the world. So it felt important to weigh in and say, “Let’s take a deeper look at this.” And the other thing is normally, we have this idea of consensus in the community, and I would say it has a certain usually constructive ambiguity, like what is consensus? How do you define that? We’ve avoided for good reason, I think, saying, “it’s 80 percent” or any kind of simple rule like that. And the reason is because there are so many different areas in editing where there are different levels of certainty and different levels of consensus. My simplest example is, which picture of the Eiffel Tower should we have as the main picture on the Eiffel Tower wiki page? Well, maybe somebody does a straw poll and it’s 60-40. Personally, if I’m in the 40 percent, I’m going to go, Most people don’t agree with me, oh well, because it isn’t that important. 

Whereas in other cases, if you’ve got a significant number of good Wikipedians who are saying, “I don’t agree with this, I don’t think this should be in wiki voice, you shouldn’t go for 60 percent.” That’s nowhere near good enough, particularly not if it has enormous implications for the reputation of Wikipedia and neutrality. We should hold ourselves to a very high standard. This is the kind of thing that over the years, we have to reexamine over and over and over. Where are we drawing these lines? And are we doing a good job of it? And should we ratchet it up and be more serious about it? And over the years, we have gotten more serious about it. And I think we should be even more serious about it.

Some of the editors said they felt that there was a consensus, that they’d debated this question for months, and that to frame the article as you wanted would be to give both sides of the debate equal weight, rather than to represent the proportional view of experts and institutions. What are your thoughts on that critique?

Yeah, I think they’re wrong. I think we have to always dig deep and examine it, and I think it’s absolutely fine to say, “The consensus of academic genocide researchers is that this was genocide.” That, as far as I can tell, is a fact, so that’s fine. Report on that fact. That doesn’t mean that Wikipedia should say it in our own voice. 

And that’s actually important more broadly that if there’s significant disagreement within the community of Wikipedians and we don’t have consensus, and if people are putting forward policy-based reasons to disagree with that, which they are, then hold on. We should always be looking for as much agreement as possible. So what can we all agree on? Oftentimes that may be stepping back, going meta and saying, “Okay, well, we can all agree to report on the facts. We’re not all going to agree on using wiki voice here. So we’re not going to do that. But we are going to report the facts that we can all agree on.”

And it’s important for two reasons. One, it’s what you want from an encyclopedia. You don’t want to be jumping to a conclusion while there’s still live debate. And two, socially within the community, it means we can all have a win-win situation where we can all point at this and say, “Yeah, we disagree but we can point to this with pride and say, ‘Actually, this is a good presentation. If you read this, you’ll understand the debate.’” Brilliant. That’s where we want to be.

When I see people attack Wikipedia for bias, it often comes down to which sources editors deem reliable. They’ll say, “Well, you don’t let us cite Breitbart, so now it’s going to be biased.” How are you thinking about how to draw the line of what is an acceptable source, and how to maintain neutrality as these decisions no longer seem neutral to people who have a completely different media diet made up of sources deemed unreliable?

It’s something we will always be grappling with. Wikipedia does not have firm rules. That’s one of the core pillars. We don’t completely ban sources. We may deprecate them and say, “Well, it’s not preferred as a source. We’d rather have something better.” And then I make no apologies at all for saying not all sources are equal. I always say, if I have a choice between The New England Journal of Medicine and Breitbart, I’m going with The New England Journal of Medicine. That’s just the way it is, and I think that’s fine. When I say we have to grapple with it and take seriously the question of bias, I think we do. But sometimes we’re going to conclude, Actually, I think we’re fine here. 

Elon Musk has been a loud voice complaining about bias on Wikipedia. Now he has Grokipedia, an AI-rewritten version of Wikipedia that draws on a bunch of sources that Wikipedia won’t allow. Have you looked at Grokipedia?

A little. Not enough. I need to do a deep dive.

What are your thoughts on it?

I think a lot of the criticism that it’s getting is not surprising to me. I use large language models a lot and I know about the hallucination problem, and I see it all the time. Large language models really aren’t good enough to write an encyclopedia. And what’s particularly true is the more obscure the topic, then the more likely they are to hallucinate. I also think in terms of the question of trust, I’m not sure anybody’s going to trust an encyclopedia that has a thumb on the scales. Which is to say, when I’m not happy about something in Wikipedia, I open a conversation and enter the discourse. I’m sure if Elon doesn’t like something, it’s just going to change. I don’t see how you can trust a process like that. You know, it is reported that Grokipedia seems to agree with Elon Musk’s political views quite well. Fine. It’s Elon, but that might not be what we all want from an encyclopedia.

Are you concerned that it could be what some people want, or that people will start to use or prefer an AI-revised version of Wikipedia that conforms to their worldview? 

Obviously you can’t dismiss that out of hand, but I actually reflect on various research that we cite in the book about trust, that if people feel like there’s a thumb on the scale, then even if they agree with that thumb on the scale, they are likely to trust it less.

I have great confidence in ordinary people. I think that if you ask people, “Would you prefer to have a news source that reflects all your own prejudices and biases and that you agree with every day?” or “Would you rather get something that is neutral and gives you insight into things you might not agree with?” I don’t think it’d be a contest. Most people would prefer the latter. That doesn’t mean they automatically click on it, and they may prefer their preferred outlet. That’s fine. That’s humanity. But I don’t think we’re about to all go off into our little mind bubbles permanently. 

How are you thinking about Wikipedia and AI more generally? The internet is increasingly full of AI-generated slop, and the foundation noted earlier this year that bots scraping the site were straining Wikipedia’s servers. Do you see AI presenting a threat, possible benefit, both? 

Both. AI slop on the internet I don’t think is a huge issue for Wikipedia because we’ve spent, you know, now nearly 25 years studying sources and debating the quality of sources. And so I think Wikipedians aren’t likely to be fooled by, you know, sort of fluff content that is generated by AI.

Obviously, crawling Wikipedia and hammering our servers, that’s not cool. So we hope we find a reasoned solution to that. The money that supports Wikipedia is the small donors giving an average of just over $10. They’re not donating to subsidize billion-dollar companies crawling Wikipedia. So you know, “pay for what you’re using” seems like a fair request. 

Then the other thing that I think is super interesting are questions around how might we, the community, might use the technology in a new way. I’m not a very good programmer, but I’m a programmer and I just wrote a little thing that I can feed it a short Wikipedia entry that maybe has five sources and feed it the five sources and say, “Is there anything in the sources that should be in Wikipedia but isn’t? Or is there anything in Wikipedia that isn’t supported by the sources?” I haven’t even had time to play with it, but even at a first pass, I thought, this is actually not terrible.

Going back to why Wikipedia works, editors do seem to largely trust each other to be working in good faith, but it also seems like they have a lot of trust or respect for Wikipedia’s rules and processes in a way that feels rare in online communities. Where does that come from? 

I think it probably has to do with everything being genuinely community-driven and genuinely consensus-driven. The rules aren’t imposed, the rules are people writing down accepted best practices. Certainly in the early days, that was absolutely how it worked. We would be doing something for a while and then we would notice, like, Oh, actually, you know, best practice is this, so we should maybe write that down as a guide for people, and it becomes policy at some point. That helps to build trust in the rules, that they’re genuinely not imposed top-down, that they are the product of our values and a process and the purpose of Wikipedia.

When he was 19 years old, Brendan Foody started Mercor with two of his high school friends as a way for his other friends, who also had startups, to hire software engineers overseas. It launched in 2023 as essentially a staffing agency, albeit a highly automated one. Language models reviewed resumes and did the interviewing. Within months, Mercor was bringing in $1 million in annualized revenue and turning a modest profit. 

Then, in early 2024, the company Scale AI approached Mercor with a big request: They needed 1,200 software engineers. At the time, Scale was one of the only well-known names in the historically back-of-house business of producing AI training data. It had grown to a valuation of nearly $14 billion by orchestrating hundreds of thousands of people around the world to label data for self-driving cars, e-commerce algorithms, and language-model-powered chatbots. Now that OpenAI, Anthropic, and other companies were trying to teach their chatbots to code, Scale needed software engineers to produce the training data. 

This, Foody sensed, could herald a larger change in the AI industry. He’d heard about growing demand for specialized data work, and now here was Scale asking for a thousand coders. When the engineers he recruited started complaining about missed pay (Scale has a reputation among data workers for chaotic platform management and is being sued in California over wage theft, among other infractions), Foody decided to cut out the middleman. 

In September, Foody announced that Mercor had reached $500 million annualized revenue, making it “the fastest growing company of all time.” The previous titleholder was Anysphere, which makes the AI coding tool Cursor. In a sign of the times, Cursor recently noted that its users produce the exact sort of training data labs are paying for, and The Information recently reported that OpenAI and xAI are interested in buying it. 

Mercor’s most recent fundraising round valued the company at $10 billion. Foody and his two cofounders are 22 years old, making them the youngest self-made billionaires. At least one of their early employees has already left to start an AI data company of her own. 

While discussions of AI infrastructure typically focus on the gargantuan buildout of data centers, an analogous race is happening with training data. Labs have already exhausted all the easily accessible data, adding to questions about whether early rapid progress through sheer increases in scale will continue. Meanwhile, most recent improvements have come through new training techniques that make use of smaller datasets tailor-made by experts in particular fields, like programming and finance, and AI companies will pay premium prices for it. 

There are no good statistics on how much labs are spending, but rough estimates from investors and industry insiders place the figure at over $10 billion this year and growing, the vast majority coming from five or so companies. These companies have yet to find a way to make money from AI, but the people selling them training data have. For now, they are some of the only AI companies turning a profit. 

The data industry has long been the most undervalued and unglamorous aspect of AI development, according to a 2021 study by Google researchers, seen as regrettably necessary janitorial work to be done as quickly and cheaply as possible. Yet modern machine learning could not exist without its ecosystem of data suppliers, and the two spheres move in tandem. 

The enormous datasets that proved the viability of machine learning in the early 2010s were made possible by the emergence several years before of Amazon Mechanical Turk, an early crowdsourcing platform where thousands of people could be paid pennies to label images of dogs and cats. The push to develop autonomous vehicles fed the growth of a new batch of companies, among them Scale AI, which refined the crowdsourcing approach through a dedicated work platform called Remotasks where workers used semi-automated annotation software to draw boxes around stop signs and traffic cones. 

The turn to language model chatbots after the launch of ChatGPT initiated another transformation of the industry. ChatGPT got its humanlike fluency from a training approach called reinforcement learning from human feedback, or RLHF, which involved paying contractors to rate the quality of chatbot responses. A second model trained on these ratings, then rewarded ChatGPT whenever it did something that this second model predicted humans would like. Providing these ratings was a more nuanced affair than past iterations of crowdsourced data work, particularly as the chatbots got more advanced; it takes someone with medical training to judge whether medical advice is good. 

Scale supplied much of the human ratings, but a new company, Surge AI, self-funded by a data scientist named Edwin Chen, quietly grew to become the industry’s other major provider. In Chen’s past jobs at Google, Twitter, and Facebook, he had been dismayed at the poor quality of the data he received from vendors, full of mislabelings done for minimal pay by people who lacked relevant backgrounds. The vendors, Chen said, were just “body shops,” throwing people at the problem and trying to substitute quantity for quality. 

Where Scale had its Remotasks platform, Surge has Data Annotation Tech: smaller, more targeted in its recruiting, and with tighter quality controls. It also paid better, around $30 an hour, though like Scale, Surge is also being sued in California for misclassification and unpaid wages. Demand from OpenAI and the labs trying to catch up was immense. The company has been profitable since it launched, and last year, it reportedly took in more than $1 billion in revenue, surpassing Scale’s reported $870 million. Earlier this year, Reuters reported that Surge is considering taking funding for the first time, looking for a $1 billion investment at a $15 billion valuation. According to Forbes, Chen still owns approximately 75 percent of it.

Data about which chatbot responses people prefer is a crude signal, however. Models are prone to learning simple hacks like “tell the user they’ve made an excellent point” instead of something as complex as “check for factual consistency with reliable sources.” Even when domain experts are doing the judging, the results often just sound more expert but are still too unreliable to actually be useful. Models ace bar exams but invent case law, pass CPA tests but pick the wrong cells in a spreadsheet. In July, researchers at MIT released a study finding that 95 percent of the businesses that have adopted generative AI have seen zero return. 

AI companies hope that reinforcement learning with more granular criteria will change this. Recent improvements in math and coding are a proof of concept. OpenAI’s o1 and DeepSeek’s R1 showed that given a bunch of math and coding problems and a few step-by-step examples of how humans thought their way to solutions, models can become quite adept at these domains. As they trial-and-error their way to correct solutions, models weigh possible approaches, backtrack, and display other problem-solving techniques developers have called “reasoning.” 

The problem is that math and coding problems are idealized, self-contained tasks compared to what a software engineer might encounter in the real world, so scores on benchmarks don’t reflect actual performance. To make models useful, AI companies need more data that is reflective of real tasks an engineer might do — hence the rush to hire software engineers. 

The other problem is that math and coding might be the easiest possible domains for AI to conquer. For reinforcement learning to work, models need a clear signal of success to optimize for. This is why the method works so well for games like Go: Winning is a clear, unambiguous outcome, so models can try a million ways to achieve it. Similarly, code either runs or it doesn’t. The analogy isn’t perfect; ugly, inefficient code can still run, but it provides something verifiable to optimize for. 

Few other things in life are like this. There is no universal test for determining whether a legal brief or consulting analysis is “good.” Success depends on the context, goals, audience, and countless other variables.

“There seems to be a belief in the community that there’s a single reward function, that if we can just specify what we want these AI systems to do, then we can train them to [do it],” said Joelle Pineau, chief AI officer at Cohere, an enterprise-focused AI lab. But, she said, the reality is more varied and nuanced.

“[Reinforcement learning] wants one reward function. It’s not very good about finding solutions when you have multiple conflicting values that need to coexist, so we may need a very different paradigm than that.”

In lieu of a new paradigm, AI companies are attempting to brute force the problem by paying — via companies like Mercor and Surge — thousands of lawyers, consultants, and other professionals to write out in painstaking detail the criteria for what counts as a job well done in every conceivable context. The hope is that these lists, often called grading rubrics, will allow models to reinforcement-learn their way to competence in the same way they have begun doing with software engineering. 

Rubrics are extremely labor-intensive to produce. People who work on them said that it is not unusual to spend 10 hours or more refining a single one, which might include more than a dozen different criteria. Companies guard the details of their training methods closely, but an example OpenAI released for its recent medical benchmark offers a good indication of what they’re like. Asked a question about an unresponsive neighbor, the model gets rewarded if its response includes advice to check for a pulse, locate a defibrillator, perform CPR, and 16 other criteria. There are nearly 50,000 such criteria in the benchmark, with different ones applying to different prompts. Labs are ordering tens to hundreds of thousands of rubrics with millions of criteria between them per training run, according to people in the data industry.

These rubrics need to be “super granular,” according to Mercor’s Foody. Producing consulting rubrics, Foody said, would start by creating a taxonomy of all the industries a consulting company operates in, then all the types of consulting it does in each of those industries, then all the types of reports and analyses a consultant might produce in each of those categories.

Performing these tasks typically requires doing things on computers, and each of those things needs a rubric, too. Sending an email requires a lot of steps — opening a browser, beginning a new message, typing it out, and so on. But what if your only verifier for success was whether the email was sent or received? It’s important to check for more actions than just one, according to Aakash Sabharwal, Scale’s VP of engineering. 

Models learn to perform these tasks in simplified versions of software called reinforcement learning environments, often described as AI “gyms,” where models can stumble around until they figure out how to do the clicking and dragging required to score well on the grading rubric. The market for these environments is booming, too.

As with rubrics, each one needs to be tailored to its use. “Sometimes it’s a DoorDash or a Salesforce clone, but a lot of times it’s just an enterprise-specific environment,” said Alex Ratner, cofounder and CEO of Snorkel AI. Snorkel makes annotation software but recently launched a human data service of its own. 

Ratner cites a recurring irony in AI development known as Moravec’s paradox, named for a researcher working on computer vision in the 1980s who observed that the things that come easiest to humans are often the most difficult for machines. At the time, conventional wisdom was that machine vision would be solved before chess; after all, only a select few humans have the talent and training to be grandmasters, whereas even children can see. Now models can solve complex one-off coding challenges, but they flounder on more basic real-world engineering tasks without close human supervision, misusing tools and making obvious errors. 

“That kind of real work, with ambiguous, intermediate metrics of success that seem way more mundane than a coding competition, that is where models struggle,” Ratner said. “That’s the counterintuitive frontier, and that’s where people are trying to lean in, ourselves included, with building more complex environments, more nuanced rubrics.” 

According to vendors, the most in-demand fields are the ones that sit at the sweet spot of verifiability and economic value. Software engineering continues to be the largest, followed by finance and consulting. Law is popular, though so far it is proving to be less verifiable and thus amenable to reinforcement learning. Physics, chemistry, math are all in demand. Really, it’s nearly anything you can imagine. There are ads for nuclear engineers and animal trainers. 

“It’s everything from clinical hospital settings to legal deep research to — we got a request for woodworking the other day,” Ratner said. “It’s every nook and cranny of human expertise.”

Encoding all of humanity’s skill and know-how into checklists is an enormous, possibly quixotic undertaking, but the frontier labs have billions to spend, and the sheer scale of their demand is reconfiguring the data industry. New entrants seem to appear by the day, and everyone is touting successively more pedigreed experts getting paid ever higher rates. 

Surge touts its Fields Medalist mathematicians, Supreme Court litigators, and Harvard historians. Mercor advertises its Goldman analysts and McKinsey consultants. Handshake AI, another fast-growing expert provider, boasts of its physicists from Berkeley and Stanford and the ability to draw alumni from more than 1,000 universities. 

Garrett Lord, the CEO and cofounder of Handshake, started picking up signals about the changing data market last year, when incumbent data providers came around asking for experts. Handshake had experts. Lord founded the company in 2014 as a sort of LinkedIn-meets-Glassdoor for college students and recent grads looking for internships and first jobs. More than a thousand college career centers pay for access, as do companies looking to recruit from Handshake’s 20 million alumni, grad students, masters, and PhDs. Early this year, Lord entered the AI data market himself, launching essentially a second company within his existing one, called Handshake AI. 

Then, in June, Meta hired away Scale’s CEO and took a 49 percent stake in the company. Rival labs fled, wary that Scale would no longer be a neutral provider — could they trust the data now that it was being provided by a quasi-Meta subsidiary? It was like breaking a billion-dollar piñata over all the data startups. Handshake saw demand triple overnight. 

In November, Handshake surpassed a $150 million run rate, exceeding the original decade-old business. There is more demand than the company can meet, Lord said. “We’ve gone from three to 150 people in five months,” Lord said. “We’ve had 18 people start on a Monday. We’re running out of desks.” 

The ravenous demand of AI model-builders is pulling any company that might have data to offer into its gravitational field. Turing, which began as a staffing agency but pivoted to training data after OpenAI approached the company in 2022, also saw demand spike following the Scale deal. As did Labelbox, which makes annotation software but last year launched its own expert-annotator service, called Alignerr, where buyers can search for experts, called “Alignerrs,” who’ve been vetted by Labelbox’s AI interviewer, named Zara. 

Staffing agencies, content moderation subcontractors, and other adjacent businesses are also reorienting around the labs. Invisible Technologies started 10 years ago as a personal assistant bot that directed tasks to workers overseas, but it started posting twentyfold revenue increases as AI labs hired those workers to produce data. This year, it brought on an ex-McKinsey executive as CEO, took on venture funding, and is positioning itself as an AI training company. The company Pareto followed the same trajectory, launching in 2020 by offering executive assistants based in the Philippines and now selling AI training data services. 

The company Micro1 began in 2022 as a staffing agency for hiring software engineers, who had been vetted by AI, but now it’s a data labeling company too. In July, Reuters reported that the company had seen annualized revenue go from $10 million to $100 million this year and was finalizing a Series A funding round valuing the company at $500 million. 

Even Uber is angling to get a piece of the action. In October, it bought a Belgian data labeling startup and is in the process of rolling out an annotation platform to US workers, so drivers can annotate when they aren’t driving.

Then there is a long list of smaller, niche players. The company Sapien is paying data labelers in crypto. Rowan Stone, CEO of Sapien, told The Verge in July that the data labeling company — which specializes in vertical models focused on just one thing and has Scale cofounder Lucy Guo on its advisory board — is “absorbing the collective knowledge of humanity.” They aren’t even the only human data startup paying in crypto tokens. 

Stellar, Aligned, FlexiBench, Revelo, Deccan AI — everyone is touting their talent networks, their experts in the loop, their data enrichment pipelines. The company Mechanize rose above the scrum on a wave of viral outrage by announcing in April that its goal was “the full automation of all work.” How will it accomplish this provocative goal? By selling training data and environments, like everyone else. 

Like Nvidia, the dominant designer of AI chips, these companies sell the picks and shovels for the AI gold rush, capturing the billions in debt-financed spending flowing out of the frontier labs as they race to achieve superintelligence. It’s a safer business than prospecting, and it is much easier to start selling data than to design new chips, so startups are proliferating. 

“It’s like everyone and their mother realized, ‘Hey, I’m doing a human data startup,’” said Adam J. Gramling, a former Scale employee who said he received approximately 300 recruiting messages on LinkedIn when he announced his departure in one of Scale’s recent rounds of layoffs. “This Cambrian explosion happened, and now let’s see who survives.” 

The data industry may be growing quickly, but it is a historically tumultuous business. The industry is littered with former giants felled by a sudden change in training techniques or customer departure. In August 2020, the Australian data annotation company Appen’s market cap surpassed the equivalent of $4.3 billion USD; now, it’s less than $130 million, a 97 percent decline. For Appen, 80 percent of its revenue came from just five clients — Microsoft, Apple, Meta, Google, and Amazon — which made even a single client departure an existential event. 

Today’s market is also highly concentrated. On a recent podcast, Foody compared Mercor’s customer concentration to Nvidia, where four customers represent 61 percent of its revenue. If investors tire of giving money to model-builders, or the labs take a different approach to training, the effects could be devastating. All of the AI developers use multiple data suppliers already, and as the exodus from Scale showed, they are quick to take their money elsewhere.

All this lends itself to a fiercely competitive atmosphere. On podcasts and in interviews, the CEOs take swipes at the business models of their rivals. Chen still thinks most of his competitors are “body shops.” Foody refers to Surge and Scale as legacy crowdsourcers in an era of highly paid experts. Handshake’s Lord says his rivals are spending thousands on recruiters spamming physicists on TikTok, but they’re all already on his platform. All three say Scale had quality problems even before it was tainted by Meta’s investment. Every time one of these barbs is reported, a Scale spokesperson snipes back, accusing Foody of seeking publicity or mocking Chen for his lengthy fundraising round. Scale is also currently suing Mercor, claiming it poached an employee who stole clients on their way out the door.

For now, there is more than enough money flowing from the labs for everyone. They want rubrics, environments, experts of every conceivable type, but they’re still buying the old types of data too. “It’s always increasing,” says Surge’s Chen. “These ever-increasing new forms of training, they’re almost complementary to each other.” 

Even Scale is growing after its post-Meta setback, and major customers have come back, at least in some capacity. Interim CEO Jason Droege said in an onstage interview in September that the company is still working with Google, Microsoft, OpenAI, and xAI. To better compete in the enterprise AI space, Scale has also started a program called the “Human Frontier Collective” for white-collar professionals in STEM fields like computer science, engineering, mathematics, and cognitive science. 

Scale told The Verge that both its data and applications businesses are each generating nine figures of revenue, with its data business growing each month since the Meta investment and its application business doubling from the first half to the second half of 2025. It also said that the third quarter of 2025 was its public sector business’s best quarter since 2020, partly due to government contracts. Scale also reportedly expects revenue for this year to more than double, to $2 billion. (The company declined to comment on the figure on the record.)

It has diversified into selling evaluations, the tests that AI developers use to see where their models are weak and need more training data, according to Bing Liu, Scale’s head of research. The business strategy: Companies will ideally use the evaluations to see where their own models are lacking in data — and then, ideally, buy those types of data from Scale.

The 11-digit valuations of just-launched data companies could be seen as signs of an AI bubble, but they could also represent a bet on a certain trajectory of AI development. (Both can also be true.) The goal held out by the AI labs when justifying their enormous expenditures is an imminent breakthrough to artificial general intelligence, something, to use the definition in OpenAI’s charter, that is “highly autonomous” and can “outperform humans at most economically valuable work.” 

The term is amorphous and disputed, but one thing artificial general intelligence should be able to do is, well, generalize. If you train it to do math and accounting, it should be able to do your taxes without further rounds of reinforcement learning on tax law, state-specific tax rules, the most recent edition of TurboTax, and so on. A generally capable agent should not need massive amounts of new data to handle each variety of task in every domain. 

“The future where the AI labs are right is one where as performance goes up, the need for human data goes down, until you can take the human out of the loop entirely,” said Daniel Kang, assistant professor of computing and data science at the University of Illinois Urbana-Champaign, who has written about the demand for training data. Instead, the opposite seems to be happening. Labs are spending more on data than ever before, and improvements are coming from bespoke datasets tailored to increasingly specific applications. Given current training trends, Kang predicts that getting high-quality human data in each discrete domain will be the primary bottleneck for future AI progress. 

In this scenario, AI looks more like a “normal technology,” Kang said. Normal technology here being something like steam engines or the internet — potentially transformative, but also not computer god. (This is also, he hypothesized, why companies are less keen to trumpet their spending on data than they are on data centers: It cuts against their fundraising narrative.) In the AI-as-normal future, companies will need to buy new data whenever they want to automate a particular task, and keep buying data as workflows change. 

The data companies are betting on that too. “The labs very much want to say that we’re going to have superintelligence that generalizes as soon as possible,” said Foody. “The way it’s playing out in practice is that reinforcement learning has a limited generalization radius, so they need to build evals across all the things that they want to optimize for, and their investments in that are exploding very quickly.” 

Other companies, predicting that the frontier models will not “just hit this point of generalization where it’s just magic and you can do everything,” in the words of Ryan Wexler, who manages AI infrastructure investments at SignalFire, are positioning themselves to cater to the many companies that will need to tune models to suit their purposes. 

SignalFire invested in Centaur AI, a medical and scientific data company. Rather than the frontier labs, most of Centaur’s customers are medical institutions like Memorial Sloan Kettering or Medtronic with highly specific applications and low margins for error. Last year, the smart mattress company Eight Sleep wanted to add “snore detection” to its bed’s suite of capabilities. Existing models struggled, so the company hired Centaur to enlist more than 50,000 people to label snores. 

“The attempts to make the God model, I don’t know what will happen there, but I’m very confident that demand will keep growing among everyone else,” said Centaur’s founder and CEO, Erik Duhaime. “Everyone was sold some dream that this will be easy, plug and play,” Duhaime said. “Now they’re realizing, ‘Oh, we need to customize this thing for our use case.’” 

Matt Fitzpatrick, the CEO of Invisible, is also focusing on its enterprise services. If you look at “spend curves over time,” he said, the enterprise is “where a lot of this will move.” Since January, the company has overhauled its business to focus more on attracting enterprise clients, with about 30 percent of its data annotation pool now being people with PhDs and master’s degrees. Fitzpatrick describes the company as a “digital assembly line” where experts “anywhere on Earth” can be called in to generate data. Invisible is currently often being asked to provide environments for software development and contact centers, he said. 

If AGI is to be achieved one order of contact-center training rubrics at a time, the future looks bright for data vendors, which is perhaps why a new grandeur has entered the language of the CEOs. Turing’s CEO predicts that AI data annotator will become the most common job on the planet in the coming years, with billions of people evaluating and training models. Handshake’s Lord sees the nascent formation of a new category of work, comparing it to Uber drivers a decade ago. 

“We’re going to need a huge build-out of data and evals across every industry in the economy,” Foody said. At Mercor, he says, the customer support team responds to tickets the AI agent can’t manage, but also updates its rubrics so it can field those questions next time. “If you zoom out,” he said, “it feels like the entire economy will become a reinforcement learning environment.” 

If investors don’t find this vision as enticing as a country of geniuses in a data center, as Anthropic’s Dario Amodei described the impending transformation, they can take consolation in the fact that someone, at least, has found a way to make money off AI.