In May 2024 I shared my thoughts on LinkedIn about the potential of AI agents and how they could be used to build self-managing AI systems. The case study I discussed looked at ways to monitor hundreds of news articles and pass the most relevant ones to a key account manager.
Around the same time I came across an academic paper that applied the same idea to one of the hardest jobs in translation: novels. “(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts”, by Minghao Wu, Longyue Wang and colleagues, built a team of AI agents that works like a translation company. The authors revised it in 2025 for the journal Transactions of the Association for Computational Linguistics.
It’s worth a close look, because the company it simulates looks a lot like a language service provider. Here’s what it does, what it found, what it got wrong and what it means, with three figures to explore.
What is TransAgents?
A multi-agent system for literary translation. Each agent is a large language model (GPT-4 Turbo in the study) given a role, and together they reproduce how a human translation company handles a book: a CEO, a Senior Editor, a Junior Editor, a Translator, a Localization Specialist and a Proofreader.
To make the simulation more realistic, the authors generated 30 virtual profiles for each role, complete with nationality, education, years of experience, specialisms and a rate per word. The work runs in two stages: preparation, where the team is assembled and a set of translation guidelines is written, and execution, where each chapter is translated, localised, proofread and given a final review.
Click any step to see what the agent does, and the job it mirrors in a human LSP.
A translation company made of agents
Preparation
Execution, for every chapter
How do the agents work together?
The paper uses two patterns, and both will be familiar to anyone who has run a translation project.
Addition-by-Subtraction. One agent adds everything that might be relevant. A second removes whatever is redundant. They go back and forth until the second has nothing left to cut. The team uses it to build the glossary and the book summary.
Trilateral collaboration. One agent does the work, a second critiques it, and a third decides whether it needs another round. The authors found the third role critical to quality, because it stops the loop.
Two ways the agents work together
How was it tested?
On Chinese web novels translated into English: the 240 chapters in the test set of the WMT 2023 shared task on discourse-level literary translation. The authors compared TransAgents with GPT-4 on its own, a fine-tuned LLaMA model, and the human reference translation.
They used three kinds of evaluation:
- d-BLEU, the standard automatic score, which counts how closely a translation matches a reference. TransAgents scored worst of all the systems.
- Monolingual Human Preference. Readers in the United States who were interested in Chinese web novels compared translations without seeing the original. At least ten people judged each pair.
- Bilingual LLM Preference. GPT-4 compared each translation directly with the Chinese source.
On both preference measures, readers and the model preferred TransAgents’ translations over GPT-4’s and over the human reference. It did best in genres that need background knowledge, such as historical and cultural settings, and worse in contemporary ones.
The low d-BLEU score and the high preference scores aren’t as contradictory as they look. d-BLEU rewards closeness to one reference translation, and TransAgents wrote more freely. Two experienced professional translators who reviewed a sample chapter described its style as the most expressive and engaging, and also noticed it sometimes left parts of the original out.
What would it cost?
The paper’s cost comparison is the headline most people remember. Using the American Translators Association’s suggested minimum of US$0.12 a word, a chapter of about 1,404 words would cost $168.48 to translate by hand. TransAgents translated the whole test set for about $500, or $2.08 a chapter: roughly 80 times cheaper.
Try your own numbers, and see what happens when you add the human review the study suggests you’d need.
Agents, humans, or both?
What went wrong?
The paper is honest about its weaknesses, and they matter as much as the results.
- Content omission. Both GPT-4 and TransAgents left out significant chunks of the source. The omissions didn’t break the plot, but they could thin out character development or change the emotional effect of a passage. For a client, a translation that reads beautifully and quietly drops sentences is a serious risk.
- Weaker in contemporary genres. The advantage came in historical and cultural material.
- Evaluation limits. The authors list them: how documents were split for review, who the evaluators were, the scale of the evaluation, and the quality of the human reference it was compared with.
- One language pair, one genre. Chinese web fiction into English is a long way from a pharmaceutical label or a legal contract.
What does it mean for language service providers?
The striking thing about TransAgents is how closely it copies an LSP’s own workflow. Every role in the diagram is a role a translation company sells. Once those roles can be written as prompts, the question for an LSP becomes which parts of its process carry the value that clients will keep paying for.
The study points at the answer. The agents were cheap, fluent and often preferred, and they still dropped content that nobody would notice without checking against the source. On this evidence, the parts that still need people are quality assurance, accountability, and judgement about what a client actually needs. Those are the services to build and sell around.
When I first wrote about this paper I said I was sure the tool vendors were watching it very closely. For more on what this shift means for language companies, see Nobody asked the language industry, my comparison of LLMs for LSP sales, and YouTube’s live auto-dubbing. If you run a language company and want to work through what it means for yours, that’s the kind of work I do in localization consulting.
Sources: Wu, M., Xu, J., Yuan, Y., Haffari, G., Wang, L., Luo, W. and Zhang, K. (2024, revised 2025) “(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts”, arXiv:2405.11804, to appear in Transactions of the Association for Computational Linguistics; American Translators Association, atanet.org.