Which AI Is Best at European Languages? We Tested 9 Models in 8 Languages

In a single low-cost run, nine AI models translated the same recent press release and wrote original texts in eight European languages, exposing clear gaps between the leaders and the rest.

OpenAI’s GPT-6.1 Sol made the fewest errors overall and also produced the strongest writing. The best open models were Meta’s Muse Spark Open and Moonshot AI’s Kimi K3.1 Pro, while Europe’s own Mistral AI Large 4 finished sixth in translation and last in writing.

Which AI Is Best at European Languages?

GPT-6.1 Sol led on both tests we ran. Its eight translations accumulated just 16 error points in total, compared with 22 for Google’s Gemini 4 Carbon, and the judge rated seven of them publishable without further work. It also posted the highest writing score of 94.5 out of 100.

The two closed models finished at the top. On the 100-point quality scale the best open model, Muse Spark Open, scored 99.0 against 99.7 for GPT, yet it made roughly three times as many error points. Errors follow standard MQM weighting (1 for minor, 5 for major, 25 for critical). The speed figures are the median time per translation, so one unusually slow language doesn’t distort the result.

How Did We Test the AI Models?

We gave every model the same English text, a European Parliament press release on the Sakharov Prize published on October 22, 2026, too recent to be in their training data. The Parliament also publishes its own version in every EU language.

Each model translated it into eight languages and wrote two texts directly in each: an apology email from a Berlin furniture shop and an explanation of inflation for a 15-year-old, with prices in local money. We ran all nine through OpenRouter, each pinned to its maker’s own service where one exists, with the same low reasoning setting for all.

Claude Opus 5.5 judged every batch blind, seeing only letters, with the MQM error framework, which Google researchers found more reliable than crowd-worker ratings. Because the annotator itself is an AI, we checked its scores. The Parliament’s own translations entered the same batches as a control.

We checked the results three ways. chrF++, an automatic measure of closeness to the Parliament’s versions, placed the top five models in the same group. Re-judging the French and Hungarian sets from scratch shifted each translation’s score by about one point out of 100 on average (excluding Mistral’s two re-runs) and left the same winner. A native Hungarian speaker also scored the Hungarian translations blind.

Which AI Translates Best?

GPT-6.1 Sol, followed by Gemini 4 Carbon. Muse Spark Open (46 points) and Kimi K3.1 Pro (53) were the closest open models. Seven models had all eight translations publishable after a light edit or better. GLM 5.3 and Gemma 4 31B managed five each, and Gemma’s Hungarian was the only translation the judge labelled unpublishable.

Kimi, GLM 5.3, DeepSeek and Gemma all used “liberté de pensée” in French, while the prize’s official French name is “liberté de l’esprit”. Qwen 3.8 left an English word in its French and Polish versions and a Turkish word in its Hungarian. Mistral Large 4 dropped a French word into its Italian.

Which AI Writes Best in Each Language?

GPT-6.1 Sol led writing as well, with 94.5, ahead of Gemini 4 Carbon (91.6) and Kimi K3.1 Pro (90.8). Kimi wrote the strongest Spanish and Swedish pieces, and Gemini the strongest Hungarian.

Writing exposed local habits more than translation did. Three emails kept the English label “Subject:”, from DeepSeek in Italian and Hungarian and from Mistral in Polish. Others addressed customers in a way a local shop would not, such as by title and surname alone in Polish.

Which Open Model Is Best for European Languages?

Among open-weight models, Muse Spark Open for translation and Kimi K3.1 Pro for writing. Meta’s 816bn-parameter Muse Spark Open made 46 error points and finished each translation in a median of 18 seconds. Kimi K3.1 Pro, whose weights came out on November 4, wrote better but took 90 seconds.

DeepSeek V4.1 Flash is the bargain. All 24 of its answers cost $0.046, and it made 72 error points. Our map of Chinese LLMs covers it, Kimi, Qwen and GLM. Google’s small Gemma 4 31B is the weakest choice here, mostly because of its Hungarian.

How Did Mistral, Europe’s Own Model, Do?

Mistral Large 4 finished below the leaders. Mistral Large 4 came sixth in translation, with 126 error points, and last in writing, with 80.8. It was also the slowest, at 138 seconds per translation.

Most of that time went on thinking. Even on our low setting, Mistral used a median of about 10,800 hidden reasoning tokens per translation. It ran past our 12,000-token limit on six of its eight translations, so our script retried them with double the budget, and two answers cut off in our first run had to be redone. The weights of the trillion-parameter model came out on October 29.

Europe’s public alternative is not ready yet. OpenEuroLLM’s first model is unfinished, so we could not include it.

Did AI Beat the Parliament’s Own Translators?

On faithfulness to the English source, most of them did. The Parliament’s versions collected 266 error points, more than every model except Gemma. About 40% of those points came from added or cut material that reads as deliberate local editing.

The Italian and Polish versions added a list of countries where the Court has issued arrest warrants, and the Italian one gave the date of the ceremony. The Spanish one changed a phrase in Roberta Metsola’s quote, which the judge called unacceptable in a press release.

The rest were real slips, such as a wrong Polish term for crimes against humanity and a Swedish line that turned the award into a nomination. Our Hungarian reviewer called the Parliament’s version the most readable of the ten but the loosest with the source.

Which AI Is Best for Hungarian?

Hungarian separated the models more sharply than any other language in the test. GPT-6.1 Sol and Gemini 4 Carbon finished essentially tied on translation, one error point apart, while Gemini led the writing scores. Gemma 4 31B scored only 65.2 out of 100, inventing words such as “kézrendóriparok” for arrest warrants and using the wrong first name for Andrei Sakharov.

A native Hungarian speaker also scored the ten Hungarian translations blind from 1 to 5. Full marks went only to GPT-6.1 Sol and Gemini 4 Carbon, with the lowest mark going to Gemma. The order matched the judge’s closely, with a rank correlation of 0.89, and the same slips were spotted, including the Turkish word in Qwen’s version.

Hungarian is also the most expensive language to process. The same text needs about twice as many tokens as in English, which we explain in our piece on the AI language tax.

Author: Akos Szima

See Also:

European LLMs in 2026: The Complete Map

What Is the Best Open-Source LLM in 2026?

What Is OpenEuroLLM? Europe’s €37m Open-Source AI Project and Its Unfinished First Model

How We Checked This:

All answers were generated on November 11, 2026 through OpenRouter, with no provider fallbacks; the run cost $2.43. Each model used reasoning effort “low”, because at the default setting Kimi and GLM spent 24,000 reasoning tokens on the Hungarian translation without answering. The judge’s scores follow MQM weights (minor 1, major 5, critical 25) per 587 source words. Writing scores average four criteria (language, naturalness, register, task) out of 10, times 10. We counted an English “Subject:” label as a major error and still scored the text. chrF++ was computed with sacrebleu against the Parliament’s versions. Download every score, with sources (CSV).

What This Test Does Not Show

The results come from one source text and two writing tasks: a 587-word European Parliament press release on the Sakharov Prize, a customer apology email, and a short explanation of inflation for a 15-year-old. Other text types, such as legal documents or open conversation, were not tested.

Eight languages were included: German, French, Spanish, Italian, Polish, Dutch, Swedish and Hungarian.

Judging was done by Claude Opus 5.5 using the MQM error framework. Scores were checked with chrF++, by re-judging the French and Hungarian sets, and by a native Hungarian speaker. The ranking stayed the same. The Parliament’s own translations scored 266 error points under the same method. Much of that total came from added or removed material.

All models ran with the same low reasoning setting.

Change log: November 13, 2026, first published.

Frequently Asked Questions:

This test was designed and run by MRKT 3.0. The judge, Claude Opus 5.5, is made by Anthropic, which has no model in the test. MRKT 3.0 has no commercial relationship with any of the model makers.