Aleph Alpha has released a model it calls sovereign, while the sale of the company is still open.
On October 3 the Heidelberg firm put Kolibri on Hugging Face under the label “Sovereign AI made in Germany.” The weights are public under Apache 2.0, the licence that lets a licensee download, modify and ship them. The model handles German and English, was trained in Europe, and is built to run on infrastructure the customer controls.
On September 16, Cohere and Aleph Alpha signed a definitive business combination agreement, five months after the plan was announced on April 24. Cohere says the unified company will operate globally as Cohere and “effectively create the first transatlantic sovereign AI solution.” The transaction remains subject to final regulatory approvals. Aleph Alpha’s October 5 note still says the agreement was “recently announced,” and that until closing it continues to operate independently.
Ilhan Scheer, chief executive on the Kolibri release and co-chief executive in the September materials, said, “Kolibri demonstrates that we have the talent and expertise in Germany to develop competitive AI models. For us, AI sovereignty means freedom of choice by retaining the ability to build and advance this technology.”
What is Aleph Alpha’s Kolibri?
Kolibri is a mixture-of-experts model, a design that keeps a large pool of specialist blocks and switches on only a few for each token.
Aleph Alpha says it combines 78 billion total parameters with around 3 billion active per token. The model card lists 78.1 billion in total and 3.46 billion active. Its native context length, the amount of text it was trained to read in one pass, is 262,144 tokens. Aleph Alpha has validated quality up to 1,048,576 tokens, and recommends at most 262,144 for serving efficiency and complex tasks.
The checkpoint is about 78GB in FP8, a compressed number format used to cut memory. The card’s minimum is two A100 80GB cards, two H100 SXM5s, or one H200, B200 or B300.
The pre-training mix on the card is about 62.5 percent English, 23.9 percent German and 13.6 percent code. The release rounds the German share to around 23 percent. All training data was screened against a blocklist of more than 4.5 million URLs, drawing on sources including the European Commission’s Piracy Watch List.
Where the Agreement Leaves It
Reports indicate that the combined business will operate as Cohere, with dual headquarters in Toronto and Berlin, and that Heidelberg will focus on research. Aidan Gomez stays chief executive. Scheer becomes chief operating officer on closing, and Samuel Weinbach becomes chief research officer.
In April it was reported that the company had a combined value of about $20 billion. Aleph Alpha shareholders would receive one Cohere share for every nine they held, which is about 10 percent of the group. The companies have not published updated terms. Schwarz Group, the Lidl owner, is investing €500 million in the new entity. Aleph Alpha’s 2023 Series B raised $500 million. That was capital in, not a published valuation, which is why the €450 million figure on this site’s ranking should not be read as a priced round.
Gomez said at the signing, “no government or enterprise should have to choose between capable AI and control over their technology.”
Karsten Wildberger, Germany’s digital minister, said on May 20, “when capital, compute, customers, and political will come together, sovereignty stops being a concept, it becomes a possibility.” A licensee can download Kolibri from Heidelberg this week. Who maintains it after regulatory approval and closing is the open question.
Author: Grace Sharp
See Also:
Karsten Wildberger on Germany’s AI Sovereignty Push
What Is Sovereign AI? Definition, the Money Behind It, and Europe’s Reality
