Breaking News
Finnish consortium unveils first member of planned family of European LLMs
Helsinki-based Silo AI has released the first member of a family of large-language models that is to cover all 24 official languages of the EU.
Silo AIPoro 34B, a large-language model (LLM) for Finnish and English, has been released by a consortium launched by Silo AI.
Trained on a dataset of one trillion tokens by LUMI, a supercomputer situated in Kajaani, Finland, the model is also proficient in a number of programming languages, such as Java and Python, and can perform basic translations between English and Finnish.
Silo AI said Poro has completed 30 per cent of the planned training but is already achieving state-of-the-art results, outperforming existing monolingual models on the widely used benchmark for Finnish, FIN-bench.
The capability stems from how it addresses the relative lack of data associated with low-resource languages such as Finnish. It leverages shared patterns across languages to cross-train low-resource languages with high-resource ones, an approach that allows it to achieve greater performance on the low-resource language than a monolingual approach.
The release marks the first step toward creating a family of open-source large-language models that covers all 24 official languages of the EU, an endeavour undertaken in collaboration with the natural language processing research team at the University of Turku.
Called Poro 34B, the model is already outperforming existing monolingual models on the FIN-bench benchmark for Finnish, despite having completed only 30 per cent of its training.
Silo AIAs large-language models are shaping how people access information and interact with technology, it is important to make sure the models capture the values, cultures, languages and regulatory environments in Europe, Peter Sarlin, CEO of Silo AI, underscored in an interview with VentureBeat.
“This initiative helps to ensure that underlying models are based on data and information representing the citizens and organisations of the region, and overall compliance with regulation, data privacy and other vital concerns,” he explained in a news release. “And eventually we need sovereignty on how downstream applications and value creation happen. This requires trusted and secure approaches to independent base models that enable fine-tuning for domain-specific needs.”
“From a European perspective, it is also critical that models are designed from the outset to prioritise [multilingualism] and an equitable approach to all languages,” added Sampo Pyysalo, research fellow at the University of Turku.
SiloGen, the generative artificial intelligence division of Silo AI, has also collaborated with the Finnish public broadcasting company YLE.
Having access to large-language models that “are based on our languages and truly reflect our local culture” is of utmost importance to YLE, said CEO Merja Ylä-Anttila.
“We are more than happy to be part of the exploration on how public service media companies around Europe can participate in the development of trustworthy AI technologies, including language models, that take the rich diversity of languages and cultures into consideration,” she stated.