EuroLLM: Multilingual Language Models for Europe
akhaliq
Abstract
The quality of open-weight LLMs has seen significant improvement, yet they remain predominantly focused on English. In this paper, we introduce the EuroLLM project, aimed at developing a suite of open-weight multilingual LLMs capable of understanding and generating text in all official European Union languages, as well as several additional relevant languages. We outline the progress made to date, detailing our data collection and filtering process, the development of scaling laws, the creation of our multilingual tokenizer, and the data mix and modeling configurations. Additionally, we release our initial models: EuroLLM-1.7B and EuroLLM-1.7B-Instruct and report their performance on multilingual general benchmarks and machine translation.
Community
Nice work!
But why are the translations English-centric?
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data (2024)
- A Survey of Large Language Models for European Languages (2024)
- Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation (2024)
- BOTS-LM: Training Large Language Models for Setswana (2024)
- Ruri: Japanese General Text Embeddings (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
It's always great to see wider multi-lingual efforts. Nice!
Will you folks also release the dataset you created in some form? Even subsets for the non-English languages would be very valuable.
Models citing this paper 3
Datasets citing this paper 0
No dataset linking this paper