BMI belongs to the Department of Electrical, Computer, and Biomedical Engineering (Faculty of Engineering) of the University of Pavia, Italy. Established in 1982, it is a leading center for education, research, and IT innovative solutions in the healthcare area. Nowadays about 30 people are working at BMI, focusing their research on:
Our research interests have led us to frequently explore the realm of Natural Language Processing, including Transformers. Here we host public weights for our biomedical language models. There are several options to choose from, please check the details below.
Model | Domain | Type | Details |
---|---|---|---|
Igea | Biomedical | CausalLM Pretrain | Small language model trained after sapienzanlp/Minerva with more than 5 billion biomedical words in Italian. Three versions available: 350M params, 1B params, and 3B params. Use the quantized GGUF version for CPU-only, limited-hardware machines. |
BioBIT * | Biomedical | MaskedLM Pretrain | BERT model trained after dbmdz/bert-base-italian-xxl-cased with 28GB Pubmed abstracts (as in BioBERT) that have been translated from English into Italian using Neural Machine Translation (GNMT). |
MedBIT * | Medical | MaskedLM Pretrain | BERT model trained after BioBIT with additional 100MB of medical textbook data without any regularization. |
MedBIT-R3+ (recommended) * | Medical | MaskedLM Pretrain | BERT model trained after BioBIT with additional 200MB of medical textbook data and web-crawled medical resources in Italian. Regularized with LLRD (.95), Mixout (.9), and Warmup (.02). |
* model developed for the Italian Neuroscience and Rehabilitation Network in partnership with the Neuroinformatics Lab of IRCCS Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy