LFM2.5-Encoders for Fast Long-Context Inference on CPU
They match the quality of larger models but stay fast as inputs get longer. This means you can run document-scale jobs on the hardware you already have, even on CPU. Here's what you get: Strong for their size: match or beat larger encoders on GLUE, SuperGLUE, and multilingual tasks.
Key Takeaways
- 8,192-token context with latency that grows slowly as inputs get longer.
Fast on CPU : about 3.7× faster than ModernBERT-base at long context.
- That's why we built a general-purpose model instead of reusing the retrievers.
Encoders power many modern production NLP applications: classifiers, intent routers, safety filters.
- Masked language modeling: we mask 30% of the tokens during training.
We train both models in two stages: General language competence : a short-context masked-language objective on a large web corpus at a 1,024-token context.
- The three ahead of it are all larger, including a 3.5B model nearly 10 times its size.
LFM2.5-Encoder-230M beats ModernBERT-base and every EuroBERT model, while being smaller than most of them.
- At 8,192 tokens, ModernBERT-base takes over a minute and a half per forward pass versus about 28s for LFM2.5-Encoder-230M.
Stats & Key Facts
- #8,192-token context with latency that grows slowly as inputs get longer.
- #Masked language modeling: we mask 30% of the tokens during training.
- #We train both models in two stages: General language competence : a short-context masked-language objective on a large web corpus at a 1,024-token context.
- #Long-context adaptation : extending context to 8,192 tokens on the full data mix, strengthening factual, legal, and multilingual competence.
8,192-token context with latency that grows slowly as inputs get longer. Fast on CPU : about 3.7× faster than ModernBERT-base at long context. With these, you can build intent routers, policy linters, PII detectors, and text classifiers that run cheaply, all day.
Why we built a general-purpose encoder Last month we released LFM2.5-Retrievers , built for multilingual search. LFM2.5-Encoders come from the same family but serve a broader purpose. They're pre-trained with a masked-language objective, so you can fine-tune them for classification, token-level tasks, and search alike.
Search is just one thing an encoder enables. That's why we built a general-purpose model instead of reusing the retrievers. Encoders power many modern production NLP applications: classifiers, intent routers, safety filters.
For more details please read the original article at Hugging Face.
Continue Learning
Comments
Comments appear only after moderation. Your email identifies your submission to the moderator and is never displayed here.
No approved comments yet.