Hot French startup ZML releases free product to speed inference across lots of AI chips
ZML, a hot French AI startup endorsed by Turing Award winner Yann LeCun, has now released ZML/LLMD, software that could make running AI less costly. The days of Nvidia's unparalleled market dominance aren't over, but challengers and choices are arising from all directions. ZML , a hot French AI startup endorsed by Turing Award winner Yann LeCun, has released inference -performance software that allows a variety of open-source large language models to run on a variety of chips - including Nvidia's, AMD's, Google's TPU, Apple Metal and Intel Arc.
Key Takeaways
- With ZML/LLMD , the newly launched LLM inference server, the company's ambition is to break existing silos and make different chips available for AI use cases at their maximum available speed, and sometimes faster, ZML founder Steeve Morin told TechCrunch.
- Such a software assist may help novel AI chipmakers, many of which happen to be from Europe, Morin observed, citing Axelera , Fractile , Kalray , OLIX , Q.
- Inference has been an area of such intense investment, that the trend has been hailed the " inference gold rush .
" So ZML has competition such as Baseten , recently valued at $13 billion; Inferact , from the creators of open source project vLLM ; as well as RadixArk , the commercial company behind SGLang .
- Thanks to his track record as VP of engineering of Zenly , which Snapchat acquired for nine figures in 2017 , Morin raised $20 million from venture firms including Harry Stebbings' 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel's Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.
Unlike ZML's first public project, the inference-focused ML framework released in 2024 and updated in March , ZML/LLMD is not open source.
- This also builds the case that Europe's AI startups can now build from home .
Stats & Key Facts
- #" So ZML has competition such as Baseten , recently valued at $13 billion; Inferact , from the creators of open source project vLLM ; as well as RadixArk , the commercial company behind SGLang .
- #Thanks to his track record as VP of engineering of Zenly , which Snapchat acquired for nine figures in 2017 , Morin raised $20 million from venture firms including Harry Stebbings' 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel's Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.
With ZML/LLMD , the newly launched LLM inference server, the company's ambition is to break existing silos and make different chips available for AI use cases at their maximum available speed, and sometimes faster, ZML founder Steeve Morin told TechCrunch. As AI becomes integrated into our work and everyday lives, optimizing inference - aka, the processing of prompts - has been outpacing model training in importance, but often feels patchy behind the scenes, with software and architecture barriers that lead to vendor lock-in, Morin said. The promise of achieving peak performance across a variety of chips is a technological feat, but it could also be a market disruptor, amid mounting fears over AI-related costs.
ZML hopes to provide enterprises and clouds with the option to use a mix of chips, some of which might be less costly or consume less energy. "The idea is to give people back the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated," Morin said. Such a software assist may help novel AI chipmakers, many of which happen to be from Europe, Morin observed, citing Axelera , Fractile , Kalray , OLIX , Q.
ANT , SiPearl , SpiNNcloud , and VSORA . But more than their region of origin, what matters to him is that ZML can work with them on "things that haven't been done before anywhere in the world. " That doesn't mean Morin is bearish on Nvidia.
For more details please read the original article at TechCrunch AI.
Continue Learning
Comments
Sign in to join the conversation