The Open ASR Leaderboard Adds Its First Global South Language
Hugging Face and Voice Arena have partnered to introduce Monsoon en-IN and Monsoon hi-IN to the Open ASR Leaderboard. Spoken by more than half a billion people, Hindi becomes the first Indic language added to the platform's multilingual benchmark. The new evaluation datasets feature 4,888 speakers across four public and private splits.
Key Takeaways
- The Open ASR Leaderboard has expanded its evaluation scope by introducing its first Global South language through a partnership between Hugging Face and Voice Arena.
The update adds two evaluation sets, Monsoon en-IN and Monsoon hi-IN, bringing Indian English and Hindi to the platform.
- Spoken by more than half a billion people, Hindi represents the first Indic language included on a multilingual tab that previously covered only European languages.
The datasets are separated into public and private splits to support self-scoring while preventing benchmark-specific optimization.
- To address this issue, the Monsoon datasets include 4,888 speakers across four speaker-disjoint splits with 12 recorded attributes per speaker.
The collection varies along nine design axes, including geography, age, gender, devices, acoustic environments, and speech rate, ensuring automated speech recognition systems are evaluated across real-world conditions.
- Voice Arena and Hugging Face launched new evaluation sets named Monsoon en-IN and Monsoon hi-IN on the Open ASR Leaderboard.
Hindi is spoken by more than half a billion people and is the first Indic language added to the leaderboard.
- Each speaker in the dataset has 12 recorded attributes to measure automated speech recognition performance across diverse populations.
Stats & Key Facts
- #The new evaluation datasets feature 4,888 speakers across four public and private splits.
- #To address this issue, the Monsoon datasets include 4,888 speakers across four speaker-disjoint splits with 12 recorded attributes per speaker.
- #The evaluation datasets include 4,888 speakers across four speaker-disjoint public and private splits.
- #Each speaker in the dataset has 12 recorded attributes to measure automated speech recognition performance across diverse populations.
The Open ASR Leaderboard has expanded its evaluation scope by introducing its first Global South language through a partnership between Hugging Face and Voice Arena. The update adds two evaluation sets, Monsoon en-IN and Monsoon hi-IN, bringing Indian English and Hindi to the platform. Spoken by more than half a billion people, Hindi represents the first Indic language included on a multilingual tab that previously covered only European languages.
The datasets are separated into public and private splits to support self-scoring while preventing benchmark-specific optimization. Traditional speech recognition benchmarks often fail to expose accuracy disparities across different demographics because test sets record what was said rather than who said it. To address this issue, the Monsoon datasets include 4,888 speakers across four speaker-disjoint splits with 12 recorded attributes per speaker.
The collection varies along nine design axes, including geography, age, gender, devices, acoustic environments, and speech rate, ensuring automated speech recognition systems are evaluated across real-world conditions. Voice Arena and Hugging Face launched new evaluation sets named Monsoon en-IN and Monsoon hi-IN on the Open ASR Leaderboard. Hindi is spoken by more than half a billion people and is the first Indic language added to the leaderboard.
For more details please read the original article at Hugging Face.
Continue Learning
Comments
Comments appear only after moderation. Your email identifies your submission to the moderator and is never displayed here.
No approved comments yet.