What I Learned from Whisper Fine-Tuning Event

Bofeng Huang

TL;DR

OpenAI’s Whisper promises to “approach human level robustness and accuracy on English speech recognition”. In this blog, we will explain how to further improve its performance on other languages. It allowed us to win first prize in the Whisper Fine-Tuning Event held by HuggingFace 🤗 and Lambda Labs, both on French language and German language. The models and the demos are available on the Hugging Face Hub.

Preface

Iam a machine learning engineer at Zaion, a French company that is leading the European market of customer relation solutions. Zaion’s goals include providing accurate and precise transcription of customer service conversations. Thus, it is crucial for us to have a reliable speech recognition system, that is robust with different real-world environments.