The Current State of Automatic Speech Recognition: A Report
Read the 2022 State of Automatic Speech Recognition [Free Report]
3Play Media conducts annual research to determine the current state of automatic speech recognition technology. The study looks at the general state of speech-to-text technology and evaluates how the top speech recognition engines perform, specific to the task of captioning and transcription. Our research found that the accuracy of the technology has improved measurably since the company’s last report, published in January of 2021.
“As the AI models driving ASR continue to evolve, many of the engines we evaluated have shown significant strides in their transcription accuracy over the last two years.”
3Play Media tested all engines using a large dataset representative of 3Play Media’s diverse customer base. Accuracy was evaluated against two measurements: Word Error Rate (WER) and Formatted Error Rate (FER), which includes formatting errors like grammar, speaker identification, and non-speech elements in addition to word errors.
In both WER and FER measurements, Speechmatics with 3Play modeling and post-processing led the pack, followed by Speechmatics alone and Microsoft. Rev, Google VM, and Voicegain followed, each with respectable scores which were close enough that these vendors are hard to differentiate. Despite exciting improvement across the board, all engines performed well below the industry standard of 99% accuracy, confirming that ASR on its own still falls short of being “good enough” for compliance with closed captioning legal requirements.
Read The 2022 State of Automatic Speech Recognition: ➡️
“As the AI models driving ASR continue to evolve, many of the engines we evaluated have shown significant strides in their transcription accuracy over the last two years,” Chris Antunes, Co-CEO and Co-Founder, 3Play Media, said. “We run this report every year because we use ASR in our own transcription process, and we have a vested interest in making sure we’re utilizing the best engine on the market. Speechmatics remains a clear industry leader in both pre-recorded and live automated transcription, and applying 3Play’s mappings and post-processing resulted in an exciting improvement in word error rate of over 8%.”
The study showed a wide range in accuracy among the technology tested, with the highest and lowest performing engines differing by over 15 percentage points. This suggests that different engines are optimizing for different goals, and some ASR engines will not perform well for transcription. Compared to other uses of speech-to-text technology like automated assistants that are able to train on a specific voice, transcription is a very difficult task, with variables like diverse sentence structure and spontaneous speech, specialized terminology, and complex patterns including multiple speakers, accents, and background noise.
Accuracy is critical in captioning for a number of reasons, most important being that individuals who are d/Deaf or hard of hearing rely on captions as an accommodation. Accurate captions also improve viewer engagement – studies show that captions improve watch time, brand recognition, and comprehension. And, while customer experience has emerged as a critical driver for businesses, so has digital accessibility legislation: in 2021 alone, there were 10 accessibility lawsuits filed per day.
—
Read the full 2022 State of Captioning Report to learn more about our findings and discover what you can expect from both 3Play Media and ASR engines in 2022 and beyond.
Further Reading
Subscribe to the Blog Digest
Sign up to receive our blog digest and other information on this topic. You can unsubscribe anytime.
By subscribing you agree to our privacy policy.