3Play’s Patent Playbook: Revolutionizing Live Captioning Technology with Failover and Failback

3Play’s Patent Playbook blog series tells the stories behind our patented technology. Learn how 3Play Media’s Research and Development (R&D) teams are spearheading innovation in accessibility tech and creating breakthroughs in the media accessibility industry at large.

The 2023 State of Automatic Speech Recognition [Free Report]

To kick off our Patent Playbook series, we’re delving into how 3Play Media is revolutionizing live captioning technology with our patented failover and failback solutions. We will take a look at the state of live captioning prior to our team’s developments, learn the story behind the Hybrid Live Captioning Systems and Methods patent, and discover the impact our technology has made on the media accessibility industry.

The State of Live Captioning Before 3Play

For years, the live captioning industry relied on complex technologies and manual methods that provided inconsistent captioning experiences. Captioning workflows were often hobbled together in a piecemeal fashion, with a heavy reliance on physical encoder and decoder equipment. Individual captioners, vendors, and producers were part of a tangled web of unclear responsibilities and outdated tech.

Live captioners or vendors are commonly booked with back-to-back live events, some of which may be scheduled at the last minute. If an event goes over time, they may finish captioning it at the expense of another event, creating a gap in coverage. Should they need to take a necessary break to grab a bite or even use the restroom, captioners are left with a difficult dilemma that puts both accessibility and personal needs at risk.

Live event and broadcast producers need their events to be accessible and compliant. Coordination of live captioning coverage often falls to them, a task that can be stressful and time-consuming. Even once a producer secures a captioner or vendor, they must test connections, set up any necessary physical equipment, and hope for the best. Most of the time, the process is smooth. But should captions fail for some reason—such as an internet disruption, power failure, encoder/network outage, or a schedule with back-to-back events—producers must answer for it, even if they did everything right.

The complexity of live captioning workflows is additionally compounded by captioner capacity. A significant increase in live events and live broadcast programming over the past few years has started to outpace the number of trained, qualified live captioners. Securing captioners for live events and broadcast can be a struggle for producers and networks who need captions to meet ADA and FCC requirements.

No matter the nature of the live captioning issue, everyone involved is often left scrambling to figure out a solution to an already flawed process. That process is exactly what 3Play has actively worked to improve.

Innovating Live Captioning Technology

Innovation is a key part of 3Play Media’s core values. Since our inception at MIT in 2007, we have been driven to create more efficient ways to provide media accessibility services across a variety of industries. 3Play’s tech-forward mindset has resulted in a number of patents, including Hybrid Live Captioning Systems and Methods, better known as our ever-reliable Live Captioning Failover and Failback.

Live Captioning Failover and Failback ensures a live stream is never without captions. 3Play’s Live Automatic Captions always run in the background, while our system simultaneously monitors the status of the Live Professional Captioning stream. If the feed is lost for any reason, 3Play fails over to auto captions, then fails back to professional captions once the feed stabilizes.

So what inspired our team to develop this technology for live captioning?

According to Roger Zimmerman, Chief of Research & Development (R&D) at 3Play Media, a reliable failover and failback solution was always part of 3Play’s live captioning technology roadmap.

3Play’s R&D team was always cognizant of the fact that while human-produced captions were higher quality than auto captions, they remained susceptible to problems like lost connections. Creating a backup method that kept live automatic captions going in the background became a North Star for the team to align around.

“As soon as we decided to release Live Professional Captions, [we knew] failover and failback had to be there,” says Zimmerman, who noted that the development process happened fairly quickly, with the R&D team’s ideas and logic for live captioning failover taking shape shortly thereafter.

By the end of the development process, 3Play’s R&D team had created one of the most exciting and innovative advancements in modern accessibility technology: an agile live captioning solution that automatically monitors live captioning status and provides a seamless backup method in the event of an outage.

The Patent: Hybrid Live Captioning Systems and Methods

3Play’s Hybrid Live Captioning Systems and Methods patent discloses, among other technologies, live captioning failover and failback based on programmatic detection methods. Determining if and when a live captioner’s connection is lost and regained can be done in a number of ways, for example: counting time since the last professional captions appeared, sensing whether audio is still present, and being directly notified by a captioner when they’ve lost a connection.

Much of our live failover and failback solution is powered by 3Play’s proprietary logic that enables our technology to go further than standard live captioning methods, providing a “nice insurance policy for well-known problems” in the live captioning world, according to Zimmerman.

A Consistent Solution to Live Captioning Woes

Live Automatic Captions will never replace Live Professional Captions, but they are a great stopgap to ensure continued coverage when outages occur with human-enabled workflows. We believe that blending human expertise with innovative technology is how we can best inspire further technological developments and achieve sustainable, accessible solutions that propel the media accessibility industry forward. Our patented technology is an important step in this direction. 

In the time since our R&D team’s development of Live Captioning Failover and Failback, it is clear that the impact goes deeper than functioning as a simple backup option. This solution creates a wealth of benefits for customers, live captioners, and the accessibility industry:

• Customers now have the peace of mind that their live events always have captions running and remain accessible for their viewers, without having to stress over the stability of a captioning feed.
• Captioners have greater breathing room for those moments when their internet connection is unstable or even if they need a quick personal break.
• The accessibility industry now sees live captioning capabilities growing to new heights. 3Play’s technology allows for consistent accessibility in the end user’s experience and promotes the development of inclusive solutions that spark change and advancement in outdated captioning processes.

Beyond 3Play’s technology, our innovative, holistic approach to the live captioning process is changing the industry for the better. With our scaled workforce, qualified live captioner coverage for live events and broadcast is consistent and stable. Live event and broadcast producers have more transparency into scheduling and captioner coverage than ever before with in-platform visibility. And advanced technological solutions like Tuned ASR and Live Virtual Encoding allow producers the freedom to take their caption workflows beyond outdated technologies. Learn more about our live captioning solutions and how you can ensure a seamless, accessible experience for your viewers.