satellite

CaptionSync by AST Accuracy Analysis

We submitted files to CaptionSync by AST in order to measure their true accuracy and provide empirical evidence of their product quality.

Read on for the results.
arrow pointing down

We submitted 7 files with good quality audio, of varied subject matters, and with a duration of 2+ minutes to CaptionSync.

What We Measured

  • Spelling and Grammatical Errors
    Since punctuation errors are subjective, we measured accuracy for both including and excluding punctuation and case.

  • Word Error Rate (WER)
    The WER measures the number of substitutions, deletions, and insertions, divided by the number of words in the reference.
  • Order & Delivery Process
    We measured the time elapsed from ordering to order confirmation and time elapsed from order received to delivery.

  • Revision and Support Experience
    We also measured the time elapsed for editing and correcting the returned transcripts from CaptionSync.

CaptionSync's average accuracy rate for the 5 files submitted ranged between 93.49% to 95.00% accuracy.

The industry standard for caption quality is a 99% accuracy rate. This means that there is a 1% chance of error or leniency of 15 errors total per 1,500 words.

The FCC, DCMP, and WCAG have all set captioning quality standards to ensure captions are comprehensible. Failure to uphold these standards could affect your brand, messaging, or result in a costly lawsuit.

For more information on caption quality, read Best Practice for Caption Quality.

Graph shows 3Play Media's Measured Accuracy Rate at 99.6% and 99% being the industry standard. CaptionSync's measured average accuracy range is between 93.49%-95%.

Here's a comparison of the files we submitted alongside their accuracy rate.

CatpionSync’s average accuracy rate for including punctuation for the 7 files submitted: 93.5%

Even when we omit punctuation, CaptionSync’s accuracy falls short of 99%, and the average accuracy rate for the 7 files comes to: 95%

*Why include or omit punctuation? Punctuation errors can be subjective, so we measured both cases.

At 3Play Media, our accuracy rate is measured including punctuation.

Table showing the accuracy rate for 7 videos when punctuation errors are included versus when they are omitted

Common Errors

Throughout files, we saw errors such as extra and incorrect words inserted into sections with no dialogue, false starts, misplaced commas, and spelling errors.

The most common error we saw from CaptionSync was [inaudible] tags when the audio was clear enough to transcribe.

Missed Words

CaptionSync’s Errors:

  • Misses words
  • Uses [inaudible] tags unnecessarily

Why Spelling Matters:

  • Affects your branding
  • Affects the meaning of the content
  • Affects viewer comprehension

Incorrect Words

CaptionSync’s Errors:

  • Incorrect words
  • Extra words
  • False starts
  • Incorrect tense

Why This Matters:

  • Incorrect wording creates confusion for the reader
  • Impacts reading comprehension

Punctuation

CaptionSync’s Errors:

  • Missing commas
  • Missing apostrophes
  • Missing hyphens

Why punctuation matters:

  • Affects meaning of the content
  • Affects reading comprehension

Missing Speaker IDs

CaptionSync’s Errors:

  • Missing speaker identifications

Why speaker IDs matters:

  • Helps distinguish speaker
  • Important when there are multiple speakers

The Bottom Line on Quality

CaptionSync demonstrates quick turnarounds and low-cost rates. However, based on our findings, CaptionSync does not demonstrate – nor can they guarantee – 99% accuracy.

On average, CaptionSync produces 93-95% accuracy.

Don’t Let Inaccurate Captions Hurt Your Content

3Play Media guarantees a 99% accuracy rate for every file.

(In fact, we have a measured accuracy rate of 99.6%!)

Get Started with 3Play Media