Video Subtitles and Localization

Let’s talk about localization and subtitles – not captions. From Wikipedia:

“Subtitles” assume the viewer can hear but cannot understand the language or accent, or the speech is not entirely clear, so they only transcribe dialogue and some on-screen text. “Captions” aim to describe to the deaf and hard of hearing all significant audio content—spoken dialogue and non-speech information such as the identity of speakers and, occasionally, their manner of speaking – along with any significant music or sound effects using words or symbols.

So far I worked on two projects that involved subtitles – Web We Want and Firefox: Choose Independent – and this is what I learned in the process.

The Process

Step 1: Provide Source Content

You need the video (obviously) and English subtitles with the correct timing (less obvious). This means that the picture might no be final (sometimes they call this picture lock quality), but the audio track and its timing do.

Step 2: How Do I Localize Subtitles?

Currently the most common format for subtitles is SubRip, and it’s really simple: sequential number of the subtitle, timing (start-end), text.

For example this is the beginning of the Web We Want .srt file:

1
00:00:00,000 --> 00:00:00,864
THE WEB WE WANT

2
00:00:00,864 --> 00:00:03,388
THE WEB WE WANT
an open letter

3
00:00:05,503 --> 00:00:07,406
I am not a data point

4
00:00:07,642 --> 00:00:09,503
to be bought and sold.

Amara is a great tool to localize subtitles: you can use an interactive editor/timeline and you can adapt the timing of each sentence to your needs while watching the video. Potentially you can also choose to automatically sync subtitles between Amara and Youtube.

Step 3: Host the Video on Youtube

At this point subtitles are available, someone just needs to load them on YouTube and display the video on a web page.

Sounds simple, doesn’t it? What could go wrong?

Potential Localization Issues

The first issue is timing: if the English text requires 2 seconds to be read, that might not be enough to read the same sentence in Spanish, German or other verbose languages. That’s something you need to keep in mind from day one while producing the video and audio track.

Split sentences: sometimes a sentence is split into multiple parts because of its length. Unfortunately the sentence structure might be completely different in the target language, and some details will get lost in translation. Consider for example the final frame of the Choose Independent video:

Choose Independent frame

«It’s how we keep our independence online…» (pause, switch to burning fox) «…burning bright»

This works nicely in English since “burning” is perfectly synced with the picture, but most locales won’t be able to obtain the same result. Small bits, but still reducing the impact of the message.

In general, watching a subtitled video is a sub-optimal experience. If you plan to have content in video format, don’t focus your communication exclusively on that.

Why Youtube Is not Great

For the Firefox Independent page we force YouTube to load subtitles with cc_load_subtitles.

A few hours after the launch, I started receiving complaints from localizers saying that YouTube was loading subtitles with the wrong localization for some languages: for example nl for fy-NL (Dutch instead of Frisian), de for rm (German instead of Romansh), en-US for cy (English instead of Welsh).

This is how YouTube’s embed works:

  • If you’re logged in into Google, you’ll get the locale you choose for YouTube. For example I get English subtitles, even if I’m using a browser in Italian (a quick test is to watch the video in private mode).
  • If you’re not logged in, you’ll get the first good locale based on the Accept Language header sent by your browser. For example, for ‘dsb’ (Lower Sorbian) this equals to “dsb, hsb, de, en-US, en”, and German is the first available language on the list.

For some unknown reasons YouTube wasn’t founding a match in Accept Language and falling back to a different language for those locales, even if the subtitles were localized and loaded. My only guess would be a mismatch in the locale format understood by YouTube, like fy_NL vs fy-NL/fy.

Then I found out that we can send a parameter called ‘cc_lang_pref’ to force the language (not exactly well documented), and that fixes cy, fy-NL, rm. For example, if you open https://www.mozilla.org/it/firefox/independent you’ll get the Italian subtitles even if you’re using a browser in a different language. Since we have a good locale detection on top of mozilla.org, this makes sense and let us use the most of our localization teams’ work.

But then YouTube becomes not smart: if the requested locale is missing, it doesn’t rely on Accept Language, but just falls back to English. So Welsh (cy) is now getting subtitles in the expected language, Lower Sorbian (dsb) is getting English instead of German and can only manually switch language. Far from great.

Talking about YouTube, I find quite silly a claim as «I support 163 languages” but don’t provide the list of them anywhere. I have at least 5 locales code that are not supported or recognized (ast, dsb, es-AR, es-CL, hsb).

How Do We Fix It?

I think the solution will be to move away from YouTube and its limitations, use native or alternative video players (example), subtitles in VTT format and take full control of the entire chain. But these are discussion and experiments that still need to start.

Comments are closed.