Subtitles and closed captions

Add closed captions, subtitles, etc. to your videos

INTRODUCTION

This section introduces the new HTML5 <track> element, useful for adding closed captions, subtitles, descriptions, and metadata to your videos. It comes with a new JavaScript API.

The WebVTT format used for describing a track file is also presented in this chapter.

Most of the major desktop browsers now support HTML5 captioning

• See this compatibility table for the up-to-date status of the different browsers. Also, please check on caniuse.com for information about the <track> element support by browsers.

Some definitions

• closed captions describe all relevant audio present in the video (fire, rain, birds, gun fights, etc.).
• subtitles are only for spoken words.

The accessibility features of TV programs often propose both options for people with hearing deficiencies. 

TYPICAL USE: ADD A SUBTITLE/CAPTION TRACK TO A <VIDEO> ELEMENT

Important warning!!

The <track> element cannot be used with a file:// URL. Please use http:// and a Web server. Your server must use a special MIME format for the .vtt files: text/vtt;charset=utf-8 (set by default on most servers now).

Examples of the lines to add to an Apache Web server:

1. <Files mysubtitle.vtt>
2. ForceType text/vtt;charset=utf-8
3. </Files>

It might be worth mentioning that most browsers work fine with WebVTT even if the MIME Type is not set,but Internet Explorer doesn’t, in fact it will ignore WebVTT entirely unless the MIME Type is set correctly!

Here is an example of a video element that includes a <track> element in the .vtt (WebVTT) format (line 7), original version at http://www.jwplayer.com/html5/track/, and we also cooked a version easier to study on jsbin: http://jsbin.com/luyazi/2/edit.

As seen in this screenshot, the example uses a <track> element to insert basic captions to the video: sounds and music are described, in addition to standard subtitles that correspond to what the different movie characters say.

1. <video height="272" width="640"
2.        poster="http://content.bitsontherun.com/thumbs/q1fx20VZ-640.jpg"
3.        crossorigin="anonymous"
4.        controls>
5.    <source src="http://demo.jwplayer.com/html5-report/sintel.mp4"
6.            type="video/mp4">
7.    <source src="http://demo.jwplayer.com/html5-report/sintel.webm"
8.            type="video/webm">
9.    <track src="http://demo.jwplayer.com/html5-report/sintel-captions.vtt"
10.           kind="captions" label="Closed Captions" default>
11. </video>

Notice that the <track> element at line 9 has an attribute named kind that indicates the type of the track that is included. Possible values are: subtitles, captions, descriptions, chapters or metadata.

 The <track> element also has an attribute default that indicates that we want this track to be displayed by default when reading the video.

We also used  an attribute named crossorigin that is necessary just to run this demo, as the server that hosts the video from this example requires it.

MULTIPLE TRACKS MAY BE INCLUDED IN A VIDEO ELEMENT

Multiple tracks are needed to support different langages, video captions for the hearing-impaired, subtitles, etc.

Find below an example (from the specification) that includes multiple <track> elements (subtitles for three languages and captions only for English):

1. <video src="brave.webm">
2.    <track kind=subtitles src=brave.en.vtt
3.           srclang=en
4.           label="English">
5.    <track kind=captions src=brave.en.hoh.vtt
6.           srclang=en
7.           label="English for the Hard of Hearing">
8.    <track kind=subtitles src=brave.fr.vtt
9.           srclang=fr
10.           lang=fr 
11.           label="Français">
12.    <track kind=subtitles src=brave.de.vtt
13.           srclang=de
14.           lang=de
15.           label="Deutsch">
16. </video>

Notice the use of some new attributes in the <track> element:

• label: the label value will be displayed in the GUI control that is included in the default HTML5 video player,
• srclang:  gives the language for the text track data. The value must be a valid BCP 47 language tag. This attribute must be present if the element's kind attribute is in the subtitles state.

• The WebVTT format

  The  W3C Web Media Text Tracks Community Group is working on the "WebVTT: The Web Video Text Tracks Format" specification, a format used for defining files that will contain text for captions and subtitles, and much more... The WebVTT files are used with the src attribute of the <track> element, that can be used inside a<video>...</video>.
  In the example presented in the previous section (located at http://jsbin.com/luyazi/2/edit), we used a file calledsintel-captions.vtt:
  1. <video height="272" width="640"
  2.        poster="http://content.bitsontherun.com/thumbs/q1fx20VZ-640.jpg"
  3.        crossorigin="anonymous"
  4.        controls>
  5.    ...
  6.    <track src="http://demo.jwplayer.com/html5-report/sintel-captions.vtt"
  7.           kind="captions" label="Closed Captions" default>
  8. </video>
  And here is an extract of the corresponding sintel-captions.vtt file:
  1. WEBVTT
  3. 00:00:01.000 --> 00:00:02.042
  4. (drumbeat)
  6. 00:00:07.167 --> 00:00:12.025
  7. (plaintive violin solo playing)
  9. 00:00:15.000 --> 00:00:18.183
  10. (wind whistling)
  12. 00:00:24.167 --> 00:00:27.025
  13. (orchestra music swells)
  15. 00:00:43.033 --> 00:00:43.192
  16. (weapons clash)
  18. 00:00:44.000 --> 00:00:44.175
  19. (gasps)
  21. 00:00:44.183 --> 00:00:45.158
  22. (grunts)
  24. 00:00:45.167 --> 00:00:47.058
  25. (groaning)
  27. 00:00:54.192 --> 00:00:55.150
  28. (blade rings)
  30. 00:00:55.158 --> 00:00:57.008
  31. (bellowing)
  33. 00:00:57.017 --> 00:00:58.067
  34. (grunting)
  36. 00:00:59.075 --> 00:01:00.133
  37. (panting)
  39. 00:01:05.108 --> 00:01:06.125
  40. (cries out in agony)
  42. 00:01:08.050 --> 00:01:09.058
  43. (panting)
  45. 00:01:12.092 --> 00:01:13.142
  46. (panting)
  48. 00:01:14.017 --> 00:01:18.125
  49. (orchestra plays ominous low notes)
  51. 00:01:31.058 --> 00:01:35.133
  52. (plaintive violin solo returns)
  54. 00:01:46.158 --> 00:01:49.058
  55. This blade has a dark past.
  57. 00:01:51.092 --> 00:01:54.108
  58. It has shed much innocent blood.
  60. 00:01:57.083 --> 00:02:00.000
  61. You're a fool for traveling alone
  62. so completely unprepared.
  64. 00:02:01.100 --> 00:02:03.033
  65. You're lucky your blood's still flowing.
  67. 00:02:04.183 --> 00:02:06.075
  68. Thank you.
  This format is rather simple, but we still recommend reading this excellent article from Mozilla Developer Network that explains in details all the different options.
  Each "element" in this file has a starting and ending time, plus a value (the text that will be displayed), followed by a blank line (blank lines are separators between elements).
  Each element is called "a cue", and may optionally have an ID that will be useful when using the track element JavaScript API, in particular the getCueById() method of TextTrack objects. How to use these will be taught in the "advanced HTML5" course, soon on W3Cx
  Example of numeric IDs:
  1. 9
  2. 00:00:21.000 --> 00:00:22.000
  3. to hear from <u>you</u>
  5. 10
  6. 00:00:22.500 --> 00:00:25.000
  7. We want to hear what inspires you as a developer
  IDs may be also defined as strings, and values can use HTML as well:
  1. Opening
  2. 00:00:00.000 --> 00:00:30.000
  3. Welcome to our <i>nice film</i>
  The displayed text can span over multiple lines, but blank lines are not allowed, as they would be interpreted as a separator:
  1. 00:01:57.083 --> 00:02:00.000
  2. <p>You're a fool for traveling alone</p>
  3. <p>so completely unprepared.</p>

Practical example: adding subtitles to a video

INTRODUCTION

We will look at a simple example. First, you need to have a video on one of the formats/codecs supported by the browsers you target. A recommended codec is H264, but other formats, such as webm, may have some advantages if the browser supports them. For example, webm allows to start playing the video after a much shorter buffering time. In other words, try to provide the video encoded with more than one codec, if possible.

For this, do use any sort of open source, free or commercial video encoding software, such as Handbrake (free, open source) or Super (free). There are also online video encoding services, and you can even upload your video to YouTube, let it encode your video in several resolutions and codecs, and use a browser extension like VideoDownloadHelper (for Firefox) to download the video in the formats you like.

So, let's suppose you have a video like this one below (we included it on YouTube for practical reasons). This video does have subtitles (you can activate them in the YouTube player), but the goal of this lesson is to explain how we made them without using the YouTube embedded tools, that do not allow to export the subtitle file in the webVTT format.

And you've got it also in mp4/H264 and in webm formats. Here is how you can embed it in your page using the video element:

1. <video id="myVideo" width=500 controls>
2.   <source  
3.       src="videos/SameOldSongAndDanceRogerLathaudPart1MidRes.mp4"
4.       type="video/mp4">
5.   <source   
6.       src="videos/SameOldSongAndDanceRogerLathaudPart1MidRes.webm"
7.       type="video/webm">
8.  
9.   <track src="videos/subtitles.vtt"   
10.          kind="subtitles" srclang="en" default>
11. </video>

At line 9, we added a <track> element to add english subtitles, as the guitar teacher there is speaking in French. We will now explain how we created this subtitle track.

ADDING SUBTITLES TO THE VIDEO

Now, we need to create a WebVTT file for this video. How can we synchronize an English translation of what the guitar teacher says in French?

There are many tools available to add subtitles to a video, free and commercial ones. Most of them are native applications you need to install on your computer. However, it exists a free and very practical tool for doing this 100% in a Web browser: http://www.universalsubtitles.org, also known as http://www.amara.org.

Go to this Web site, click on the "subtitle a video" link, then follow the different tutorials/instructions. It will ask for a YouTube URL, so it's better to first upload your video to YouTube (even in private mode). Once you entered in the URL of your video, you will have an online subtitles/caption editor. Enter your subtitles and sync them until you are happy with the results.

Once your subtitles/captions are ok, you will be able to upload them to YouTube, or -this is what we wanted first- download them as WebVTT format:

TRY YOUR SUBTITLED/CAPTIONED VIDEO

Styling and positioning the captions/subtitles

ONLINE EXAMPLE USED IN THIS SECTION

In this section, we will look at different possibilities for positioning and styling the text displayed as captions/subtitles while playing a video.

This video example shows each use case. We also "cooked" an easier version to study on JS Bin (look at the HTML in JS Bin).

The screenshots below are taken from this video.

And here is the WebVTT file. Notice the new attributes that have been added on the right end of the duration values:

1. WEBVTT
3. 00:00:01.000 --> 00:00:05.000
4. These captions test some features of the WebVTT formats
6. 00:00:06.000 --> 00:00:10.000 line:5%
7. This cue is positioned at the top of the video
9. 00:00:11.000 --> 00:00:15.000 position:5% align:start
10. This cue is positioned at the left side of the video.
12. 00:00:16.000 --> 00:00:20.000 position:95% align:end
13. And this one ate the right side.
15. 00:00:21.000 --> 00:00:25.000 size:33%
16. This cue is only a third of the width of the video, hence the multiple line breaks.
18. 00:00:26.000 --> 00:00:30.000
19. This cue contains <b>bold</b> text.
21. 00:00:31.000 --> 00:00:35.000
22. This cue contains <i>italic</i> text.
24. 00:00:36.000 --> 00:00:40.000
25. This cue contains <u>underlined</u> text.
27. 00:00:41.000 --> 00:00:45.000
28. This cue contains <b><i><u>bold, italic, underlined</u></i></b> text.
30. 00:00:46.000 --> 00:00:50.000
31. <c.myclass>This cue contains the class "myclass".
32. Browsers that support ::cue CSS should make it red.</c>
34. 00:00:51.000 --> 00:00:55.000
35. The following cue contains two voices.
36. Tarzan should be blue and Jane green.
38. 00:00:56.000 --> 00:01:00.000
39. <v Tarzan>Me Tarzan...
40. <v Jane>That would make me Jane!
42. bigtext
43. 00:01:01.000 --> 00:01:05.000
44. This cue has a unique id.
45. Using CSS, its font size should be 150%.
47. 00:01:06.000 --> 00:01:10.000
48. The <00:01:06.333>text <00:01:06.666>in <00:01:07.000>this <00:01:07.333>cue<00:01:07.666>should <00:01:08.000>grow
49. <00:01:08.333>one <00:01:08.666>word <00:01:09.000>at <00:01:09.333>a<00:01:09.666>time
51. 00:01:11.000 --> 00:01:15.000
52. That's it! For now...

HOW TO POSITION THE SUBTITLES

The video example tests nearly all the possibilities for positioning subtitles/captions, styling (using HTML element wrapping with <b>, <i>,  etc.), voicing (subtitles corresponding to different characters will be displayed in different colors) and CSS styling.

It is possible to locate the cues in the video viewport using absolute or relative values. The attributes that position the text are located on the same line as the cue definition, like at line 9 of the previous WebVTT example file:

1. 00:00:11.000 --> 00:00:15.000 position:5% align:start
2. This cue is positioned at the left side of the video.

There are several possible values:

• line:5% means "vertical position at a line 5% of the height of the video viewport (it will be located at the top of the video, proportional to its vertical size).
• position:5% align:start means "regular location at the bottom of the video, the start of the sentence will be located at 5% of the width of the video", i.e., near the left side.
• position:95% align:end means "regular location at the bottom of the video, the end of the sentence will be at 95% of the horizontal width of the video".
• size:33% The size of each line will be one third of the size of the video. Since the sentence won't fit, it will be displayed in multiple lines.

And so on. Please look at the video as it is self-explanatory.

USE OF <B>, <I>, <U> FOR STYLING SUBTITLES / CAPTIONS

USING CSS CLASSES FOR STYLING

It is possible to style using CSS classes as part of a cue value, using the <c> element. You can specify the CSS class that should be applied by adding "." followed by the name of your CSS class. Here is an example:

1. <c.myclass>This cue contains the class "myclass".
2. Browsers that support ::cue CSS should make it red.</c>

CSS rules used in this example:

1. <style type="text/css">
2.      ::cue(.myclass) { color: red; }
3.      ::cue(v[voice="Tarzan"]) { color: blue; }
4.      ::cue(v[voice="Jane"]) { color: green; }
5.      ::cue(#bigtext) { font-size: 150%; }
6. </style>

The ::cue pseudo element selector is used to match "cues" in the webVTT file. You add parenthesis and a secondary CSS selector to match cues that have a particular id, or a particular CSS class, etc. Look at the CSS above and at the extract from the webVTT file, play the video, you will understand how this works...

Support differs from one browser to another, see this compatibility table for more infos. Notice that most of the enhanced players presented further on in the course will provide full support.

USING VOICING FOR STYLING: THE <V> ELEMENT

Screenshot taken at 1:02 of this example online:

Using the <v> tag, you will distinguish different voices that should be displayed in different colors (depending on the HTML5 video player implementation). See the CSS presented in the previous section to see how to specify the colors for the different voices.

Extract from the source code:

1. 00:00:56.000 --> 00:01:04.000
2. <v Tarzan>Me Tarzan...
3. <v Jane>That would make me Jane!

Chapters: use another specific <track>

Ading chapters is very similar to adding subtitles/captions. Look at line 5, where we use an extra <track>element with a kind="chapters" attribute.

1. <video poster="webvtt_talk.png" style="width:100%" preload="metadata">
2.     <source src="webvtt_talk.webm">
3.     <source src="webvtt_talk.mp4">
4.     <source src="webvtt_talk.ogv">
5.     <track id="nav" src="webvtt_talk_navigation.vtt" kind="chapters"srclang="en"></track>
6.     <track id="cc" src="webvtt_talk_captions.vtt" kind="captions"
7.           label="captions" srclang="en" default>
8.     </track>
9. </video>

Here is an example of WebVTT files with defined chapters. Each "CUE" at line 3, 7, 11 etc. can bear any name. We used "Chapter 1, Chapter 2, Ending, etc..." but you are free to give them the name you like best. 

What make them special is the fact that the track has an attribute kind="chapters". In 2015, the <video>elements rendered in standard browsers do ignore chapters, but enhanced HTML5 players such asSublimeVideo Player or JW Player take them into account.

It's also easy to process them, and for example, to generate a custom navigation menu, using the <track>JavaScript API (which will be covered in an "advanced HTML5" course, soon on W3Cx).

Example of a WebVTT file that defines chapters:

1. WEBVTT FILE
2.  
3. Chapter 1
4. 00:00:00.000 --> 00:00:10.700
5. Title Slide
6.  
7. Chapter 2
8. 00:00:10.700 --> 00:00:47.600
9. Introduction by Naomi Black
10.  
11. Chapter 3
12. 00:00:47.600 --> 00:01:50.100
13. Impact of Captions on the Web
14.  
15. Chapter 4
16. 00:01:50.100 --> 00:03:33.000
17. Requirements of a Video text format
18.  
19. Ending
20. 00:03:33.000 --> 00:04:57.766
21. Simple WebVTT file
22.  
23. Greetings 6
24. 00:04:57.766 --> 00:06:16.666
25. Styled WebVTT file

JW Player displays chapters:

SublimeVideo's player takes into account chapters:

Tools for creating WebVTT subtitles and for reusing existing subtitles

Many tools are available to make and edit HTML5 video and caption/subtitles:

• Tools for converting existing file formats to  WebVTT files, such as the .srt format, which is popular in the DivX/AVI/Mkv scene. This article reviews 3 of them, but there are many more. Do a quick Web search and you will get plenty of results.
• Tools for creating subtitles/captions from scratch. Use these to generate a WebVTT file. You can embed the <track> element in your own videos, on your own Web pages, or upload the WebVTT file to YouTube. We recommend Universal Subtitles. It's a free online tool, very easy to use. Check the "add subtitles/captions to your video" unit of this course. 
• Enhanced HTML5 video players that use the <video>, <source> and <track> elements under the hood. They also provide many extra features, including support for subtitle/caption formats other than WebVTT (by converting on the fly).
• JavaScript libraries for converting to the WebVTT format on the fly, such as JS_videosub

The <track> JavaScript API

INTRODUCTION

• The <track> element comes with a powerful API that is used to develop a lot of interesting features such as:
  • Dynamically building a navigation menu that shows the different chapters of the video,
  • Synchronizing page content with timestamps in the WebVTT file (for example: show a map next to the video, that shows the location corresponding to the video content),
  • Displaying all the subtitles/captions at once as HTML in the page,
  • Making an app for creating on the fly subtitles/captions,
  • Etc.
  How to do this will be taught in the "advanced HTML5" course, soon on W3Cx.

EXAMPLES OF USE

Add a navigation menu to start playing the video at given chapters

• This example shows a video with an enhanced progress bar that displays the different chapters as small "clickable" squares. Furthermore, using the JavaScript API of the <track> element, this Web site builds a navigation menu (on the right of the video):
  

Sync video with Google Map and Google Street View

• Check this demo by Sam Dutton: it shows a video that comes with a WebVTT file that contains longitudes and latitudes. When the video plays, JavaScript functions are called at given times and get the longitude and latitude. A Google Map and a Google Street views are updated in real time.
  

SYNC GUITAR TABLATURES AND MUSIC SCORE WITH A VIDEO

• This example shows how we manage to render music scores in real time as the video plays. 
  Some JavaScript code listens to the ontimeupdate event while the video is playing. We use the currentTimeproperty of the video to know exactly where we are in the video. Finally, we also rely on an external library to render in an HTML5 canvas the bars corresponding to the current video explanations. We render in real time guitar pro tablatures using the alphatab.net library.

EXTERNAL RESOURCES (OPTIONAL, IF YOU WISH TO LEARN MORE)

• The W3C specification for the <track> element
• The living W3C standard for the WebVTT format
• Examples of text tracks with JavaScript scripting
• HTML5 video with SRT subtitles
• Creating and validating WebVTT subtitles
• Silvia Pfeiffer's HTML5 Video Accessibility slides give more examples of working with markup, as well as showing how to build chapter tracks for navigation and description tracks for screen readers.