HTML5 -Creating Chapter menu Continue...

Example 5: creating a chapter menu with image thumbnails, using JSON cues

Instead of using text (eventually with some HTML for styling, multi lines, etc.), it is also possible to use JSON objects as cue values that can be manipulated from JavaScript. JSON means "JavaScript Object Notation". It's an open standard for describing JavaScript object as plain text.

Here is an example of cue on a WebVTT file, that uses JSON instead of plain text. JSON is useful for describing "structured data"', and processing such data from JavaScript is easier than parsing plain text.

1. WEBVTT
3. Wikipedia
4. 00:01:15.200 --> 00:02:18.800
5. {
6.    "title": "State of Wikipedia",
7.    "description": "Jimmy Wales talking ...",
8.    "src": "http://upload.wikimedia.org/...../120px-Wikipedia-logo-v2.svg.png",
9.    "href": "http://en.wikipedia.org/wiki/Wikipedia"
10. }

This JSON object (in bold green) is a JavaScript object encoded as a text string. If we listen to cue events or if we read a WebVTT file like in the previous examples, we can get this text content using  cue.text property. For example:

1. var videoElement = document.querySelector("#myvideo");
2. var textTracks = videoElement.textTracks; // one for each track element
3. var textTrack = textTracks[0]; // corresponds to the first track element
5. var cues = textTrack.cues;
6. var cue = cues[0]; // first cue
8. // cue.text is in JSON format, with JSON.parse we turn it back
9. // to a real JavaScript object
10. var obj = JSON.parse(cue.text); 
12. var title = obj.title; // "State of Wikipedia"
13. var description = obj.description; // Jimmy Wales talking...
14. etc...

This is a powerful way of embedding metadata, especially when used in conjunction with the cue and track events that can be listened to.

FIRST EXAMPLE: MAKE A NICER CHAPTER MENU BY EMBEDDING A RICHER DESCRIPTION OF CHAPTER MARKERS

Earlier we saw an example that could display chapter markers as clickable text on the right of a video.

This example used only standard plain text content for the cues:

1. WEBVTT
2.  
3. chapter-1
4. 00:00:00.000 --> 00:00:26.000
5. Introduction
6.  
7. chapter-2
8. 00:00:28.206 --> 00:01:02.000
9. Watch out!
10.  ...

We used this example to manually capture the images from the video that correspond to each of the seven chapters:

• We clicked on each chapter link on the right, then paused the video,
• then we used a screen capture tool to grab each image that corresponds to the beginning of chapter,
• Finally, we resized the images with Photoshop to approximately 200x400 pixels.

(For advanced users: it's possible to semi-automatize this process using the ffmepg command line tool, see for example this and that).

Here are the images that correspond to each of the seven chapters of the video from the previous example:

      

To associate these images with its chapter description, we will use JSON objects as cue contents:

elephants-dream-chapters-en-JSON.vtt:

1. WEBVTT
2.  
3. chapter-1
4. 00:00:00.000 --> 00:00:26.000
5. {
6. "description": "Introduction",
7. "image": "introduction.jpg"
8. }
9.  
10.  
11. chapter-2
12. 00:00:28.206 --> 00:01:02.000
13. {
14. "description": "Watch out!",
15. "image": "watchOut.jpg"
16. }
17. ...

Before explaining the code, we propose that you try this example at JSBin that uses this new .vtt file:

HTML code:

1. ...
2. <video id="myVideo" preload="metadata" controls crossOrigin="anonymous">
3. <source src="http://...../elephants-dream-medium.mp4"
4.          type="video/mp4">
5. <source src="http://...../elephants-dream-medium.webm"
6.          type="video/webm">
7. <track label="English subtitles"
8.         kind="subtitles"
9.         srclang="en" src="http://...../elephants-dream-subtitles-en.vtt" >
10. <track label="Deutsch subtitles"
11.         kind="subtitles"
12.         srclang="de" src="http://...../elephants-dream-subtitles-de.vtt"default>
13. <track label="English chapters"
14.          kind="chapters"
15.         srclang="en" src="http://...../elephants-dream-chapters-en-JSON.vtt">
16. </video>
17. <h2>Chapter menu</h2>
18. <div id="chapterMenu"></div>
19.  ...

It's the same code we had in the first example, except that this time we used the new WebVTT file that describes the chapters with JSON cues. For the sake of simplicity, we also removed the buttons and all the code for displaying a clickable transcript of the subtitles/captions on the right of the video.

JavaScript code:

1. var video, chapterMenuDiv;
2. var tracks, trackElems, tracksURLs = [];
3.  
4. window.onload = function() {
5.    console.log("init");
6.    // When the page is loaded
7.    video = document.querySelector("#myVideo");
8.    chapterMenuDiv = document.querySelector("#chapterMenu");
10.    // Get the tracks as HTML elements
11.    trackElems = document.querySelectorAll("track");
12.    for(var i = 0; i < trackElems.length; i++) {
13.       var currentTrackElem = trackElems[i];
14.       tracksURLs[i] = currentTrackElem.src;
15.    }
17.    // Get the tracks as JS TextTrack objects
18.    tracks = video.textTracks;
20.    // Build the chapter navigation menu for the given lang and kind
21.    buildChapterMenu('en', 'chapters');
22. };
23.  
24. function buildChapterMenu(lang, kind) {
25.   // Locate the track with language = lang and kind="chapters"
26.   for(var i = 0; i < tracks.length; i++) {
27.      // current track
28.      var track = tracks[i];
29.      var trackAsHtmlElem = trackElems[i];
31.      if((track.language === lang) && (track.kind === kind)) {
32.         // the track must be active, otherwise it will not load
33.         track.mode="showing"; // "hidden" would work too
34.  
35.         if(trackAsHtmlElem.readyState === 2) {
36.            // the track has already been loaded
37.            displayChapterMarkers(track);
38.         } else {
39.            displayChapterMarkersAfterTrackLoaded(trackAsHtmlElem, track);
40.         }
41.      }
42.    }
43. }
44.  
45. function displayChapterMarkers(track) {
46.    var cues = track.cues;
48.    // We must not see the cues on the video
49.    track.mode = "hidden";
51.    // Iterate on cues
52.    for(var i=0, len = cues.length; i < len; i++) {
53.       var cue = cues[i];
54.  
55.       var cueObject = JSON.parse(cue.text);
56.       var description = cueObject.description;
57.       var imageFileName = cueObject.image;
58.       var imageURL = "http://mainline.i3s.unice.fr/mooc/" + imageFileName;
60.       // Build the marker. It's a figure with an img and a figcaption inside.
61.       // The img has an onclick listener that will make the video jump
62.       // to the start time of the current cue/chapter
63.       var figure = document.createElement('figure');
64.       figure.classList.add("img");
66.       figure.innerHTML = "<img onclick='jumpTo("
67.                          + cue.startTime + ");' class='thumb' src='"
68.                          + imageURL + "'><figcaption class='desc'>"
69.                          + description + "</figcaption></figure>";
70.       // Add the figure to the chapterMenuDiv
71.       chapterMenuDiv.insertBefore(figure, null);
72.     }
73. }
74.  
75. function displayChapterMarkersAfterTrackLoaded(trackElem, track) {
76.   // Create a listener that will only be called when the track has
77.   // been loaded
78.   trackElem.addEventListener('load', function(e) {
79.      console.log("chapter track loaded");
80.      displayChapterMarkers(track);
81.   });
82. }
83.  
84. function jumpTo(time) {
85.   video.currentTime = time;
86.   video.play();
87. }
88.  

Explanations:

• Lines  4-18: when the page is loaded, we get all track HTML elements and their corresponding TextTrack objects.
• Line 19: after that we can build the chapter navigation menu. All is done in the window.load callback, so we do nothing before the DOM is ready.
• Lines 24-43: the buildChapterMenu function first locates the chapter track for the given language, then checks if this track has been loaded by the browser. Once it has been confirmed that the track is loaded, the function displayChapters is called. 
• Lines 45-65: the displayChapters(track) function will iterate on all cues of the chapter track passed as parameter. For each cue, its JSON content is turned back into a JavaScript object (line 52) and the image filename and description of the chapter/cue is obtained (lines 53-54). Then an HTML description for a chapter is built and added to the div element with id=chapterMenu. Here is the HTML code for one menu marker:
  
  
  1. <figure class="img">
  2.    <img onclick="jumpTo(0);" class="thumb"src="http://...../introduction.jpg">
  3.    <figcaption class="desc">
  4.       Introduction
  5.    </figcaption>
  6. </figure>

Notice that we added a click listener to each thumbnail image. Clicking a chapter thumbnail will make the video jump to the chapter time location (the example above is for the first chapter with start time = 0).

We also added CSS classes "img", "thumb" and "desc", which make it easy to style and position the different thumbnails using CSS.

CSS source code extract:

1. #chapterMenuSection {
2.    background-color: lightgrey;
3.    border-radius:10px;
4.    padding: 20px;
5.    border:1px solid;
6.    display:inline-block;
7.    margin:0px 30px 30px 30px;
8.    width:90%;
9. }
10.  
11. figure.img {
12.   margin: 2px;
13.   float: left;
14. }
15.  
16. figcaption.desc {
17.   text-align: center;
18.   font-weight: normal;
19.   margin: 2px;
20. }
21.  
22. .thumb {
23.   height: 75px;
24.   border: 1px solid #000;
25.   margin: 10px 5px 0 0;
26.   box-shadow: 5px 5px 5px grey;
27.   transition: all 0.5s;
28. }
29.  
30. .thumb:hover {
31.   box-shadow: 5px 5px 5px black;
32. }

A menu marker is shown below (it's also animated - move the mouse cursor over a thumbnail to see its shadow animating):

Introduction

SECOND EXAMPLE WITH THE CLICKABLE TRANSCRIPT AND THE CHAPTER MENU

This example is the same as the previous one except that we have kept the features that we saw previously: the buttons for displaying a clickable transcript. The code is longer; it's just a combination of the "clickable transcript" example from a previous lesson, and the one from this lesson.

Try it at JSBin