The HTML5 <track> element, combined with the Text Track API and the <video> element provides a number of interesting opportunities to build rich metadata-driven media Web applications. Further, WebVTT, a simple line-based file format for timed text that is currently in development on the WHATWG is an excellent candidate to be used as a so-called HTML5 text track.
More information can also be found on the blog posts of Silvia Pfeiffer (http://blog.gingertech.net/2011/05/01/html5-multi-track-audio-or-video/ and http://blog.gingertech.net/2011/03/29/webvtt-explained/).
On top of these interesting developments, we have combined WebVTT, the HTML5 <track> element, and the Text Track API with Media Fragment URIs and RDF-based metadata. More specifically, timed metadata identified by Media Fragment URIs and described in RDF format is embedded in a WebVTT file.
An example of timed metadata identified by a Media Fragment URI and described in RDF format (Turtle) is shown below:
<http://foo.com/media.webm#t=4,19&xywh=pixel:223,0,177,167> foaf:depicts <http://dbpedia.org/resource/Barack_Obama>.
The subject in the above RDF triple is a Media Fragment URI, pointing to a rectangular box within the time range [4,19[ seconds. However, this metadata can also be embedded within a WebVTT file:
1 00:00:04,000 --> 00:00:19,000 <http://foo.com/media.webm#t=4,19&xywh=pixel:223,0,177,167> foaf:depicts <http://dbpedia.org/resource/Barack_Obama>.
This way, a Web application developer can rely on so-called cue-events. The latter are events that are generated each time a certain cue (i.e., one line or temporal fragment within the WebVTT file) becomes active. Hence, the programming model for timed metadata-driven Web applications has shifted from pull-based to push-based. Combining this with HTTP adaptive streaming will enable the development of very powerful synchronized formal metadata delivery applications such as the one described in http://www.w3.org/2010/11/web-and-tv/papers/webtv2_submission_36.pdf.
We are currently extending the NinSuna platform so that it can deliver WebVTT text tracks, which on their turn rely on information coming from RDF databases. In the meantime, you can try the small demo at http://ninsuna.elis.ugent.be/html5/syncmeta/webvtt-turtle.html.