SMIL and Interactivity

John I. Gelder

Department of Chemistry

Oklahoma State University

Streaming Media Seminar I

Thursday, March 15, 2001

In the time I have available I hope to introduce you to how to synchronize two media streams together and add buttons for interactivity all using SMIL. I have given a few talks about this topic before. The first talk was at a national meeting of the American Chemical Society on August, 1999. The other was a presentation was last summer in July, 2000. The July presentation also had a series of slides. These talks may be interesting as you are learning more about SMIL and what it can do. The talk last summer has some relationship to what I'm talking about today. The one in 1999 is a little further out in terms of what I was discussing and trying to do and will not be a topic of todays discussion, although it is a direction I'm preparing to re-visit now that software has caught up with the idea I had then.

Today I'm going to introduce you to SMIL (Sychronized Multimedia Integration Language). This language is a markup language like HTML. Most important a SMIL file is just a text file with the '.smi' extension.

Such a file is created with a text editor.

A SMIL file contains a series of tags just as an HTML file is composed of tags.

The basic SMIL file contains the following tags

 

<smil>

<head>

<layout>

</layout>

</head>

<body>

<seq>

<par>

</par>

</seq>

</body>

</smil>

 

So how do we build up this basic skeleton of a SMIL file so that it does something for us? As we will learn in the next few minutes the <layout> </layout> tag contains information about where different media elements appear within the RealPlayer window. The <body> </body> tag contains the different media streams that appear in the RealPlayer window. There are some specific rules that must be followed within both the <layout> and <body> tag that we will discuss. The important thing to remember is that it will take a some time to generate your own SMIL file. You will frequently get frustrated as you learn that what I will show you does have a few details that must be worked out for your specific application. I know enough to produce a working SMIL file of the type I require for my vision. I'm learning more everyday.

To get started we need a streaming video file. That is up to you to get. So how do you get a video file? Capture some video using a video camera, either analog or digital. Capture the video to a computer hard drive using a video capture card (for analog video) or FireWire (digital video). An application such as Adobe Premier (assistance is availalble in the Faculty Resource Center in 402 CLB) will do fine to transfer a video from a camera videotape to harddrive. (There are some important details about the hardware to perform video capture, such as RAM, the speed of the harddrive and the video capture card. Bill or I can answer those questions if you have any.) Capturing a video from an analogy device such as a Super8 camera using Premier requires the captured file have an Ô.aviÕ extension and be capture uncompressed. After the file is captured some editing may be required. Once the edited, uncompressed avi file is available, RealProducer is used to compress the file to a '.rm' file.

Alternatively if the video does not require any editing and you have limited harddrive space you can capture directly from the camera, through the video capture card, to RealProducer to get the '.rm' file.

Here is a sample rm file that will open in RealPlayer as a standalone '.rm' file. I can open it using RealPlayer. Sorry we are going to have to use a Chemistry Example. We will now use this '.rm' file in our SMIL file (shown above) to see what happens.

When I add the '.rm' file it is placed between the parallel tag in the body portion of the file.

 

<smil>

<head>

<layout>

</layout>

</head>

<body>

<seq>

<par>

<ref src="P1episode.rm"/>

</par>

</seq>

</body>

</smil>

 

Let's look at this file and see what happens.

It looks exactly like the original '.rm' file opened in RealPlayer.

But suppose I wanted to place this video into an image to make it a little more attractive. I could create an image file with a Ô.gifÕ extension using a software program like FireWorks by MacroMedia. Then place this '.gif' image into my SMIL file as shown below.

 

 

<smil>

<head>

<layout>

</layout>

</head>

<body>

<seq>

<par>

<ref src="P1episode.rm"/>

<ref src="Background1.gif"/>

</par>

</seq>

</body>

</smil>

<smil>

<head>

<layout>

</layout>

</head>

<body>

<seq>

<ref src="Background1.gif"/>

<ref src="P1episode.rm"/>

</seq>

</body>

</smil>

 

Let's look at what the SMIL file on the left looks like. Notice that this time since the two media files are placed within the <par> </par> tags they are played in parallel. If I'd removed the <par> </par> tags and placed the two media files between the <seq> </seq> tags the result would have been much different. Here is the SMIL file shown on the right in the table above. Notice how quickly the RealPlayer window changed size as it moved sequencially through the '.gif' file and then to the '.rm' file. We only saw the Background image file for a moment and the '.rm' file loaded and played.

So if we want to place the '.rm' file over the Background '.gif' file and locate the streaming video at a particular point in the Background image we need to provide some additional information in the SMIL file. This information is entered between the <layout> </layout> tag.

Within the <layout> </layout> tag we must define the size of the RealPlayer window and than two regions, the region the Background.gif image will occupy and the region the streaming video will occupy. In the <region/> element we must identify the name of the region so we can refer to that named region later within the <body> </body> tag and the dimension and top, left location of the media within the RealPlayer window. Below I've added the <region/> element in our SMIL file.

 

 

<smil>

<head>

<layout>

<root-layout background-color="black" width="720" height="480"/>

<region id="backregion" top="0" left="0" width="720" height="480" z-index="0"/>

<region id="videoregion" top="65" left="370" width="320" height="240" z-index="1"/>

</layout>

</head>

<body>

<seq>

<par>

<ref src="P1episode.rm" region="videoregion"/>

<ref src="Background1.gif" region="backregion" fill="freeze"/>

</par>

</seq>

</body>

</smil>

 

Notice within the <region/> element I've given the region an id (name), the top and left coordinate (relative to the root layout window) and a z-index value (a relative layer for each media element).

Let's look at what the SMIL file looks like now.

You'll notice I have another region in my Background.gif image below the streaming video region. This region is there to display some text information so as the user views the streaming video particular information appears in this area of the windo. I have previously prepared a series of slides containing text information. Using RealPresenter I placed the slides into a time line that is synchronized with the streaming video. In the SMIL listing below I've added another region element definition within the <layout> </layout> tag and a new ref src element within the <body> </body> tag.

 

 

<smil>

<head>

<layout>

<root-layout background-color="black" width="720" height="480"/>

<region id="backregion" top="0" left="0" width="720" height="480" z-index="0"/>

<region id="videoregion" top="65" left="370" width="320" height="240" z-index="1"/>

<region id="textregion" top="355" left="267" width="442" height="106" z-index="1"/>

</layout>

</head>

<body>

<seq>

<par>

<ref src="P1episode.rm" region="videoregion"/>

<ref src="slides.rp" region="textregion" fill="freeze"/>

<ref src="Background1.gif" region="backregion" fill="freeze"/>

</par>

</seq>

</body>

</smil>

 

Let's look at what the SMIL file looks like now.

Now that we have the video and the slides playing in parallel over the Background.gif I would like to add some interactivity in the form of buttons. These buttons will jump to a particular point in the video and play. This way it is not necessary to guess where the important points in the video are located. The buttons have two aspects. First I'll create a place for the button on the Background.gif, then I'll create a hotspot rectangle in the SMIL file so when the user clicks on the hotspot is jumps the video to a particular point in the timeline.

The new Background.gif looks like this and the new SMIL file is shown below;

 

 

<smil>

<head>

<layout>

<root-layout background-color="black" width="720" height="480"/>

<region id="backregion" top="0" left="0" width="720" height="480" z-index="0"/>

<region id="videoregion" top="65" left="370" width="320" height="240" z-index="1"/>

<region id="textregion" top="355" left="267" width="442" height="106" z-index="1"/>

</layout>

</head>

<body>

<seq>

<par>

<ref src="P1episode.rm" region="videoregion"/>

<ref src="slides.rp" region="textregion" fill="freeze"/>

<ref src="Background1.gif" region="backregion" fill="freeze"/>

<anchor href="command:seek(3:00.0)" target="_player" coords="20,93,227,117"/>

<anchor href="command:seek(4:43.0)" target="_player" coords="20,123,247,147"/>
<anchor href="command:seek(6:29.0)" target="_player" coords="32,153,103,177"/>
<anchor href="command:seek(8:39.0)" target="_player" coords="112,153,183,177"/>
<anchor href="command:seek(10:19.0)" target="_player" coords="192,153,263,177"/>
<anchor href="command:seek(12:39.0)" target="_player" coords="272,153,343,177"/>
<anchor href="command:seek(14:55.0)" target="_player" coords="32,183,103,207"/>
<anchor href="command:seek(16:01.0)" target="_player" coords="112,183,183,207"/>
<anchor href="command:seek(17:57.0)" target="_player" coords="192,183,263,207"/>
<anchor href="command:seek(19:07.0)" target="_player" coords="272,183,343,207"/>
<anchor href="command:seek(19:32.0)" target="_player" coords="47,213,118,226"/>
<anchor href="command:seek(11:59.0)" target="_player" coords="128,213,199,226"/>
<anchor href="command:seek(17:40.0)" target="_player" coords="208,213,279,226"/>
<anchor href="command:seek(14:32.0)" target="_player" coords="288,213,359,226"/>
<anchor href="command:seek(20:00.0)" target="_player" coords="20,234,247,258"/>
<anchor href="command:seek(22:21.0)" target="_player" coords="32,263,103,287"/>
<anchor href="command:seek(24:31.0)" target="_player" coords="112,263,183,287"/>
<anchor href="command:seek(25:56.0)" target="_player" coords="192,263,263,287"/>
<anchor href="command:seek(27:33.0)" target="_player" coords="272,263,343,287"/>
<anchor href="command:seek(28:33.0)" target="_player" coords="32,293,103,317"/>
<anchor href="command:seek(28:59.0)" target="_player" coords="112,293,183,317"/>
<anchor href="command:seek(30:29.0)" target="_player" coords="192,293,263,317"/>
<anchor href="command:seek(30:41.0)" target="_player" coords="47,323,118,336"/>

</ref>

</par>

</seq>

</body>

</smil>

 

Here is the new SMIL file. The <anchor href> element contains a 'command:seek(mm:s)' which identifies the point in the video in minutes and second that the button is linked too. The 'coords=' provides the rectangle that defines the hotspot in coordinates relative to the background.gif image.

So now we have a relatively sophisticated SMIL file with interactivity.