Making MPEG Movies

by Mark Hays

Introduction
Comments on Images
Making the MPEG Movie
Making the HTML
Special Effects
Credits
MPEG links
Manpages

Introduction

This document describes the making of MPEG movies for your home page on the departmental Linux system using the Berkeley MPEG encoder. First of all, what is an MPEG? To find out, click on the image below.

This 43 KB MPEG shows the time evolution of a knot.

If all went well, two external windows titled "MPEG Play" and "MPEG Player Controls" popped up and you saw a movie of a knot. On my screen, it looks something like the following:

picture of the MPEG player control bar

image of the MPEG player window

The MPEG player probably uses a private colormap on your display. This means that the movie looks funny when the mouse cursor is not in the "MPEG Play" window (as in the image above). When the cursor is in the window, the movie will look correct, but the rest of your screen will probably look funny. This is normal so don't panic.

Here is a quick tour of the Berkeley MPEG player controls: the first item is just a box showing you the current frame number and average frame rate. The next four items rewind, pause, single step, and play the movie. The loop button controls whether or not the movie plays once and stops, or loops continuously. Finally, the exit button terminates the player. To run the player by hand (as you develop your movie, for example), type

mpeg_play <filename.mpg>

The rest of this document is concerned with making MPEG movies like this one. In particular, we'll spend quite a bit of time discussing the Berkeley MPEG encoder. There are other movie formats and players out there (QuickTime and AVI, for example), but I will not discuss them here.

To create these sorts of movies, you need to do several steps, in order:

Make the data that is the basis for the animation.
Convert the data into images.
Assemble the images into an MPEG file.
Add HTML code to your home page to publish the MPEG file.

Step 1 is the hardest part. Unfortunately, this document cannot help you with it. In step 2, you have quite a bit of leeway. Virtually any image type can be used to create your movie, though there are a few things to watch out for; these issues are discussed in the next section. Step 3 is more or less mechanical and is described in detail below. In step 4 you also have a large number of options; I'll give some example HTML code in the appropriate section.

Comments on Images

Although the MPEG encoder will accept virtually any type of image as input, there are a few restrictions on what you can do. The biggest restriction is that

All of your images must be the same size

The MPEG encoder will coredump if this is not the case (after printing a warning message).

Another restriction to keep in mind is that

The encoder expects your image dimensions to be a multiple of 16 pixels

This restriction is due to the fact that MPEG movies do motion vector prediction on 16 by 16 pixel chunks of future and past frames. If your images dimensions are not multiples of 16, the encoder will crop your images to the nearest multiple of 16. Some commonly used dimensions are 192, 224, and 256.

Decoding an MPEG movie is CPU intensive. The amount of time it takes to decode a complex frame is roughly a quadratic function of the image size. Generally speaking, most machines can play 192x192 movies with no trouble. If you are animating simple blick and white line drawings (function plots, for example), you can use larger images while maintaining an acceptable frame rate.

It is important to remember that net.surfers must download your entire movie before it starts playing on their local machine. The size of an MPEG file is basically a quadratic function of the image size; therefore, so is its download time. It is a good idea to state the size of your movie in the vicinity of its hyperlink; this way, the people out there have some idea of how long they might have to wait before they get to see anything. In summary,

Keep the pixel dimensions of your movie as small as possible

Like JPEG images, MPEG movies are designed to display "real scenes," as opposed to computer generated graphics. This means that flesh tones and other "natural" colors are given precedence in the encoding. For example, the RGB triple (0,0,255) for bright blue comes out as a very dim blue in an MPEG movie. This is due to the fact that the MPEG standard was intended to mimic television broadcast standards. There is a great deal more information on this in the MPEG FAQ.

The colors in your movie will not be the same as those in your original frames

While you are making your images, it is probably worthwhile to make small MPEGs consisting of a few of your frames so you can see how the colors are transformed. A hint: desaturated colors (as opposed to pure, saturated ones) might come out brighter.

Finally, there are a few file formats to avoid making movies from:

TIFF: There are several versions of the TIFF library (used by the various image conversion utilities) available on the net. Unfortunately, each version seems to have peculiarities and bugs that make it annoying to work with. If you somehow end up with broken TIFF files, your best bet may be to take them to a Macintosh and try to convert them to GIF format there.
PostScript: Although it is possible to rasterize a PostScript file into a regular image format, it is very time consuming to do so. Basically, you can run ghostscript each PostScript image and have it output in GIF or PPM format. See the gs manpage for more details.
JPEG: GIF images achieve data compression in a "lossless" manner. For example, you can convert GIF images to PPM format and back any number of times without loss of image quality. JPEG images, on the other hand, use "lossy" compression. If you convert a JPEG to GIF and back, the JPEG that comes out will, in general, be of lower quality than the one you started with. Since MPEG frames are compressed in a similar manner, you get degraded twice if you use JPEGs: once to decode the JPEG and again when the MPEG encoder reencodes the image data.

There is some information on locally available image manipulation software in the SWIG area off of the SWIG home page.

Making the MPEG Movie

Now that you have all of your images ready (I'll assume that they are in GIF format for concreteness), it is time to assemble them into an MPEG file. This process involves two steps:

Create an input file that describes, among other things, the location of your images and the format that they are in.
Run the encoder to create the movie.

Below, we'll take a tour of a sample input file to get a feel for the first step. Lines starting with "#" in the input file are treated as comments. There are a few other restrictions detailed in the full-length sample. Line numbers in the full-length version are given on the left margin in the following excerpts.

The first section of the input file simply declares the name of the output MPEG file:

    26  #######################################################################
    27  # 1. Name of output file -- should end in .mpg
    28  #
    29  
    30  OUTPUT          movie.mpg

The second section gives the name of the directory in which the image files to assemble reside (in this case they live in a subdirectory named "gifs" under the directory where the input file lives):

    32  #######################################################################
    33  # 2. directory to get all input files from
    34  #
    35  # This should precede the INPUT section (ie, don't move this section)
    36  # Use "." for the current directory.
    37  #
    38  
    39  INPUT_DIR       gifs

In the third section, you list the names of the files to be assembled. This specification is delimited by the keyword "INPUT" and continues through the keyword "END_INPUT." Both of these keywords must be by themselves on a line.

    41  #######################################################################
    42  # 3. specify input filenames
    43  #
    44  # There are a bunch of ways to specify the input files.
    45  # from a simple one-per-line listing, to the following 
    46  # way of numbering them.  See the manual for more information.
    47  #
    48  # '*' is replaced by the numbers 01, 02, 03, 04
    49  # if I instead do [01-11], it would be 01, 02, ..., 09, 10, 11
    50  # if I instead do [1-11], it would be 1, 2, 3, ..., 9, 10, 11
    51  # if I instead do [1-11+3], it would be 1, 4, 7, 10
    52  # the program assumes none of your input files has a name ending in ']'
    53  # if you do, too bad!!!
    54  #
    55  # For example, the line
    56  #       image*.gif [100-135]
    57  # would expand to: image100.gif image101.gif ... image135.gif
    58  #
    59  # You can also do things like
    60  #       `cat list`  # "list" is a file containing a list of filenames
    61  # and
    62  #       `ls *.gif`  # ls returns a list of matching files in INPUT_DIR
    63  #
    64  # You can also go low-tech and have one filename per line
    65  #
    66  
    67  INPUT
    68  `ls *.gif`
    69  END_INPUT

There are a number of ways to specify the input images. First, you can simply list all of the files, one per line, in this section. There are shortcuts, however. If your image filenames are numbered, you can do things like
image*.gif [100-755]
The encoder automatically expands this to
image100.gif image101.gif... image755.gif
You can also use shell commands to generate the file list. These commands are delimited in backquotes. For example,
`ls *.gif`
expands to all the files ending in ".gif" in INPUT_DIR.

The MPEG encoder really only understands a few image formats -- PPM, and PNM being the most useful. The encoder does not directly understand the GIF format, for example. How, then, do we coerce the encoder into reading our GIF images? The answer is that we convert all of our images into one of the formats that the encoder understands. The fourth section of the input file tells the encoder which of the (two or three) formats we are converting to:

    71  #######################################################################
    72  # 4. Converted file format
    73  #
    74  # mpeg_encode really only accepts 3 different file formats, but using a
    75  # conversion statement it can effectively handle ANY file format
    76  #
    77  # You must specify the type of the input files.  The choices are:
    78  #    YUV, PPM, JMOVIE, Y, JPEG, PNM
    79  #       (must be upper case)
    80  #
    81  # Generally, you'll use PPM or PNM. Then you use the INPUT_CONVERT
    82  # statement to convert all of your frames to BASE_FILE_FORMAT
    83  #
    84  
    85  BASE_FILE_FORMAT        PNM

Usually, you can just stick with PNM, but sometimes, you might want to use PPM or JPEG. Note that the PNM format includes all of PBM, PGM, and PPM.

The next section tells the encoder how to convert your images into the BASE_FILE_FORMAT you chose above:

    87  #######################################################################
    88  # 5. Specify how to convert input files to BASE_FILE_FORMAT
    89  #
    90  # Each occurrence of '*' will be replaced by the input file
    91  #
    92  # e.g., if you have a bunch of GIF files, then this might be:
    93  #       INPUT_CONVERT   giftopnm *
    94  #
    95  # e.g., if you have a bunch of files like a.Y a.U a.V, etc., then:
    96  #       INPUT_CONVERT   cat *.Y *.U *.V
    97  #
    98  # e.g., if you are grabbing from laser disc might have something like
    99  #       INPUT_CONVERT   goto frame *; grabppm
   100  #
   101  # 'INPUT_CONVERT *' means the files are already in the base file format
   102  #
   103  # Make sure /usr/local/netpbm is in your path!
   104  #
   105  
   106  INPUT_CONVERT   giftopnm *

If you use Net-PBM to do the conversion, make sure that /usr/local/netpbm is in your PATH variable; otherwise, the encoder won't be able to find the conversion programs. As an alternative, you can give the full pathname to the conversion program you'd like to use.

This is one place where you definitely do not want to use ImageMagick's "convert" program. It is much larger and slower than the myriad of tiny Net-PBM programs.

The last section of the input file you might want to look at decribes to the encoder the desired quality of the output movie versus its file size. You might want to skip this part on first reading and just make some movies. If you aren't happy with the quality you get, you can come back and experiment with these three parameters.

   108  #######################################################################
   109  # 6. Image quality/compression parameters
   110  #
   111  # these specify the q-scale for I, P, and B frames
   112  # (values must be between 1 and 31)
   113  # These are the Qscale values for the entire frame in variable bit-rate
   114  # mode, and starting points (but not important) for constant bit rate
   115  #
   116  #  1 is best quality/lowest compression
   117  # 31 is best compression/lowest quality
   118  #
   119  # 1-1-1   gives 300K MPEG for 250 200x200 frames with good quality
   120  #         took 2 min to encode on ame2
   121  # 8-10-25 gives  45K MPEG for 250 200x200 frames with lower quality
   122  #         took 1 min 40 sec to encode on ame2
   123  #
   124  # 8-10-25 is a good starting point
   125  #
   126  
   127  IQSCALE         8
   128  PQSCALE         10
   129  BQSCALE         25

These three numbers control the quality factors for the movie's I, P, and B frames. A value of 1 gives the highest quality and lowest compression; a value of 31 gives the lowest quality and best compression. See the User's Guide and the MPEG-1 FAQ for more information.

There are other arcane parameters later in the file; normally you won't need to worry about them. The PostScript version of the User's Guide listed in the manpage section below gives all the details.

Having created a suitable input file, we can now make our movie. This part is simple: just type

mpeg_encode mpeg.in
Replace "mpeg.in" with whatever you named your input file. The encoder should print out reams of diagnostic information, including the number of the frame it is working on, as well as a running estimate of how much time it will take to complete the encoding process. At the end of the run, it displays statistics on I, P, and B frames, including compression ratios for each.

By the way, the issuance of error messages isn't a particular strong point of the encoder; however, this encoder is much easier to use and more flexible than the one that preceded it.

Making the HTML

This section describes the necessary HTML support code for making an MPEG movie available from your home page. A typical link to an MPEG looks like this:

This 43 KB MPEG shows the time evolution of a knot.

There is a "live" image that you can click on to download and view the movie. The image above is actually one of the frames of the movie itself. Next to the image is a description of the animation with a duplicate link to the movie. The size of the movie is stated in the caption. The HTML code to do this is simple:

<a href="IMG/mpg1.mpg">
  <img src="IMG/mpg1.gif" align=left hspace=10>
</a>

<br><br><br>
This <a href="IMG/mpg1.mpg">43 KB MPEG</a>
shows the time evolution of a knot.
<br><br><br><br><br><br>

The first href creates a link to the MPEG movie file (stored in IMG/mpg1.mpg). Href's can make anything "live", including images. In this case, the href wraps an image stored in IMG/mpg1.gif. The "align=left" tells the browser to put the image on the left side of the page. The "hspace=10" puts a 10 pixel horizontal border around the image -- this has the effect of shifting the text a little to the right. Without the hspace, the caption ends up a little too close to the image. Finally, the </a> ends the href.

Next come a bunch of <br>'s. If you take them out, the caption will appear at the top of image. If you want the text centered, you'll need to put in several of these forced line breaks. The number of <br>'s required depends on the size of the image and the font used for the text (which is not really under your control, unfortunately). The text contains an href to the same movie file. Finally, there are several more <br>'s. These eat up the remaining space in the image so that the next paragraph of text starts below the image.

Special Effects

The following movie illustrates some special effects.

This 348 KB 192x192 MPEG shows the time evolution of a knot (and James T. Kirk going where no one has gone before). A 492 KB 256x256 version is also available.

This movie started with a still image of the background, the UFP logo, and the Enterprise (I found these out on the www). The knot sequence is as before. Effects used include:

Overlaying images: All of the images are overlaid onto the background image.
Image fades: The UFP logo when it first appears in blick.
Color fades: The UFP logo when it changes from dark to illuminated.
Scaling: The enterprise is rescaled on most every frame.
Plotting: The phasers are actually a line plot.
Rotating colormaps: The moving-colors effect on the "SWIG" logo.

The composite images were created with about 700 lines of IDL code. I think that IDL can do morphing as well, but I haven't had time to play with it yet. You can grab these (undocumented) IDL scripts from here. If you are going to use this code, there are several restrictions on your images (these apply to making video tapes of your animations as well):

Your images must be 8 bit and color mapped (GIFs, for instance)
The images in each sequence must have the exact same colormap, in the same order.
Collectively, the images must use no more than 256 colors.

If you start with 24 bit images (SGI RGBs or JPEGs), it is very time consuming to satisfy these criteria. The ImageMagick tools "convert" and "mogrify" can be used to perform these operations. If you do not need special effects, 24 bit images are fine; in fact, the MPEG encoder internally works on 24 bit data. It is possible to use 24 bit images and achieve the same effects; however, it is a much more CPU intensive process. $picture of the math department building$

This 99 KB 160x120 MPEG depicts daily life in the vicinity of Lowell and Santa Rita.

This 326 KB 320x240 MPEG is a larger version of the previous one.

Credits

The data for the animations on this page was provided by Dr. Isaac Klapper. The animation code was written by Anu Rao in C using the GL graphics library on an SGI.

The sample input file for mpeg_encode is a slightly modified version of the one that comes with the Berkeley mpeg_encode source distribution.

Beavis and Butthead are property of MTV.

Manpages:

Berkeley MPEG manpages:

mpeg_encode	MPEG encoder
mpeg_play	MPEG player
User's Guide	for mpeg_encode in PS

Software Interest Group

Spring 2008

Tentative talks:

Interested in speaking?