fmII
Thu, Jan 08th home | browse | articles | contact | chat | submit | faq | newsletter | about | stats | scoop 19:17 UTC
in
Section
login «
register «
recover password «
[Article] add comment [Article]

 Video and Audio Streaming with Flash and Open Source Tools
 by Klaus Rechert, in Tutorials - Sat, May 7th 2005 00:00 UTC

Flash has always been developed and used for multimedia purposes, but until version 6, the possibilities for audio streaming were limited, and there was no video support. With versions 6 and 7, Macromedia introduced video support and a new file format to support various ways of streaming. This article covers only a streaming variant called "progressive download", which does not need server support.


Copyright notice: All reader-contributed material on freshmeat.net is the property and responsibility of its author; for reprint rights, please contact the author directly.

FLV Streams

Streaming is built upon a new file format called FLV, which separates the streamable content and the Flash movie. The results are a very compact Flash movie acting as a multimedia player and a container for streamable content, from which the Flash movie loads a stream on demand.

A single FLV stream contains one audio stream and one video stream. Flash supports uncompressed sound and various compressed formats like MP3 and ADPCM, as well as the proprietary Nellymoser audio codec. With Flash version 6, Macromedia also introduced video support. In version 6, only the Sorenson H.263 video codec was supported. This is a slightly modified version of the open H.263 standard. The latest Flash version, 7, introduced a second video format called "Screen Video", which is a simple, lossless video format, especially developed for screen capturing.

Converting and Creating Content

One method for creating FLV streams is converting existing audio and video content with FFmpeg. FFmpeg is mature and excellent software for converting audio and video to and from various formats.

Converting a video can be done simply with:

 
$ ffmpeg -i infile.[avi|mpeg] stream.flv

FFmpeg uses the Sorenson H.263 video format for encoding video data. There is no support for the Screen Video format at this time. While the Screen Video format is mainly useful for screen capturing applications, Sorenson's H.263 is a multipurpose video codec with good compression rates, suitable especially for encoding motion pictures.

Another project dealing with FLV streams is called libflv. While FFmpeg is a general audio and video converting suite, libflv is focused on working with FLV streams. The project is still in a very early stage, but is able to encode videos in the Screen Video format and allows simple FLV stream manipulations like (de-)multiplexing of audio and video streams. A simple GTK-based screen capturing application can be found in the example directory.

Building a Simple Multimedia Player

After we've created some streamable content, a Flash multimedia player is needed. One huge advantage of a Flash-based player over other plugin-based multimedia players is that there are no constraints about its look and how it is integrated in your site's design.

MING is an Open Source library which is able to create Flash files with almost all recent Flash features, including Action Script and sound and video support. It also has language bindings for a bunch of script and programming languages. The examples presented in this article are written in PHP4. Porting them to other supported languages like C/C++, Java, Python, or Perl should be trivial.

To run the following example, a current CVS snapshot of MING is needed. It is available either via SourceForge's anonymous CVS service or pre-packaged at http://klaus.geekserver.net/ming/.

First, we create a new movie instance and set the dimensions and background color:

	ming_useswfversion(7);
	$movie=new SWFMovie(7);
	$movie->setDimension($width, $height);
	$movie->Background($r,$g,$b);

The new Flash movie can now be filled with Flash objects called characters. For the multimedia player example, we create a video canvas object and add it to the movie. The add() method takes a character, inserts it to the current frame, and returns a handle to the object. This can be used to move, rotate, resize, or remove an object. If the object is going to be used with ActionScript, a name can be assigned to it.

	$stream = new SWFVideoStream();
	$stream->setDimension($width, $height);
	$item = $movie->add($stream);
	$item->moveTo($x, $y);
	$item->setname("video");

The SWFVideoStream() constructor can also take an FLV file as an argument. In this case, the video stream will be embedded to the Flash file. However, this approach has some drawbacks. First, the resulting Flash movie will become at least as big as the stream. Also, the stream's frame rate must not exceed the Flash movie's frame rate, and each Flash file is limited to 16,000 frames, which means that the embedded stream can contain at most 16,000 frames.

A multimedia player application should be able to load and play streams dynamically. Therefore, the SWFVideoStream() constructor is called with no arguments. Thus, only an empty video canvas will be created, which will be controlled by the following ActionScript code:

	connection = new NetConnection(); 
	connection.connect(null);
	stream = new NetStream(connection);
	video.attachVideo(stream); 	
	stream.setBufferTime(10); 
	stream.play('http://localhost/mystream.flv');

The ActionScript first creates a pseudo connection by passing null to the connect() method of the NetConnection object. In contrast, a real connection to a Macromedia streaming server can be made by passing a valid URL to the method. Having a NetConnection instance, a new NetStream object can be created and attached to the empty video canvas. This object handles streaming and provides methods for controlling the stream. The above example loads an FLV stream from the local Web server with a downloadbuffer of 10 seconds. The ActionScript code can be compiled and added to the movie with:

	$action = new SWFAction($action_string);
	$movie->add($action);

Until now, the Flash movie just loads and plays a certain FLV stream. To control its behavior, a simple user interface consisting of some buttons and a seek-slider is needed. Flash has its own compressed lossless bitmap format called DBL. MING provides a small utility called png2dbl to convert PNG images to DBL. Such images are used for the player's control buttons:

	$button = new SWFButton();
	$flags = (SWFBUTTON_UP | SWFBUTTON_HIT | SWFBUTTON_OVER | SWFBUTTON_DOWN);
  	$button->addShape(ImageShape("images/pause.dbl"), $flags);
        
	$action = new SWFAction("stream.pause();");
 	$button->addAction($action, SWFBUTTON_MOUSEDOWN);
	
	$button_ref = $movie->add($button);
	$button_ref->moveTo($x, $y);

The above example creates a pause button for the multimedia player. An interactive button is created in two steps. First, its look has to be defined by adding shapes for certain mouse events. In Flash, a shape is the basic representation for graphic objects. For each mouse event, a differently-shaped object can be assigned to the button. In the above example, the button always looks the same. In the second step, the button's action can be defined by assigning ActionScript to a special event.

One drawback of using progressive download streaming without server support is that there is no possibility of getting the stream's total length. Therefore, the seek-slider's functionality is limited to seeking within the already-loaded parts of the stream.

The dragable part of the seek-slider is realized as a movie-clip object. A movie-clip runs as an independent movie in the Flash movie. It has an independent timeline, can handle scripts, and handles external events itself.

	$mc = new SWFSprite();
	$shape = new SWFShape();
        $shape->setLine(4,25,0,0,128);
        $shape->movePenTo(0, 5);
        $shape->drawLineTo(0, 10);
        $mc->add($shape);
	$mc->nextFrame();
	
	$slider = $movie->add($mc);
	$slider->moveTo($xMin, $y);

A movie clip (SWFSprite) has methods similar to those of a movie object. The add() method inserts a Flash object to the current frame, and nextFrame() finishes the current frame and creates a new one. The movie clip is also a normal Flash object which can be added to a movie and placed on the stage. The functionality of the seek-slider is defined by three small scripts. The first two actions make the movie-clip dragable:

	$a = new SWFAction("startDrag(this, $xMin, $y, $xMax, $y, 1); drag = true;");
        $slider->addAction($a, SWFACTION_MOUSEDOWN);
        
        $a = new SWFAction("stopDrag(); drag=flase;");
        $slider->addAction($a, SWFACTION_MOUSEUP);

The third, lengthier script sets the stream position, depending on the slider's x-position if the slider is actually moved by the user, or sets the slider's x-position, depending on the stream's current time:

	// width in px
	width = xMax - xMin;
	
	paused = false;
        if(drag) { 
		// pause stream while seeking
        	_global.stream.pause(true);
                paused = true;
			
                x = _root._xmouse - xMin;
                seekTo = (_global.streamLen / width) * x;
                _global.stream.seek(seekTo);
       	}
       	else {
		pos = (_global.stream.time * (width / _global.streamLen)) + xMin; 
                this._x = pos; 
                this._y = y; 
      	}
      
      	// restart paused stream
   	if(paused)
	{
               _global.stream.pause(false); 
        }

This script is assigned to the $slider handle with the SWFACTION_ENTERFRAME event.

After we've added all the elements to the Flash movie, the first frame has to be closed with the nextFrame() call. Since we don't need another frame, the movie can also be finished:

	$movie->nextFrame();
        $movie->save("FLVPlayer.swf");

The resulting multimedia player (requires Flash 7)
Example Video: Copyright by Thilo Weigel, University of Freiburg

Conclusion

With Flash, it is easy to create a lightweight, fully customized, embedded video and audio player. There are powerful Open Source tools available for creating content and creating Flash movies. This article introduced the basic concepts of Flash streaming and working with MING. The mediaplayer presented here provides only the most basic features and was only intended as a simple example. It can be extended in many ways, which is left to the reader.


Author's bio:

Klaus Rechert is involved in the MING and in libflv development, and is developing e-learning applications at the University of Freiburg. The source code of the example player can be found at libflv project pages.


T-Shirts and Fame!

We're eager to find people interested in writing articles on software-related topics. We're flexible on length, style, and topic, so long as you know what you're talking about and back up your opinions with facts. Anyone who writes an article gets a t-shirt from ThinkGeek in addition to 15 minutes of fame. If you think you'd like to try your hand at it, let jeff.covey@freshmeat.net know what you'd like to write about.

[Comments are disabled]

 Referenced categories

Topic :: Multimedia :: Sound/Audio :: Players
Topic :: Multimedia :: Video :: Display

 Referenced projects

libflv - A tool to create FLV streams for use with Flash/SWF movies.
Ming - SWF (flash) output library / PHP module

 Comments

[»] Great tip!
by pentapenguin - Feb 20th 2006 21:16:22

I had no idea it was so easy to make something like this in Flash! This could come in very handy on some of my websites. This is the coolest tip I've seen in a while. Thanks! :)

--
-pentapenguin

[reply] [top]


[»] opensource list
by www.Debainhelp.co.uk - Jan 26th 2006 01:16:16

any one is having the list of tools for converting rm to mp3 tools

[reply] [top]


    [»] Re: opensource list
    by jammy - Feb 12th 2006 23:10:17


    > any one is having the list of tools for

    > converting rm to mp3 tools

    anyone please post a resources for converting rm to mp3 me too looking for the same software or any online tool

    [reply] [top]


[»] flash is not free
by Jaromil - dyne.org - May 7th 2005 06:55:15

You may obtain a copy of a flash player only for yourself, you cannot redistribute it (for example into a liveCD) which ends up in a bad situation: Macromedia holds all the control in what is being reproduced.

Imagine you're an artist, then everything you do with Flash can be shown to someone else only after signing a contract (EULA) with the Macromedia corporation that practically owns the reproducibility of all your creations.

I discourage everyone from using Flash technology.

and BTW: GNU GPL alternatives to Flash are coming up fast, stay tuned ;)

ciao

--
jrml

[reply] [top]


    [»] Re: flash is not free
    by Karl Trygve Kalleberg - May 7th 2005 19:57:18


    > and BTW: GNU GPL alternatives to Flash
    > are coming up fast, stay tuned ;)

    Would you be referring to GPLFlash, or a completely new alternative?

    [reply] [top]


      [»] Re: flash is not free
      by Jaromil - dyne.org - May 16th 2005 05:30:46


      >
      > Would you be referring to GPLFlash, or a
      > completely new alternative?
      >

      completely new, still developed since long time and with a very active community behind: Blender

      and something interesting you will find in Freej as well, scriptable with Javascript syntax, still under active development FreeJ

      from the point of view of a programmer ;) there are even more possible alternatives, while i phear that Macromedia corp. is going to hold on Flash the same way Sun corp. is doing with Java... see this article for more details.

      ciao

      --
      jrml

      [reply] [top]


      [»] Re: flash is not free
      by vegelr - Nov 23rd 2005 22:26:26


      >
      > % and BTW: GNU GPL alternatives to
      > Flash
      > % are coming up fast, stay tuned ;)
      >
      >
      >
      >
      > Would you be referring to GPLFlash, or a
      > completely new alternative?
      >
      GPLFlash is something i like to work as there is freedom in the work i do and look for

      vegeler

      [reply] [top]


      [»] Re: flash is not free
      by andreo - Feb 17th 2006 21:24:00


      >

      > % and BTW: GNU GPL alternatives to

      > Flash

      > % are coming up fast, stay tuned ;)

      >

      >

      >

      >

      > Would you be referring to GPLFlash, or a

      > completely new alternative?

      >


      GPLFlash2 shutting down! (gplflash1 continues)
      After going through the code of gplflash2, it was decided it was too messy and a new design was needed. But before a the design process took off, a new project caught our eye: gnash. It is not perfect, but it got some advantages over gplflash2, and most of the developers of gplflash2 will move to gnash.it is mentioned at the url
      http://gplflash.sourceforge.net/

      --
      andreo

      [reply] [top]


        [»] Re: flash is not free
        by CrazyGFreak - Apr 27th 2006 15:49:50


        >
        > %
        > % % and BTW: GNU GPL alternatives to
        > % Flash
        > % % are coming up fast, stay tuned ;)
        > %
        > %
        > %
        > %
        > % Would you be referring to GPLFlash, or
        > a
        > % completely new alternative?
        > %
        >
        >
        >
        > GPLFlash2 shutting down! (gplflash1
        > continues)
        > After going through the code of
        > gplflash2, it was decided it was too
        > messy and a new design was needed. But
        > before a the design process took off, a
        > new project caught our eye: gnash. It is
        > not perfect, but it got some advantages
        > over gplflash2, and most of the
        > developers of gplflash2 will move to
        > gnash.it is mentioned at the url
        > http://gplflash.sourceforge.net/
        >
        is that in hd quality possible, too? looks real nice. i am busy with it to test around thanks.

        [reply] [top]


    [»] Re: flash is not free
    by Petit - Dec 23rd 2005 15:55:26


    > You may obtain a copy of a flash player

    > only for yourself, you cannot

    > redistribute it (for example into a

    > liveCD) which ends up in a bad

    > situation: Macromedia holds all the

    > control in what is being reproduced.

    Right! If you are doing some nifty work to be presented on the web, You'll be sure that almost everyone will be able to enjoy your artistic work in their browser. In that case you wouldn't have to redistribute the player.


    > I discourage everyone from using Flash

    > technology.

    >

    > and BTW: GNU GPL alternatives to Flash

    > are coming up fast, stay tuned ;)

    So I don't agree ;-) Being able to use open source software to produce Flash movies ( aka SWF ) and content as Flash streaming or progressively downloading sound and video is a great thing.
    Using the alternatives certainly means your audience will have to download not only content, but some kind of player.

    Well, I really like alternatives, but I think Flash is a beautiful format. If I can afford the Adobe/MM Flash tools I will use them - if not I'll go for open source tools and libraries.


    --
    B Wildered

    [reply] [top]




© Copyright 2008 SourceForge, Inc., All Rights Reserved.
About freshmeat.net •  Privacy Statement •  Terms of Use •  Trademark Guidelines •  Advertise •  Contact Us • 
ThinkGeek •  Slashdot  •  Linux.com •  SourceForge.net  •  Jobs