Archive for the 'Actionscript' Category

Producing video with flash/air: png sequence rendering from swf playback

I’ve been quite busy at school here in Barcelona. Busy in fun ways too: recently I put flash to good use in a ‘creative project’ my mates and I had to pull through in the last couple of days before exams kicked in. We hacked together this short animation to welcome freshmen to next year’s csr classes:

View on Vimeo.

You have to love flash – you can pull out quite some tricks in no time. All scenes were done in flash authoring, and the swf files were rendered with camera shake and flicker to png sequences by a small air app. The non-flash part was simply to put everything in an avi container, compress it and mux it with audio, stuff that was done with virtualdub and megui.

Rendering swf playback to an image sequence

When working with video, I personally enjoy image sequences because they’re so easy to manage. My uncontested favourite format for image editing is png, mostly because it’s compact and lossless. In the case of dumping video frames from flash, it also has the advantage of compressing very well rasterised vector graphics.

The app I put together is some sort of a batch processor that you compile with the rendering ‘script’ inlined in its code; its beyond simple but worked so well that I thought about posting it here in case someone needs to do something similar.

Here’s the cleaned-up api:

public class PNGSequenceRenderer extends Sprite
	 * Sets the dimensions of the input swf material.
	 * @param	width (pixels)
	 * @param	height (pixels)

	public function setInputSize ( width : Number, height : Number ) : PNGSequenceRenderer;

	 * Sets the dimensions of the outputed png sequence.
	 * @param	width (pixels)
	 * @param	height (pixels)

	public function setOutputSize ( width : uint, height : uint ) : PNGSequenceRenderer;

	 * Sets the scale mode for the output, either 'show all' or 'no border'.
	 * @param	showAll true for 'show all', false for 'no border'.

	public function setScaleMode ( showAll : Boolean ) : PNGSequenceRenderer;

	 * Sets the level of camera shake. ( e.g. 0 - disabled, 1 - moderate, 2 - heavy. )
	 * @param	factor Camera shake amount.

	public function setCameraShake ( factor : Number = 0 ) : PNGSequenceRenderer;

	 * Sets the level of flicker. ( e.g. 0 - disabled, 1 - moderate, 2 - heavy. )
	 * @param	factor Level of shutter flicker.

	public function setFlicker ( factor : Number = 0 ) : PNGSequenceRenderer;

	 * Sets the output background transparency and fill color.
	 * @param	transparency Whether to draw on a transparency-enabled BitmapData.
	 * @param	color Background color for rendering.

	public function setBackground ( transparency : Boolean = false, color : uint = 0 )
		: PNGSequenceRenderer;

	 * Enqueues a scene for rendering.
	 * @param	url The location of the movie to render.
	 * @param	cameraShakeMultiplier Local camera shake adjustment.
	 * @param	flickerMultiplier Local flicker adjustment.
	 * @param	from First frame to render.
	 * @param	to Frame upon which to end the scene early.

	public function enqueueScene ( url : String, cameraShakeMultiplier : Number = 1,
		flickerMultiplier : Number = 1, from : uint = 0, to : uint = 0xffffff )
		: PNGSequenceRenderer;

	 * Begins loading scenes and outputing frames in the specified folder.
	 * @param	outputFolder The folder where all outputed files are to be put.
	 * If no other limit is specified, each scene is rendered until a pink
	 * (solid #ff00ff) frame is reached.

	public function render ( outputFolder : String ) : void;

How to

What you do with this is to extend your app from the Renderer class, and put together the rendering script in the constructor body, something like:

public class Main extends PNGSequenceRenderer
	public function Main () : void
		setCameraShake ( 1 );
		setFlicker ( 1 );
		setInputSize ( 800, 400 );
		setOutputSize ( 853, 480 );
		enqueueScene ( 'C:\\prj\\swfs\\scene01.swf' );
		enqueueScene ( 'C:\\prj\\swfs\\scene02.swf' );
		render ( 'C:\\prj\\output\\' );

Compiling the project for the air 1.5 target will launch the adl and run your app, which will begin loading and playing your scenes one by one, and dumping the png sequence in the output folder. During the process you’ll be getting some basic feedback on the screen, mainly which scene and frame is being rendered.

Note: Before you jump into testing it out, remember to have a pink solid fill frame (#ff00ff) at the end of each scene, as the renderer expects a frame with a pink middle pixel in order to proceed with the queue. Also, compile in release mode as the PNG encoder runs twice to three times faster without debug stuff in the bytecode.

Once you have the image sequences ready, you can edit them in virtually any video editing software package (if you need to), and then package everything in a video container. The latter will come out pretty voluminous, so you’ll need to compress it with a codec, such as h.264. Finally, you mux it with the audio and you are ready for distribution.

For the windows crowd, virtualdub is a great piece of software for playing with video, and also pulls off the packaging of the sequences into a container with a couple of mouse clicks. Also, check out megui for the video and audio compression and muxing, it makes it all real easy.


The code requires the PNGEncoder class from as3corelib, as well the standard normal output Park-Miller prng class (described here and used for the flicker and camera shake effects).

Bookmark and Share

Speedy Voronoi diagrams in as3/flash

Voronoi diagram is a decomposition of space, here a 2d plane, on a nearest neighbour basis. The diagram splits the plane in regions, each corresponding to a site (a point of interest), and containing all the points closer to this site than to any other.

Voronoi diagrams have many applications, in fields as different as botanics and astrology. A few examples of computer graphic uses for Voronoi diagrams include procedural generation of organic forms, crazy image compression, beautiful fractals, noise, etc.

What got me in Voronoi mood however is seeing this this algorithmic collage by Santiago Ortiz from Bestiario. The work’s sheer visual impact shows off the one quality of Voronoi diagrams I appreciate the most – they look awesome. Watching the masks move for a couple of minutes got me terribly into porting Steve Fortune’s algorithm for Voronoi decomposition: I had to see for myself how much stuff could happen on the screen before the player choked.

Fortune’s algorithm

Fortune’s algorithm, ideally, computes a Voronoi diagram in O(NlogN) time, where N is the number of sites, or the minimum possible time for the job, as O(NlogN) is the time-complexity of optimal sorting algorithms. It sweeps through the plane, maintaining a front of parabolas that progressively traces out the diagram, by spliting and adding curves for each site encountered, and removing arcs whenever a region vertex is determined with certainty. The Wikipedia article has it all explained pretty well, but if you need additional info, give this and this articles a try.

I began with this c++ implementation, which is a simplified take on the algorithm. The latter uses a binary tree for the composition of the beachline, whilst here we’ve got a double linked list. However, I do think that the double-link approach would do better for most visual applications in flash, where the cost of not having quick lookup of beachline parabolas might be offset by not having to reogranise the tree, and by the low overall number of particles. I ended up modifying and optimising most of the code however, and I bet this is the reason for the artifacts that show up every now and then in the last of the three demos.

Below you have a demo showing off 1000 regions, in line with nodename‘s mona-lisa tessalator:

Then 2000 points going around with their speed determined by the values of an image:

Finally, a simple audio visualiser: 

Anyway, this really is just a study of how quickly the regions of a diagram can shape up without cheating. There is no bounding box intersection, nor sophisticated output options. Clearly, output of an optimal and informative data structure will be worth the performance kick, and if I get to do my graph theory homework, I’ll be coming back to this.


Fortune’s algorithm is already well implemented and made available by astatic, also nodename shows it off with his Voronoi toy, even though you don’t see the sources. You get a different tool for the same job by HIDIHO!, he goes through Delaunay triangulation, and then translates to a Voronoi decomposition, the two being dual graphs.


Bookmark and Share

A quick note on line and segment intersection (as3)

Just a quick note: Keith Hair has a very sweet and useful series of posts about line intersections that have saved me more than a few hours of headbanging. In particular, his line intersection function is a marvelous paste-and-go solution, and I really feel like posting a small optimisation that accelerates the constraint checks at the end, which, as Nicolas Barradeau points out, do need some attention.

Instead of posting the whole function body here, here’s a recipe. First, get the original code from Keith’s post. Then, replace the function declaration with the following:

function lineIntersectLine (
	A : Point, B : Point,
	E : Point, F : Point,
	ABasSeg : Boolean = true, EFasSeg : Boolean = true
) : Point

Finally, instead of the if (as_seg) block, paste this:

//	Deal with rounding errors.

if ( A.x == B.x )
	ip.x = A.x;
else if ( E.x == F.x )
	ip.x = E.x;
if ( A.y == B.y )
	ip.y = A.y;
else if ( E.y == F.y )
	ip.y = E.y;

//	Constrain to segment.

if ( ABasSeg )
	if ( ( A.x < B.x ) ? ip.x < A.x || ip.x > B.x : ip.x > A.x || ip.x < B.x )
		return null;
	if ( ( A.y < B.y ) ? ip.y < A.y || ip.y > B.y : ip.y > A.y || ip.y < B.y )
		return null;
if ( EFasSeg )
	if ( ( E.x < F.x ) ? ip.x < E.x || ip.x > F.x : ip.x > E.x || ip.x < F.x )
		return null;
	if ( ( E.y < F.y ) ? ip.y < E.y || ip.y > F.y : ip.y > E.y || ip.y < F.y )
		return null;

By having both an ABasSeg and EFasSeg parameters, you’ll be also able to easily intersect segments with lines:

Bookmark and Share

A diff/patch solution for Flash: a proof of concept Monte Carlo algorithm for comparing text versions in as3

EDIT 2: was fixed again, another issue with RangeErrors from ByteArray.writeUTF (). Well, forget certainty levels, but this time all should really be working just fine with strings of any length.

EDIT 1: underwent some serious debugging and unit testing these days (thanks to Rich, see the comments below), so I’ve 95% certainty that the sources below (updated) are free from bugs. Anyway, the bugs we fixed were silly copy/paste havoc, so if you DO find something wrong about the Patch class, I doubt it’ll be difficult to fix.

Also check out the .reversed flag on the Patch object, it’s helpful when you need to ‘unpatch’ a string.


diff and patch

diff is a file comparison tool in Unix-like environments, born in the 1970s, which takes two plain text files and outputs the spots where they differ. Its counterpart, patch, is a tool that applies diff output to files in order to keep them updated; this technique was used to synchronise content, often source code, on multiple workstations (today Subversion uses the same trick), and later became the paradigm for revision control systems in collaborative content applications.

Generally speaking, diff works by solving the longest common subsequence problem; when dealing with texts, that’s finding the longest common substring. The operation could be repeated on the left and right of this biggest common chunk, and so on, until there’s nothing else to match. The optimal solution is the minimal set of edits (inserts and deletes) that would suffice to construct the new text from the old, or the most compact possible correct diff output.


I came across google-diff-match-patch when reading about the O(ND) diff. The author of the library, Neil Fraser, has done an amazing job in bringing what’s probably the most sophisticated diff algorithm to the world of web development, by creating a common API for javascript, java and python (hello App Engine!), and also c++. The library is already ported to as3; you need to ask Neil for the code though, it’s still not in the repository.

Neil’s google-diff-match-patch provides you with a diff tool and patch tool for strings. The latter is extremely sweet and clever, because it works on a best effort basis: in case the patch tool is ran on a string which is not identical to the ‘original’ text used by diff to make the patch, it looks up the areas to be patched by approximate string matching of the patch context, and is thus pretty much capable of patching texts which have been even heavily modified in the meantime by another user or action.

There’s something that worries me about the O(ND) diff when applied on a character-by-character basis on long texts however, especially in the context of js virtual machines or the avm. I do not fully understand how the solver works, but I am afraid that it may be a little too combersome a computation in settings which are not completely theoretical. In other words, if you run the Google diff on two big enough texts with lots of common and differing chunks alternated throughout, the algorithm will choke, even for minutes.

Why so serious?

In my view, looking for an optimal diff solution is overkill. In fact, if one allows for some degree of suboptimal results, the computational effort needed to pull the trick can be radically decreased.

Well, I though that a problem such as that of finding common substrings in two strings would be a great way to get some experience with Monte Carlo problem-solving, and decided to put together a quick and dirty proof of concept implementation of a plain text diff based on random sampling. Starting from Neil’s articles and the google-diff-match-patch sources, I aimed to build a diff which can relatively quickly find a near-to optimal solution, so to prevent the app from freezing, while avoiding the need for marsalling.

The following benchmark contrasts the performance and result quality (with no post-processing nor semantic clean-up) of my naive Monte Carlo diff and the Google O(ND) diff. The two texts are the first and last revision of my ‘Hello world!’ post:

There are a few possible explainations for why the google diff slows down this much. First of all, the O(ND) diff’s name suggests its time-complexity (N = A+B, the sum of the lengths of the texts being compared, and D being the shortest edit path). Another is in the nature of the port I have at my disposal, which shoots out a ton of warnings, and is nowhere near optimised for the avm [EDIT: Indeed, running the same texts in the diff demo of google-diff-match-patch (in Chrome) yields timings that are about three times lower]. Most importantly, the algorithm takes so much more time because of the much higher quality of its solution: still, most of this effort will be for nothing after an efficiency and semantic clean-up, yet there is no real way to request less detail in the first place.

My point is that you don’t need an optimal solution to communicate text changes; you need a reasonably compact one, but asap. A quick patch can always be reduced server-side if needed, by recursively running diff on all insertion/deletion pairs of edits.

What’s behind this ‘Monte Carlo’ diff

Small substrings from the shorter text are taken, and if they exist in the longer text, a binary search begins on their left and right expanding them in the two directions. When an entire common chunk is found, its location and length are recorded and excluded from the search, hence reducing the problem size. The first two substrings are taken from the begining and the end of shorter string, so to quickly find and rule-out the common prefix or suffix (if any), after which the search goes on at random locations.

Arbitrary decision rules determine when the search will stop, and then a simple solver kicks in to resolve conflicts among the discovered common substring. Such conflicts usually involve chunk overlapping, but can also consist in N-to-1 relationships in between chunks in two texts, usually when those two have repetitive common parts (copy-pasting is good reason for such a thing to happen). Because some conflicts will require that common chunks get discarded, before running the solver, the commonalities are sorted (and thus prioritised) by length: hence naively solving the longest common subsequence problem. Conflicts are cheap to resolve however, and this step adds very little overhead to the whole algorithm.

Finally, as an optional post-processing step, semantic adjustment takes place, aligning all chunks to whitespace and punctuation, hence yielding human readable output.

What you can do with this

The demo below gives you a basic interface to play around with the API. The scenario mimics the google-diff-match-patch Shakespeare demo. The first two fields contain the old and new versions of the text to be diffed. The following row displays the computed patch in the google-diff-match-patch ‘emulated’ unified diff format, and a human-readable version of the solution. Finally, the final row provides an input field for a text to be patched, and a result field for the patch operation.

The approximate string matching functionality of the patch tool is built directly over Neil’s implementation of the Bitap algorithm; I have ported Neil’s code to as3, doing my best to optimise it for the avm.

Worth noting is that the API is fully compatible with the google-diff-match-patch library through this pseudo-unified serialisation format; hence you can be running the Google diff on the server-side, and this quick diff in the client with no complications.

Grounds for improvement

In terms of performance, there is a lot that could be done with the algorithm’s search logic; currently it abuses the substring and indexOf methods, and they are indeed slow. An inconclusive charAt () binary search with a final substring comparison will probably render the algorithm much faster in most situations. Another thing to look into is to direct the random sampling process towards areas that are more likely to contain commonalities and preventing it from sampling the same substring more than once.


These are the current sources. The code is still in the works though, so let me know if you spot any trouble.

Bookmark and Share

Random numbers: standard normal distribution in flash/as3

Posting about measuring clock drift got me in the mood of poking random numbers a bit more. Well, about an year ago I decided I needed normally distributed prng output, but never really used it for anything. I am sharing a solution in my best hope that someone will spare themselves the headbanging part.

The normal distribution is one of those things one should be acquainted with if about to deal with statistics, simulations or even procedural art. Anyhow, the main problem with normal pseudo-random numbers in flash is obtaining them, because the usual output of every prng follows the uniform distribution and one has to map that output to the distribution of interest. The reasonable solutions I have explored are three; one is using lookup tables, which is self-explanatory and very inexpensive, but unfortunately also painfully boring.

Another is making use of the central limit theorem, which in real life boils down to this. One can take, say, 10 numbers extracted from a uniformly distributed population (read Math.random) and take their average, whose sampling distribution will approximate to some extent the normal distribution.

I fancy using the Box-Muller transform or, more specifically, the Marsaglia polar method, which is a less computationally expensive variation of the former. Both take a pair of uniformly distributed inputs and map them to two (independent and uncorrelated) standard normal outputs. Effectively, this means that one could take any prng, be it Math.random, a Park-Miller lcg (again, check out the one on Polygonal labs), a Mersenne twister or whatever, and map its output to the standard normal distribution.

For better performance, the mapping algorithm could be built into the prng so that all operations are performed within one call. This is even more valid for the Marsaglia method, because it relies on rejective sampling, which means that it rejects a pair of random inputs every now and then and requests another. The code snippets below demonstrate the Marsaglia transform built into a Park-Miller prng, whose core logic consists in just a couple of lines:

public class ParkMiller
	 *	Seeds the prng.
	private var s : int;
	public function seed ( seed : uint ) : void
		s = seed > 1 ? seed % 2147483647 : 1;

	 *	Returns a Number ~ U(0,1)
	public function uniform () : Number
		return ( ( s = ( s * 16807 ) % 2147483647 ) / 2147483647 );

When you call uniform(), the seed value is multiplied by 16807 (a primitive root modulo) and set to the remainder of the product divided by 2147483647 (a Mersenne prime, 2^31-1, or the int.MAX_VALUE). This new value is returned as a Number in the range (0,1).

The histogram below illustrates the uniform distribution of the prng output:

The uniform pseudo-random values will be fed to the Marsaglia transform, whose simple algorithm is well described in its Wikipedia article. An as3 implementation with inlined uniform() getters could look like this:

	 *	Returns a Number ~ N(0,1);
	private var ready : Boolean;
	private var cache : Number;
	public function standardNormal () : Number
		if ( ready )
		{				//  Return a cached result
			ready = false;		//  from a previous call
			return cache;		//  if available.

		var	x : Number,		//  Repeat extracting uniform values
			y : Number,		//  in the range ( -1,1 ) until
			w : Number;		//  0 < w = x*x + y*y < 1
			x = ( s = ( s * 16807 ) % 2147483647 ) / 1073741823.5 - 1;
			y = ( s = ( s * 16807 ) % 2147483647 ) / 1073741823.5 - 1;
			w = x * x + y * y;
		while ( w >= 1 || !w );

		w = Math.sqrt ( -2 * Math.log ( w ) / w );

		ready = true;
		cache = x * w;			//  Cache one of the outputs
		return y * w;			//  and return the other.

The following histogram displays the distribution of this new getter:

Performance-wise, there is some ground for reducing the algorithm’s overhead. One could, for ranges of values, approximate the square root with, for example, an inlined Babylonian calculation. The natural logarithm can also be easily approximated for values not too close to zero. Still, every approximation comes at the cost of some precision, and the above implementation will be fast enough for most uses; on my notebook I get a million numbers for about 350 milliseconds.

Finally, the Ziggurat algorithm is an alternative to the Marsaglia transform, and has the promise of better perfomance if well optimised. Still, I personally haven’t managed to make it work all that great in as3.


Bookmark and Share