Motion Detection In Flash (AS2 and AS3) and C#

Currently working on some motion detection with flash/c# and webcams right now. Here’s a basic overview of some motion detection source files and tricks. Most motion detection is based on snapshots and finding brightness of a pixel with all combined colors, then comparing that to previous snapshots to detect enough variance and thus movement. If you have a webcam hooked up, this sample in Flash AS3 highlights this well showing the camera on the left, then the brightness snapshots on the right. It also has an indicator to the amount of movement due to much brightness.

C#

Here is a nice example of motion detection using various motion detection algorithms in C#. This is built on the very slick AForge.NET Computer Imaging Library.

If you ever wanted your own motion detection or recording it is all possible with the basics of checking brightness and snapshots in the most simple form checking how much change or variance their was to bright pixels or the count of bright pixels compared to previous snapshots.

// Calculate white pixels

private int CalculateWhitePixels( Bitmap image )
{
    int count = 0;
    // lock difference image
    BitmapData data = image.LockBits( new Rectangle( 0, 0, width, height ),
        ImageLockMode.ReadOnly, PixelFormat.Format8bppIndexed );
    int offset = data.Stride - width;
    unsafe
    {
        byte * ptr = (byte *) data.Scan0.ToPointer( );
        for ( int y = 0; y < height; y++ )
        {
            for ( int x = 0; x < width; x++, ptr++ )
            {
                count += ( (*ptr) >> 7 );
            }
            ptr += offset;
        }
    }

    // unlock image
    image.UnlockBits( data );
    return count;
}

Flash AS2

In Flash this is also possible here is a good article from Flash8 that explains cycling through each pixel to compare the image data and implement motion detection from a webcam.

For instance the basics here show how you can compare each pixel and the change in the brightness for each pixel:

//accuracy
tolerance=10;

//color of the current pixel in the current snapshot
nc=now.getPixel(x,y);

//red channel
nr=nc>>16&0xff;

//green channel
ng=nc>>8&0xff;

//blue channel
nb=nc&0xff;

//brightness
nl=Math.sqrt(nr*nr + ng*ng + nb*nb)

//color of the same pixel in the previous snapshot
bc=before.getPixel(x,y);

//red channel
br=bc>>16&0xff;

//green channel
bg=bc>>8&0xff;

//blue channel
bb=bc&0xff;

//brightness
bl=Math.sqrt(br*br + bg*bg + bb*bb);

//difference in brightness between now and before
d=Math.round(Math.abs(bl-nl));

if(d>tolerance)
{
//there was a change in this pixel
}

Flash AS3

Here is a link to grab a conversion of the AS2 Flash motion detection above to AS3.

Source of AS3 motion detection here.

Grant Skinner has done some interesting things with motion detection with Flash and webcams in the past in the incomplet gallery.

C# or other hardware accelerated capable kits are faster but AS3 and Flash with the new AVM2 virtual machine  should be about 10 times faster than AS2 as much of the improvement in performance and the virtual machine is on iteration speed increases such as loops (i.e. pixel loop).

AS3 Content Aware Resizing or Image Seam Carving

Image Manipulation Using ‘Seam Carving’ From SIGGRAPH 2007

Watch this interesting “seam carving” technology to resize images, image item removal, and manipulating images that go well beyond crop, resizing and retargeting.

Be sure to watch the end where you can edit the images and remove objects from the photography or image.

3D Flash Perlin Noise Terrains with AS3 and Sandy

Rafajafar posted a great tutorial on 3D Flash Perlin Noise dynamic terrain generations with AS3 and Sandy.

Sandy is a flash 3d engine that has been out just a little longer than Papervision3D. The code for Sandy is very clear and it is a great engine. I have experimented with it and found it to be a bit slower it seems that papervision but it has great tools, including an extra 3ds file importer in addition to the Collada and ASE formats that most flash 3d engines support.

Using Perlin Noise is very common in 3d engines to draw large terrains and realistic terrains for 3d games or simulations. This technique makes it very easy to make dynamic terrains or randomly generated terrains, water effects, fire, clouds, whatever your need.

The demo has terrain generation class that can be used to build the dynamic terrain generation and the perlin noise material movement.

Samples using the TerrainCreator class to make different terrains. This could be used to also make level editors and other components of the game making pipeline in flash 3d.

//Water
ter = new TerrainCreator("MyTerrain", 1000000,
128, 9, 6000, 6000, 10,60,60, false,false, 15,
BitmapDataChannel.BLUE, false);

//Gentle Hills
ter = new TerrainCreator("MyTerrain", 10, 128, 9,
6000, 6000, 10,120,120, false,false, 15, 7, true);

//Smooth Hills
ter = new TerrainCreator("MyTerrain", 10, 128, 9,
6000, 6000, 10,120,120, true,true, 30, 7, true);

//Mountainous
ter = new TerrainCreator("MyTerrain", 30, 128, 3,
6000, 6000, 15,100,100, false,false, 10, 7, true);

Click here for the full tutorial and source files on 3D Flash Perlin Noise dynamic terrain generations with AS3 and Sandy.

Now if we could just fast forward 2-3 years on processor power and speed, or get hardware acceleration for Flash...

3D Materials, Texturing and Mapping in AS3 and the Limits of Flash and Software Rendering

Who’s got the textures and cool chrome shiny 3d objects in Flash? There have been lots of materials work recently from papervision list developers and away3d developers (away3d is a branch of Papervision3D) and recently it is heating up a bit.

Early on on the papervision3D excitement, flash possibilities in 3d with AS3 specifically, much of that was due to 3d in flash but also the ability to texture and have bump mapping, toon rendering/cell shading and other neat effects like baked lighting, faked real-time lighting, video and movieclips on flash 3d objects and animated textures.

But when it comes to environmental mapping and true 3d reflection that might be simply stretching Flash to a limit that might require hardware acceleration but that isn’t stopping some.

I am not sure if environmental mapping will every be possible on a large scale without hardware acceleration. Pushing the limits could help influence Adobe to the market direction. But then again I never thought I would see the level of 3d in flash that we have and maybe in 2-3 years with multi-core processors it will be possible.

Here’s a snapshot of the current materials and environmental mapping (fake and real attempts).

UnitZeroOne first environment mapping /bump mapping:


Some toon renderings from UnitZeroOne

Recent work by mr doob

Wood

Metal

Weird

Weird 2

Recent work by actionscript architect

Perlin noise algorithm to animate texture real-time into water effect

More environmental mapping effects by the away3d materials developer Fabrice. Fabrice and the away3d developers are really taking off with the papervision3d core. I am seeing lots of engine limits tested and some great work at away3d.

Bumpy Meteor

Flat Lighting on bitmapMaterial

Chrome Ball (dont’ zoom in too far :) )

Did your processor melt yet?

I think that for games and flash effects faking it or real environmental mapping will have to be judged by what is needed for your purpose. I think that Flash player on software rendering can only go so far. So if you have real-time environmental reflections and surroundings it doesn’t always make your gameplay better and it won’t make your demo better if it means removing assets in other areas to make up for the performance drain of software rendering and the pressures it puts on the processor or browser plug-in.

You can still make really killer effects with baked animations, fake environmental mapping, faked dynamic real-time lighting and other effects. Flash, nor silverlight, will not be able to match hardware rendered shaders, per pixel lighting and physics anytime soon. But people are making good progress on this. I think it would be great if hardware acceleration were added to both Silverlight and Flash, with that, a brand new massive game market online, and it will be game on!

3D Pathfinding in AS3

The actionscript architect Paul Spitzer has posted two killer demos of pathfinding using A* in AS3/Flash 9 using the Away3D engine (derivative of Papervision3D that is taking off) and you can download the source for both. These demos not only highlight the pathfinding but also provide editing of the graph to be traversed which could easily be made into a game world editor.

Paul adds to some path finding work with the likes of Zeh using Tweener (the best animation kit for AS3 yet) with the bezier papervision pathfinding and the good work going on at polygonal labs in data structures for games and their usage in pathfinding for gaming in flash.

At the actionscript architect’s blog check out the true 3d path finding in all three dimensions, and a 2d based pathfind using A* algorithm in 3d.

Here’s the 3d pathfinding with true 3d (x,y,z) points used in the path.

Here’s the 2d pathfinding with 3d interface.

Paul has been doing very cool vector work from video on 3d in flash using papervision3D, to being one of the first people making demos for WPF and Silverlight such as the Northface Demo.

If you are working on path finding or gaming in AS3 be sure to check out Paul’s work, Zeh’s bezier pathing in papervision and polygonal labs often.

Polygonal Labs Motor Physics Tests with AS3

Polygonal Labs is building a Motor Physics Engine (unreleased) and here is a test that has boxes stacked up that you shake. I am looking forward to the Motor Physics Engine by Michael of Polygonal Labs after going through APE Physics engine by Alec Cove which is pretty slick.

The tools so far by polygonal labs such as the AS3 Data Structures for Game Developers have been excellent and made it into many games and engine tests of mine and his blog is a really well presented and the topics are always well presented and researched.

Be sure to check out the 50 boxes test with Motor Physics at Polygonal Labs

Also if you don’t regularly visit polygonal labs you are missing out on some really good research and tools. The recent post with the graph-based A* sample using the AS3 Data Structures for Game Developers was killer.

APE Physics Engine for AS3

Photo Sharing and Video Hosting at PhotobucketBe sure to check out the APE physics engine for flash

The library contains particles to assign to movieclips that appy the physics.  These base objects can be extended in many ways to create complexity from simplicity.

APE (Actionscript Physics Engine) is an AS3 open source 2D physics engine for use in Flash and Flex. APE is written and maintained by Alec Cove.

CircleParticles
-fixed or non-fixed
-variable mass, elasticity, and surface friction

RectangleParticles
-rotatable
-fixed or non-fixed
-variable mass, elasticity, and surface friction
-corner particles that can be attached to other particles with SpringConstraints

WheelParticles
-traction and angular velocity
-fixed or non-fixed
-variable mass, elasticity, and surface friction
-edge particles that can be attached to other particles with SpringConstraints

SpringConstraints
-stiffness
-collidable or non-collidible
-collidable constraints have variable width and scale

Licensing
-non restrictive LGPL

 [demo]

Hopefully we will have a tutorial soon here using APE and some other excellent libraries released recently with the onslaught of AS3 code and kits that will be fluid in their frequency. Another coming physics library is from polygonal labs called the Motor Engine.

AS3 Character Rigging Classes from the Algorithmist

Photo Sharing and Video Hosting at PhotobucketThe Algorithmist posted an AS3 character rigging class library. The kit includes some really specific classes for building characters and is quite well done.

The library includes all you need to rig up characters in flash and includes the source, samples for flex and what the hell more could you ask for?

The Algorithmist calls it the Singularity Package and it contains:

Singularity Package Contents (character rigging classes)

All rigging classes are in the Singularity package, which must be added to your path when building a new Flex project. The current package organization for the rigging classes is as follows,

Singularity.Rigs
Arm – 2-link bone chain representing right or left arm in a humanoid biped.

BaseBone – Base class encapsulating functionality common to Bones and Connectors.

Biped – Used to create and animate humaniod biped characters.

Bone – Represents a single bone with support for fast propagation of FK transformations in bone chains. The Bone class is optimized for linking bones together in chains.

Chain – Manager class for a single chain of bones.

Clavicle – 1-link connector representing right or left clavicle in a humanoid biped.

Connector – Optimized container for multiple single-bone chains with a single input and multiple terminators. Chains or additional connectors linked forward at each terminator.

  Foot – 1-link connector representing right or left foot in a humanoid biped.

  Hand – Multi-link connector representing right or left hand in a humanoid biped.

  Head – 1-link connector representing a humanoid biped head.

  IBone – Bone interface – represents all functionality a bone must implement.

  IChain – Chain interface – represents all functionality a bone chain must implement. Chains and Connectors implement IChain and are considered interchangeable when propagating FK.

Leg – 2-link bone chain representing right or left leg in a humanoid biped.

Neck – 1-link connector representing the neck of a humanoid biped.

Pelvis – Multi-link connector representing the pelvis of a humanoid biped.

SimpleSpine – 1-link connector providing the simplest representation of a humanoid biped spine. Used for very simple (but fast) game characters.

Template – Templates are used to skin (or draw bones) for segmented characters. Refer to SimpleSkinTest for examples.

When developing physics and character based animation rigging can help to really speed things along in 3d and here in this case in 2d flash.  This allows the developer/animator more freedom to work on the animation and not about redrawing.  It also helps to create characters that can be animated from the beginning rather than an afterthought which usually leads to less troublesome issues. 

The AS3 rigging classes are used for skelton rigging and skinning of 2D characters. The rigging class library is organized around the development of highly specific articulated rigs. The current focus is on humaniod bipedal characters. In a 3D animation package or game engine, a bone hierarchy would normally be represented as a tree structure. In this class library, bones and chains are organized in a structure that is easier to deconstruct and understand by OOP programmers. This structure has some performance advantages, particularly when propagating FK in a 2D rig.

The only drawback to this kit is it is not fully open source and commercial use must be approved.

The Singularity AS3 parametric curve library requires the Flash 9 player and a development environment supporting Actionscript 3. All demonstration programs were created with FlexBuilder 2. The AS 3 character rigging classes are copyrighted and licensed for personal, experimental usage. Please contact me to obtain authorization for commercial use. The download includes the entire Singularity package (which includes the parametric curve library).

Download library (.zip) here. Before running any example programs, place the Singularity package in your build path.

AS3 Mouse Gesture Recognition

Didier Brun of ByteArray just keeps throwing down.  Here is another great showcase of the power of AS3 and the tools that can be built with it, strongly based on alogorithms proven in other hardcore languages. This project is a Mouse Gesture Recognition lib for AS3 (demo)

Photo Sharing and Video Hosting at Photobucket 

Mouse gesturing can be used for recognition of letters, numbers or even just programmable actions by gesture.  Imagine an application that allows you to just make a gesture (even if its not a drawing program or letter recognition) that would open when you make an open gesture, or close a file with a slash gesture. I have many uses for this planned. Very inspiring stuff.

A bit on the algorithm from Didier at ByteArray, I just don’t think it can be made any more simple:

Photo Sharing and Video Hosting at Photobucket

  1. Each letter is defined by a n ‘ 8-directions gesture sequence
  2. The mouse moves are saved with the same 8-directions sensibility
  3. A Levenshtein distance is calculated from each letter to the user moves
  4. The algorithm return the best candidate (lowest levenshtein cost)

Usage

Example for the B :

gesture=new MouseGesture(stage);
gesture.addGesture(“B”,”260123401234″);
gesture.addEventListener(GestureEvent.MATCH,matchHandler);

function matchHandler(e:GestureEvent):void{
trace (e.datas+” matched !”)
}

This project is OPEN SOURCE under RPL License.
2007-05-17 v1.0 mouse_gesture_v1_0.zip

AS3 Random Number Generator

Polygonal Labs has an excellent random number generator.  There is an AS2 and AS3 version and it has much better random numbers tools such as seeds and ranges.

When developing games you often need a PRNG that, once initialized with a seed value, produces always the same sequence of random numbers after it. The Math.rand() function in Flash is a PRNG, but it doesn’t let you define a seed value (I think it’s picked randomly when the swf starts, but there isn’t much information about it) so another solution has to be found.

One of the best PRNGs is the Mersenne Twister algorithm, but in my opinion it’s total overkill for flash games (also slow and difficult to implement). However, after spending some time with google I found an interesting site explaining the implementation of the Park-Miller-Carta PRNG, which belongs to the group of Linear congruential generators (LCGs).

read more >>

Download & Usage

Park-Miller-Carta Pseudo-Random Number Generator:
AS2 Version
AS3 Version

First define a seed value (default is one) and then call nextInt() to get the next integer value, nextDouble() to get a floating point number between 0..1 and nextIntRange() or nextDoubleRange() to generate numbers within a certain range.

Game Development Common Algorithms /Techniques

Here’s some algorithms and techniques that most game developers should be aware of when developing games for Flash, Director or other gaming tools (primarily 2d or fake 3d).

A* (A Star)
http://en.wikipedia.org/wiki/A*_search_algorithm

Mode 7
http://en.wikipedia.org/wiki/Mode_7

Collision Detection
http://en.wikipedia.org/wiki/Collision_detection

Painter’s Algorithm (papervision/software blitter)
http://en.wikipedia.org/wiki/Painter’s_algorithm

Sprites
http://en.wikipedia.org/wiki/Sprite_(computer_graphics)

Isometric Projection aka Axonometric Projection aka Orthographic Projection (essentially lack of true perspective in 3d objects)

http://en.wikipedia.org/wiki/Orthographic_projection
http://en.wikipedia.org/wiki/Axonometric_projection
http://en.wikipedia.org/wiki/Isometric_projection

Well versed in mathematics such as Trigonometry and Calculus

http://en.wikipedia.org/wiki/Cosine
http://en.wikipedia.org/wiki/Sine
http://en.wikipedia.org/wiki/Tangent
http://en.wikipedia.org/wiki/Radian
http://en.wikipedia.org/wiki/Pythagorean_theorem
http://en.wikipedia.org/wiki/Linear_equation
http://en.wikipedia.org/wiki/Quadratic_equation
etc…

If you want to make games, math is required. Wiki should only be a starting point. Sometimes the best way to understand something is to implemented it or try to implement it yourself. There are lots of samples and tutorials on the web regarding these but its always good to understand what makes it all work, usually that is math. Right after “42″ it is the answer to Life the Universe and Everything.

We will feature another set for primarily 3d games such as quaternions, matrix maps, transformation matrices , 3d projection and more! Over time we hope to have a tutorial for all 2d in flash and 3d techniques in papervision or 3d framework such as OpenGL or DirectX.

Follow

Get every new post delivered to your Inbox.