Motion Detection In Flash (AS2 and AS3) and C#

Currently working on some motion detection with flash/c# and webcams right now. Here’s a basic overview of some motion detection source files and tricks. Most motion detection is based on snapshots and finding brightness of a pixel with all combined colors, then comparing that to previous snapshots to detect enough variance and thus movement. If you have a webcam hooked up, this sample in Flash AS3 highlights this well showing the camera on the left, then the brightness snapshots on the right. It also has an indicator to the amount of movement due to much brightness.

C#

Here is a nice example of motion detection using various motion detection algorithms in C#. This is built on the very slick AForge.NET Computer Imaging Library.

If you ever wanted your own motion detection or recording it is all possible with the basics of checking brightness and snapshots in the most simple form checking how much change or variance their was to bright pixels or the count of bright pixels compared to previous snapshots.

// Calculate white pixels

private int CalculateWhitePixels( Bitmap image )
{
    int count = 0;
    // lock difference image
    BitmapData data = image.LockBits( new Rectangle( 0, 0, width, height ),
        ImageLockMode.ReadOnly, PixelFormat.Format8bppIndexed );
    int offset = data.Stride - width;
    unsafe
    {
        byte * ptr = (byte *) data.Scan0.ToPointer( );
        for ( int y = 0; y < height; y++ )
        {
            for ( int x = 0; x < width; x++, ptr++ )
            {
                count += ( (*ptr) >> 7 );
            }
            ptr += offset;
        }
    }

    // unlock image
    image.UnlockBits( data );
    return count;
}

Flash AS2

In Flash this is also possible here is a good article from Flash8 that explains cycling through each pixel to compare the image data and implement motion detection from a webcam.

For instance the basics here show how you can compare each pixel and the change in the brightness for each pixel:

//accuracy
tolerance=10;

//color of the current pixel in the current snapshot
nc=now.getPixel(x,y);

//red channel
nr=nc>>16&0xff;

//green channel
ng=nc>>8&0xff;

//blue channel
nb=nc&0xff;

//brightness
nl=Math.sqrt(nr*nr + ng*ng + nb*nb)

//color of the same pixel in the previous snapshot
bc=before.getPixel(x,y);

//red channel
br=bc>>16&0xff;

//green channel
bg=bc>>8&0xff;

//blue channel
bb=bc&0xff;

//brightness
bl=Math.sqrt(br*br + bg*bg + bb*bb);

//difference in brightness between now and before
d=Math.round(Math.abs(bl-nl));

if(d>tolerance)
{
//there was a change in this pixel
}

Flash AS3

Here is a link to grab a conversion of the AS2 Flash motion detection above to AS3.

Source of AS3 motion detection here.

Grant Skinner has done some interesting things with motion detection with Flash and webcams in the past in the incomplet gallery.

C# or other hardware accelerated capable kits are faster but AS3 and Flash with the new AVM2 virtual machine  should be about 10 times faster than AS2 as much of the improvement in performance and the virtual machine is on iteration speed increases such as loops (i.e. pixel loop).

Advertisements

Tweener Robert Penner Easing Equation Visualization Cheat Sheets

Zeh has posted an excellent cheat sheet for Tweener Easing Penner equations based on the original cheat sheet by Nate Chatellier. This is a very nice visual lookup to see what easing equation you want to use on your motion animations. Usually it is a trial-error approach without some visualization like this unless you were going to Robert Penner’s site visualization tool. It is not specific to Tweener in visualizing the Penner easing equations as many kits use them such as Fuse (AS2), TweenLite, Animation Package etc.

View the large demo here

Download from Tweener on Google Code

Download the source

View Robert Penner’s Easing Visualization Tool

Thanks Zeh/Nate one again and always Robert Penner!

Vector Wars Update: HD Video for Flash9 and Silverlight



Flash Performance and HD (H.264) Video Progress

Fast performance is needed for good video combined with interactivity especially. Quickly comparing AS2 to AS3 shows that AS3 and the AVM2 virtual machine in Flash9 is much faster. If you are combining Flash and video especially when it is time to go HD, you will need performance.

Compare AS2 to AS3 rendering: AS2 & AS3 Note the as2 item only has 10000 pixel operations, the as3 item has 50,000 and is considerably more usable, in fact the AS2 version isn’t usable since it is slow. [source] This is a good test because it checks the movement of every pixel on the screen which leads me to video performance.

Flash HD in the FLV format is pretty nice looking today in Flash9 with AS3. But to keep up there are more formats and a move to support H.264/MPEG-4 AVC I am sure for online TV/Movie market.

The reason to bring up performance and video is because the video battle on the web is heating up even more between Adobe and Microsoft with Flash Video and Silverlight Video. The recent released info regarding HD on Flash is to combat Silverlight HD video that is looking pretty good.

Silverlight Performance and HD Video VC-1

View this Halo 3 video in Silverlight HD:

View it->

Recently Silverlight came out last week officially on Wednesday, really the 1.0 release is just a video and javascript release but when you view the videos below you will see that they have a short coup de tat on Flash video at the moment in terms of HD quality video.

Silverlight also has been pushing TV on the internet. See these samples that are actual TV on web pages already using Silverlight.

  1. Home Shopping Network Online
  2. ET Emmy’s coverage
  3. They also have WWE and MLB.com baseball already using Silverlight video.
  4. Netflix online video uses Silverlight

Flash video is cool and you can do great fun things like neave.tv (flash 8) and high definition FLV video in flash9 with AS3 and on occasion AS2 if it is just video with full screen flash but when you are talking TV and movie quality, it is still up for grabs but both are looking good especially Silverlight and that Halo 3 video.

Flash video is updating to H.264

Tinic Uro reports the beta player of flash that has HD now. Currently the FLV format is the web leader (you tube, Google video) but is not as good quality as Silverlight until they update to H.264 at least in terms of standards video. This is probably 6 months off?

Video literally changed in the internet in the last two weeks with the Flash announcements and Silverlight launching. I am surprised at Silverlight’s adoption rate and partnerships so far with the 1.0 offering which lacks many programming tools, but I see why on the video front why they did it. When 1.1 comes out it has all the programmer toys, and it will attract a lot of attention I think. It is all about performance and HD video right now.

Yes yes… Quicktime has done HD/H.264 video for a while but Quicktime is just a video format really, it will be able to be used in the new flash video as well as other formats but it doesn’t have the interactive platform behind it like Flash and Silverlight have.

The difference is the choice of format and standards. Silverlight with VC-1 and Flash going to H.264 codec. They are largely competing standards that are both “HD”.

The good news is we have competition to bring really high quality video to the web, the mashups with HD video in interactive games, demos, advertising etc will be very fun.

Flash9 AS3 Market Now At 91% Penetration Availability

Adobe released the numbers on Flash9/AS3 capabilities/penetration of the market and AS3 is now even more ready for primetime at 91%.

AS3 and the new Flash9 virtual machine AVM2 is ready to be used and performance gained from it. When you develop AS2 in Flash9 it uses the older, clunkier AVM1, this does not give the performance benefit that using AS3 and the new virtual machine will give you.

Currently AS3 apps are about 6-10 times faster than the AS3 counterparts. Also, AS2 code and the community is drying up and moving to AS3 where it is much more capable and ready for better performance of things like pseudo 3d engines like papervision, sandy and away3d and it can handle all types of binary manipulation and sockets that lead to very coo apps like emulators, pdf creators, compression tools, and of course with more performance, better games and more room for more content.

Flash9 AS3 penetration as of June 1, 2007:

Flash Player 6 Flash Player 7 Flash Player 8 Flash Player 9
Mature Markets1 99.3% 99.3% 98.5% 90.3%
US/Canada 99.4% 99.4% 98.7% 90.5%
Europe2 99.3% 99.1% 98.2% 90.5%
Japan 99.8% 99.8% 99.0% 89.8%
Emerging Markets3 97.7% 97.6% 94.4% 89.4%

[source]

This is as of June, it is September and most likely due to their projections that we are at 92% which is almost critical mass.

But ok, cmon, is it really worth it in performance?

Well back in late 2006 Carlos Ulloa did some early benchmarking which you can see here. As it turns out AS3 versions of the same effect from AS2 are at minimum 6-20 times faster. What can you do creatively with that much more power?

AS2 & AS3 Versions of the star test to compare AS2 to AS3 so you can judge for yourself [source]

My recommendation, get your AS3 learn on. See what BIG SPACESHIP thinks of AS3 here. Big Spaceship just recently got added to thefwa’s hall of fame – one of three agencies all time.

For future reuse of code and knowledge and to get the most performance out of our creative projects, it is the best option NOW. Your competitors are moving to it and already making more compelling offerings, how long are you going to hack it with just AS2 skills?

A good place to start learning is at kirupa and the senocular posts.

The trajectory of the flash player penetration should complete to around 95-96% by teh end of the year judging by the current player and flash 8 penetration trajectory, Flash9 has gone from 85% to 91% in 3 months, it slows down to about 2% a month once it hits 90% until a critical mass of around 98% for each player. Essentially by 2008 everyone should have moved on to AS3, if you haven’t, well…

FlashDevelop 3 Alpha Released: Best Flash IDE Gets an Upgrade

Photo Sharing and Video Hosting at PhotobucketObligatory posting on FlashDevelop3 Alpha being released.  To people like me alpha means “old school” released.  I am ready to use it today!

Download it here

If you are not using FlashDevelop for your coding portion of your flash application, you might be missing out.  There is a reason some of the best actionscripters (ahem actioncoders) use it.

Fluid Flash Layouts

Photo Sharing and Video Hosting at PhotobucketHere’s how to do Fluid Flash layouts quickly without Flex.

Fluid layouts are not new but fhard to come by documentation.

To create a Flash fluid layout, essentially what you do is create a Flash movie that is set to 100% in HTML/SWFObject/XHTML but scaling set to noScale. Then in the flash script you position each element as needed on a resize handler.

This works great for applications built in Flash that will need to scale such as an IDE, a word processor, an image editing programs or programs that might have floating panels and tools.

1) Make your flash movie HTML scaled to the browser:

If you are using deconcept’s excellent SWFObject (be sure to download it) then see this sample for the HTML embedding with js swfobject.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">

<head>

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

<title>blipgames.com</title>

<script type="text/javascript" src="swfobject.js"></script>

<style type="text/css"> /* hide from ie on mac */

 html {

  height: 100%;

  overflow: hidden;

 }

#flashcontent {

  height: 100%;

 }

 /* end hide */

body {

  height: 100%;

  margin: 0;

  padding: 0;

}

</style>

</head>

<body>

 <div id="flashcontent">

  <strong>You need to upgrade your Flash Player</strong>

 </div>

<script type="text/javascript">

  // <![CDATA[

var so = new SWFObject("myMovie.swf", "myMovie", "100%", "100%", "8", "#efefef");

  so.addParam("scale", "noscale");

  so.write("flashcontent");

// ]]>

 </script>

</body>

</html>

2) Inside your flash code:

AS2

Stage.align = "TL"; // Top left align the stage

Stage.scaleMode = "noScale"; // Set noScale for containing stage.// CREATE A LISTENER OBJECTsizeListener = new Object();// ATTACH THE ON RESIZE EVENT TO THE LISTENERsizeListener.onResize = function()

{

// In the event call a method to move the items

// or, call each item here to be sized

movieToBeCentered._x = Stage.width/2;

movieToBeCentered._y = Stage.height/2;

movieToBeCentered._width = Stage.width * .5;

// add other movie clips to position...

};

// ADD THE LISTENER

Stage.addListener(sizeListener);

AS3

package

{

 import flash.display.*;

 import flash.events.*;

 import flash.utils.*;

public class AppMain extends MovieClip

 {

  public function AppMain()

  {

   stage.addEventListener(Event.RESIZE,onResize);

   stage.align = "TL"; // Top left align the stage

   stage.scaleMode = "noScale";  // items do not stretch to fit

  }
  public function onResize(event:Event)

  {

   // In the event call a method to move the items

   // or, call each item here to be sized

   movieToBeCentered.x = stage.width/2;

   movieToBeCentered.y = stage.height/2;

   movieToBeCentered.scaleX = stage.width * .5;

   // ... etc

  }

 }

}

This allows you to see easily how you can make a fluid layout in flash that scales with more of an alignment lock rather than the whole movie scaling.

A demo and toolkit will be available soon with an easy class library soon. For now you can go here to get a demo and source from the http://www.tutorio.com/tutorial/liquid-flash-layout site.