AS3 Real-Time Raytracing

Forrest Briggs throwing down with a real-time raytracer in AS3. Also a C++ OpenGL version sample on the page.

Real-time pixel manipulation in flash is getting faster, but is still probably going to have to be faked in AS3, maybe AS4 will provide us per pixel speeds that Andre Michelle has been harping on since flash 8.5. Native operations can be much faster in that area. AIF might look to change some of that but that is Flash 10.

Here is the code for the as3 raytracer. Read more at laserpirate.

package
{
import flash.display.Sprite;
import flash.display.Bitmap;
import flash.display.BitmapData;
import flash.events.Event;
import flash.utils.getTimer;
import flash.events.MouseEvent;
import flash.text.TextField;
import flash.text.TextFormat;

public class RayTracer extends Sprite
{
private var t:Number;
private var dt:Number = .01;
private var frameTimeTxt:TextField;

public static const BUFFER_WIDTH:int = 160;
public static const BUFFER_HEIGHT:int = 120;
public static const BUFFER_SCALEDDOWN:int = 320 / BUFFER_WIDTH;

public static const HALF_BUFFER_WIDTH:int = BUFFER_WIDTH / 2;
public static const HALF_BUFFER_HEIGHT:int = BUFFER_HEIGHT / 2;

private var outputBitmapData:BitmapData;
private var outputBitmap:Bitmap;

public var FOV:Number = 20;

public var sphereCenterX:Array = [0, 0, 0, 0];
public var sphereCenterY:Array = [0, -.2, .4, 100.5];
public var sphereCenterZ:Array = [4, 4, 4, 10];
public var sphereRadius:Array = [.35, .35, .25, 100];
public var sphereR:Array = [255, 0, 0, 20];
public var sphereG:Array = [0, 150, 0, 20];
public var sphereB:Array = [0, 0, 255, 20];
public var sphereReflects:Array = [false, false, false, true];
public var sphereReflectiveness:Array = [0,0,0,.3];
public var sphere2dX:Array = new Array(sphereCenterX.length);
public var sphere2dY:Array = new Array(sphereCenterX.length);
public var sphere2dR:Array = new Array(sphereCenterX.length);

public var numSpheres = sphereCenterX.length;

var skyR:int = 20;
var skyG:int = 20;
var skyB:int = 20;
var skyColor:int = (skyR< <16) + (skyG<<8) + skyB; var ambientIllumination:Number = .1; var canvas:BlankClip; var theta:Number = 0; var mouseIsDown:Boolean = false; var mouseDownTheta:Number = 0; var mouseDownX:Number = 0; public function RayTracer() { outputBitmapData = new BitmapData(BUFFER_WIDTH, BUFFER_HEIGHT, false); outputBitmap = new Bitmap(outputBitmapData); addChild(outputBitmap); //outputBitmap.smoothing = true; outputBitmap.width= 320; outputBitmap.height = 240; canvas = new BlankClip; addChild(canvas); canvas.buttonMode = true; canvas.useHandCursor = true; frameTimeTxt = new TextField(); frameTimeTxt.defaultTextFormat = new TextFormat("Arial"); frameTimeTxt.x = 8; frameTimeTxt.y = 8; frameTimeTxt.width = 640; frameTimeTxt.textColor = 0xFFFFFF; frameTimeTxt.selectable = false; addChild(frameTimeTxt); t = 0; addEventListener(Event.ENTER_FRAME, update, false, 0, true); canvas.addEventListener(MouseEvent.MOUSE_DOWN, mouseDownHandler); canvas.addEventListener(MouseEvent.MOUSE_UP, mouseUpHandler); } public function mouseDownHandler(e:*):void { mouseIsDown = true; mouseDownX = stage.mouseX; mouseDownTheta = theta; } public function mouseUpHandler(e:*):void { mouseIsDown = false; } public function update(e:*) { // start frame timer and update global time var timer:Number = getTimer(); t += dt; // handle mouse rotation if( mouseIsDown ) theta = mouseDownTheta - .0015 * (stage.mouseX - mouseDownX); theta += dt; // do some funky animation sphereCenterX[0] = .5*Math.sin(theta*5); sphereCenterZ[0] =1 + .5*Math.cos(theta*5); sphereCenterX[1] = .5*Math.sin(theta*5 + 2 * Math.PI / 3); sphereCenterZ[1] = 1 + .5*Math.cos(theta*5 + 2 * Math.PI / 3); sphereCenterX[2] = .5*Math.sin(theta*5 + 4 * Math.PI / 3); sphereCenterZ[2] = 1 + .5*Math.cos(theta*5 + 4 * Math.PI / 3); // reused variables var x:int; var y:int; var i:int; var j:int; var r:int; var g:int; var b:int; var dx:Number; var dy:Number; var rayDirX:Number; var rayDirY:Number; var rayDirZ:Number; var rayDirMag:Number; var reflectRayDirX:Number; var reflectRayDirY:Number; var reflectRayDirZ:Number; var intersectionX:Number; var intersectionY:Number; var intersectionZ:Number; var reflectIntersectionX:Number; var reflectIntersectionY:Number; var reflectIntersectionZ:Number; var rayToSphereCenterX:Number; var rayToSphereCenterY:Number; var rayToSphereCenterZ:Number; var lengthRTSC2:Number; var closestApproach:Number; var halfCord2:Number; var dist:Number; var normalX:Number; var normalY:Number; var normalZ:Number; var normalMag:Number; var illumination:Number; var reflectIllumination:Number; var reflectR:Number; var reflectG:Number; var reflectB:Number; // setup light dir var lightDirX:Number = .3; var lightDirY:Number = -1; var lightDirZ:Number = -.5; var lightDirMag:Number = 1/Math.sqrt(lightDirX*lightDirX +lightDirY*lightDirY +lightDirZ*lightDirZ); lightDirX *= lightDirMag; lightDirY *= lightDirMag; lightDirZ *= lightDirMag; // vars used to in intersection tests var closestIntersectionDist:Number; var closestSphereIndex:int; var reflectClosestSphereIndex:int; // compute screen space bounding circles //canvas.graphics.clear(); //canvas.graphics.lineStyle(1, 0xFF0000, .25); for(i = 0; i < numSpheres; ++i) { sphere2dX[i] = (BUFFER_WIDTH / 2 + FOV * sphereCenterX[i] / sphereCenterZ[i]); sphere2dY[i] = (BUFFER_HEIGHT /2 + FOV * sphereCenterY[i] / sphereCenterZ[i]); sphere2dR[i] = (3 * FOV * sphereRadius[i] / sphereCenterZ[i]); //canvas.graphics.drawCircle(sphere2dX[i]*BUFFER_SCALEDDOWN, sphere2dY[i]*BUFFER_SCALEDDOWN, sphere2dR[i]*BUFFER_SCALEDDOWN); sphere2dR[i] *= sphere2dR[i]; // store the squared value } // write to each pixel outputBitmapData.lock(); for(y = 0; y < BUFFER_HEIGHT; ++y) { for(x = 0; x < BUFFER_WIDTH; ++x) { // compute ray direction rayDirX = x - HALF_BUFFER_WIDTH; rayDirY = y - HALF_BUFFER_HEIGHT; rayDirZ = FOV; rayDirMag = 1/Math.sqrt(rayDirX * rayDirX + rayDirY * rayDirY +rayDirZ * rayDirZ); rayDirX *= rayDirMag; rayDirY *= rayDirMag; rayDirZ *= rayDirMag; /// trace the primary ray /// closestIntersectionDist = Number.POSITIVE_INFINITY; closestSphereIndex = -1 for(i = 0; i < numSpheres; ++i) { // check against screen space bounding circle dx = x - sphere2dX[i]; dy = y - sphere2dY[i]; if( dx * dx + dy * dy > sphere2dR[i] ) continue;

// begin actual ray tracing if its inside the bounding circle

lengthRTSC2 = sphereCenterX[i] * sphereCenterX[i] +
sphereCenterY[i] * sphereCenterY[i] +
sphereCenterZ[i] * sphereCenterZ[i];

closestApproach = sphereCenterX[i] * rayDirX +
sphereCenterY[i] * rayDirY +
sphereCenterZ[i] * rayDirZ;

if( closestApproach < 0 ) // intersection behind the origin continue; halfCord2 = sphereRadius[i] * sphereRadius[i] - lengthRTSC2 + (closestApproach * closestApproach); if( halfCord2 < 0 ) // ray misses the sphere continue; // ray hits the sphere dist = closestApproach - Math.sqrt(halfCord2); if( dist < closestIntersectionDist ) { closestIntersectionDist = dist; closestSphereIndex=i; } } /// end of trace primary ray /// // primary ray doesn't hit anything if( closestSphereIndex == - 1) { outputBitmapData.setPixel(x, y, skyColor); } else // primary ray hits a sphere.. calculate shading, shadow and reflection { // location of ray-sphere intersection intersectionX = rayDirX * closestIntersectionDist; intersectionY = rayDirY * closestIntersectionDist; intersectionZ = rayDirZ * closestIntersectionDist; // sphere normal at intersection point normalX = intersectionX - sphereCenterX[closestSphereIndex]; normalY = intersectionY - sphereCenterY[closestSphereIndex]; normalZ = intersectionZ - sphereCenterZ[closestSphereIndex]; normalX /= sphereRadius[closestSphereIndex]; // could be multiply by precacluated 1/rad normalY /= sphereRadius[closestSphereIndex]; normalZ /= sphereRadius[closestSphereIndex]; // diffuse illumination coef illumination = normalX * lightDirX + normalY * lightDirY + normalZ * lightDirZ; if( illumination < ambientIllumination ) illumination = ambientIllumination; /// trace a shadow ray /// var isInShadow:Boolean = false; for(j = 0; j < numSpheres; ++j) { if( j == closestSphereIndex ) continue; rayToSphereCenterX = sphereCenterX[j] - intersectionX; rayToSphereCenterY = sphereCenterY[j] - intersectionY; rayToSphereCenterZ = sphereCenterZ[j] - intersectionZ; lengthRTSC2 = rayToSphereCenterX * rayToSphereCenterX + rayToSphereCenterY * rayToSphereCenterY + rayToSphereCenterZ * rayToSphereCenterZ; closestApproach = rayToSphereCenterX * lightDirX + rayToSphereCenterY * lightDirY + rayToSphereCenterZ * lightDirZ; if( closestApproach < 0 ) // intersection behind the origin continue; halfCord2 = sphereRadius[j] * sphereRadius[j] - lengthRTSC2 + (closestApproach * closestApproach); if( halfCord2 < 0 ) // ray misses the sphere continue; isInShadow = true; break; } /// end of shadow ray /// if( isInShadow ) illumination *= .5; /// trace reflected ray /// if( sphereReflects[closestSphereIndex] ) { // calculate reflected ray direction var reflectCoef:Number = 2 * (rayDirX * normalX + rayDirY * normalY + rayDirZ * normalZ); reflectRayDirX = rayDirX - normalX * reflectCoef; reflectRayDirY = rayDirY - normalY * reflectCoef; reflectRayDirZ = rayDirZ - normalZ * reflectCoef; closestIntersectionDist = Number.POSITIVE_INFINITY; reflectClosestSphereIndex = -1 for(j = 0; j < numSpheres; ++j) { if( j == closestSphereIndex ) continue; rayToSphereCenterX = sphereCenterX[j] - intersectionX; rayToSphereCenterY = sphereCenterY[j] - intersectionY; rayToSphereCenterZ = sphereCenterZ[j] - intersectionZ; lengthRTSC2 = rayToSphereCenterX * rayToSphereCenterX + rayToSphereCenterY * rayToSphereCenterY + rayToSphereCenterZ * rayToSphereCenterZ; closestApproach = rayToSphereCenterX * reflectRayDirX + rayToSphereCenterY * reflectRayDirY + rayToSphereCenterZ * reflectRayDirZ; if( closestApproach < 0 ) // intersection behind the origin continue; halfCord2 = sphereRadius[j] * sphereRadius[j] - lengthRTSC2 + (closestApproach * closestApproach); if( halfCord2 < 0 ) // ray misses the sphere continue; // ray hits the sphere dist = closestApproach - Math.sqrt(halfCord2); if( dist < closestIntersectionDist ) { closestIntersectionDist = dist; reflectClosestSphereIndex=j; } } // end loop through spheres for reflect ray if( reflectClosestSphereIndex == - 1) // reflected ray misses { r = sphereR[closestSphereIndex] * illumination; g = sphereG[closestSphereIndex] * illumination; b = sphereB[closestSphereIndex] * illumination; } else { //trace("ref hit"); // location of ray-sphere intersection reflectIntersectionX = reflectRayDirX * closestIntersectionDist + intersectionX; reflectIntersectionY = reflectRayDirY * closestIntersectionDist + intersectionY; reflectIntersectionZ = reflectRayDirZ * closestIntersectionDist + intersectionZ; // sphere normal at intersection point normalX = reflectIntersectionX - sphereCenterX[reflectClosestSphereIndex]; normalY = reflectIntersectionY - sphereCenterY[reflectClosestSphereIndex]; normalZ = reflectIntersectionZ - sphereCenterZ[reflectClosestSphereIndex]; normalX /= sphereRadius[reflectClosestSphereIndex]; // could be multiply by precacluated 1/rad normalY /= sphereRadius[reflectClosestSphereIndex]; normalZ /= sphereRadius[reflectClosestSphereIndex]; // diffuse illumination coef reflectIllumination = normalX * lightDirX + normalY * lightDirY + normalZ * lightDirZ; if( reflectIllumination < ambientIllumination ) reflectIllumination = ambientIllumination; r = sphereR[closestSphereIndex] * illumination + .5 * sphereR[reflectClosestSphereIndex] * reflectIllumination; g = sphereG[closestSphereIndex] * illumination + .5 * sphereG[reflectClosestSphereIndex] * reflectIllumination; b = sphereB[closestSphereIndex] * illumination + .5 * sphereB[reflectClosestSphereIndex] * reflectIllumination; if( r > 255 ) r = 255;
if( g > 255 ) g = 255;
if( b > 255 ) b = 255;

} // end if reflected ray hits

} /// end if reflects
else // primary ray doesn’t reflect
{
r = sphereR[closestSphereIndex] * illumination;
g = sphereG[closestSphereIndex] * illumination;
b = sphereB[closestSphereIndex] * illumination;
}

outputBitmapData.setPixel(x, y, (r<<16) + (g<<8) + b); } // end if primary ray hit } // end x loop } // end y loop outputBitmapData.unlock(); // compute FPS var fps:Number = 1.0/((getTimer() - timer) / 1000.0); frameTimeTxt.text = "Drag to rotate. FPS: " + int(fps); } } }[/sourcecode]

AS3 Papervision3D 2.0 and 2D BitmapData Effects are Evolving

Andy Zupko is probably doing some of the coolest / useful work in performance and possible effects combining 2D and 3D. Using 2D BitmapData and papervision 3D it turns out you can create a parallel dimension of coolness that cannot fully exist by themselves.

Papervision 2.0 with these effects and if it is as pluggable as it seems is very good for games that lighting is a key component or effects. Imagine a game that can customize weapons with 2d effects in 3d, or rocket boosters, or fireworks or all kinds of inspiring things like changing the mood or environment such as fog, lighting etc… If you start taling about adding physics to all this it just gets too fun. Effects have always been there and around, but making this possible to have a semi-standard way to do this and if it is pluggable, this can lead to many engine advancements.

I think the PV3d team additions of Tim Knip and Andy Zupko have been very good and zupko era in PV3d has begun. Tim Knip is also very active and helping to really organize the ascollada formats and performance stuff like only drawing what is on screen.

Who needs Hydra now? j/k although having this now in papervision leads me to see a very fun 2008 ahead for Flash, it is also, if as pluggable as it seems, a bit like a shaders kit.

All those older great 2d effects merging into 3d from the good old days (some still going very strong) of praystation, yugop, levitated, neave (great 2d tv effects in neave.tv) , flight404 (moved to processing) and many others. And a new era of zupko [pv3d], mr. doob, unitzeroone [pv3d], fabrice [away3d] and many more a new 2d effects in 3d platform is emerging. This kit for papervision3d by zupko and Hydra is making the future glowing full of bright points, and lots of effect explosions.

Let’s hope papervision3d 2.0 it is released soon and it has zupko’s effects code in there.

 

Flash 10: Hydra and AIF (Adobe Image Foundation) and Hardware Rendering

Adobe has announced the release of the inital developer views of AIF and Hydra in “Astro” the next version of the flash player (10).

AIF (Adobe Image Foundation) like AIR (Adobe Integrated Runtime) is a new technology just out of the gate but it does show that Adobe is into innovating and the vector wars. AIR is beta2 and Flash player 9 “moviestar” updates for video are coming along nicely but here we have more news out of MAX in Chicago that AIF is now available.

What is AIF? It is a new imaging and effects technology to help people create their own filters for Flash (blur, drop shadow etc are defaults). Hydra the new language for this is reminiscent of processing.org (if you haven’t been to flight404.com since the 90’s then processing is all it is about there) and Cg from nVidia to write and test shaders. The fact that it is based on GLSL (OpenGL Shading Language) will help it easily port shaders coming in 3d gaming into Flash which is really sweet. The direction slowly is that Flash and maybe Silverlight will become more of gaming platforms and this is a nice point in that direction.

From Adobe here is what AIF is:

Introduction to the Adobe Image Foundation Toolkit Technology Preview

The Adobe Image Foundation (AIF) Toolkit preview release includes a high-performance graphics programming language that Adobe is developing for image processing, codenamed Hydra, and an application to create, compile and preview Hydra filters and effects. The toolkit contains a specification for the Hydra language, several sample filters, and sample images provided by AIF team members. The AIF technology delivers a common image and video processing infrastructure which provides automatic runtime optimization on heterogeneous hardware. It currently ships in After Effects CS3 and will be used in other Adobe products in the future. The next release of Flash Player, codenamed Astro, will leverage Hydra to enable developers to create custom filters, effects and blend modes.

Hydra is a programming language used to implement image processing algorithms in a hardware-independent manner. Some benefits of Hydra include:

  • Familiar syntax that is based on GLSL, which is C-based
  • Allows the same filter to run efficiently on different GPU and CPU architectures, including multi-core and multiprocessor systems in a future update
  • Abstracts out the complexity of executing on heterogeneous hardware
  • Supports 3rd party creation and sharing of filters and effects
  • Delivers excellent image processing performance in Adobe products

Reaction is that this is a strong Adobe direction to move towards more capable technology as in AS3 and Hydra and allow more customizable possibly hardware rendered and accelerated shader like technology for Flash filters. The new AVM2 and AS3 allow for faster processing and pixel based operations that you need for buildng filters and or shaders.

This is pretty interesting, it isn’t full blown hardware rendering which would just be excellent. So far hardware accelerated full screen stretching in Flash 9 Moviestar beta and now filters will have an element of hardware capable rendering, it should help performance. Full hardware acceleration seemingly will not happen in Flash 10 so the 3d engines and new 3d elements from Adobe are all software rendered still. However dual and multi-core processing will help rendering of 3d in flash BUT video cards are more prevalent than dual or multi-core for some time. Basic hardware rendering even for a low bar could greatly change the flash platform.

It is still a while off yet but it was good to know that performance and shaders/filters are getting attention but hardware rendering not just yet for 3d and basic drawing/rendering. One thing is for sure, in 2007 developing interactive for the web is being shook up and changing rapidly.

Here’s a video taken by Aral Balkan of the Astro presentation at MAX