Rick Barraza
Silverlight UX Development
Sunday August 30, 2009
Advanced Render Techniques with WriteableBitmaps
Silverlight Advanced Rendering Techniques: Part 2
In the first part of this series on advanced rendering techniques ( Fluid Dymamics in Silverlight), I talked about generating writeableBitmaps, using a source image as a brush, and a very basic blur and fade technique. For this tutorial, we’re going to start with a base project and progressively improve it to support line and graphic rendering, dynamic coloring, a much better blur and fade implementation, and finally, use the dynamic writeableBitmap as the distortion input for a Pixel Shader. So get ready, we have a ton of ground to cover…
Step 1: Setting up the Base Project
Here is the base project (dynamically creating a writeableBitmap and drawing pixels based on the mouse position) you should understand before continuing.Download Base Solution Here
While we are using a lot of the same techniques from the previous post, there are a few changes that I've highlighted in the snapshot. Namely, we are no longer using a source image in the render() function of a writeableBitmap, but we are now manipulating the pixels directly. Here is a snapshot of the important code that illustrates this:
Step 2: From Pixels to Evaporating Color
Obviously, there are a few design issues if we only set a single pixel per mouseMove() event. Most importantly, it's just boring, and that’s the biggest problem right there. This looks much better.Download Line Solution Here
We can increase the coolness factor if instead of drawing a single pixel, we track the previous and current mouse positions and draw a line between those two points. There are at least three ways we can do this. We have already shown how it’s possible to create a Shape Geometry and render it directly into the writeableBitmap. In fact, this is exactly what we’re doing here to get the fade to black effect:
So with a little bit of tweaking, you could imagine a drawLine() function that simply creates a Line object and renders it into the WriteableBitmap directly. However, in the spirit of "setting pixels directly", I decided on an alternate approach that would be more "teach you to fish" instead of "giving you a fish".
I'm a sucker for a good line drawing algorithm. Setting the correct pixels between two moving points is actually a bit more tricky than you may first imagine. So if you are trying to set pixels directly between two points, it might be good to familiarize yourself with the Bresenham algorithm. You can find a great discussion with code samples and illustrations at http://www.cs.unc.edu/~mcmillan/comp136/Lecture6/Lines.html. If you look at the drawLine() function in the code for the above project, you'll see a modified implementation of this algorithm and an updated setPixel() function that takes advantage of some speed optimizations by precalculating the pixel index.
Finally, I should also highlight the changes in the mouse move event you may have noticed in the code snippet above. In the base project, we are always setting the pixel to white. However for this line project, I wanted to color the lines based on position of the mouse. The blue value increases as you move along the x axis, red value increases as you move along the y axis,and the green value is just the inverse of the blue value. I do basic bit shifting to mash them together into a 32-bit color value I'll use later in the setPixel() function to color the pixels that form the current line.
Step 3: Adding Glow and Fade
We start turning up the polish by adding the glow effect we introduced in the previous post with a couple more enhancements.Download Glow Solution Here
First, I should mentioned some differences in the XAML. Take a look at the markup:
The Ellipse named blueBlur is a new visual element I am also rendering at the end of each line I draw, to add a little more punch to the glow. Notice that I named its gradient stops so I can programatically change the color of the glow to match the color of the line. Also, while the previous post on rendering techniques showed a very basic Image structure for acheiving a blur and fade (there was only one Image with a Blur effect on the stage and one writeableBitmap in code that was both bound to it and rendering it), here we are stacking two Images with different blurs on top of each other. One will be the line you actually draw and the other is its Glow effect. These two images will also use two separate semi-opaque black rectangles to fade out at different speeds.
If you look at the ColorBlur project's code and compare it to the Line project's code, you'll see the biggest difference is the addition of a second writeableBitmap named bmpBlur which is set as the source for the nested fadeBase image that serves as the background image. Also, the rendering loop has been expanded to wire up the multiple bitmaps and images appropriately. The key snippet and comments are shown below:
There is one technique burried up in that code above that I should call out. We've been fading to black in most of the examples by constantly drawing a semi-opaque black rectangle over our image. But for this more advanced glow and fade technique where we have nested images and transparency is essential, we need a fade to transparent technique. So in the code loop above, we loop through each pixel, isolate the last 8 bits of the color (the alpha) and drop it down progressively while preserving the R, G, and B values.
Step 4: Writeable Bitmap as a Pixel Shader Displacement
While Steps 1 through 3 were focused on building an engaging experience using writeableBitmaps, it's only a small jump to make this workable as a Pixel Shader displacement input as well.Download Pixel Shader Solution Here
The pixel shader I'm using here is a very simple displacement shader and you can find it in the project download. Adding it to the project in Step 3 only requires a couple changes, but I do want to point out a couple work arounds:
The way I usually setup a WriteableBitmap to serve as one of the inputs for a PixelShader is simply naming the ImageBrush in XAML and setting it's ImageSource to the WriteableBitmap in code. The only gotcha I want to call out is for some reason, I needed to continually reset the ImageSource of the ImageBrush whenever I would invalidate the source WriteableBitmap.
You can find a lot of good references on Pixel Shaders in WPF and now in Silverlight. Make sure to check out Rene Schulte's Blog (again) and Nikola Mihaylov's Blog for some great tutorials on using Shaders in Silverlight and WPF. If you're just starting out with Pixel Shaders, then I would suggest first reading Greg Schechter's Series of articles, which is what first read when I was getting started.
While there are a lot of ways to interact with Pixel Shaders, I have found that this writeableBitmap approach is a good hack to know and keep in your bag of tricks when more conventional methods won't work (or would be prohibitively harder to implement). For example, when you want a Pixel Shader to have multiple inputs based on touch in Silverlight, using a WriteableBitmap in this way may mean the difference between a high performance, elegant experience and a less impressive experience.
The Warp Mona Lisa experience at the top of this lesson was created using Steps 1 through 4 and custom graphics. If you make anything cool with this, drop me a line and let me know. In the next part of our Advanced Rendering Techniques in Silverlight, we should be returning to the subject of Vector Fields,so stay tuned!
Rick Barraza
twitter.com/rickbarraza
Posted by rickbarraza | Aug 30 2009, 09:56:27 PM PST
Permalink
