Rick Barraza
Silverlight UX Development
Friday November 13, 2009
The Secret Behind the Page Flip Technique
Two Flavors of Page Flip
The Page Flip technique has been around for several years in Flash. However, not all implementations are equal. Usually, the premium page flip components that are sold to developers are substantially different than the free page flip tutorials you might find online. The biggest difference is in the range of motion of the page corner being flipped. Most free page flip tutorials have a fixed radius motion path instead of a wider range of motion. In other words, no matter where you drag your mouse, the page corner will stay fixed to a curve as it travels from one side to the other. The math behind this technique is rather straightforward and variations of its implementation are found all over the place. The problem, though, is that the premium page flip technique allows a far greater range of motion while flipping the page (as shown above), which yields a much richer experience. However, nobody ever talks about how this one is done!The Page Flip in Silverlight
So what we're going to do today is explain the mathematics behind a premium page flip experience engineered in Silverlight. The Silverlight implementation shown above was solved using trigonometry, clever masking and a little "smoke and mirrors" magic for the shadows. As with many of my blog posts, the effect can usually be split into the mechanics of the effect(the math part) and the presentation (the rendering tricks). Since Silverlight and Flash handle the mechanics of masking, clipping and rendering transforms differently, this tutorial is aimed primarily at Silverlight developers. However, the mathematical secrets behind the Page Flip technique are applicable to interactive developers in any language and this tutorial focuses primarily on explaining the rotational mechanics of the solution.Step 1: Follow the Mouse, Constrain the Corner Point
[ Download BlogPageFlipStudy01.zip ]
PageFlipStudy01 sets up the framework for what we need to do. Please review it carefully, since the next 3 studies are built on top of each other. If you open it up, you'll notice the solution only has MainPage.xaml and a Dot.xaml control I created to visualize the solution variables so far. If you look over the MainPage.xaml.cs code, you'll see that the loaded() function mainly initializes variables and setups up the Mouse event handlers. All the logic happens in the CompositeTarget.Rendering() animation loop. Here are the key variables, as illustrated above:
M: The position of the Mouse as set by the MouseMove() event F: The mouse Follower that allows us to gracefully ease toward the raw mouse position C: The Constraint point which defines where the page corner will eventually be R1: The Radius point on the constraining circumference of the arc that defines the maximum page width allowed. SC: Spine Center SB: Spine Bottom EB: Edge Bottom
And here is the main constraint logic in the animation loop:
The most important variable at this point is our calculated constraint (dot C above). I never use the positional data from the Mouse events directly, but usually create a Follower variable that constantly eases toward the raw mouse position. However, for the page flip effect to work, the Constraint point that positions the page corner needs to respect the maximum width of the page. In other words, the line SB > C can never be longer than the line SB > EB.
Step 2: Adding the Second Constraint and Defining the Critical Triangle
[ Download BlogPageFlipStudy02.zip ]
In PageFlipStudy02, we need to add an additional radius constraint R2 to define the maximum distance the page corner can be from the spine top (dot ST above). In other words, the line ST > C cannot be longer than the line that would connect ST and R2. Here is the additional constraint code for the second constraining radius:
Now that the corner point C has been constrained between the two radii, we can get to the heart of the solution; the critical triangle. The critical triangle is creating by taking the bisector of line C > EB, which is marked by dot T0. Shoot out a line perpendicular to the bisector toward the page bottom, and you get T2. You then close up the triangle with T1. This critical triangle will be used to control both the location and rotation of the Page object and the Rectangle that can serve as your clipping boundary. Here is the logic that calculates the tangent:
You may want to experiment with this skeleton and then experiment with the original page flip solution at the start of this article and see if you figure out how the properties of the critical triangle are used in a rendered solution.
Step 3: Adding the Page and Rotating the Corner
[ Download BlogPageFlipStudy03.zip ]
The biggest change in PageFlipStudy03 is the addition of a pageBack.xaml control that contains the page graphic we want to display. If you look at the control, you'll notice that both the positional and rotational point of the control is located at its bottom left corner. This makes sense, since when the page is finally flipped, the corner you're trying to flip is its bottom left corner. In MainPage.xaml.cs, the positioning and rotating of the page is calculated like so:
The important thing to realize is that the angle defined by T2 to C defines the amount of rotation needed for the page control if you anchor it to the constrained corner point C. You can determine the angle in Radians by performing a Math.atan2() operation on the variables shown above. Once you have this tangent value, you can set the page control's rotation angle to this value after converting it from Radians to Degrees (by multiplying by Math.PI/180.0), and the Page being flipped will always intersect with T2.
Step 4: Defining the Clipping Region
[ Download BlogPageFlipStudy04.zip ]
Up to this point, you have all the logic required to constrain and rotate the page properly. The last essential step, however, is defining a clipping region that will also be animated to define what portion of the page should be visible during the flip. Since the focus of this tutorial is to understand the underlying math of the technique, though, I'll keep the rectangle visible so you can see exactly what is going on.
The red Rectangle above that can give me the points for a clipping path was created in such a way that it's renderTransformOrigin lines up to wherever T2 is. You calculate its position and rotation like so:
Inevitably, though, when you want to implement this solution, the red Rectangle object used above will need to be converted to a collection of Path figure segments within a PathGeometry that defines the clipping region for the page control (it's easier doing it this way than trying to perform a complex rotation on a clipping path, believe me). The good news is that Silverlight's GeneralTransform object makes this very easy:
That's it! The purple region above where the blue page and the red rectangle overlap defines how much of the page is visible as your flip it. Using the calculated points above, you can convert this rectangle to a proper pathGeometry that clips the page control, making it look like the page is being revealed by being flipped over. This trigonometry defines the mathematical underpinnings of the premium page flip effect. From here, you can use this structure to add your shadows and add multiple pages. Flipping from left to right is simply a reversal of the geometry and flipping from the top is a basic inversion. However, I'll leave it for you to add the shading and multiple page logic. Have fun!
Rick Barraza
twitter.com/rickbarraza
November 13, 2009
Thursday September 10, 2009
Rendering Vector Fields in Silverlight
Silverlight Advanced Rendering Techniques: Part 3
[ Download VectorFieldProject.zip ]Now that we have explored how to render particles directly into a WriteableBitmap (see: Advanced Render Techniques with WriteableBitmaps), we can introduce the concept of a vector field and see how these two techniques work together.
Static versus Dynamic Vector Fields
A Vector Field is basically a map that stores information about how much change in the x and y direction is represented by any given point in a map. With a static vector field, you may calculate these values once (as we will do using a regular photo as our source) and simply use them as the velocity control of other moving elements. In a dynamic vector field, you may also choose to dissipate the actual Vector values in the grid over time using some algorithm (for example, using Jos Stam’s technique for “Real Time Fluid Dyamics”) to get a more liquid effect as each vector bleeds into the vectors around it.However, for this introduction we will only be exploring static vector fields. Once you understand how a vector field works in general and how you can use one to animate moving particles, you may want to explore the more advanced references in the Fluid Dynamics post or follow the link above.
Step 1: Setting up a Photo Source
If you look at the code for the demo above, you will see that I’m defining a WriteableBitmap called bmpSource that will render whatever image is currently being shown in the SourceContainer canvas. This lets me switch out the photos I will use as my inputs easily by simply selecting which child photo I make visible. One other gotcha I ran into is that you need to wait until at least one of the source images is fully Opened (not just having the Image object Loaded). Once the ImageOpened() event has been fired, you can grab a reference to the photo in XAML, render it (or in this case, its visual parent) into a WriteableBitmap, and start accessing its pixel data directly.Step 2: Converting the Photo to Greyscale
In the setupGreyscale() function, we loop through each pixel in bmpSource and calculate its greyscale value. We then draw this greyscale pixel into the WriteableBitmap called bmpGreyscale so we can display the result on screen.Since Pixel color data is stored as a 32-bit integer in WriteableBitmaps, we usually need to extract the Alpha, Red, Green and Blue values represented in this one integer and break them into their individual byte values. This makes it easier to manipulate them or average them out to calculate their greyscale value as we're doing below:
You should also notice that since I already have the pixel value broken down into its color components, I go ahead and create a C# Color object and store it in the array sourceColors[] for later. This will come in handy in our render loop, and I’d rather calculate and store the values once then continuously perform the byte operations every time I need to color values in different formats.
Step 3: Edge Detection and Calculating Vectors
Now here comes the tricky part. The reason we converted the source photo into greyscale was to run an edge detection algorithm against the pixels. Here is the code:
There is a fantastic openFrameworks tutorial that describes the hows and whys of this technique and you can find it here http://wiki.openframeworks.cc/index.php?title=Making_VectorFields_from_images and it definitely served as both a guide and inspiration for this project in Siverlight. In summary, though, what we’re doing is this: For every non-edge pixel (starting at the second row and column), we are looking at the 3x3 grid of pixels that surround it to calculate a Point that represents the directional change at that point. In other words, we look at the 3 pixel row above and below our current pixel to assign a value to the amount of vertical change (the y value) influencing that pixel. We then look at the 3 pixel column to the left and right of our current pixel to assign a value to the amount of horizontal change ( the x value) influencing that pixel. In our code, we put these two directional change values into a Point and then store that Point into an array.
This array of points that analyzes the greyscale changes in X and Y for each pixel and converts it into a Point that represents directional data is our Vector Field.
While the Vector Field now stores directional change values for every point of the original photo, it would be too expensive (and conceptually challenging) to represent this directly. So for the static vector field display in our demo, we are taking a simplified approach which we are also using for our animated particle cloud. We create a simplified grid of particles (in our case, spaced 3 pixels apart) and query the Vector Field for the direction vector for only that particle's position. If the vector values for either x or y are greater than our threshold, we will drop that particle down and color it based on the original source photo. Using a variation of the line() function we’ve been using in the previous post, we then draw a line into bmpVectorField from our particle’s starting point to where it would be after applying its vector (see code below).
Step 4: Use Vector Field for Particle Control
Up until now, we have used a photo as our source data, duplicated it into a greyscale image to calculate its edges, and converted this greyscale variance for each pixel into a vector field that represents directional change for any point we query. On top of this, we have created a simplified grid of particles for those vectors whose values are higher than our given threshold values and then we've drawn these velocities into a static image.
So to create the particle cloud, we simply animate this collection of particles along their vectors over time in an animation loop (instead of all at once as we did in bmpVectorField) and add any extra special effects we can think of.
We’re using the same fade and blur techniques we introduced in the previous post to give us a greater sense of motion and color. Every particle knows its original starting point, and then moves along its vector path for the length of its life. Each particle has a random life limit, so they’re not all resetting at the same time. We are also using the horizontal position of the mouse to add additional directional noise to each particle, for some visual variety. The vertical position of the mouse affects how quickly the particle trails fade to black. I wanted to add a little more eye candy, so you can toggle glow on or off and also put in a couple lines of code to push any particles away from the mouse. All of this code is found in the main updateVectorField() function that gets called from CompositeTarget.Rendering().
In Conclusion
We’ve covered a fair amount of ground and presented a scattering of various design solutions during these past three tutorials. We started off demonstrating a finished Fluid Dynamics simulation in Silverlight and then slowly built upon the new WriteableBitmap to learn some advanced rendering techniques. Finally, we spent some time today looking at Vector Fields and how they can be used with our new found particle rendering code.Along the way, I’ve tried to incorporate references, links and techniques from various languages, including Flash, Processing and OpenFrameworks. I’m a big believer in experience designers being multi- lingual and studying the deeper patterns that exist across all of these languages. Most of the byte manipulations for color, the procedural easing alogrithms, the line functions, and even the trigonometry for pushing particles arounds are all techniques that are common across multiple Interactive Design languages and once you understand the principles of them, you can usually translate them easily from one language to the other. This is a far more important skill to have as an experience designer than simply memorizing a solution approach specific to a single design language. Being multilingual and seeing the patterns behind interactive programming solutions is at least one way to help ensure you against the future.
With the release of Silverlight 3 and its powerful new features (including the WriteableBitmap we've been exploring in this series), I feel it has finally demonstrated a maturity and richness that deserves to be taken seriously by the interactive design community. I'm happy I've had a chance to hopefully show off some of the great experiences that are now possible in Sivlerlight 3, and I can't wait to see what the future holds for Silveright 4.
Rick Barraza
twitter.com/rickbarraza
September 10, 2009
Sunday August 30, 2009
Advanced Render Techniques with WriteableBitmaps
Silverlight Advanced Rendering Techniques: Part 2
In the first part of this series on advanced rendering techniques ( Fluid Dymamics in Silverlight), I talked about generating writeableBitmaps, using a source image as a brush, and a very basic blur and fade technique. For this tutorial, we’re going to start with a base project and progressively improve it to support line and graphic rendering, dynamic coloring, a much better blur and fade implementation, and finally, use the dynamic writeableBitmap as the distortion input for a Pixel Shader. So get ready, we have a ton of ground to cover…
Step 1: Setting up the Base Project
Here is the base project (dynamically creating a writeableBitmap and drawing pixels based on the mouse position) you should understand before continuing.Download Base Solution Here
While we are using a lot of the same techniques from the previous post, there are a few changes that I've highlighted in the snapshot. Namely, we are no longer using a source image in the render() function of a writeableBitmap, but we are now manipulating the pixels directly. Here is a snapshot of the important code that illustrates this:
Step 2: From Pixels to Evaporating Color
Obviously, there are a few design issues if we only set a single pixel per mouseMove() event. Most importantly, it's just boring, and that’s the biggest problem right there. This looks much better.Download Line Solution Here
We can increase the coolness factor if instead of drawing a single pixel, we track the previous and current mouse positions and draw a line between those two points. There are at least three ways we can do this. We have already shown how it’s possible to create a Shape Geometry and render it directly into the writeableBitmap. In fact, this is exactly what we’re doing here to get the fade to black effect:
So with a little bit of tweaking, you could imagine a drawLine() function that simply creates a Line object and renders it into the WriteableBitmap directly. However, in the spirit of "setting pixels directly", I decided on an alternate approach that would be more "teach you to fish" instead of "giving you a fish".
I'm a sucker for a good line drawing algorithm. Setting the correct pixels between two moving points is actually a bit more tricky than you may first imagine. So if you are trying to set pixels directly between two points, it might be good to familiarize yourself with the Bresenham algorithm. You can find a great discussion with code samples and illustrations at http://www.cs.unc.edu/~mcmillan/comp136/Lecture6/Lines.html. If you look at the drawLine() function in the code for the above project, you'll see a modified implementation of this algorithm and an updated setPixel() function that takes advantage of some speed optimizations by precalculating the pixel index.
Finally, I should also highlight the changes in the mouse move event you may have noticed in the code snippet above. In the base project, we are always setting the pixel to white. However for this line project, I wanted to color the lines based on position of the mouse. The blue value increases as you move along the x axis, red value increases as you move along the y axis,and the green value is just the inverse of the blue value. I do basic bit shifting to mash them together into a 32-bit color value I'll use later in the setPixel() function to color the pixels that form the current line.
Step 3: Adding Glow and Fade
We start turning up the polish by adding the glow effect we introduced in the previous post with a couple more enhancements.Download Glow Solution Here
First, I should mentioned some differences in the XAML. Take a look at the markup:
The Ellipse named blueBlur is a new visual element I am also rendering at the end of each line I draw, to add a little more punch to the glow. Notice that I named its gradient stops so I can programatically change the color of the glow to match the color of the line. Also, while the previous post on rendering techniques showed a very basic Image structure for acheiving a blur and fade (there was only one Image with a Blur effect on the stage and one writeableBitmap in code that was both bound to it and rendering it), here we are stacking two Images with different blurs on top of each other. One will be the line you actually draw and the other is its Glow effect. These two images will also use two separate semi-opaque black rectangles to fade out at different speeds.
If you look at the ColorBlur project's code and compare it to the Line project's code, you'll see the biggest difference is the addition of a second writeableBitmap named bmpBlur which is set as the source for the nested fadeBase image that serves as the background image. Also, the rendering loop has been expanded to wire up the multiple bitmaps and images appropriately. The key snippet and comments are shown below:
There is one technique burried up in that code above that I should call out. We've been fading to black in most of the examples by constantly drawing a semi-opaque black rectangle over our image. But for this more advanced glow and fade technique where we have nested images and transparency is essential, we need a fade to transparent technique. So in the code loop above, we loop through each pixel, isolate the last 8 bits of the color (the alpha) and drop it down progressively while preserving the R, G, and B values.
Step 4: Writeable Bitmap as a Pixel Shader Displacement
While Steps 1 through 3 were focused on building an engaging experience using writeableBitmaps, it's only a small jump to make this workable as a Pixel Shader displacement input as well.Download Pixel Shader Solution Here
The pixel shader I'm using here is a very simple displacement shader and you can find it in the project download. Adding it to the project in Step 3 only requires a couple changes, but I do want to point out a couple work arounds:
The way I usually setup a WriteableBitmap to serve as one of the inputs for a PixelShader is simply naming the ImageBrush in XAML and setting it's ImageSource to the WriteableBitmap in code. The only gotcha I want to call out is for some reason, I needed to continually reset the ImageSource of the ImageBrush whenever I would invalidate the source WriteableBitmap.
You can find a lot of good references on Pixel Shaders in WPF and now in Silverlight. Make sure to check out Rene Schulte's Blog (again) and Nikola Mihaylov's Blog for some great tutorials on using Shaders in Silverlight and WPF. If you're just starting out with Pixel Shaders, then I would suggest first reading Greg Schechter's Series of articles, which is what first read when I was getting started.
While there are a lot of ways to interact with Pixel Shaders, I have found that this writeableBitmap approach is a good hack to know and keep in your bag of tricks when more conventional methods won't work (or would be prohibitively harder to implement). For example, when you want a Pixel Shader to have multiple inputs based on touch in Silverlight, using a WriteableBitmap in this way may mean the difference between a high performance, elegant experience and a less impressive experience.
The Warp Mona Lisa experience at the top of this lesson was created using Steps 1 through 4 and custom graphics. If you make anything cool with this, drop me a line and let me know. In the next part of our Advanced Rendering Techniques in Silverlight, we should be returning to the subject of Vector Fields,so stay tuned!
Rick Barraza
twitter.com/rickbarraza
Thursday August 13, 2009
Fluid Dynamics in Silverlight
It's been crazy busy here at Cynergy so I haven't written much in a while. But after a few months of Microsoft Surface work and a side detour into Flash territory, I finally had some free time to jump back into Silverlight 3 and I’m loving the new WriteableBitmap object.
My plan is to create a series of tutorials concerning more advanced rendering techniques in Silverlight 3. However, I wanted to start today with sharing a Silverlight port of a project I’ve been building on the Surface, a real time fluid dynamics effect based off of Jos Stam's paper and the amazing work of Mehmet Akten in OpenFrameworks. I'm a big fan of his and you can find reference links to his original implementation at the end of the article.
Dynamic Visuals Versus Byte Arrays and Why It Matters
Usually, code for interactive experiences generally involves dynamically attaching a User Control or a Visual to the screen in an animation loop of some kind ( for example, see my previous tutorial
Sketching in C#).
However, with Silverlight 3 and Windows 7 providing a multi-touch experience, the amount of dynamic elements on the stage based on multiple inputs can quickly get out of hand. This problem is made even more obvious with the massive multi-user opportunity that Microsoft Surface provides with over 50 simultaneous user inputs! Now coding a single cursor particle emitter is rather straight forward
(See Project Rosetta Tutorial). Both Flash and Silverlight allow thousands of elements animating across the screen right out of the box. However, try having 50 fingers on a Surface all shooting out hundreds of particles per second each, and you’ll quickly bring the best machines to a crawl. Even with the limited number of multiple contact points that current touch screens allow, this will also become a very quick issue for Silverlight interactive animations as well. It might be a good time to look into what all this WriteableBitmap stuff is about.
The fluid simulation you see above takes an entirely different approach. Instead of creating, attaching, animating and deleting independent User Controls dozens of times a second, all the visual logic is calculated in memory in a byte array and the only the final calculation is rendered into a WriteableBitmap, which we then display on screen. Long story short, it is significantly faster for computers to do some array math and shoot out a single picture than for us to force the program to literally juggle thousands and thousands of Visuals at a time, with all the unnecessary overhead this would require.
“So Anyways, Let’s Get Down to the Nitty Gritty”
I’m not going to cover the basics of WriteableBitmaps since I think there are already amazing resources on Silverlight 3 and WriteableBitmaps online. For example, check out Rene Schulte and his fantastic series of posts at www.kodierer.blogspot.com.
I would also strongly recommend you check out Pete Brown’s experiments with Byte Streams and other
awesomeness here. Both of these guys always share amazing stuff and push me to do more.
However, there is a rendering technique that I’m using in the Fluid Dynamics demo above that I would like to share and haven't seen it talked about too much in Silverlight 3. If you click on the stage and enter Vector Lines mode, you’ll see what I’m talking about. The elusive fade to black. While I can’t share the full Fluid Dynamics code externally yet, I will walk you through a much more direct example that highlights this particular render technique.
On dynamic elements, this one is a no brainer. You simply drop the opacity of your dynamic UIElements until they’re transparent and then delete or respawn them to conserve memory. However, applying a fade to black on a dynamically generated image in Silverlight is not nearly as straight forward.
Too up the ante some, I’ll also show you how to do the much cooler “blur and fade” technique, since you end up getting both with just a little more work. Hey, I have to give you at least something cool, right? So here’s how it goes.
The MainPage.XAML is pretty straightforward:
Notice that the IMAGE object named imgStamp is pointing to a 30x30 pixel image of a star I created in Photoshop. I’m going to use this as my stamp (or brush). However, I can hide this particular IMAGE instance that is in XAML by collapsing its visibility and it won't effect anything. Also, its pretty important to note that the image that will be bound to my WriteableBitmap in code, imgOut, has a BlurEffect on it. The WriteableBitmap we generate will basically be constantly taking a photo of itself, so the blur will continue to grow if we don't fade it back down.
The actual code isn't too long, but there are some tricky things going on here. I’ll discuss them in the comments inline:
And here is what this simple snippet of code get's you:
Again, this is just a fast and easier demo of one of the new render techniques that are now available with WriteableBitmap and Silverlight 3. But there are a lot of things we can do to make this much, much better. How about not using an image brush at all but drawing lines directly into the byte array? What about expanding this to support multi-touch? What if want to control the blur or fade to transparent? And what exactly is a vector field, anyway? These are some of the issues we'll be presenting next.
Rick Barraza
twitter.com/rickbarraza
August 14, 2009
Appendix: Tackle it Yourself
I'm not going to lie to you, wrapping your head around the fluid dynamics solution isn't going to be the easiest thing you've done. You may be able to port a straightforward solution to C# quickly enough, but I learned so much by jumping into it and really working out the guts of the process, I strongly suggest anyone interested in this type of thing spend a weekend and do it for themselves. Here is what I used and where I suggest you get started if you're up for it:
"Real Time Fluid Dynamics for Games" by Jos Stam
The paper that seems to have started it all that Jos Stam presented in 2003 at the Game Developers Conference. You can find the
PDF here.
However, I wouldn't have even heard of this if I hadn't been blown away by this
great video that Mehmet Akten posted of his Fluid Dynamics implementation and render techniques in Open Frameworks.
Open Frameworks, by the way, is a whole post in itself! However, if you've been playing in C# and WPF but want to really see what the Microsoft stack can do with C++ and OpenGL, you can drink from the fire hose here:
www.openframeworks.cc.
Finally, coming from a Flash background, I found Eugene's ActionScript 3 Port of Memo's Open Frameworks code a great help. You can find his
AS3 rewrite here.
Hope those links help. They definitely got me where I wanted to be and have been a fountain of great ideas the past couple weeks. As I mentioned above, I'll be spending the next couple posts discussing what I learned on the actual rendering side of things, so stay tuned. Have fun!
Tuesday April 14, 2009
Toward a New Experience Architecture
For almost two years now, I've been working under the title Senior Experience Architect and have enjoyed great and challenging conversations with many of my peers regarding our rather odd arrival into the world of traditional software development. Being both designer and developer, we tend to end up getting shot at by both sides if we don’t watch ourselves closely, since our existence tends to challenge the current status quo. Read more...
Thursday March 26, 2009
EMOTIV Thought Control, WPF and Magic
I first got a glimpse of the Emotiv headset in a small video shown at MIX08. The panel was on the future of Human / Computer Interfaces and Dave Wolf was discussing my work with Project Maestro and the Wii. But when I saw the brief clip of the Emotiv headset and it's mind control interface, I was instantly smitten and wanted to learn more.
Read more...
Tuesday March 24, 2009
Sketching in C#: Legos versus Clay
There is a messy quality to clay. An ability to have happy accidents, what I like to call reckless creativity, that is harder to duplicate when working in a Lego world. So one of the first things I did when coming into the Microsoft UX world was to figure out how to have fun with Silverlight. How to code in such a way that I'm not thinking of Design Patterns or OOP or reusability, but how to capture that same fast, sketch like quality I used to enjoy when coding dynamic Flash interactions... Read more...
Thursday March 19, 2009
Microsoft Surface Physics and SXSW
Microsoft made some great anouncements at MIX09 and I'm very glad that we can start talking about them publicly. However, it has also been a busy couple of months for Cynergy Labs, and I will be blogging a series of tutorials over the coming weeks that will cover some of our discoveries, UX techniques and results. Here is a video of some of the Surface explorations Microsoft asked us to show last week at the South By Southwest Interactive Festival and the MIX09 conference. Read more...
Sunday April 20, 2008
Bitmap Images and Particle Engines with Silverlight
Needless to say, its been one hectic year so far! Between Project Maestro at SXSW, Teaching at MIX08, Winning the PhizzPop nationals, Artists in Residency and getting ready for the Web 2.0 Expo, this is the first chance I've had to push up a new post! But this is a great time for Silverlight development and I'll be keeping you updated with some amazing things just over the horizon!
First things first, we're going to talk about Streaming data and particles today. Here's the project the code snippet comes from:
Instructions: Move the slider to increase the particle buzz. Upload any 24 bit .BMP file. All binary streams are processed client side, there is no server manipulation required.
Here is the main loop that takes the BMP file from a client side dialog box, parses it on the client side to extract the important informatio, and populate an array of custom color objects that will be used to draw the tiles:
[...]
using System.Windows.Media.Imaging;
using System.IO;
using System.Windows.Browser;
using System.Windows.Resources;
drawSqaure[] tiles;
double totalCol = 0;
double totalRow = 0;
double imageWidth = 0;
double imageHeight = 0;
[...]
private void OpenBMP()
{
OpenFileDialog ofd = new OpenFileDialog();
if (ofd.ShowDialog() == DialogResult.OK)
{
Stream stream = ofd.SelectedFile.OpenRead();
output = ofd.SelectedFile.Name;
// Find out how big the file is. Create a Byte array to hold it.
// Read the values you need from the Bitmap data stream...
int nBytes = Convert.ToInt32(stream.Length);
byte[] ByteArray = new byte[nBytes];
int nBytesRead = stream.Read(ByteArray, 0, nBytes);
stream.Close();
// BMP Header files start with B and M in Ascii
if ((ByteArray[0] == 66) && (ByteArray[1] == 77))
{
output += " [24bit BMP File] ";
}
else
{
output = " ERROR: File is not a BMP";
txtOut.Text = output;
return;
}
// BMP files will pad the stream with leading zeros if the
// image width in pixels is not divisble by 4
imageWidth = (double)(convert4Bytes(ByteArray[18], ByteArray[19],
ByteArray[20], ByteArray[21]));
double rowPad = imageWidth % 4;
imageHeight = (double)(convert4Bytes(ByteArray[22], ByteArray[23],
ByteArray[24], ByteArray[25]));
int dataOffset = convert4Bytes(ByteArray[10], ByteArray[11],
ByteArray[12], ByteArray[13]);
int bitDepth = convert2Bytes(ByteArray[28], ByteArray[29]);
int imageDataSize = convert4Bytes(ByteArray[34], ByteArray[35],
ByteArray[36], ByteArray[37]);
if ( (imageWidth > 600) || (imageHeight > 600) ) {
output = "ERROR: Image too large. Please select another file.";
txtOut.Text = output;
return;
}
if (bitDepth != 24)
{
output += "ERROR: Color depth must be set at 24 bits.";
txtOut.Text = output;
return;
}
// I will be calculating out the color average of 10x10 pixel
// squares from the image and storing them in a drawSqaure object.
// A 24bit BMP uses 3 bytes per pixel for the color. This code
// can easily be expanded to support other bit depths as well.
int resolution = 10;
int bytesPerPixel = 3;
totalCol = Math.Floor( imageWidth / resolution);
totalRow = Math.Floor( imageHeight / resolution);
int totalTiles = (int)(totalCol * totalRow);
tiles = new drawSqaure[totalTiles];
double rowOffset = 0;
double pointer = 0;
int index = 0;
// Here is the important part. I have now stored the Bitmap
// image's stream in my byte array and I have created an array
// of 'drawSquare' objects to be populated with the average
// of the 10 x 10 pixel square color values I will next calculate.
for (int r = 0; r < totalRow; r++)
{
// BMP files send their pixel data from the bottom left corner
// of the image UP. Also, while their pixel data is traditional
// 0-255 per RGB color, the ORDER is reversed, so it comes in
// BGR, not RGB.
rowOffset = 54 + (r*(resolution *
((imageWidth * bytesPerPixel) + rowPad) ));
for (int c = 0; c < totalCol; c++)
{
pointer = rowOffset + (c * resolution * bytesPerPixel);
drawSqaure d = new drawSqaure();
int b = 0;
int g = 0;
int red = 0;
for (int i = 0; i < resolution; i++)
{
for (int y = 0; y < resolution; y++)
{
int calc = Convert.ToInt32(pointer +
(i*((imageWidth*bytesPerPixel)+rowPad)) + (y*3));
b += (int)ByteArray[calc];
g += (int)ByteArray[calc+1];
red += (int )ByteArray[calc + 2];
}
}
// Since the above nested for loops will sample a 10 x 10 pixel
// square, I need to divide the sum of each Color by 100 to
// get its color average per channel.
d.red = red / 100;
d.blue = b / 100;
d.green = g / 100;
tiles[index] = d;
index++;
}
}
txtOut.Text = output;
ByteArray = null;
buildUI();
}
}
The drawSquare object simply holds the averaged color values per canvas tile we will add to the screen:
public class drawSqaure
{
public int red { get; set; }
public int blue { get; set; }
public int green { get; set; }
}
And here is what the buildUI() loop looks like:
private void buildUI()
{
int index = 0;
int nodeSize = 2;
nodes.Children.Clear();
for (int r = 0; r < totalRow; r++)
{
for (int c = 0; c < totalCol; c++)
{
node n = new node();
n.SetValue(NameProperty, "n_" + c.ToString() + "_" + r.ToString());
n.X = origin.X - (totalCol*(nodeSize*.5)) + (c * nodeSize);
n.Y = origin.Y - 30 + (totalRow*(nodeSize*.5)) - (r * nodeSize);
n.anchor = new Point(n.X, n.Y);
n.vX = 0;
n.vY = 0;
drawSqaure d = tiles[index];
index++;
n.setColor(Color.FromArgb(0xFF, Convert.ToByte(d.red),
Convert.ToByte(d.green), Convert.ToByte(d.blue)));
nodes.Children.Add(n);
}
}
index = 0;
tiles = null;
}
private Int32 convert4Bytes(byte a, byte b, byte c, byte d)
{ return ( (d<<24) | (c<<16) | (b<<8) | (a)); }
private Int32 convert2Bytes(byte a, byte b)
{ return ((b << 8) | (a)); }
The node user control is empty except for a 2x2 rectangle that we can change the fill with through its setColor() function as well as the X and Y public properties that I've gone over many times before.
Finally, the animation engine looks something like this:
void sb_Completed(object sender, EventArgs e)
{
foreach (node n in nodes.Children)
{
n.vX += ((double)r.Next(100) - 50.0)/ 5.0;
n.vY += ((double)r.Next(100) - 50.0) / 5.0;
n.vX *= buzz;
n.vY *= buzz;
n.X += n.vX;
n.Y += n.vY;
n.X += (n.anchor.X - n.X) * .07;
n.Y += (n.anchor.Y - n.Y) * .07;
}
sb.Begin();
}
Of course, I added a couple more bells and whistles in mine, such as controlling the value of the buzz variables above by the position of the slider, but I'll leave you to customize your project anyway you want. The above outlines the essentials required to get it up and running.
Hope you like it. Drop me a note if you do anything cool with this...
Friday January 11, 2008
Connecting to the Wii Control with WPF
Using Brian Peek's Wiimote Library, we'll show you how to get quickly up and running with a Wii Control, WPF and some Infrared emitters to create a multi-point tracking system like we used in Project Maestro.
Read more...
Thursday December 13, 2007
DD#03: Custom Animation Loops
Sorry for the delay in posts, we were offline competing in the Microsoft PhizzPop L.A. challenge all last week. The good news is we won, so we're off to SXSW in March to compete in the Finals!
I'm a big fan of creating your own custom animations and have been pretty vocal about it in both the Flex space and now in Silverlight/WPF. So today we'll look at a basic custom animation process I've been using and explore a little bit the 3D logic in the animation loop.
Here’s what we’re building today:
Instructions: No interaction required. Just sit back and enjoy :)
We'll start off as we normally do, with an empty canvas in our in main Page.XAML to hold the elements we'll be attaching in code:
<Canvas x:Name="nodes" Width="480" Height="480" Canvas.Left="11" Canvas.Top="11" >
<Canvas.Clip>
<RectangleGeometry Rect="0, 0, 480, 480"/>
</Canvas.Clip>
</Canvas>
We'll also have a custom user control, named nodeName.xaml, that looks like this:
<Canvas xmlns="http://schemas.microsoft.com/client/2007"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
Width="100"
Height="20"
>
<TextBlock Foreground="#FFFFFF" Width="100" Height="20" Text="silverlight" FontFamily="Arial"
FontSize="9" RenderTransformOrigin="0,0">
<TextBlock.RenderTransform>
<TransformGroup>
<ScaleTransform x:Name="scale" ScaleX="1" ScaleY="1"/>
<RotateTransform x:Name="rotate" Angle="-60"/>
</TransformGroup>
</TextBlock.RenderTransform>
</TextBlock>
</Canvas>
And has a code behind file that looks like this:
namespace SilverlightDirtyDozen
{
public class nodeName : Control
{
private double _FibX;
private double _FibY;
private double _objDepth;
private double _perspective_ratio;
private double _rotation;
public double FibX
{
get { return _FibX; }
set { _FibX = value; }
}
public double FibY
{
get { return _FibY; }
set { _FibY = value; }
}
public double objDepth
{
get { return _objDepth; }
set { _objDepth = value; }
}
public double perspective_ratio
{
get { return _perspective_ratio; }
set { _perspective_ratio = value; }
}
private ScaleTransform st;
private RotateTransform rt;
FrameworkElement lroot;
public double rotation
{
get { return rt.Angle; }
set { rt.Angle = value; }
}
public double scale
{
get { return st.ScaleX; }
set { st.ScaleX = value; st.ScaleY = st.ScaleX; }
}
public nodeName()
{
System.IO.Stream s = this.GetType().Assembly.GetManifestResourceStream("SilverlightDirtyDozen.nodeName.xaml");
lroot = this.InitializeFromXaml(new System.IO.StreamReader(s).ReadToEnd());
st = lroot.FindName("scale") as ScaleTransform;
rt = lroot.FindName("rotate") as RotateTransform;
}
}
}
The most important thing to note is that I'm using two named transforms ('scale' and 'rotate') in the XAML and grabbing a reference to them in the code behind file to manipulate them through code.
Those are the only additional elements we need. Everything else will be handled in the main code behind file of Page.xaml.
I should probably also apologize right now for getting a little math geeky on the actual animation engine since I was bored and wanted to port over a basic ActionScript 1.0 3D engine to C#/Silverlight. So, lets just jump into it...
We'll break this into three parts; the initialization, the setup, and the looper. Here is the initialization stuff:
public partial class dd03 : Canvas
{
double gRatio = 1 / 1.618033989;
double gAngle;
double rad = 50;
double rGrowth = 1.01;
double toRadians = Math.PI / 180;
double depth = 1000;
Point origin = new Point(240, 240);
int totalNames = 50;
int screenDepth = 150;
int z_axis_rotation = 1;
int y_axis_rotation = 0;
int x_axis_rotation = 0;
double convertToRadians = Math.PI / 180;
double rot = 0;
Storyboard looper;
int counter = 0;
public void Page_Loaded(object o, EventArgs e)
{
// Required to initialize variables
InitializeComponent();
looper = new Storyboard();
looper.SetValue<string>(Storyboard.NameProperty, "looper");
this.Resources.Add(looper);
looper.Completed += new EventHandler(looper_Completed);
setupNodes();
}
Again, creating an animation loop is very easy, faking 3D is the messy part. Most of the variables are for creating a 3D environment and placing nodes in a Fibonacci sunflower pattern, not creating a dynamic animation engine. To create a basic engine, you just need to create a Storyboard object (named looper in this code), initialize it, name it, and add it to the page resources. Once we call looper.Begin(), it will play once (at the established framerate, which is why I use a Storyboard instead of a Timer event) and then call it's Completed event Handler. All you gotta do is tell the looper_Completed event handler to looper.Begin() again, and you have yourself an infinite loop that will play at the correct frame rate. We'll talk about kill switches in a subsequent Dirty Dozen, if you want to not leak processor so egregiously or use a custom drag drop animation with drag or elastic..
So before telling our Storyboard object named looper to begin, you'll notice we first setup our elements on our empty node canvas in the setupNodes() function:
private void setupNodes()
{
gAngle = 360 - (360 * gRatio);
for (var x = 0; x < totalNames; x++)
{
nodeName temp = new nodeName();
temp.SetValue<string>(NameProperty, "node_" + x.ToString());
getLoc(temp);
temp.objDepth = -x;
temp.perspective_ratio = screenDepth / (screenDepth - x);
nodes.Children.Add(temp);
}
looper.Begin();
}
private void getLoc(nodeName passObj)
{
rot = rot + gAngle;
rad *= rGrowth;
passObj.FibX = Math.Cos(rot*toRadians)*rad;
passObj.FibY = Math.Sin(rot*toRadians)*rad;
passObj.rotation = rot;
}
Note that these two functions only create the user controls but don't set any rendering values on them, only logical properties we will use to calculate and render later (such as an objDepth, perspective_ratio, FibX, FibY, rotation, etc.) All the rendering changes and placement occur in the looper_completed function, which finally gets started as the last line of setupNodes().
Here is what that function looks like. Remember, it's the Storyboard's completed event handler, so by telling itself to begin again as the last line of code in its function, the function will always loop at the established framerate of your Silverlight movie:
void looper_Completed(object sender, EventArgs e)
{
// this is some fast, ugly code to change the rotations
// after a couple seconds.
if (counter > -1)
{
if (counter > 400)
{
z_axis_rotation = 2;
x_axis_rotation = 1;
counter = -2;
}
counter++;
}
// Here is all the trig for 3D. Never changes between projects...
double sin_x = Math.Sin(x_axis_rotation * convertToRadians);
double cos_x = Math.Cos(x_axis_rotation * convertToRadians);
double sin_y = Math.Sin(y_axis_rotation * convertToRadians);
double cos_y = Math.Cos(y_axis_rotation * convertToRadians);
double sin_z = Math.Sin(z_axis_rotation * convertToRadians);
double cos_z = Math.Cos(z_axis_rotation * convertToRadians);
double left = 0;
double top = 0;
// loop through every node and set their values based on their
// logical position using the trig values from above.
foreach (nodeName n in nodes.Children)
{
// these equations never change between projects either...
double rotatedY = (n.FibY * cos_x) - (n.objDepth * sin_x);
double rotatedDepth = (n.objDepth * cos_x) + (n.FibY * sin_x);
n.FibY = rotatedY;
n.objDepth = rotatedDepth;
double rotatedX = (n.FibX * cos_y) - (n.objDepth * sin_y);
rotatedDepth = (n.objDepth * cos_y) + (n.FibX * sin_y);
n.FibX = rotatedX;
n.objDepth = rotatedDepth;
rotatedX = (n.FibX * cos_z) - (n.FibY * sin_z);
rotatedY = (n.FibY * cos_z) + (n.FibX * sin_z);
// Now we start applying the calculated values with our objects
n.FibX = rotatedX;
n.FibY = rotatedY;
n.perspective_ratio = screenDepth / (screenDepth + n.objDepth);
n.scale = n.perspective_ratio;
n.Opacity = n.perspective_ratio;
n.SetValue<double>(Canvas.ZIndexProperty, n.objDepth);
left = (n.FibX * n.perspective_ratio) + origin.X;
top = (n.FibY * n.perspective_ratio) + origin.Y;
n.SetValue<double>(Canvas.TopProperty, top);
n.SetValue<double>(Canvas.LeftProperty, left);
}
looper.Begin();
}
I'm not going to lie to you. If you haven't played with Trig since High School or been tinkering in Flash, this may be a bit of math to take in, but once you do it a couple times it starts sinking in no problem. I could have just as easily only done this in the loop:
void looper_Completed(object sender, EventArgs e)
{
foreach (nodeName n in nodes.Children)
{
double left = Convert.toDouble(n.getValue(Canvas.LeftProperty)) + 1;
n.setValue<double>(Canvas.LeftProperty, left);
}
looper.Begin();
}
If I just wanted each node to move off the screen to the right, but that experience would have been hilariously boring. The 3D Fibonacci ring is a lot more interesting, so I thought I would share the whole shebang with you.
In summary then, ( after taking all the 3D and Fibonacci math out) to create a custom animation loop:
1. Create a Storyboard variable.
2. In your startup code, initialize it
3. Then name it
4. Add a Completed Event Handler to it (with all your goodie code in there)
5. Add the storyboard to your this.Resources()
6. Call storyboardname.Begin() when you want the engine to start running.
7. To keep the storyboard looping, add another storyboardname.Begin() at
the end of the Completed event handler.
We'll take a little break from animations in the next Dirty Dozen and look at importing and working with custom fonts. Until then, SLapp happy.
Tuesday November 20, 2007
DD#01: Dynamically Attaching Controls in Silverlight
We’ll start with this assumption: You know the basics of XAML, you know how to at least get to the code behind file, and most importantly, you know where to look online for detailed help if you get stuck (‘cause these can go pretty fast ;)
Ready? Here’s what we’re building today:
Instructions: Click on the stage below to add a circle dynamically
Requires Silverlight 1.1
As a quick get your bearings for those of you coming from Flash trying to map up Silverlight to what you already know (Non flashers, ear muff it for a paragraph or so): We’re basically trying to figuring out the equivalent of attaching movieClips dynamically at run time and interacting with them in XAML/C#. Whereas in Flash you’re used to starting off with a single framed timeline and the ability to add ActionScript code to that initial frame, you start off here with a TestPage.xaml file (the visual definition of your stage in markup) and it’s code behind file, TestPage.xaml.cs (All my Sivlerlight 1.1 examples will use C# as the language of choice). Now in Flash, you create a movieClip, export it, and then attach it to the stage with movieClip.attachMovie(). These Flash movieClips come with their own stage and actionscript. In Silverlight, you can use User Controls in a similar way. Like Flash movieClips, Silverlight User Controls comes with their own XAML file (the visuals) and Code Behind file (the script).
So for the Silverlight example above, I created my stage like so:
<Canvas x:Name="parentCanvas"
xmlns="http://schemas.microsoft.com/client/2007"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
Loaded="Page_Loaded"
x:Class="SilverlightDirtyDozen.Page;assembly=ClientBin/SilverlightDirtyDozen.dll"
Width="502"
Height="502"
>
<Canvas.Background>[…]</Canvas.Background>
<Rectangle Width="500" Height="500" Stroke="White" StrokeThickness="10" Canvas.Left="1" Canvas.Top="1">
<Rectangle.Fill>[…]</Rectangle.Fill>
</Rectangle>
<Canvas x:Name="circles" Width="480" Height="480" Canvas.Left="11" Canvas.Top="11" >
<Canvas.Clip>
<RectangleGeometry Rect="0, 0, 480, 480"/>
</Canvas.Clip>
</Canvas>
</Canvas>
The important element is the empty Canvas element toward the bottom named circles. This is where we will add our circle objects when the user clicks on the main stage.
The Rectangle isn’t important as far as what we’re doing here, it’s mainly aesthetic. I also added a clipping region to the empty circles canvas, purely for aesthetic reasons as well. A Clip object is a great, fast way to setup Opacity masks if you want to control render bounds.
Off of this main project in Visual Studio, I add a New Item > Silverlight User Control and name it DropCircle.xaml. I design it like so:
<Canvas xmlns="http://schemas.microsoft.com/client/2007"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
Width="1" Height="1" >
<Ellipse x:Name="ellipse" Width="100" Height="100" Canvas.Left="-50" Canvas.Top="-50" Fill="#ccD5FCFC"
Stroke="#FFFFFF" StrokeThickness="2"/>
<Ellipse Width="6" Height="6" Canvas.Left="-3" Canvas.Top="-3" Fill="#FFFFFF"/>
<TextBlock x:Name="display" Width="100" Canvas.Top="6" Canvas.Left="0" Text="0,0" FontFamily="Arial"
FontWeight="Bold" FontSize="11" Foreground="#FFFFFF"/>
</Canvas>
Notice that for demonstration purposes, I left the Width and Height of the Use Control’s canvas to 1px, though the primary Ellipse object has a 100px diameter. I also cheated a little here and offset the Ellipse by setting its Left and Top properties to negative half its diameter. This will keep thing’s centered around the upper left corner of the Canvas. We’ll get to the TextBlock in a moment.
With just this much done, we can already get instances of DropCircle.xaml to added to the TestPage.xaml stage on left mouse button down by adding the following code to TestPage.xaml.cs:
namespace SilverlightDirtyDozen
{
public partial class Page : Canvas
{
public void Page_Loaded(object o, EventArgs e)
{
// Required to initialize variables
InitializeComponent();
this.MouseLeftButtonDown += new MouseEventHandler(Page_MouseLeftButtonDown);
}
void Page_MouseLeftButtonDown(object sender, MouseEventArgs e)
{
DropCircle dc = new DropCircle();
dc.SetValue(Canvas.TopProperty, e.GetPosition(circles).Y);
dc.SetValue(Canvas.LeftProperty, e.GetPosition(circles).X);
circles.Children.Add(dc);
}
}
}
Since DropCircle and Page share the same namespace and are compiled into the same DLL, I can create new logical instances of DropCircle easily. However, that alone doesn’t visually display the object on the screen. I need to added it to some visual element already in the TestPage (including TestPage itself if I so wanted). In this case, I’m adding each DropCircle to my empty circles canvas I created earlier, so they’re all in one place. The reason for this will become more clear in a subsequent Dirty Dozen lesson.
Using the MouseEventArgs parameter e to get the position on the stage where the mouse clicked is a nice way to dynamically set the TopProperty and LeftProperty of our new object. But what if I also wanted to display that information inside the DropCircle itself? Here is where it gets a little more interesting.
The user control can easily be modified to expose various methods and properties, but the default User Control code needs a little fixing. Here’s how:
FrameworkElement lroot;
TextBlock display;
public DropCircle()
{
System.IO.Stream s = this.GetType().AssemblyGetManifestResourceStream(
"SilverlightDirtyDozen.DropCircle.xaml");
lroot = this.InitializeFromXaml(new System.IO.StreamReader(s).ReadToEnd());
display = lroot.FindName("display") as TextBlock;
}
You need to create a FrameworkElement as a self reference inside the user control if you want the class to be able to interacted with its own named XAML elements. Here, I created a FrameworkElement variable called lroot and had it catch a reference to itself at the point of creation. For any important xaml elements inside the UserControl I want it manipulate, I create variables of the appropriate type and create a reference to them by using lroot.FindName(). In this case, the TextBlock I named display in the XAML file is searched for by name off of the User Control and has a reference created for it and stored in a TextBlock variable I created also named display. I can now manipulate it through code.
Since I now have all the references I need, let’s keep adding to DropCircle.xaml.cs to create a function that will accept a string to display in the TextBlock:
public void setDisplay(string text)
{
display.Text = text;
display.SetValue(Canvas.LeftProperty, -display.ActualWidth/2);
}
Here, we have a public function that can accept a text string and will set and center the text inside by using the ActualWidth of the TextBlock once it’s Text property is changed.
Now, the last thing left to do is call this function from TestPage.xaml.cs when we add the DropDots. The final MouseLeftButton down event looks like this:
void Page_MouseLeftButtonDown(object sender, MouseEventArgs e)
{
DropCircle dc = new DropCircle();
dc.SetValue(Canvas.TopProperty, e.GetPosition(circles).Y);
dc.SetValue(Canvas.LeftProperty, e.GetPosition(circles).X);
string location = e.GetPosition(circles).X.ToString() + " , " + e.GetPosition(circles).Y.ToString();
dc.setDisplay(location);
circles.Children.Add(dc);
}
That should do it. When TestPage.xaml is loaded, you can click on it and add a DropCircle dynamically and have it update its display programatically. This first lesson starts nice and easy, but it forms the initial step of many of the other tricks to come in building rich experiences in Silverlight.
But how do we get these dynamic User Controls to move both internally and as a collection? Dirty Dozen #02 will begin to address that with Dynamic Storyboards next.
The Silverlight Dirty Dozen - An Introduction
A couple years ago when I was teaching ActionScript programming, I came up with a series of lessons called the ActionScript Dirty Dozen. The general idea was thus. Most programming books take a very comprehensive, non biased approach to their subject matter (and well they should!). But many designers take a more holistic, intuitive approach to problem solving and pattern recognition. So once the basics are learned, there’s usually a gap of direction in terms of what to do next.
Now coding has the innate disposition toward precision; a cold, quantifiable efficiency. Design, on the other hand, has a this fuzzy, messy, highly subjective element to it, and many times the practitioners of either craft adopt a similar bent. The Dirty Dozen stems from the later, more designer focused inclination and takes a very personal and subjective approach to teaching user experience code. It takes the position: I may have hundreds of tools at my disposal, and those books may be a great reference for hundreds more, but the tips and tricks I used every day, the techniques that get grimy and gritty with use, are these twelve here...
So the next series of blog entries will present my current Dirty Dozen for Silverlight 1.1 projects. I’ll try and present these Dozen tips and tricks in growing order of complexity and often times they build upon each other.
Official Warning: This is very much a work in progress, if for no other reason than Silverlight 1.1 is still Alpha Bits and things can drastically change before the beta. I don’t know what the shelf life is on these, but right now they’re what I’m using all the time. The warning didn’t scare you away? Yeah, me neither. So let’s get cracking.
Friday November 02, 2007
Flash to Silverlight Project 01: Cynergy Matrix with Editor
Ok, so the first and most basic thing I want to know as an interactive designer is how to create something on the screen and power it with code. So what better way to jump into Silverlight than by duplicating a typical ActionScript project and seeing where the metaphors stand and where they fall. The dots are controls (Silverlight version of MovieClips) that are added on the mouseMove and have a dynamic storyboard attached at creation to move them into the position. Opening the grid editor lets you customize your own string of up to 10 characters and stores the data in a collection of bitkeys per character. Why bitkeys? I'm just a sucker for bitshifting and wanted to see how to do it in C#, being still relatively fresh to .NET 3.0.
Unlike my serious Flex and .NET developer friends, I would say my traditional approach to experience coding is more like my approach to design; I start bold and sloppy and refine with iterations. Doing it with functions and math is pretty much the same thing as doing it with pixels and layout if the language is forgiving enough. C# is much more... exacting than ActionScript 1.0 was, so I still have to figure out how to keep reckless creativity compliant with Silverlight's more rigid code environment.
Silverlight Application Below
UPDATED FOR SILVERLIGHT 2.0 BETA
Click to activate object.
Instructions: Move the mouse to spell out messages with the dots you drop. Open the Grid Editor to build your own message. You can customize a message up to 10 characters long. Build your characters from scratch or use the alphabet templates supplied.
Wednesday October 31, 2007
Lost In Translation
There are a lot of tutorials out there for Silverlight, but many of them fall into two camps: Preaching to the choir or treating interactive designers like code phobic pixel pushers. Now, I’m a designer and a developer, as an apparently silent but influential minority of you are. In design mode, I draw the vision and in development mode I make it happen. But personally, I don’t trust my designs to mere markup tags and WYSIWYG editors. I never have, and I don’t think most people like me have. Give me Notepad over Dreamweaver, any day. It’s how I roll. That’s why Flash was such a revolutionary tool. A single environment that actually understood what we wanted; a place to create beautiful things and bring them to life with a powerful and forgiving language.
Some of us designers have been coding our own dynamic interfaces in Flash long before the .NET platform even existed. So why does Silverlight assume that a strong divide between designer and developer must exist and require two different solutions and work environments?
The party line that ‘Designers do their thing over here, and hand it off for developers to do their stuff over there’ doesn’t hold water with me. Not for what Silverlight is trying to do, which is leap frog into a territory that already has Flash as its dominant species.
What it should be focusing on is aggressive innovation, and aggressive innovation is always the result of magicians mixing form and function in a revolutionary way. This is what Flash allowed and this is why they won five years ago. This is no longer what Flex does, because what gets someone to first place is not always the same thing that will keep them there. But I’m afraid Silverlight is copying the compartmentalized but integrated workflow of Flex instead of the innovative Form & Function miracle that was early Flash.
So, as someone who is already used to working in an environment that lets me be both artist and developer at once, if Blend or Designer don’t let me manage at least C# code directly in them, they’re not going to be powerful enough, period. So the other option for me is to work in Visual Studio directly.
Now Orcas is a powerhouse. As a developer, it gives me everything I want, but I happened to have the time, inclination and past Microsoft experience to go up its rather steep learning curve. But it’s definitely programmer focused and a big pill to swallow for designers wanting to experiment with Silverlight.
Which leads me up to this blog. I’ve spent a long time teaching interactive programming to graphic designers, and an even longer time working in Flash. What I want to do with this blog is present Silverlight and build projects approaching them with all the prejudices and assumptions that a seasoned Flash designer fluent in Actionscript 1.0 would have. If anything, it may serve to document our mindset and how we think a little better. At best, it could be a simplified Rosetta Stone for those Interactive Designers looking to translate their Flash based skill set to this new platform.
