Windows Phone 7: Translation, rotation, scaling and the effect of ‘BitmapCache’ on performance

One thing that was pending on my “todo” list from a long time was implementing the ‘Game Of Life’ on Windows Phone 7. I got the basic version running in a couple of hours and I’ve been thinking about improving it since and adding new features to it. Maybe I’ll submit it to the marketplace someday, who knows. I have also been reading about gestures, multi-touch, pinch-to-zoom and related topics and most importantly trying to understand the math behind each of those gestures, and the math is so elegant that it fascinates me!

Game Of Life
Game Of Life is a simple cellular automaton in which you create a pattern in a world made of rectangular grids, start the evolution process and then watch the pattern change as the world evolves. The world is made up of dead cells initially, you create a pattern by selecting a few cells and giving them “life”. The world then evolves according to a set of rules (source: Wikipedia):

1. Any live cell with fewer than two live neighbours dies, as if caused by under-population.
2. Any live cell with two or three live neighbours lives on to the next generation.
3. Any live cell with more than three live neighbours dies, as if by overcrowding.
4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.

It’s a fascinating concept and some very interesting patterns emerge out of it, read more about Game Of Life on Wikipedia here.

Gestures
Gestures like translation, scaling and rotation seemed complex to me at first, but they’re pretty simple once you understand the math that makes the magic happen. I came across two great articles that really dig deep into these concepts. Both articles use the Silverlight Toolkit for Windows Phone 7 and the gesture support that it provides. I was a little surprised to know that the silverlight toolkit actually uses XNA’s touch api’s to detect gestures. These are the articles that you must read:

MSDN Magazine : Touch Gestures on Windows Phone by Charles Petzold. This is the implementation I chose to use in my app Game Of Life. The code is available for download in the article.

Windows Phone 7: Correct pinch zoom in silverlight by Francesco De Vittori. I was using this before I stumbled on Charles’ article :) Also, Charles’ implementation seemed more intuitive to me.

Charles Petzolds’ implementation makes use of Matrix transforms. It took me some time to understand matrix transforms and realize their importance in 2D/3D graphics. The only time I used matrix multiplication was to solve silly linear equations in college. You’ll find many a articles on the internet about matrix transforms but the best resource I found again comes from Charles Petzold. Chapter 22 – From Gestures to Transforms in his free book Programming Windows Phone 7. Download it now if you haven’t already, it’s a great resource.

The two paths had to meet. Using gestures in Game Of Life was the best way to learn them.

In my first implementation of Game Of Life, I used the Manipulation Delta events to support pinch-to-zoom. In this I used the DeltaManipulation.Scale.X and DeltaManipulation.Scale.Y values in the ManipulationDeltaEventArgs class (passed to the ManipulationDelta event handler) to modify the ScaleX and ScaleY values of a Scale transform on the object to be scaled. If that sentence didn’t make sense, don’t worry, this is the easiest and the worst implementation of pinch-to-zoom. The scaling was not smooth or accurate. To understand what a perfect pinch-to-zoom should be read this – Pinch Zooming using XNA on WP7: Getting it right. The article explains this with reference to XNA but the concept remains the same everywhere.

In my second attempt I used the Silverlight Toolkit for Windows Phone and Francesco’s implementation of pinch-to-zoom. It was working pretty nicely with a few quirks here and there. Though the implementation is easy to understand, I felt there were too many variables used for storing state information and the code looked a bit messy, but it worked.

It is while tuning this implementation that the MSDN Magazine article was published. Charles Petzold’s implementation felt intuitive and was much cleaner. The only part I had to figure out was the matrix transforms he used. And that I’ll explain a bit here.

This is what the app looks like,

The rectangular grids are added programmatically to a canvas. Here is the XAML for the canvas,

So during the Drag Delta and Pinch Delta events the transforms under currentTransform are manipulated to achieve the desired effect and once the drag or pinch gestures are completed a function, TransferTransforms(), is called:

void TransferTransforms()
{
    previousTransform.Matrix = Multiply(previousTransform.Matrix, currentTransform.Value);

    //Set current transforms to default values
    scaleTransform.ScaleX = scaleTransform.ScaleY = 1;
    scaleTransform.CenterX = scaleTransform.CenterY = 0;

    rotateTransform.Angle = 0;
    rotateTransform.CenterX = rotateTransform.CenterY = 0;

    translateTransform.X = translateTransform.Y = 0;
}

Matrix Multiply(Matrix A, Matrix B)
{
    return new Matrix(A.M11 * B.M11 + A.M12 * B.M21,
            A.M11 * B.M12 + A.M12 * B.M22,
            A.M21 * B.M11 + A.M22 * B.M21,
            A.M21 * B.M12 + A.M22 * B.M22,
            A.OffsetX * B.M11 + A.OffsetY * B.M21 + B.OffsetX,
            A.OffsetX * B.M12 + A.OffsetY * B.M22 + B.OffsetY);
}

To understand what is happening here we need to look at the XAML first. The RenderTransform on the MainCanvas is a double-barrelled (term used by Charles in his article) transform. Basically, a transform group within a transform group. So the effective transform on MainCanvas is equal to the effect of previousTransform plus the effect of currentTransform. Well, it’s not exactly “plus”. First, previousTransform is applied on MainCanvas to change its state, and then currentTransform is applied on that state to change it further. If you download and look into Charles’ code, you’ll see that during drag delta and pinch delta events, only currentTransform values are modified. Once the drag or pinch gesture is complete, TransferTransform() is called to “transfer” the values of currentTransform to previousTransform. And this transfer happens by the way of matrix multiplication, which is what the function Mulitply() does. It’s not over yet. After the multiplication the values of currentTransform (i.e scaleTransform, rotateTransform and translateTransform) are reset. This is important because, as we discussed before, the effective transform on MainCanvas is effect of previousTransform “plus” the effect of currentTransform. Transferring the values of currentTransform to previousTransform and then resetting currentTransform keeps the effect same.

CacheMode and BitmapCache
Now I got the translation, scaling and rotation to work correctly, but the performance was not very great. There was a huge lag in pinching and dragging, and this is where BitmapCache comes in. Every UIElement has a property called CacheMode, which can be set to BitmapCache.

In XAML, CacheMode=”BitmapCache”

In code, uiElem.CacheMode = new BitmapCache();

When the CacheMode of a UIElement is set to BitmapCache, a snapshot of the UIElement is taken and is stored in the video memory. So the element is not redrawn every time, instead all the operations are performed on the cached bitmap. This is super fast and is particularly useful when working with transforms on controls. Change the MainCanvas element in the XAML to include CacheMode,

and this will have a dramatic effect on the performance as you will see in the video below.

Please leave a comment if you have anything else to add.

Until next time..