Building Touch Interfaces for Windows Phones, Part 4

10 Comments January 20, 2011

The first three articles in this series presented three different ways to respond to touch input in Windows phone apps: mouse events, Touch.FrameReported events, and manipulation events. In this, the fourth and final installment, we’ll discuss a means for processing touch input that trumps all three – namely, the GestureListener class in the Silverlight for Windows Phone Toolkit.

In order to use the GestureListener class, you’ll first need to download and install the toolkit. Then, in order to use GestureListener in an app, you’ll need to add a reference to the assembly named System.Phone.Controls.Toolkit to the project. You’ll find the assembly in your development machine’s “\Program Files\Microsoft SDKs\Windows Phone\v7.0\Toolkit\Nov10\Bin” folder. (The “Nov10” segment of the path will vary depending on which version of the toolkit you download and install.)

The mechanisms discussed in the previous articles support touch input, but they don’t feature general support for gestures. A “gesture” is an action or series of actions performed with one or more fingers to convey commands to an application. There is no formal standard for gestures, but certain gestures have become so common on phones and other devices with touch-screens that they are de facto standards. “Standard” gestures include, but are not limited to:

  • Tap
  • Double-tap
  • Tap-and-hold
  • Drag or pan
  • Flick
  • Pinch

The toolkit’s GestureListener class, which, along with the companion GestureService class, is built on top of Windows Phone 7’s XNA Framework, supports all of these gestures. The API is event-driven, meaning you don’t have to do much more than register a handler for, say, Flick events, in order to know when a flick has occurred. And the events are fired on a per-UI-element basis, so you frequently don’t have to do any hit testing to figure out which element was targeted by the gesture.

To use GestureListener, you attach an instance of it to a UI element and register handlers for the events you’re interested in. Those events include:

  • GestureBegin and GestureCompleted, which fire at the beginning and end of every gesture
  • Tap, which fires when a UI element is tapped
  • DoubleTap, which fires after a Tap event when a UI element is double-tapped
  • Hold, which fires when a UI element is touched and the finger remains on the element without moving for approximately one second
  • DragStarted, DragDelta, and DragCompleted, which fire as a finger moves across the screen
  • Flick, which fires just before a DragCompleted event if the finger was still moving when it left the screen
  • PinchStarted, PinchDelta, and PinchCompleted, which fire as two fingers in contact with the screen move relative to each other

Each of these events carries with it all the information you need to respond to gestures, without making any assumptions about how you might interpret a given gesture. For example, Pinch events provide to you information regarding the distance between fingers, which is useful if you’re implementing pinch-zooms, as well as information regarding the angle of rotation between the fingers, which is useful for using two fingers to rotate elements on the screen.

If a picture’s worth a thousand words, then a code sample ought to be worth at least a hundred. Here’s a very simple example of GestureListener at work – one that allows the user to move a rectangle with a finger:

 

// MainPage.xaml

<Rectangle Width="100" Height="100" Fill="Red">

    <toolkit:GestureService.GestureListener>

        <toolkit:GestureListener DragDelta="OnDragDelta"

            GestureBegin="OnGestureBegin" GestureCompleted="OnGestureCompleted" />

    </toolkit:GestureService.GestureListener>

    <Rectangle.RenderTransform>

        <TranslateTransform />

    </Rectangle.RenderTransform>

</Rectangle>

 

// MainPage.xaml.cs

private void OnGestureBegin(object sender, GestureEventArgs e)

{

    Rectangle rect = sender as Rectangle;

    rect.Tag = rect.Fill; // Save the original fill color

    rect.Fill = new SolidColorBrush(Colors.Yellow);

}

 

private void OnDragDelta(object sender, DragDeltaGestureEventArgs e)

{

    Rectangle rect = sender as Rectangle;

    TranslateTransform transform = rect.RenderTransform as TranslateTransform;

 

    // Move the rectangle

    transform.X += e.HorizontalChange;

    transform.Y += e.VerticalChange;

}

 

private void OnGestureCompleted(object sender, GestureEventArgs e)

{

    Rectangle rect = sender as Rectangle;

    rect.Fill = rect.Tag as Brush;

}

 

The <toolkit:GestureService.GestureListener> element assigns a GestureListener object to the Rectangle. (“toolkit” is an XML namespace prefix that equates to “clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone.Controls.Toolkit”.) The <tookit:GestureListener> element creates the GestureListener instance and registers handlers for three events: DragDelta, GestureBegin, and GestureCompleted. The latter two event handlers turn the rectangle to yellow and then back to red. I could have used the DragStarted and DragCompleted events instead, but I wanted to rectangle to turn yellow the moment it was touched rather than after it began to move.

Here’s a slightly more sophisticated example. In the previous article, I presented a sample that allowed the user to pan a panoramic image horizontally. That sample also used animation and animation easing to continue panning if the finger was still moving when it broke contact with the screen. I modified the app to use GestureListener events instead of manipulation events. For good measure, I replaced the panoramic image with another from a recent trip to Dubai (see below), and I added logic to center the image when it’s double-tapped.

screen

Pertinent code and XAML are reproduced below. In the XAML, you can see that a GestureListener is attached to the Image and handlers are registered for GestureListener’s DragDelta, Flick, and DoubleTap events. The DragDelta handler simply moves the image by an amount that equals the finger’s travel in the horizontal direction, while the Flick handler launches an animation to continue the motion. In the sample built around manipulation events, recall that we checked the IsInertial and FinalVelocities.LinearVelocity properties of the ManipulationCompletedEventArgs to determine how much (if any) inertia to apply. With GestureListener, it’s a bit simpler. As the finger traverses the screen, GestureListener fires DragDelta events, and when the finger leaves the screen, GestureListener fires a DragCompleted event. Moreover, it precedes DragCompleted with a Flick event IF the finger was still moving. No Flick event, no inertia. Couldn’t get much easier than that.

 

// MainPage.xaml

<Grid x:Name="ContentPanel" Width="2048" Height="480">

    <Image Source="Dubai.jpg" Width="2048" Height="480" CacheMode="BitmapCache">

        <toolkit:GestureService.GestureListener>

            <toolkit:GestureListener DragDelta="OnDragDelta"

                Flick="OnFlick" DoubleTap="OnDoubleTap" />

        </toolkit:GestureService.GestureListener>

        <Image.RenderTransform>

            <TranslateTransform x:Name="PanTransform"/>

        </Image.RenderTransform>

        <Image.Resources>

            <Storyboard x:Name="Pan">

                <DoubleAnimation x:Name="PanAnimation"

                    Storyboard.TargetName="PanTransform"

                    Storyboard.TargetProperty="X" Duration="0:0:1">

                    <DoubleAnimation.EasingFunction>

                        <CircleEase EasingMode="EaseOut" />

                    </DoubleAnimation.EasingFunction>

                </DoubleAnimation>

            </Storyboard>

        </Image.Resources>

    </Image>

</Grid>

 

// MainPage.xaml.cs

private void OnDragDelta(object sender, DragDeltaGestureEventArgs e)

{

    Image photo = sender as Image;

    TranslateTransform transform = photo.RenderTransform as TranslateTransform;

 

    // Compute the new X component of the transform

    double x = transform.X + e.HorizontalChange;

 

    if (x > 0.0)

        x = 0.0;

    else if (x < Application.Current.Host.Content.ActualHeight - photo.ActualWidth)

        x = Application.Current.Host.Content.ActualHeight - photo.ActualWidth;

 

    // Apply the computed value to the transform

    transform.X = x;

}

 

private void OnFlick(object sender, FlickGestureEventArgs e)

{

    Image photo = sender as Image;

 

    // Compute the inertial distance to travel

    double dx = e.HorizontalVelocity / 10.0;

    TranslateTransform transform = photo.RenderTransform as TranslateTransform;

 

    double x = transform.X + dx;

 

    if (x > 0.0)

        x = 0.0;

    else if (x < Application.Current.Host.Content.ActualHeight - photo.ActualWidth)

        x = Application.Current.Host.Content.ActualHeight - photo.ActualWidth;

 

    // Apply the computed value to the animation

    PanAnimation.To = x;

 

    // Trigger the animation

    Pan.Begin();

}

 

private void OnDoubleTap(object sender, GestureEventArgs e)

{

    Image photo = sender as Image;

    TranslateTransform transform = photo.RenderTransform as TranslateTransform;

 

    // Compute distance to travel to center the image

    double x = (Application.Current.Host.Content.ActualHeight - photo.ActualWidth) / 2.0;

 

    if (x != transform.X)

    {

        // Apply the computed value to the animation

        PanAnimation.To = x;

 

        // Trigger the animation

        Pan.Begin();

    }

}

 

The DoubleTap handler uses the same animation that the Flick handler uses, but it uses it to scroll horizontally to the image’s center. Try it: run the app on your phone (or in the emulator), and double-tap the image.

My final example is one that uses a variety of gestures to provide a rich and interactive UI. It allows you to translate, scale, and rotate a penguin. It takes advantage of the fact that pinch gestures are first-class citizens in the GestureListener universe, and that pinch gestures generate information that can be used for scaling and rotating. The PinchGestureEventArgs accompanying a PinchDelta event features a handy DistanceRatio property that provides a measure of the distance between fingers – perfect for scaling. It also includes a TotalAngleDelta property that you can use for rotating. It’s up to you to interpret these values and decide whether you want to scale, rotate, or both.

In my example, you’re either in “zoom mode” or “rotate mode” at any given time, and you can switch modes using hold and tap gestures. By default, making a pinch gesture with two fingers scales the penguin up and down. But if you touch the penguin and hold your finger there for a second or so, the app switches to rotate mode and the penguin becomes red-tinted to indicate as much (see below). Now pinch gestures rotate the penguin, and when you’re finished rotating, you can give the penguin a gentle tap to go back to zoom mode. The red tint will go away as visual confirmation that pinches are once again interpreted as zoom commands. At any time, you can double-tap anywhere on the screen to reset the scaling, translation, and rotation factors.

screen1

The logic that makes all this work is both simple and straightforward. The app uses two GestureListeners: one attached to the LayoutRoot Grid listening for pinch and double-tap gestures emanating from anywhere on the screen, and another attached to the Grid that contains the penguin Canvas listening for drag, hold, and tap gestures emanating from the penguin. Translating, scaling, and rotating are accomplished by manipulating a CompositeTransform attached to the same Grid.

 

// MainPage.xaml

<Grid x:Name="LayoutRoot" Background="#FF101010">

    <toolkit:GestureService.GestureListener>

        <toolkit:GestureListener PinchDelta="OnPinchDelta"

            PinchStarted="OnPinchStarted" DoubleTap="OnDoubleTap" />

    </toolkit:GestureService.GestureListener>

      .

      .

      .

    <Grid x:Name="ContentPanel" Grid.Row="1" Margin="12,0,12,0"

        RenderTransformOrigin="0.5,0.5">

        <toolkit:GestureService.GestureListener>

            <toolkit:GestureListener DragDelta="OnDragDelta" Hold="OnHold" Tap="OnTap" />

        </toolkit:GestureService.GestureListener>

        <Grid.RenderTransform>

            <CompositeTransform x:Name="PenguinTransform" />

        </Grid.RenderTransform>

        <Canvas x:Name="PenguinCanvas" Width="340" Height="322">

            <Ellipse Fill="#FF050505" Stroke="#FF000000" x:Name="OuterBody"

                Width="243" Height="286" Canvas.Left="46" Canvas.Top="21"/>

              .

              .

              .

        </Canvas>

        <Canvas x:Name="HighlightCanvas" Opacity="0.0" Width="340" Height="322">

            <Ellipse Fill="Red" Width="243" Height="286" Canvas.Left="46" Canvas.Top="21"/>

              .

              .

              .

        </Canvas>

    </Grid>

</Grid>

 

// MainPage.xaml.cs

public partial class MainPage : PhoneApplicationPage

{

    private double _cx, _cy;

    private double _angle;

    private bool _rotate = false;

 

    // Constructor

    public MainPage()

    {

        InitializeComponent();

    }

 

    private void OnPinchStarted(object sender, PinchStartedGestureEventArgs e)

    {

        // Record the current scaling and rotation values

        _cx = PenguinTransform.ScaleX;

        _cy = PenguinTransform.ScaleY;

        _angle = PenguinTransform.Rotation;

    }

 

    private void OnPinchDelta(object sender, PinchGestureEventArgs e)

    {

        if (_rotate) // Rotate the penguin

        {

            PenguinTransform.Rotation = _angle + e.TotalAngleDelta;

        }

        else // Scale the penguin

        {

            // Compute new scaling factors

            double cx = _cx * e.DistanceRatio;

            double cy = _cy * e.DistanceRatio;

 

            // If they're between 1.0 and 4.0, inclusive, apply them

            if (cx >= 1.0 && cx <= 4.0 && cy >= 1.0 && cy <= 4.0)

            {

                PenguinTransform.ScaleX = cx;

                PenguinTransform.ScaleY = cy;

            }

        }

    }

 

    private void OnDragDelta(object sender, DragDeltaGestureEventArgs e)

    {

        // Move the penguin

        PenguinTransform.TranslateX += e.HorizontalChange;

        PenguinTransform.TranslateY += e.VerticalChange;

    }

 

    private void OnDoubleTap(object sender, GestureEventArgs e)

    {

        // Reset when a double-tap occurs

        PenguinTransform.ScaleX = PenguinTransform.ScaleY = 1.0;

        PenguinTransform.TranslateX = PenguinTransform.TranslateY = 0.0;

        PenguinTransform.Rotation = 0.0;

        HighlightCanvas.Opacity = 0.0;

        _rotate = false;

    }

 

    private void OnHold(object sender, GestureEventArgs e)

    {

        // Tint the penguin red and switch to rotate mode

         HighlightCanvas.Opacity = 0.4;

        _rotate = true;

    }

 

    private void OnTap(object sender, GestureEventArgs e)

    {

        // Remove the tint and switch to zoom mode

        HighlightCanvas.Opacity = 0.0;

        _rotate = false;

    }

}

 

If you’d like to see the application in action, you can download the source code for this project and the other projects featured in this article.

Of all the touch APIs featured in Silverlight for Windows Phone, GestureListener offers the most bang for the buck. It’s not perfect: it can only fire gesture events for one UI element at a time, so if you want to build a multi-touch UI that supports the simultaneous manipulation of two or more UI elements, you’ll need to use Touch.FrameReported events. But for most touch applications, GestureListener should prove more than enough to get the job done.


10 Comments

  • Gravatar Image
    Twitter Trackbacks for Jeff Prosise's Blog : Building Touch Interfaces for Windows Phones, Part 4 [wintellect.com] on Topsy.com January 20, 2011 3:15 PM

    PingBack from http://topsy.com/www.wintellect.com/CS/blogs/jprosise/archive/2011/01/20/building-touch-interfaces-for-windows-phones-part-4.aspx?utm_source=pingback&utm_campaign=L2

  • Gravatar Image
    Rick Engle January 21, 2011 9:49 AM

    Hey Jeff, fantastic article, very useful info!
    Rick

  • Gravatar Image
    Rick Engle January 22, 2011 10:18 AM

    Hi Jeff,
    When trapping the OnDragDelta event for the GestureListener, it’s easy to drag an item like an Image out of view. I like how in the WP7 picture viewer that you can only move a stretched image about a 3rd of the way out of view and when you let go it bounces back to the original position.

    Any idea how to restrict a drag operation so you cant drag a UIELement competely out of view?

    Rick

  • Gravatar Image
    jprosise January 22, 2011 10:39 AM

    Rick, it's just a matter of adding logic to constrain the values you assign to the transform. My picture viewer sample does that to prevent the user from scrolling too far to the left or right. Obviously, the logic is very application-specific.

  • Gravatar Image
    c0x3y February 8, 2011 3:25 PM

    Thanks for writing the article it has been a great resource. I tried using the same approach as you but the animation does not to be as fluid and smooth as the built in picture viewer.

    Any ideas how to get it smoother?

  • Gravatar Image
    Jon April 22, 2011 3:39 PM

    It would be nice if there was a way to know when a Hold has ended. I have code that repeats as long as the user keeps a finger on an icon. But no way to stop it when the user is no longer touching the screen.

  • Gravatar Image
    jprosise April 22, 2011 5:18 PM

    The GestureCompleted event should do the trick.

  • Gravatar Image
    Charles May 4, 2011 11:23 AM

    I have a question.

    how to do this thing? I use native photo viewer application on WP7. It can drag picture and display some blank, then the photo will auto move to edge.

    Do you have any idea about this effect?

  • Gravatar Image
    pratik June 22, 2011 4:01 AM

    Hi Jeff,

    If the initial scale is less 1 (i.e. ScaleX and ScaleY

  • Gravatar Image
    Om January 14, 2012 4:20 PM

    Hi Jeff,
    Thanks for this nice article.
    I downloaded the code and I am looking for a way to set limit to the area where penguine can be dragged. e.g When I run the sample code, I can drag the penguine over the StackPanel and also outside of the screen.

    Is there a way to not let the elements drag over say StackPanel and not letting them to go out of Visible Screen area?

    You can answer here as well if you prefer: http://stackoverflow.com/questions/8859955/drag-image-in-a-window-phone-7-silverlight-app

    Thanks a lot.

    -Om

Have a Comment?

Archives

Tags