Skeleton Custom Control for Kinect SDK WPF

posted in: Uncategorized | 0

I’ve been using the Microsoft Research SDK for Kinect for a little over a week in my spare hours, it is tonnes of fun and so much more stable then the previous frameworks I had been using. One thing that I think everyone will want to do is show a nice skeleton of up to two people being tracked by the Kinect.


I’ve created a Custom WPF Control ready for you to use or to style to your taste in Blend. It uses András Velvárt Bone Behaviour to create awesome little bones between the tracked joints.

So how do you use it?

Step 1 – Add a reference to your WPF project to the following:

  • Microsoft.Research.Kinect.dll – this is the core dll for using the Kinect SDK
  • System.Windows.Interactivity.dll – this comes with Blend 4, you will want the .Net version (not silverlight) to allow the behaviours to work
  • Coding4Fun.Kinect.Wpf.dll – the guys from channel9/Coding4Fun have a little helper, I use the scaleto method.
  • SoulSolutions.Skeleton.dll – the custom control itself

Step 2 – Open up your xaml page and reference the controls namespace


Step 3 – Add an instance of the control to your page (or two)

<s:SkeletonControl x:Name="skeleton" />

Step 4 – Wireup the Kinect’s events to your skeleton(s)

in your code behind wire up the SkeletonData on the SkeletonFrameReady event (Full code sample at the end if this isn’t familiar)

private void NuiSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
    SkeletonFrame allSkeletons = e.SkeletonFrame;

    //get the first tracked skeleton
    skeleton.SkeletonData = (from s in allSkeletons.Skeletons
                        where s.TrackingState == SkeletonTrackingState.Tracked
                        select s).FirstOrDefault();

Making it look awesome in Blend

The control is ready for you to style in Blend (or Visual Studio if you’re a code ninja). The simple properties wire up in the default template are:

  • Foreground – wired up to the Fill on the 20 Ellipses tracking the joints, default is a B&W gradient
  • BoneFill – wired up to the Fill on the Bones that link the joints, default is light gray
  • Width – need to explicitly set a width in this version, default is 640
  • Height – again need to explicitly set a height, default 480
  • Background
  • BorderBrush
  • BorderThickness

Don’t stop here, simply make a copy of the default template and modify to your taste:


Add more then one

This is where it all starts to make sense, drop two on your page, wire up the second skeleton to the second tracked user, ready for some Red vs Blue Kinect Skeleton combat?

Some basic Poses

Looking for your help and pushing this into a more collaborative project, it does make sense for your skeleton to know about it’s current pose, I’m not sure ultimately the best best approach on this but I’ve included 7 poses I used at a recent event to control a 2.5 Gigapixel image:

  • Jump
  • LeftArmOut
  • RightArmOut
  • RightArmUp
  • LeftArmUp
  • Crouched
  • HandsTogether

Future Thoughts

I have kept this control very simple on purpose, I know people want something to start working with and to pull in their direction. Feel free to modify this code to your likely, love to know if you use it and what you do with it. These are the things I will be looking to improve on:

  1. It would be great to not have to specify the height and width, I need to do more testing to see if setting the control to the ActualHeight and ActualWidth works in all cases, let me know what i need to change if you spot it
  2. Bones don’t render in Blend design time.
  3. Bones behaviour doesn’t come across in the Blend “Edit a Copy”
  4. I’d like to simply bind the control to a skeleton data rather then having to write code behind, may need to write a little wrapper control for the Kinect and use element to element binding
  5. The whole Pose and Gesture thing is huge, I’ve seen two radically different approaches so far, look forward to seeing even more. I’d like to keep the skeleton separate somehow.


The code

Since you made it this far here are the two source code files written out and afterwards a link to download the full source with sample app. I will do my best to update the linked sourced code but these listing will not change:

Generic.xaml – the default template

    <Style TargetType="{x:Type Skeleton:SkeletonControl}">
        <Setter Property="Foreground">
                <LinearGradientBrush EndPoint="1,1" MappingMode="RelativeToBoundingBox" StartPoint="0,0">
                    <GradientStop Color="Black"/>
                    <GradientStop Color="White" Offset="1"/>
        <Setter Property="BoneFill" Value="LightGray" />
        <Setter Property="Width" Value="640" />
        <Setter Property="Height" Value="480" />
        <Setter Property="Template">
                <ControlTemplate TargetType="{x:Type Skeleton:SkeletonControl}">
                    <Border Background="{TemplateBinding Background}"
                            BorderBrush="{TemplateBinding BorderBrush}"
                            BorderThickness="{TemplateBinding BorderThickness}">
                        <Canvas Name="MainCanvas" Width="{TemplateBinding Width}" Height="{TemplateBinding Height}" >
                            <Rectangle x:Name="NeckBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0">
                                    <Skeleton:BoneBehavior EndJointElementName="PART_ShoulderCenter" StartJointElementName="PART_Head"/>
                            <Rectangle x:Name="Spine" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0">
                                    <Skeleton:BoneBehavior EndJointElementName="PART_ShoulderCenter" StartJointElementName="PART_Spine"/>
                            <Rectangle x:Name="LowerSpine" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0">
                                    <Skeleton:BoneBehavior EndJointElementName="PART_HipCenter" StartJointElementName="PART_Spine"/>
                            <Rectangle x:Name="LCollarBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_ShoulderCenter" StartJointElementName="PART_ShoulderLeft"/>
                            <Rectangle x:Name="RCollarBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_ShoulderCenter" StartJointElementName="PART_ShoulderRight"/>
                            <Rectangle x:Name="RUpperArmBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_ElbowRight" StartJointElementName="PART_ShoulderRight"/>
                            <Rectangle x:Name="LUpperArmBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_ElbowLeft" StartJointElementName="PART_ShoulderLeft"/>
                            <Rectangle x:Name="RLowerArmBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_ElbowRight" StartJointElementName="PART_WristRight"/>
                            <Rectangle x:Name="LLowerArmBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_ElbowLeft" StartJointElementName="PART_WristLeft"/>
                            <Rectangle x:Name="RHandBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_HandRight" StartJointElementName="PART_WristRight"/>
                            <Rectangle x:Name="LHandBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_HandLeft" StartJointElementName="PART_WristLeft"/>
                            <Rectangle x:Name="LHipBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_HipCenter" StartJointElementName="PART_HipLeft"/>
                            <Rectangle x:Name="RHipBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_HipCenter" StartJointElementName="PART_HipRight"/>
                            <Rectangle x:Name="RUpperLegBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_KneeRight" StartJointElementName="PART_HipRight"/>
                            <Rectangle x:Name="LUpperLegBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_KneeLeft" StartJointElementName="PART_HipLeft"/>
                            <Rectangle x:Name="RLowerLegBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_KneeRight" StartJointElementName="PART_AnkleRight"/>
                            <Rectangle x:Name="LLowerLegBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_KneeLeft" StartJointElementName="PART_AnkleLeft"/>
                            <Rectangle x:Name="RFootBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_FootRight" StartJointElementName="PART_AnkleRight"/>
                            <Rectangle x:Name="LFootBone" Fill="{TemplateBinding BoneFill}" Height="4" Width="10" StrokeThickness="0" >
                                    <Skeleton:BoneBehavior EndJointElementName="PART_FootLeft" StartJointElementName="PART_AnkleLeft"/>
                            <Ellipse x:Name="PART_Head" Fill="{TemplateBinding Foreground}" Height="40" Canvas.Left="90" Width="40"/>
                            <Ellipse x:Name="PART_HandLeft" Fill="{TemplateBinding Foreground}" Height="20" Canvas.Top="150" Width="20"/>
                            <Ellipse x:Name="PART_HandRight" Fill="{TemplateBinding Foreground}" Height="20" Canvas.Left="195" Canvas.Top="150" Width="20"/>
                            <Ellipse x:Name="PART_HipCenter" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="105" Canvas.Top="155" Width="10"/>
                            <Ellipse x:Name="PART_FootRight" Fill="{TemplateBinding Foreground}" Height="20" Canvas.Left="140" Canvas.Top="275" Width="20"/>
                            <Ellipse x:Name="PART_FootLeft" Fill="{TemplateBinding Foreground}" Height="20" Canvas.Left="65" Canvas.Top="275" Width="20"/>
                            <Ellipse x:Name="PART_AnkleLeft" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="75" Canvas.Top="255" Width="10"/>
                            <Ellipse x:Name="PART_AnkleRight" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="135" Canvas.Top="255" Width="10"/>
                            <Ellipse x:Name="PART_ElbowLeft" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="40" Canvas.Top="100" Width="10"/>
                            <Ellipse x:Name="PART_ElbowRight" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="165" Canvas.Top="100" Width="10"/>
                            <Ellipse x:Name="PART_HipLeft" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="75" Canvas.Top="170" Width="10"/>
                            <Ellipse x:Name="PART_HipRight" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="135" Canvas.Top="170" Width="10"/>
                            <Ellipse x:Name="PART_KneeLeft" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="75" Canvas.Top="215" Width="10"/>
                            <Ellipse x:Name="PART_KneeRight" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="135" Canvas.Top="215" Width="10"/>
                            <Ellipse x:Name="PART_ShoulderCenter" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="105" Canvas.Top="55" Width="10"/>
                            <Ellipse x:Name="PART_ShoulderLeft" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="70" Canvas.Top="65" Width="10"/>
                            <Ellipse x:Name="PART_ShoulderRight" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="140" Canvas.Top="65" Width="10"/>
                            <Ellipse x:Name="PART_Spine" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="105" Canvas.Top="130" Width="10"/>
                            <Ellipse x:Name="PART_WristLeft" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="20" Canvas.Top="135" Width="10"/>
                            <Ellipse x:Name="PART_WristRight" Fill="{TemplateBinding Foreground}" Height="10" Canvas.Left="185" Canvas.Top="135" Width="10"/>


//This code is provided as is and with no warrentee. Use at your own risk, license is MS-PL, feel free to modifiy and use in your projects as needed.


using System;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Media;
using Coding4Fun.Kinect.Wpf;
using Microsoft.Research.Kinect.Nui;


namespace SoulSolutions.Kinect.Controls.Skeleton
    /// <summary>
    /// Kinect Skeleton for you to customise and simply drop into your projects and style in Blend.
    /// </summary>
    [TemplatePart(Name = PartHead, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartHandLeft, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartHandRight, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartHipCenter, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartFootRight, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartFootLeft, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartAnkleLeft, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartAnkleRight, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartElbowLeft, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartElbowRight, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartHipLeft, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartHipRight, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartKneeLeft, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartKneeRight, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartShoulderCenter, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartShoulderLeft, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartShoulderRight, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartSpine, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartWristLeft, Type = typeof (FrameworkElement))]
    [TemplatePart(Name = PartWristRight, Type = typeof (FrameworkElement))]
    public class SkeletonControl : Control
        private const string PartHead = "PART_Head";
        private const string PartHandLeft = "PART_HandLeft";
        private const string PartHandRight = "PART_HandRight";
        private const string PartHipCenter = "PART_HipCenter";
        private const string PartFootRight = "PART_FootRight";
        private const string PartFootLeft = "PART_FootLeft";
        private const string PartAnkleLeft = "PART_AnkleLeft";
        private const string PartAnkleRight = "PART_AnkleRight";
        private const string PartElbowLeft = "PART_ElbowLeft";
        private const string PartElbowRight = "PART_ElbowRight";
        private const string PartHipLeft = "PART_HipLeft";
        private const string PartHipRight = "PART_HipRight";
        private const string PartKneeLeft = "PART_KneeLeft";
        private const string PartKneeRight = "PART_KneeRight";
        private const string PartShoulderCenter = "PART_ShoulderCenter";
        private const string PartShoulderLeft = "PART_ShoulderLeft";
        private const string PartShoulderRight = "PART_ShoulderRight";
        private const string PartSpine = "PART_Spine";
        private const string PartWristLeft = "PART_WristLeft";
        private const string PartWristRight = "PART_WristRight";
        public static readonly DependencyProperty BoneFillProperty = DependencyProperty.Register("BoneFill", typeof (Brush), typeof (SkeletonControl), null);

        private FrameworkElement ankleLeftUi;
        private FrameworkElement ankleRightUi;
        private FrameworkElement elbowLeftUi;
        private FrameworkElement elbowRightUi;
        private FrameworkElement footLeftUi;
        private FrameworkElement footRightUi;
        private FrameworkElement handLeftUi;
        private FrameworkElement handRightUi;
        private FrameworkElement headUi;
        private FrameworkElement hipCenterUi;
        private FrameworkElement hipLeftUi;
        private FrameworkElement hipRightUi;
        private FrameworkElement kneeLeftUi;
        private FrameworkElement kneeRightUi;
        private FrameworkElement shoulderCenterUi;
        private FrameworkElement shoulderLeftUi;
        private FrameworkElement shoulderRightUi;
        private SkeletonData skeletonData;
        private FrameworkElement spineUi;
        private FrameworkElement wristLeftUi;
        private FrameworkElement wristRightUi;

        static SkeletonControl()
            DefaultStyleKeyProperty.OverrideMetadata(typeof (SkeletonControl), new FrameworkPropertyMetadata(typeof (SkeletonControl)));

        public Brush BoneFill
            get { return (Brush) GetValue(BoneFillProperty); }
            set { SetValue(BoneFillProperty, value); }

        public SkeletonData SkeletonData
            get { return skeletonData; }
                skeletonData = value;
                SetUiPosition(headUi, skeletonData.Joints[JointID.Head]);
                SetUiPosition(handLeftUi, skeletonData.Joints[JointID.HandLeft]);
                SetUiPosition(handRightUi, skeletonData.Joints[JointID.HandRight]);
                SetUiPosition(hipCenterUi, skeletonData.Joints[JointID.HipCenter]);
                SetUiPosition(footRightUi, skeletonData.Joints[JointID.FootRight]);
                SetUiPosition(footLeftUi, skeletonData.Joints[JointID.FootLeft]);
                SetUiPosition(ankleLeftUi, skeletonData.Joints[JointID.AnkleLeft]);
                SetUiPosition(ankleRightUi, skeletonData.Joints[JointID.AnkleRight]);
                SetUiPosition(elbowLeftUi, skeletonData.Joints[JointID.ElbowLeft]);
                SetUiPosition(elbowRightUi, skeletonData.Joints[JointID.ElbowRight]);
                SetUiPosition(hipLeftUi, skeletonData.Joints[JointID.HipLeft]);
                SetUiPosition(hipRightUi, skeletonData.Joints[JointID.HipRight]);
                SetUiPosition(kneeLeftUi, skeletonData.Joints[JointID.KneeLeft]);
                SetUiPosition(kneeRightUi, skeletonData.Joints[JointID.KneeRight]);
                SetUiPosition(shoulderCenterUi, skeletonData.Joints[JointID.ShoulderCenter]);
                SetUiPosition(shoulderLeftUi, skeletonData.Joints[JointID.ShoulderLeft]);
                SetUiPosition(shoulderRightUi, skeletonData.Joints[JointID.ShoulderRight]);
                SetUiPosition(spineUi, skeletonData.Joints[JointID.Spine]);
                SetUiPosition(wristLeftUi, skeletonData.Joints[JointID.WristLeft]);
                SetUiPosition(wristRightUi, skeletonData.Joints[JointID.WristRight]);

        //Poses to build on need more models to verify correct, works well for 6foot4 Male :)
        //Smaller children have already proved these values need to be changed to be proportional rather then absolute.

        public bool Jump
            get { return SkeletonData.Joints[JointID.FootLeft].Position.Y > -0.7 && SkeletonData.Joints[JointID.FootRight].Position.Y > -0.7; }

        public bool LeftArmOut
            get { return ((SkeletonData.Joints[JointID.HandLeft].Position.X - SkeletonData.Joints[JointID.ShoulderLeft].Position.X) < -0.5); }

        public bool RightArmOut
            get { return ((SkeletonData.Joints[JointID.HandRight].Position.X - SkeletonData.Joints[JointID.ShoulderRight].Position.X) > 0.5); }

        public bool RightArmUp
            get { return ((SkeletonData.Joints[JointID.HandRight].Position.Y - SkeletonData.Joints[JointID.Head].Position.Y) > 0); }

        public bool LeftArmUp
            get { return ((SkeletonData.Joints[JointID.HandLeft].Position.Y - SkeletonData.Joints[JointID.Head].Position.Y) > 0); }

        public bool Crouched
            get { return (Math.Abs(SkeletonData.Joints[JointID.FootLeft].Position.Y - SkeletonData.Joints[JointID.HandLeft].Position.Y) < 0.2); }

        public bool HandsTogether
                return (Math.Abs(SkeletonData.Joints[JointID.HandRight].Position.Y - SkeletonData.Joints[JointID.HandLeft].Position.Y) +
                        Math.Abs(SkeletonData.Joints[JointID.HandRight].Position.X - SkeletonData.Joints[JointID.HandLeft].Position.X) < 0.3);

        public override void OnApplyTemplate()
            headUi = GetTemplateChild(PartHead) as FrameworkElement;
            handLeftUi = GetTemplateChild(PartHandLeft) as FrameworkElement;
            handRightUi = GetTemplateChild(PartHandRight) as FrameworkElement;
            hipCenterUi = GetTemplateChild(PartHipCenter) as FrameworkElement;
            footRightUi = GetTemplateChild(PartFootRight) as FrameworkElement;
            footLeftUi = GetTemplateChild(PartFootLeft) as FrameworkElement;
            ankleLeftUi = GetTemplateChild(PartAnkleLeft) as FrameworkElement;
            ankleRightUi = GetTemplateChild(PartAnkleRight) as FrameworkElement;
            elbowLeftUi = GetTemplateChild(PartElbowLeft) as FrameworkElement;
            elbowRightUi = GetTemplateChild(PartElbowRight) as FrameworkElement;
            hipLeftUi = GetTemplateChild(PartHipLeft) as FrameworkElement;
            hipRightUi = GetTemplateChild(PartHipRight) as FrameworkElement;
            kneeLeftUi = GetTemplateChild(PartKneeLeft) as FrameworkElement;
            kneeRightUi = GetTemplateChild(PartKneeRight) as FrameworkElement;
            shoulderCenterUi = GetTemplateChild(PartShoulderCenter) as FrameworkElement;
            shoulderLeftUi = GetTemplateChild(PartShoulderLeft) as FrameworkElement;
            shoulderRightUi = GetTemplateChild(PartShoulderRight) as FrameworkElement;
            spineUi = GetTemplateChild(PartSpine) as FrameworkElement;
            wristLeftUi = GetTemplateChild(PartWristLeft) as FrameworkElement;
            wristRightUi = GetTemplateChild(PartWristRight) as FrameworkElement;

        private void SetUiPosition(FrameworkElement ui, Joint joint)
            if (ui != null && ActualWidth > 0 && ActualHeight > 0)
                var scaledJoint = joint.ScaleTo((int) ActualWidth, (int) ActualHeight, 1f, 1f);
                Canvas.SetLeft(ui, scaledJoint.Position.X - ui.ActualWidth/2);
                Canvas.SetTop(ui, scaledJoint.Position.Y - ui.ActualHeight/2);

Download the code here.

Comments, Suggestions, Feedback? Catch me on twitter @soulsolutions