Using the Kinect Depth Sensor
Written by Harry Fairhead   
Article Index
Using the Kinect Depth Sensor
Displaying the Image
A depth histogram

 

Displaying the image - Windows Forms

The next big problem is finding a way to display a depth frame.

As the data take the form of a 13-bit image the most obvious thing to do is convert it into a grey scale image.

This is where we hit a small snag. While Windows Forms supports the Format16bppGrayScale constant it doesn't actually support the type. That is you can't load up a Bitmap object with grey scale data and then display it using a PictureBox - it just shows as a big red cross.

So what can we do?

One answer, although it is a bit of a cheat, is to simply use the Format16bppRgb555 format which is supported. This treats a 16-bit value so that 5 bits are used for each of R,G and B and the topmost bit is ignored.

We can use the same technique as explained in the previous chapter to move the data into a Bitmap.

First we create a Bitmap object of the right size and color format:

Bitmap bmap = new Bitmap(
PImage.Width,
PImage.Height,
PixelFormat.Format16bppRgb555);

Next we create and lock a BitMapData object using the same size and format:

BitmapData bmapdata = bmap.LockBits(
new Rectangle(0, 0, PImage.Width,
PImage.Height),
ImageLockMode.WriteOnly,
bmap.PixelFormat);
Finally we copy the bits from the pixel data array to the BitmapData object and then unlock it to place the bits into the Bitmap object:
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(PImage.Bits,
0,
ptr,
PImage.Width *
PImage.Height);
bmap.UnlockBits(bmapdata);

To make this work we need to add:

using System.Drawing.Imaging;
using System.Runtime.InteropServices;

at the start of the program.

If you now place a PictureBox control on the form you can view the depth data:

pictureBox1.Image = bmap;

Distance2

This works quite well but it would be better packaged as a complete function:

Bitmap DepthToBitmap(DepthImageFrame imageFrame)
{
short[] pixelData = new short[imageFrame.PixelDataLength];
imageFrame.CopyPixelDataTo(pixelData);

Bitmap bmap = new Bitmap(
imageFrame.Width,
imageFrame.Height,
PixelFormat.Format16bppRgb555);

BitmapData bmapdata = bmap.LockBits(
new Rectangle(0, 0, imageFrame.Width,
imageFrame.Height),
ImageLockMode.WriteOnly,
bmap.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(pixelData,
0,
ptr,
imageFrame.Width *
imageFrame.Height);
bmap.UnlockBits(bmapdata);
return bmap;
}

 

To put this function to good use you simply need to call it in the DepthFrameReady event handler:

void DepthFrameReady(object sender, 
DepthImageFrameReadyEventArgs e)
{
DepthImageFrame imageFrame = e.OpenDepthImageFrame();
if (imageFrame != null)
{
  pictureBox1.Image = DepthToBitmap(imageFrame);
}
}

This is a quick and easy way to get a false color display of the depth data but it isn't particularly good as the B channel isn't used because of the bottom three bits always being zero.

We can improve it slightly by shifting each depth value by 3 bits to the right. The reason is that the color is determined by low order 15 bits but the depth field is the high 13-bits. This means that the top depth bit and the low order three color bits aren't used. Shifting three bits to the right matches the bits up in a much better way 

pixelData[i] =(short)(((ushort) pixelData[i])>>3);

We have to make sure that we perform a logical shift right hence the casts.

This shift has to be performed on each of the elements so we need a for loop:

for (int i = 0; i < imageFrame.PixelDataLength; i++)
{
pixelData[i] =(short)(((ushort) pixelData[i])>>3);
}

Now you should see a little more blue in the image but which is best is a matter of preference. They are both just quick and dirty ways of getting a false color depth image.

 

Distance1

 

Displaying the Image - WPF

If you want to use WPF then things are a little easier but there are still some odd problems to sort out. The WPF BitmapSource class does support a much wider range of pixel formats. including gray16.

To convert the PlanarImage to a BitmapSource is almost trivial:

BitmapSource DepthToBitmapSource(
DepthImageFrame imageFrame)
{
short[] pixelData = new short[imageFrame.PixelDataLength];
imageFrame.CopyPixelDataTo(pixelData);

BitmapSource bmap = BitmapSource.Create(
imageFrame.Width,
imageFrame.Height,
96, 96,
PixelFormats.Gray16,
null,
pixelData,
imageFrame.Width*imageFrame.BytesPerPixel);
return bmap;
}

And if you place an Image control on the form you can display the result:

void DepthFrameReady(object sender, 
DepthImageFrameReadyEventArgs e)
{
DepthImageFrame imageFrame =
e.OpenDepthImageFrame();
if (imageFrame != null)
{
image1.Source = DepthToBitmapSource(
imageFrame);
}
}

If you try this out you should see a reasonable representation but you might feel the need to change the contrast or brightness depending on the average depth of objects in the scene.

 

Distance3