Displaying the image - Windows Forms

The next big problem is finding a way to display a depth frame.

As the data take the form of a 13-bit image the most obvious thing to do is convert it into a grey scale image.

This is where we hit a small snag. While Windows Forms supports the Format16bppGrayScale constant it doesn't actually support the type. That is you can't load up a Bitmap object with grey scale data and then display it using a PictureBox - it just shows as a big red cross.

So what can we do?

One answer, although it is a bit of a cheat, is to simply use the Format16bppRgb555 format which is supported. This treats a 16-bit value so that 5 bits are used for each of R,G and B and the topmost bit is ignored.

We can use the same technique as explained in the previous chapter to move the data into a Bitmap.

First we create a Bitmap object of the right size and color format:

Bitmap bmap = new Bitmap(
 PImage.Width,
 PImage.Height,
 PixelFormat.Format16bppRgb555);

Next we create and lock a BitMapData object using the same size and format:

BitmapData bmapdata = bmap.LockBits(
 new Rectangle(0, 0, PImage.Width,
             PImage.Height),
 ImageLockMode.WriteOnly,
 bmap.PixelFormat);

IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(PImage.Bits,
 0,
 ptr,
 PImage.Width *
 PImage.Height);
bmap.UnlockBits(bmapdata);

To make this work we need to add:

using System.Drawing.Imaging;
using System.Runtime.InteropServices;

at the start of the program.

If you now place a PictureBox control on the form you can view the depth data:

pictureBox1.Image = bmap;

This works quite well but it would be better packaged as a complete function:

Bitmap DepthToBitmap(DepthImageFrame imageFrame)
{
 short[] pixelData = new short[imageFrame.PixelDataLength];
 imageFrame.CopyPixelDataTo(pixelData);

 Bitmap bmap = new Bitmap(
 imageFrame.Width,
 imageFrame.Height,
 PixelFormat.Format16bppRgb555);

 BitmapData bmapdata = bmap.LockBits(
  new Rectangle(0, 0, imageFrame.Width,
                         imageFrame.Height),
  ImageLockMode.WriteOnly,
  bmap.PixelFormat);
 IntPtr ptr = bmapdata.Scan0;
 Marshal.Copy(pixelData,
  0,
  ptr,
  imageFrame.Width *
    imageFrame.Height);
 bmap.UnlockBits(bmapdata);
 return bmap;
}

To put this function to good use you simply need to call it in the DepthFrameReady event handler:

void DepthFrameReady(object sender, 
           DepthImageFrameReadyEventArgs e)
{
 DepthImageFrame imageFrame = e.OpenDepthImageFrame();
 if (imageFrame != null)
 {
  pictureBox1.Image = DepthToBitmap(imageFrame);
 }
}

This is a quick and easy way to get a false color display of the depth data but it isn't particularly good as the B channel isn't used because of the bottom three bits always being zero.

We can improve it slightly by shifting each depth value by 3 bits to the right. The reason is that the color is determined by low order 15 bits but the depth field is the high 13-bits. This means that the top depth bit and the low order three color bits aren't used. Shifting three bits to the right matches the bits up in a much better way 

pixelData[i] =(short)(((ushort) pixelData[i])>>3);

We have to make sure that we perform a logical shift right hence the casts.

This shift has to be performed on each of the elements so we need a for loop:

for (int i = 0; i < imageFrame.PixelDataLength; i++)
{ 
 pixelData[i] =(short)(((ushort) pixelData[i])>>3);
}

Now you should see a little more blue in the image but which is best is a matter of preference. They are both just quick and dirty ways of getting a false color depth image.

Displaying the Image - WPF

If you want to use WPF then things are a little easier but there are still some odd problems to sort out. The WPF BitmapSource class does support a much wider range of pixel formats. including gray16.

To convert the PlanarImage to a BitmapSource is almost trivial:

BitmapSource DepthToBitmapSource(
                DepthImageFrame imageFrame)
{
 short[] pixelData = new short[imageFrame.PixelDataLength];
 imageFrame.CopyPixelDataTo(pixelData);

 BitmapSource bmap = BitmapSource.Create(
  imageFrame.Width,
  imageFrame.Height,
  96, 96,
  PixelFormats.Gray16,
  null,
  pixelData,
  imageFrame.Width*imageFrame.BytesPerPixel);
 return bmap;
}

And if you place an Image control on the form you can display the result:

void DepthFrameReady(object sender, 
           DepthImageFrameReadyEventArgs e)
{
 DepthImageFrame imageFrame = 
                     e.OpenDepthImageFrame();
 if (imageFrame != null)
 {
 image1.Source = DepthToBitmapSource(
                                imageFrame);
 }
}

If you try this out you should see a reasonable representation but you might feel the need to change the contrast or brightness depending on the average depth of objects in the scene.