Help Sending Data from Kinect to Arduino through PC Serial

Discussion in 'Embedded Systems and Microcontrollers' started by stef757, May 8, 2013.

  1. stef757

    Thread Starter New Member

    Apr 28, 2011
    I have been a little stumped on how to figure this issue out.

    We are downsampling data from a microsoft kinect into a 9x9 array of RGB+Depth data at 10Hz and attempting to send it to the arduino Mega via serial. The kinect data is processed in Visual Studios. Right now we are just trying to get a 3x3 section of the array to work for testing. This is the code in Visual Studios we have so far that will be handling the serial output:

    //color member variables
    private KinectSensor _Kinect;
    private WriteableBitmap _ColorImageBitmap;
    private Int32Rect _ColorImageBitmapRect;
    private int _ColorImageStride;
    private byte[] _ColorImagePixelData;
    private int[] Red = new int[81];
    private int[] Red3x3 = new int[9];
    private byte[] Red3x3Byte = new byte[9];
    private int[] Green = new int[81];
    private int[] Green3x3 = new int[9];
    private byte[] Green3x3Byte = new byte[9];
    private int[] Blue = new int[81];
    private int[] Blue3x3 = new int[9];
    private byte[] Blue3x3Byte = new byte[9];
    private int[] Depth = new int[82];
    private int[,] Depth2d = new int[9, 9];
    private int[,] Depth3x3in2d = new int[3, 3];
    private int[] Depth3x3 = new int[9];
    private byte[] Depth3x3Byte = new byte[9];
    public byte[] DepthByte = new byte[81];
    public byte[] RedByte = new byte[81];
    public byte[] GreenByte = new byte[81];
    public byte[] BlueByte = new byte[81];

    //serial port
    public SerialPort port1 = new SerialPort();

    #endregion Member Variables

    #region Constructor
    public MainWindow()

    KinectSensor.KinectSensors.StatusChanged += KinectSensors_StatusChanged;
    this.KinectDevice = KinectSensor.KinectSensors.FirstOrDefault(x => x.Status == KinectStatus.Connected);

    DepthImage.MouseLeftButtonUp += DepthImage_MouseLeftButtonUp;

    // serial communication initializations
    port1.PortName = "COM3";
    port1.BaudRate = 9600;
    port1.Handshake = System.IO.Ports.Handshake.None;
    port1.Parity = Parity.None;
    port1.DataBits = 8;
    port1.StopBits = StopBits.One;
    port1.ReadTimeout = 500;


    //sending the data
    if (port1.IsOpen)

    //9x9 arrays
    //port1.Write(this.DepthByte, 0, 1);
    //port1.Write(this.RedByte, 0, 1);
    //port1.Write(this.GreenByte, 0, 1);
    //port1.Write(this.BlueByte, 0, 1);

    //3x3 prototype arrays

    //port1.Write(this.Red3x3Byte, 0, 9);
    //port1.Write(this.Green3x3Byte, 0, 9);
    //port1.Write(this.Blue3x3Byte, 0, 9);
    port1.Write(this.Depth3x3Byte, 0, 9);

    part of the code is commented out for testing purposes since we are only trying to get the 3x3 working right now.

    Another member of my team is in charge of the kinect and wrote this code so I am not very familiar with this. I am trying to integrate it with the Arduino.

    My goal is to receive this data and save it into its own 9x9 array of the RGBD on the Arduino so that it can control LEDs and Motors using the code we have implemented already. I don't necessarily need the most elegant solution, I just need to get something to work. What is the best way to go about doing this? I'd imagine this not being extremely difficult since the packet size is very small and the timing is slow. Would it be best to append some sort of header the outgoing serial data in order to frame up the incoming data on the Arduino? How should I write my arduino code so that it receives the serial every 10 seconds, should the arduino handle timing as well, or just rely on the computer's timing of output serial data?

    I've spent a good bit of time looking at different examples from different sources I'm somewhat familiar with (RepRap Marlin firmware, MultiWii Quadrotor flight controller firmware) to see some examples, but it's a little too much complexity to take in right now. I don't have much in my Arduino code for receiving the serial right now so I will pretty much be starting from scratch.

    I appreciate your help!