Ernesto Sirolli: Want to help someone? Shut up and listen!

Kinect + Raspberry Pi

I have been trying to get the Kinect to work with Raspberry Pi (attempting to get raw depth data) but still keep on hitting roadblocks. The motor was able to tilt when using libfreenect. As for Open NI, I was able to get it to recognize the kinect but unable to get data from the camera.


Install libraries using this tutorial (use unstable) http://mewgen.com/Ge107_files/20120921%20Setting%20up%20Rasberry%20pi%20for%20the%20Xtion%20and%20kinect.html

alternate link: http://gremsi.blogspot.com/2013/04/installing-openni-sensorkinect-and.html

Status: Recognizes the kinect but cannot transfer data.

General Commands:
compile sample files
~/Desktop/kinect/OpenNI/Platform/Linux/Build $ make

build/install open ni
~/Desktop/kinect/unstable/OpenNI/Platform/Linux/CreateRedist $ sudo ./RedistMaker.Arm
~/Desktop/kinect/unstable/OpenNI/Platform/Linux/Redist $ sudo ./install.sh

Trial and Error

1. used this site to change usbinterface line in unstable sensor kinect: http://daybydaylinux.blogspot.com/2012/12/how-to-compile-openni-and-sensorkinect.html
1. cd ~/kinect/SensorKinect/Platform/Linux/Redist/Sensor-Bin-Linux-Arm-v5.1.2.1/Config/
   
   sudo vi GlobalDefaultsKinect.inimodify`;UsbInterface=2` into `UsbInterface=1`
2. Status: instead of saying usb interface not supported, it says: UpdateData failed: A timeout has occurred when waiting for new data!
2. Attempt to fix timeout issue: change line in Include/XnTypes.h for data_timeout from 2000 to 20,000 and build open ni
1. status: changing it to 20seconds and 1 minute and did not work
3. Was not able to change the FPS for depth because it only supports 15fps (http://openni-discussions.979934.n3.nabble.com/OpenNI-dev-How-to-change-FPS-td2500973.html)

Trace Back: NiSimpleRead
Attempting to trace the problem to see where it occurs.

1. Samples/NiSimpleRead/NiSimpleRead.cpp
   line 103: context.WaitOneUpdateAll(depth)
2. Include/XnCppWrapper.h
   line 9421: return xnWaitOneUpdateAll(...)
3. Source/OpenNI.cpp
   line 2601: xnWaitForCondition(...)
4. Source/OpenNI.cpp
   line 2552: xnOSWaitForCondition(...)
5. Source/XnOS.cpp
   line 236: xnOSWaitEvent(...)
6. Source/OpenNI/Linux/LinuxEvents.cpp
   line 160: pEvent->Wait(...)

Somehow xnOSWaitEvent is linked with Source/OpenNI/Linux/XnUSBLinux.cpp: xnUSBReadThreadMain. I am assuming its linked because I got an warning (after i set the usb interface to 1): USB events thread - failed to set priority. And this error message originates in the file XnUSBLinux

I'm currently trying to see if data is being transferred at all or if the Kinect is just waiting and not sending data.

Side note: you can change the option to print out the logs for OpenNI in the Data/SampleConfig.xml file.


[Update: 2/24]: Some people had some success with the beagleboard and the kinect:
  http://www.pansenti.com/wordpress/?page_id=1772
  http://instructionalrobotics.blogspot.com/2013/01/kinect-under-beagleboard-c4.html 

[Update: 2/27]:
I updated the file XnUSBLinux.cpp to printout the transfer status. In the xnUSBReadThreadMain method, I added these lines (in bold):

//more code...

else // transfer done

{

if (pBufferInfo->nLastStatus == LIBUSB_TRANSFER_COMPLETED || // read succeeded

pBufferInfo->nLastStatus == LIBUSB_TRANSFER_CANCELLED)   // cancelled, but maybe some data arrived

{

if(pBufferInfo->nLastStatus == LIBUSB_TRANSFER_COMPLETED)

{

printf("***** pBufferInfo->nLastStatus = LIBUSB_TRANSFER_COMPLETED\n");

}

else if(pBufferInfo->nLastStatus == LIBUSB_TRANSFER_CANCELLED)

{

printf("***** pBufferInfo->nLastStatus == LIBUSB_TRANSFER_CANCELLED\n");

}

if (pTransfer->type == LIBUSB_TRANSFER_TYPE_ISOCHRONOUS)

{

XnUInt32 nTotalBytes = 0;

// some packets may return empty, so we need to remove spaces, and make the buffer sequential

for (XnUInt32 i = 0; i < pTransfer->num_iso_packets; ++i)

{

struct libusb_iso_packet_descriptor* pPacket = &pTransfer->iso_packet_desc[i];

if (/*pPacket->status == LIBUSB_TRANSFER_COMPLETED && */pPacket->actual_length != 0)

{

XnUChar* pBuffer = libusb_get_iso_packet_buffer_simple(pTransfer, i);

// if buffer is not at same offset, move it

if (pTransfer->buffer + nTotalBytes != pBuffer)

{

// printf("buffer %d has %d bytes. Moving to offset %d...\n", i, pPacket->actual_length, nTotalBytes);

memmove(pTransfer->buffer + nTotalBytes, pBuffer, pPacket->actual_length);

}

nTotalBytes += pPacket->actual_length;

}

else if (pPacket->status != LIBUSB_TRANSFER_COMPLETED)

{

xnLogWarning(XN_MASK_USB, "2 Endpoint 0x%x, Buffer %d, packet %d Asynch transfer failed (status: %d)", pTransfer->endpoint, pBufferInfo->nBufferID, i, pPacket->status);

}

if(pPacket->status == LIBUSB_TRANSFER_COMPLETED)

{

printf("*****pPacket->status = LIBUSB_TRANSFER_COMPLETED. Length: %d\n", pPacket->actual_length);

}

else if(pPacket->status == LIBUSB_TRANSFER_CANCELLED)

{

printf("*****pPacket->Status = TRANSFER CANCELLED Length: %d\n", pPacket->actual_length);

}

}

if (nTotalBytes != 0)

{

// call callback method

pBufferInfo->pThreadData->pCallbackFunction(pTransfer->buffer, nTotalBytes, pBufferInfo->pThreadData->pCallbackData);

}

}

else

{

// call callback method

pBufferInfo->pThreadData->pCallbackFunction(pTransfer->buffer, pTransfer->actual_length, pBufferInfo->pThreadData->pCallbackData);

}

}

//more code...

After building and installing, I ran ./Sample-NiSimpleRead again. Click here to see the output (the lines that begin with '*****' are my print line statements). 

The main things to focus on are these lines:

*****pPacket->status = LIBUSB_TRANSFER_COMPLETED. Length: 1760 

*****pPacket->status = LIBUSB_TRANSFER_COMPLETED. Length: 1920 

*****pPacket->status = LIBUSB_TRANSFER_COMPLETED. Length: 0

The transfer seems to be completed but the length of the packet is 0. Not sure why this is happening. 

I left Sample-NiSimpleRead running longer, and noticed I was getting data back from the Kinect. The packet would hold 1760 or 1920 bytes at a time. (the log file has been updated with the new log)

[Update 3/29]
Unfortunately, I wasn't able to get it working with the Kinect. I am currently looking into other options. 

• Asus Xtion + Raspberry Pi: I was able to get frames back from an Asus Xtion.
• Kinect + Beagle Board (or Panda Board): I haven't had the chance to try this out but in theory since this is a little more powerful than the RPi, it could work with the kinect. 

I am currently working on getting and compressing the depth images using OpenNI. See OpenNI + Depth Compression

Amy Cuddy: Your body language shapes who you are