Installing OpenNI, SensorKinect, and PrimeSense on a Raspberry Pi

Source: http://mewgen.com/Ge107_files/20120921%20Setting%20up%20Rasberry%20pi%20for%20the%20Xtion%20and%20kinect.html

I did not create these instructions, I only reposted them here just incase they got removed.

_Using Win32DiskImager (for windows)

_downloaded 2012-08-16-wheezy-raspbian from website ( http://www.raspberrypi.org/downloads )

_burnt the image onto a 16Gb Class 10 (up to 95MB/s) Sandisk Extreme Pro card

_Powerup the Pi using a 5V 1Amp supply

_Note: Italics is what is typed into the Pi shell or added to scripts/code

_Power up the Pi, I am assuming you are plugged into the dvi port, if you cannot and only want to SSH then you need to know the IP address (one way is to look at the router's DHCP assigned table)

_On Pi startup menu:

- expand to use the whole card

- change the timezone to Aust -> Syd

- change local to AST-UTF-8 and default for system GB-UTF-8

- turn on the SSH server (it should be on by default)

- do an update

_Then from the shell, Overclock: (For details see http://elinux.org/RPi_config.txt )sudo nano /boot/config.txt

_If you don't want to void your warranty on the Pi then I suggest you use these settings
arm_freq=855
sdram_freq=500

_If you don't mind voiding your warranty added these lines after the last line. These are the setting I am using. if they don't work on bootup the press “shift” while booting I think to do a non-overclocked boot force_turbo=1
over_voltage=8
arm_freq=1150
core_freq=500
sdram_freq=600

_Get the ipaddress for eth0 assuming you are using ethernetifconfig eth0

_Then shutdown and power cycle the Pi so the card can be expanded and new faster config can be usedsudo shutdown now

_ After rebooting check the new CPU speed:
more /proc/cpuinfo

_Also if you are worried about the temperature you can check it by going here: cd /opt/vc/bin/

_and run this script: ./vcgencmd measure_temp

_ for all the possible commands use: ./vcgencmd commands

_ Go to the shell (via ssh use IP address is easiest, tunnel X through ssh and for windows use X server like Xming)sudo apt-get update
sudo apt-get install git g++ python libusb-1.0-0-dev freeglut3-dev openjdk-6-jdk doxygen graphviz

_ Get stable OpenNI and the drivers (this failed several times but keep trying)mkdir stable
cd stablegit clone https://github.com/OpenNI/OpenNI.git
git clone git://github.com/avin2/SensorKinect.git
git clone https://github.com/PrimeSense/Sensor.git

_ Get unstable OpenNI and the driversmkdir unstable
cd unstablegit clone https://github.com/OpenNI/OpenNI.git -b unstable 
git clone git://github.com/avin2/SensorKinect.git -b unstable
git clone https://github.com/PrimeSense/Sensor.git -b unstable

_I will do the following just for stable but all steps must be done for unstable too
Note: only do the install step for one or the other

_The calc_jobs_number() function in the scripts doesn't seem to work on the Pi, so change python scriptnano ~/stable/OpenNI/Platform/Linux/CreateRedist/Redist_OpenNi.py

_from containing this:
MAKE_ARGS += ' -j' + calc_jobs_number()

_to
MAKE_ARGS += ' -j1'

_ Must also change the Arm compiler settings for this distribution of the Pinano ~/stable/OpenNI/Platform/Linux/Build/Common/Platform.Arm

_from
CFLAGS += -march=armv7-a -mtune=cortex-a8 -mfpu=neon -mfloat-abi=softfp #-mcpu=cortex-a8

_to
CFLAGS += -mtune=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard

_Then run cd ~/stable/OpenNI/Platform/Linux/CreateRedist/
./RedistMaker.Arm
cd ~/stable/OpenNI/Platform/Linux/Redist/OpenNI-Bin-Dev-Linux-Arm-v1.5.2.23
sudo ./install.sh

Go to the Redist and run install (for stable or unstable not both)cd ~/stable/OpenNI/Platform/Linux/Redist/OpenNI-Bin-Dev-Linux-Arm-v1.5.4.0
sudo ./install.sh

_ Also edit the Sensor and SensorKinect makefile CFLAGS parameters
nano ~/stable/Sensor/Platform/Linux/Build/Common/Platform.Arm
nano ~/stable/SensorKinect/Platform/Linux/Build/Common/Platform.Arm

_ and the Sensor and SensorKinect redistribution scriptsnano ~/stable/Sensor/Platform/Linux/CreateRedist/RedistMaker
nano ~/stable/SensorKinect/Platform/Linux/CreateRedist/RedistMaker

_ for both, change
make -j$(calc_jobs_number) -C ../Build

_to
make -j1 -C ../Build

_ The create the redistributables
_Sensor (primesense)cd ~/stable/Sensor/Platform/Linux/CreateRedist/
./RedistMaker Arm

_ and SensorKinect  (note this does not work with stable OpenNI only the unstable. If fails about half way through with a problem of a missing header file) Note that on the SensorKinect git page it say that you need the unstable version of OpenNI for it to work
cd ~/stable/SensorKinect/Platform/Linux/CreateRedist/
./RedistMaker Arm

_ Then install either stable or unstable

_ install for stablecd ~/stable/Sensor/Platform/Linux/Redist/Sensor-Bin-Linux-Arm-v5.1.0.41
sudo ./install.shcd ~/stable/SensorKinect/Platform/Linux/Redist/Sensor-Bin-Linux-Arm-v5.1.2.1
sudo ./install.sh

_ install for unstablecd ~/unstable/Sensor/Platform/Linux/Redist/Sensor-Bin-Linux-Arm-v5.1.2.1
sudo ./install.sh
cd ~/unstable/SensorKinect/Platform/Linux/Redist/Sensor-Bin-Linux-Arm-v5.1.2.1
sudo ./install.sh

 _ Try running the sample reading program after pluging in sensor (check with lsusb)cd ~/stable/OpenNI/Platform/Linux/Bin/Arm-Release

sudo ./Sample-NiCRead
sudo ./Sample-NiBackRecorder time 1 depth vga
sudo ./Sample-NiSimpleRead

_ Problems I had:

_you need a powered hub to run the Xtion

_ If you get timeout errors it can be because the hub isn't giving enough power, even if it shows up in "lsusb" I had to unplug the keyboard and mouse from the hub before it would work

_ I had to try different ports on the hub to get some demos to work, unplug and plug in again in a different port

_ when I used the unstable version of the Xtion driver I got:
Open failed: Device Protocol: Bad Parameter sent!

_I was using the stable version of the kinect it wouldn't even build. I get this error about halfway through the build

g++ -MD -MP -MT "./Arm-Release/XnActualGeneralProperty.d Arm-Release/XnActualGeneralProperty.o" -c -mtune=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard -O3 -fno-tree-pre -fno-strict-aliasing -ftree-vectorize -ffast-math -funsafe-math-optimizations -fsingle-precision-constant -O2 -DNDEBUG -I/usr/include/ni -I../../../../Include -I../../../../Source -I../../../../Source/XnCommon -DXN_DDK_EXPORTS -fPIC -fvisibility=hidden -o Arm-Release/XnActualGeneralProperty.o ../../../../Source/XnDDK/XnActualGeneralProperty.cpp

In file included from ../../../../Source/XnDDK/XnGeneralProperty.h:28:0,
from ../../../../Source/XnDDK/XnActualGeneralProperty.h:28,
from ../../../../Source/XnDDK/XnActualGeneralProperty.cpp:25:

../../../../Source/XnDDK/XnProperty.h:29:21: fatal error: XnListT.h: No such file or directory
compilation terminated.
make[1]: *** [Arm-Release/XnActualGeneralProperty.o] Error 1
make[1]: Leaving directory `/home/pi/stable/SensorKinect/Platform/Linux/Build/XnDDK'
make: *** [XnDDK] Error 2
make: Leaving directory `/home/pi/stable/SensorKinect/Platform/Linux/Build'

_ See this page https://github.com/avin2/SensorKinect for the following notice on the above eror:

***** Important notice: *****
You must use this kinect mod version with the unstable OpenNI release......

_ with the unstable version of the Kinect it built and installed but I was getting the error
Open failed: USB interface is not supported!

_ So I had to edit
sudo nano /usr/etc/primesense/GlobalDefaultsKinect.ini

_ and uncomment this line and changed it to 1 instead of 2
UsbInterface=1

 _ And then got lots of these errors
UpdateData failed: A timeout has occurred when waiting for new data!

_ I tried doing this (without luck)
rmmod -f gspca_kinect

_ Other: save image

dd if=/dev/sdc of=~/2012-09-18-wheezy-raspbian_16GB_OpenNI-Stable+Unstable.img

tar -zcvf 2012-09-18-wheezy-raspbian_16GB_OpenNI-Stable+Unstable.img.tar 2012-09-18-wheezy-raspbian_16GB_OpenNI-Stable+Unstable.img.img

Ernesto Sirolli: Want to help someone? Shut up and listen!

Kinect + Raspberry Pi

I have been trying to get the Kinect to work with Raspberry Pi (attempting to get raw depth data) but still keep on hitting roadblocks. The motor was able to tilt when using libfreenect. As for Open NI, I was able to get it to recognize the kinect but unable to get data from the camera.


Install libraries using this tutorial (use unstable) http://mewgen.com/Ge107_files/20120921%20Setting%20up%20Rasberry%20pi%20for%20the%20Xtion%20and%20kinect.html

alternate link: http://gremsi.blogspot.com/2013/04/installing-openni-sensorkinect-and.html

Status: Recognizes the kinect but cannot transfer data.

General Commands:
compile sample files
~/Desktop/kinect/OpenNI/Platform/Linux/Build $ make

build/install open ni
~/Desktop/kinect/unstable/OpenNI/Platform/Linux/CreateRedist $ sudo ./RedistMaker.Arm
~/Desktop/kinect/unstable/OpenNI/Platform/Linux/Redist $ sudo ./install.sh

Trial and Error

1. used this site to change usbinterface line in unstable sensor kinect: http://daybydaylinux.blogspot.com/2012/12/how-to-compile-openni-and-sensorkinect.html
1. cd ~/kinect/SensorKinect/Platform/Linux/Redist/Sensor-Bin-Linux-Arm-v5.1.2.1/Config/
   
   sudo vi GlobalDefaultsKinect.inimodify`;UsbInterface=2` into `UsbInterface=1`
2. Status: instead of saying usb interface not supported, it says: UpdateData failed: A timeout has occurred when waiting for new data!
2. Attempt to fix timeout issue: change line in Include/XnTypes.h for data_timeout from 2000 to 20,000 and build open ni
1. status: changing it to 20seconds and 1 minute and did not work
3. Was not able to change the FPS for depth because it only supports 15fps (http://openni-discussions.979934.n3.nabble.com/OpenNI-dev-How-to-change-FPS-td2500973.html)

Trace Back: NiSimpleRead
Attempting to trace the problem to see where it occurs.

1. Samples/NiSimpleRead/NiSimpleRead.cpp
   line 103: context.WaitOneUpdateAll(depth)
2. Include/XnCppWrapper.h
   line 9421: return xnWaitOneUpdateAll(...)
3. Source/OpenNI.cpp
   line 2601: xnWaitForCondition(...)
4. Source/OpenNI.cpp
   line 2552: xnOSWaitForCondition(...)
5. Source/XnOS.cpp
   line 236: xnOSWaitEvent(...)
6. Source/OpenNI/Linux/LinuxEvents.cpp
   line 160: pEvent->Wait(...)

Somehow xnOSWaitEvent is linked with Source/OpenNI/Linux/XnUSBLinux.cpp: xnUSBReadThreadMain. I am assuming its linked because I got an warning (after i set the usb interface to 1): USB events thread - failed to set priority. And this error message originates in the file XnUSBLinux

I'm currently trying to see if data is being transferred at all or if the Kinect is just waiting and not sending data.

Side note: you can change the option to print out the logs for OpenNI in the Data/SampleConfig.xml file.


[Update: 2/24]: Some people had some success with the beagleboard and the kinect:
  http://www.pansenti.com/wordpress/?page_id=1772
  http://instructionalrobotics.blogspot.com/2013/01/kinect-under-beagleboard-c4.html 

[Update: 2/27]:
I updated the file XnUSBLinux.cpp to printout the transfer status. In the xnUSBReadThreadMain method, I added these lines (in bold):

//more code...

else // transfer done

{

if (pBufferInfo->nLastStatus == LIBUSB_TRANSFER_COMPLETED || // read succeeded

pBufferInfo->nLastStatus == LIBUSB_TRANSFER_CANCELLED)   // cancelled, but maybe some data arrived

{

if(pBufferInfo->nLastStatus == LIBUSB_TRANSFER_COMPLETED)

{

printf("***** pBufferInfo->nLastStatus = LIBUSB_TRANSFER_COMPLETED\n");

}

else if(pBufferInfo->nLastStatus == LIBUSB_TRANSFER_CANCELLED)

{

printf("***** pBufferInfo->nLastStatus == LIBUSB_TRANSFER_CANCELLED\n");

}

if (pTransfer->type == LIBUSB_TRANSFER_TYPE_ISOCHRONOUS)

{

XnUInt32 nTotalBytes = 0;

// some packets may return empty, so we need to remove spaces, and make the buffer sequential

for (XnUInt32 i = 0; i < pTransfer->num_iso_packets; ++i)

{

struct libusb_iso_packet_descriptor* pPacket = &pTransfer->iso_packet_desc[i];

if (/*pPacket->status == LIBUSB_TRANSFER_COMPLETED && */pPacket->actual_length != 0)

{

XnUChar* pBuffer = libusb_get_iso_packet_buffer_simple(pTransfer, i);

// if buffer is not at same offset, move it

if (pTransfer->buffer + nTotalBytes != pBuffer)

{

// printf("buffer %d has %d bytes. Moving to offset %d...\n", i, pPacket->actual_length, nTotalBytes);

memmove(pTransfer->buffer + nTotalBytes, pBuffer, pPacket->actual_length);

}

nTotalBytes += pPacket->actual_length;

}

else if (pPacket->status != LIBUSB_TRANSFER_COMPLETED)

{

xnLogWarning(XN_MASK_USB, "2 Endpoint 0x%x, Buffer %d, packet %d Asynch transfer failed (status: %d)", pTransfer->endpoint, pBufferInfo->nBufferID, i, pPacket->status);

}

if(pPacket->status == LIBUSB_TRANSFER_COMPLETED)

{

printf("*****pPacket->status = LIBUSB_TRANSFER_COMPLETED. Length: %d\n", pPacket->actual_length);

}

else if(pPacket->status == LIBUSB_TRANSFER_CANCELLED)

{

printf("*****pPacket->Status = TRANSFER CANCELLED Length: %d\n", pPacket->actual_length);

}

}

if (nTotalBytes != 0)

{

// call callback method

pBufferInfo->pThreadData->pCallbackFunction(pTransfer->buffer, nTotalBytes, pBufferInfo->pThreadData->pCallbackData);

}

}

else

{

// call callback method

pBufferInfo->pThreadData->pCallbackFunction(pTransfer->buffer, pTransfer->actual_length, pBufferInfo->pThreadData->pCallbackData);

}

}

//more code...

After building and installing, I ran ./Sample-NiSimpleRead again. Click here to see the output (the lines that begin with '*****' are my print line statements). 

The main things to focus on are these lines:

*****pPacket->status = LIBUSB_TRANSFER_COMPLETED. Length: 1760 

*****pPacket->status = LIBUSB_TRANSFER_COMPLETED. Length: 1920 

*****pPacket->status = LIBUSB_TRANSFER_COMPLETED. Length: 0

The transfer seems to be completed but the length of the packet is 0. Not sure why this is happening. 

I left Sample-NiSimpleRead running longer, and noticed I was getting data back from the Kinect. The packet would hold 1760 or 1920 bytes at a time. (the log file has been updated with the new log)

[Update 3/29]
Unfortunately, I wasn't able to get it working with the Kinect. I am currently looking into other options. 

• Asus Xtion + Raspberry Pi: I was able to get frames back from an Asus Xtion.
• Kinect + Beagle Board (or Panda Board): I haven't had the chance to try this out but in theory since this is a little more powerful than the RPi, it could work with the kinect. 

I am currently working on getting and compressing the depth images using OpenNI. See OpenNI + Depth Compression

Amy Cuddy: Your body language shapes who you are

Assassins: A Real World Game That Uses NFC

Project Links Github

#of Team Members: 4


This project description was written by the whole team. “Assassins” is a live-action "killer" game. Players try to eliminate each other from the game using mock weapons in an effort to become the last surviving player. Currently, players must eliminate their target in person and report their success to a webpage online or to a game administrator in person; this reporting system is tedious and relies heavily on an honor system. In our solution, each player will be assigned a target when the game begins. The assassin (a player with a target) will need to eliminate the target inconspicuously. The target’s phone will report its location randomly throughout the day. If the target notices the assassin first, he will be able to bump phones with the assassin to create a “timeout” period where the assassin will not be able to eliminate the target. However, if the assassin successfully locates the target without getting noticed, he will be able to bump phones to eliminate the target. These scenarios will be dependent on an honor system of who is noticed first. Assassin When the game begins, the assassin (who is also a target, as everyone playing has an assigned assassin and an assigned target) receives a notification with the name of their target. At random intervals throughout the day, they receive updates on the location of their target. They also are notified of their target’s location whenever their target “kills” their own target. When they successfully “kill” their target using whatever rules have been agreed upon, they then bump phones with the target, and the app registers the kill. The assassin’s assigned killer is also notified of the assassin’s location, and the assassin is given a new target. Target The target will use the application if he notices the assassin before getting eliminated. He will bump phones with the assassin, while the app is open, and initiate the transfer. This will create a “timeout” period where the assassin cannot eliminate the target for a set amount of time. We plan to use the Android SDK to build the game and NFC technology to share data between users. Location services such as Google Maps will also be used to display targets and other information. Google Cloud Messaging might be used if we need push notifications from servers to users.

Medical Viewer for Emory

Project Links
Github

# of Team Members: 5

This was my senior design project; we worked with emory to create a tablet app that lets doctors and nurses monitor patient data and annotate this data. It uses a variety of technologies and the goal is to make the application as easy to use while still showing all essential data.

Project Documents
Project Documentation/Description
This project documentation/description was written by the whole team.

MRS: Mobile Response System

Project Links

Website

# of Team Members: 4

This project description was written by the whole team.

Our project is designed to be a complete replacement for the current Student Response System provided by TurningPoint.  The turning point system requires students to purchase a $50 ‘clicker’ device or purchase a 3 month subscription to their mobile service (the semester is 3.5 months long), which is not optimum for students who are already being hit with tuition and fee increases virtually every semester.  Additionally, professors and TAs have complained about the software that comes with TurningPoint for managing sessions and collecting student responses; it has caused issues with some classes’ gradebooks in the past and has generally been very unreliable.    

The solution to this is a web-based, GTMob plugin that will make use of location services when available and alternate location-detection when not available that will allow students to effectively “bring their own device” as a personal response device.  We also plan to allow professors to upload a gradebook CSV file from Sakai, which will then be populated with the student’s PRS scores.  This file can then be downloaded from our tool on GTMob and reuploaded to Sakai, importing grades.