Unity3d and FaceAPI integration
This video shows realtime integration of 6 degrees of freedom head tracking using a simple webcam with Unity3D. As the camera is moved closer you can see the actor rotating his head exactly like the person. Headtracking is done with the non-commercial version of FaceAPI. As the human head is moved, the algorithm calculates head pose and measures pitch, yaw and roll angles. These angles are sent to Unity3D where a script maps moves to the actor, Lerpz. When the 'F' key is pressed, the player throws a lightning bolt in the direction the person is looking at, adding a further degree of user interaction. Part of this project is open source software.
In order to download the demo get the 6 degrees of freedom streamer from http://6dofstreamer.googlecode.com/files/faceapistreamer0.9.zip . The streamer may require Intel Integrated Performance Primitive libraries, so, it's better to firstly install the non-commercial version of FaceAPI, which is available here http://www.seeingmachines.com/product/faceapi/downloads/ . In case it is not installed firstly, the ippcore5.2.dll not found error might happen.
After starting the streamer and making sure your head has been detected in FaceAPI, check the results of the integration executing Lerpz demo: http://6dofstreamer.googlecode.com/files/LerpzFaceAPI0.9.zip
Lerpz should move his head according to yours. Press the 'F' button to throw a lightning bolt in the direction he is facing. Arrows will move him and the space bar will jump.
In order to purchase the integration script one can visit this page. You can download the documentation here.