No description
Find a file
2017-11-14 20:36:58 +00:00
cmake/modules Revert to old TBB finding. 2017-02-15 12:36:20 -05:00
exe More model simplification, getting there with img. 2017-11-14 20:36:58 +00:00
image_sequence Master commit of OpenFace. 2016-04-28 15:40:36 -04:00
imgs Updates to images 2017-05-09 16:35:04 -04:00
lib Clearer error reporting. 2017-11-14 20:19:11 +00:00
matlab_runners More model simplification, getting there with img. 2017-11-14 20:36:58 +00:00
matlab_version AU prediction de-coupling from landmark detection modules, allowing different PDMs to be used (or a different 68 point detector altogether). 2017-10-23 17:58:35 +01:00
samples Adding different sample files and replacing images in the wiki. 2017-05-09 13:21:12 -04:00
.gitignore Bringing the ICT-3DHP script up to date with the new interface 2017-11-14 07:46:01 +00:00
.travis.yml Removing clang compilation with travis. 2017-09-10 21:07:37 +01:00
appveyor.yml Fixes for travis and appveyor 2017-05-09 10:11:21 -04:00
CMakeLists.txt Separating gaze analysis into a separate project, getting closer to being ready to integrate GUI and CE-CLM branches. 2017-10-26 08:50:15 +01:00
Copyright.txt Corrections of the copyright. 2017-06-06 18:53:06 -04:00
install.sh Remove duplicate repo and unecessary commands. 2017-05-01 09:34:07 +02:00
OpenFace-license.txt Auto stash before merge of "develop" and "origin/develop" 2017-05-08 21:36:23 -04:00
OpenFace.sln Script fixes, and confidence correction where it should be (not in the executable scripts), starting work on capture in utilities 2017-11-08 09:02:50 +00:00
README.md Adding direct OpenBLAS code for some speedups. 2017-10-23 18:54:02 +01:00

OpenFace: an open source facial behavior analysis toolkit

Build Status Build status

Over the past few years, there has been an increased interest in automatic facial behavior analysis and understanding. We present OpenFace an open source tool intended for computer vision and machine learning researchers, affective computing community and people interested in building interactive applications based on facial behavior analysis. OpenFace is the first open source tool capable of facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation. The computer vision algorithms which represent the core of OpenFace demonstrate state-of-the-art results in all of the above mentioned tasks. Furthermore, our tool is capable of real-time performance and is able to run from a simple webcam without any specialist hardware.

The code was written mainly by Tadas Baltrusaitis during his time at the Language Technologies Institute at the Carnegie Mellon University; Computer Laboratory, University of Cambridge; and Institute for Creative Technologies, University of Southern California.

Special thanks goes to Louis-Philippe Morency and his MultiComp Lab at Institute for Creative Technologies for help in writing and testing the code, and Erroll Wood for the gaze estimation work.

WIKI

For instructions of how to install/compile/use the project please see WIKI

More details about the project - http://www.cl.cam.ac.uk/research/rainbow/projects/openface/

Functionality

The system is capable of performing a number of facial analysis tasks:

  • Facial Landmark Detection

Sample facial landmark detection image

  • Facial Landmark and head pose tracking (links to YouTube videos)

Multiple Face Tracking Multiple Face Tracking

  • Facial Action Unit Recognition
  • Gaze tracking (image of it in action)
  • Facial Feature Extraction (aligned faces and HOG features)

Sample aligned face and HOG image

Citation

If you use any of the resources provided on this page in any of your publications we ask you to cite the following work and the work for a relevant submodule you used.

Overall system

OpenFace: an open source facial behavior analysis toolkit Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency, in IEEE Winter Conference on Applications of Computer Vision, 2016

Facial landmark detection and tracking

Constrained Local Neural Fields for robust facial landmark detection in the wild Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. in IEEE Int. Conference on Computer Vision Workshops, 300 Faces in-the-Wild Challenge, 2013.

Eye gaze tracking

Rendering of Eyes for Eye-Shape Registration and Gaze Estimation Erroll Wood, Tadas Baltrušaitis, Xucong Zhang, Yusuke Sugano, Peter Robinson, and Andreas Bulling in IEEE International. Conference on Computer Vision (ICCV), 2015

Facial Action Unit detection

Cross-dataset learning and person-specific normalisation for automatic Action Unit detection Tadas Baltrušaitis, Marwa Mahmoud, and Peter Robinson in Facial Expression Recognition and Analysis Challenge, IEEE International Conference on Automatic Face and Gesture Recognition, 2015

Copyright

Copyright can be found in the Copyright.txt

You have to respect boost, TBB, dlib, OpenBLAS, and OpenCV licenses.

Commercial license

For inquiries about the commercial licensing of the OpenFace toolkit please contact innovation@cmu.edu

Final remarks

I did my best to make sure that the code runs out of the box but there are always issues and I would be grateful for your understanding that this is research code and not full fledged product. However, if you encounter any problems/bugs/issues please contact me on github or by emailing me at Tadas.Baltrusaitis@cl.cam.ac.uk for any bug reports/questions/suggestions.