Example App

Overview

The example applications shows how to use the addon, including receiving data from the addon thread using events. At startup, the app looks for .dat and .mat files in the model/patch_experts folder because they are needed for some detectors (see extra files).

Settings

The following settings are exposed in the app:

• settings/app/framerate the application framerate. See note below.
• settings/camera/index the index of the camera, in the traditional OF sense
• settings/camera/framerate the requested frame rate of the app
• settings/camera/width the requested camera width
• settings/camera/height the requested camera height
• settings/tracking/doCvTracking set to 1 to perform tracking using ofxCv
• settings/tracking/multipleFaces set to 1 perform tracking of multiple faces
• settings/tracking/maxFaces the max number of faces detected, if multipleFaces is enabled
• settings/tracking/detector:face the choice of face detector (0: HAAR_DETECTOR default value, 1: HOG_SVM_DETECTOR, 2: MTCNN_DETECTOR)
• settings/tracking/detector:landmarks the choice of landmark detector (0: CLM_DETECTOR, 1: CLNF_DETECTOR, 2: CECLM_DETECTOR default value)
• settings/tracking/persistenceMs the number of ms to wait before forgetting a tracked face
• settings/tracking/tolerancePixels the max number of pixels a face can move over two frames
• settings/camera/fx the camera's focal length fx
• settings/camera/fy the camera's focal length fy
• settings/camera/cx the camera's principal point x position in pixels
• settings/camera/cy the camera's principal point y position in pixels

⚠️ If your app runs too fast (and/or the camera frame rate is too high), OpenFace will not work well because the OpenFace thread will not be processing consecutive frames. I had good results with an app frame rate of 15 FPS and a grabbed camera size of 320x240 px.

Keyboard

• r: reset face model
• b: toggle draw of faces
• h: toggle sending of data to OpenFace addon

Further work

• It would be awesome to get AU detection to work. See the OFX_OPENFACE_DO_FACE_ANALYSIS preprocessor definition for a starting point. For some reason, the AU face analysis location is empty or invalid. Any help is appreciated.
• Figure out why gazes are not drawn/computed properly.