392280 ISY Project: Facial Motion Tracking with Optical Flow (Pj) (SoSe 2022)

Short Description:
Facial expressions convey significant non-verbal information necessary for communication and interaction. Current tools for facial behavior, such as Open Face [1], use machine learning methods to identify features such as facial landmarks and facial action coding, but often struggle with facial action units with limited or no representation in training data. In this project, you will develop a facial motion tracking method using an optical flow-based approach. Additionally, you will develop a method to identify the activation of facial action units based on the tracking algorithm. To evaluate the effectiveness of your approach, you will also establish a methodology to compare your approach with the Facial Action Unit detection from state-of-the-art facial behavior analysis systems such as OpenFace [1].

[1] https://github.com/TadasBaltrusaitis/OpenFace

Required skills:

• Python and/or C++

• Working knowledge of Machine Learning Methods

In case this would not find enough interest for a team project, this project proposal would be also offered (in reduced/modified form)

• an individual project

• a project for only 2 students