392247 Project: Temporal Modeling of Multimodal Behavioral Data for ASC Assessment (Pj) (SoSe 2026)

Autism Spectrum Condition (ASC) is associated with subtle, complex patterns of non-verbal behavior. Recent work using multimodal fusion of video data (e.g., facial expressions, gaze, head motion, audio, and heart rate) has demonstrated promising results in supporting clinical assessments of ASC. However, much of the analysis has focused on static features, overlooking the temporal dynamics that unfold during social interactions. This project aims to explore the temporal aspects of our large, balanced dataset by applying state-of-the-art deep learning techniques such as Long Short-Term Memory (LSTM) networks and transformer models. By uncovering temporal patterns in these multimodal signals, the project seeks to enhance the understanding of behavioral cues associated with ASC and potentially improve diagnostic support. - Background in machine learning and deep learning. - Familiarity with Python, PyTorch or Tensorflow - Basic understanding of sequence modeling (LSTM, transformers)

Requirements for participation, required level

Skills and Requirements:
- Background in machine learning and deep learning.
- Familiarity with Python, and experience using PyTorch or TensorFlow.
- Basic understanding of sequence modeling (LSTM, transformers) is advantageous.
- Strong analytical and problem-solving skills.