Cognitive-Causal Multi-Task Learning with Psychological State Conditioning for Assistive Driving Perception
2026-04-08 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors present CauPsi, a new approach for advanced driver assistance systems that better understands the relationships between what drivers feel and the traffic around them. Unlike past methods that treated tasks separately, their framework links tasks in a chain, mimicking how people process environmental info and then regulate behavior. It also uses driver facial and body data to estimate psychological state, which influences all recognition tasks. Tested on a dataset, their method outperformed previous models, especially in recognizing driver emotions and behaviors. They also showed that each part of their system adds value, and the psychological signals learned patterns without needing explicit labels.
multi-task learningadvanced driver assistance systemscausal modelingdriver emotion recognitiondriver behavior recognitiontraffic context recognitionpsychological state estimationprototype embeddingsself-supervised learning
Authors
Keito Inoshita, Nobuhiro Hayashida, Akira Imanishi
Abstract
Multi-task learning for advanced driver assistance systems requires modeling the complex interplay between driver internal states and external traffic environments. However, existing methods treat recognition tasks as flat and independent objectives, failing to exploit the cognitive causal structure underlying driving behavior. In this paper, we propose CauPsi, a cognitive science-grounded causal multi-task learning framework that explicitly models the hierarchical dependencies among Traffic Context Recognition (TCR), Vehicle Context Recognition (VCR), Driver Emotion Recognition (DER), and Driver Behavior Recognition (DBR). The proposed framework introduces two key mechanisms. First, a Causal Task Chain propagates upstream task predictions to downstream tasks via learnable prototype embeddings, realizing the cognitive cascade from environmental perception to behavioral regulation in a differentiable manner. Second, Cross-Task Psychological Conditioning (CTPC) estimates a psychological state signal from driver facial expressions and body posture and injects it as a conditioning input to all tasks including environmental recognition, thereby modeling the modulatory effect of driver internal states on cognitive and decision-making processes. Evaluated on the AIDE dataset, CauPsi achieves a mean accuracy of 82.71% with only 5.05M parameters, surpassing prior work by +1.0% overall, with notable improvements on DER (+3.65%) and DBR (+7.53%). Ablation studies validate the independent contribution of each component, and analysis of the psychological state signal confirms that it acquires systematic task-label-dependent patterns in a self-supervised manner without explicit psychological annotations.