Emotion recognition plays an important role in understanding human cognition and behavior, and has potential applications in brain-computer interfaces and mental health monitoring. Electroencephalography (EEG), which records the brain's electrical activity noninvasively and offers high temporal resolution, provides a promising way to recognize emotional states through neural signals.
However, accurately interpreting EEG signals remains challenging. EEG recordings contain complex relationships across different electrode channels, as well as dynamic patterns that evolve over time. Many existing deep-learning approaches divide EEG signals into separate sequences for individual channels. Although this can simplify feature extraction, it may disrupt the spatial relationships between channels and overlook the global structure of the EEG signals. Meanwhile, temporal modeling methods may struggle to capture both long-range dependencies and the continuous transitions of neural activity.
To address these challenges, the researchers developed DSE-TAM, a framework designed to learn spatial relationships and temporal dynamics jointly, rather than treating EEG channels as independent signals or relying on static representations.
The DSE-TAM framework consists of three main components: Dynamic Spatial Embedding (DSE), a Temporal Feature Extractor, and a classification module.
First, the DSE module uses graph convolutional networks to model relationships among EEG electrode channels. By introducing a learnable adjacency matrix, the model can adaptively estimate inter-channel dependencies and incorporate spatial information into the early stages of feature extraction. The resulting spatially informed EEG segments are then converted into an ordered sequence of compact tokens.
Next, the Temporal Feature Extractor combines two complementary approaches: self-attention and a state-space model based on Mamba. Self-attention helps identify relationships across different time points, capturing global temporal context, while the Mamba-based module models temporal dependencies and smoother changes in neural signals. Together, these components are designed to capture both long-range information and local temporal dynamics.
Finally, a classification module uses a [CLS] token to aggregate the learned features and predict the corresponding emotion category.
The researchers evaluated DSE-TAM on three EEG datasets: DEAP, DREAMER, and CEED. The experiments included both trial-wise cross-validation and leave-one-subject-out (LOSO) validation, allowing the model to be assessed under different evaluation settings, including its ability to generalize to previously unseen participants.
In trial-wise cross-validation, DSE-TAM achieved strong performance across the three datasets. On DEAP, it reached accuracies of 79.75% for arousal recognition and 78.27% for valence recognition. On DREAMER, it achieved 80.55% for arousal and 83.91% for valence, outperforming the benchmark methods on these tasks. On the CEED dataset, it achieved 73.75% accuracy in three-class emotion recognition, demonstrating competitive performance.
The researchers also evaluated the model using leave-one-subject-out validation, a more challenging setting in which the model is tested on participants not included in training. Although performance decreased compared with trial-wise cross-validation, DSE-TAM maintained competitive results, achieving accuracies of 61.29% and 58.80% for arousal and valence on DEAP, respectively; 61.96% and 65.11% on DREAMER; and 59.59% for three-class emotion recognition on CEED. These results highlight the challenge of generalizing EEG-based emotion recognition across individuals.
Further analyses provided insight into the contributions of the model's components. Removing the self-attention module led to the largest performance decline, with accuracy drops reaching 18.67 percentage points on DEAP, 20.28 percentage points on DREAMER, and 29.32 percentage points on CEED. Removing other components, including the spatial embedding and Mamba-based modules, also reduced performance. These findings suggest that spatial feature learning and complementary temporal modeling both contribute to the model's effectiveness.
The confusion-matrix analysis further showed that DSE-TAM improved the balance of emotion classification, particularly for some more challenging categories, such as low-arousal states in DEAP and sad-versus-neutral discrimination in CEED.
This study presents DSE-TAM as an approach to EEG-based emotion recognition that jointly considers spatial relationships among electrode channels and temporal dynamics in neural signals. By combining graph convolutional networks, self-attention, and Mamba-based state-space modeling, the framework addresses limitations associated with treating EEG channels independently and relying on a single temporal modeling strategy.
Despite its promising results, the current framework has several limitations. It does not explicitly organize EEG channels according to their spatial regions, and its Mamba module uses unidirectional state transitions, which may limit the modeling of bidirectional temporal dependencies. In addition, classification of positive emotions remains challenging on the CEED dataset. The model also assumes that all samples within a trial share the same emotion label, potentially overlooking emotional changes that occur during a trial.
Future work could explore more structured spatial modeling, bidirectional temporal architectures, and methods that account for emotion variations within individual trials. These directions may further improve the robustness and generalization of EEG-based emotion recognition.
Authors of the paper include Ke Liu, Wenrui Zhu, Rui Xu, Wenlong Wang, Zhuliang Yu, Wei Wu.
This work was supported in part by the STI 2030-Major project (grant number 2022ZD0211700 to Xinhong Zhu), the National Natural Science Foundation of China (grant numbers 62476034 and U24A20338), the Natural Science Foundation of Chongqing (grant number CSTB2025NSCQ-JM0010), the Science and Technology Research Program of Chongqing Municipal Education Commission (grant number KJZD-K202500607), and the AI Program of Shanghai Municipal Education Commission (grant number JWAIZD-4).
The paper, "Beyond Static Representations: Learning Electroencephalogram Spatiotemporal Dynamics for Enhanced Emotion Recognition", was published in the journal Cyborg and Bionic Systems on Oct 28, 2026.The article is available at https://doi.org/10.34133/cbsystems.0701 .
