# 3.4.2. Long Short-Term Memory Networks (LSTM)

**Long Short-Term Memory Networks (LSTMs)** are a special type of Recurrent Neural Networks (RNNs). RNNs are designed to process sequential data by connecting information across different points in a sequence, which is useful for tasks like language processing and time-dependent prediction, such as recognizing emotions from a stream of signals. Unlike RNNs, which struggle to remember information over long time gaps, LSTMs have a “memory” system with cells and gates. These mechanism lets the network decide which information to keep or discard, overall help capture better patterns without “forgetting”. However, LSTMs can be slow to train and need lots of data.

Yang et al. \[96][^1] employed LSTMs to combine EEG signals and facial expressions for continuous emotion recognition in deaf subjects, finding that EEG data offered richer emotional insights than facial expressions. In text-based emotion detection, Gupta et al. \[97][^2] used GloVe (Global Vectors for Word Representation) embeddings with an LSTM, achieving high accuracy (F1 score of 0.93) in capturing emotions from textual data, though it struggled with subtle emotions like surprise. Aydın \[98][^3] applied LSTMs on EEG-derived complexity markers (capturing both the local complexity and inter-hemispheric coordination) for distinguishing positive and negative emotions, reaching 98% classification accuracy. Song et al. \[73][^4] used an attention-enhanced LSTM (A-LSTM) to model multimodal physiological data from their own custom-built dataset, MPED, achieving high precision in discrete emotion classification.

Bidirectional LSTMs (BiLSTMs) enhance LSTM capabilities by processing sequences in both directions, improving context comprehension. Du et al. \[99][^5] used a Convolutional BiLSTM to detect driver emotions from facial and heart rate data in real-time, while on another study Du together with other colleagues \[100][^6] applied a BiLSTM-CRF (Conditional Random Fields) model for contactless emotion recognition with complexion and heart rate, achieving over 80% accuracy in detecting emotions like anger and doubt.

[^1]: \[96]  Y. Yang, Q. Gao, X. Song, Y. Song, Z. Mao, and J. Liu, "Facial Expression and EEG Fusion for Investigating Continuous Emotions of Deaf Subjects," _IEEE Sensors Journal_, vol. 21, no. 15, pp. 16894-16903, Aug. 2021, doi: 10.1109/JSEN.2021.3078087.

[^2]: \[97]  P. Gupta, I. Roy, G. Batra, and A. K. Dubey, "Decoding Emotions in Text Using GloVe Embeddings," 2021 _International Conference on Computing, Communication, and Intelligent Systems_ (ICCCIS), Greater Noida, India, 2021, pp. 36-40, doi: 10.1109/ICCCIS51004.2021.9397132.

[^3]: \[98]  S. Aydın, "Deep Learning Classification of Neuro-Emotional Phase Domain Complexity Levels Induced by Affective Video Film Clips," _IEEE Journal of Biomedical and Health Informatics_, vol. 24, no. 6, pp. 1695-1702, June 2020, doi: 10.1109/JBHI.2019.2959843.

[^4]: \[73] T. Song, W. Zheng, C. Lu, Y. Zong, X. Zhang, and Z. Cui, "MPED: A Multi-Modal Physiological Emotion Database for Discrete Emotion Recognition," _IEEE Access_, vol. 7, pp. 12177-12191, 2019, doi: 10.1109/ACCESS.2019.2891573.

[^5]: \[99] G. Du, Z. Wang, B. Gao, S. Mumtaz, K. M. Abualnaja, and C. Du, "A Convolution Bidirectional Long Short-Term Memory Neural Network for Driver Emotion Recognition," _IEEE Transactions on Intelligent Transportation Systems_, vol. 22, no. 7, pp. 4570-4578, July 2021, doi: 10.1109/TITS.2020.3007357.

[^6]: \[100] G. Du, Q. Tan, C. Li, X. Wang, S. Teng, and P. X. Liu, "A Noncontact Emotion Recognition Method Based on Complexion and Heart Rate," _IEEE Transactions on Instrumentation and Measurement_, vol. 71, pp. 1-14, 2022, Art no. 5018614, doi: 10.1109/TIM.2022.3194858.
