3.4.2. Long Short-Term Memory Networks (LSTM)

Long Short-Term Memory Networks (LSTMs) are a special type of Recurrent Neural Networks (RNNs). RNNs are designed to process sequential data by connecting information across different points in a sequence, which is useful for tasks like language processing and time-dependent prediction, such as recognizing emotions from a stream of signals. Unlike RNNs, which struggle to remember information over long time gaps, LSTMs have a “memory” system with cells and gates. These mechanism lets the network decide which information to keep or discard, overall help capture better patterns without “forgetting”. However, LSTMs can be slow to train and need lots of data.

Yang et al. employed LSTMs to combine EEG signals and facial expressions for continuous emotion recognition in deaf subjects, finding that EEG data offered richer emotional insights than facial expressions. In text-based emotion detection, Gupta et al. used GloVe (Global Vectors for Word Representation) embeddings with an LSTM, achieving high accuracy (F1 score of 0.93) in capturing emotions from textual data, though it struggled with subtle emotions like surprise. Aydın applied LSTMs on EEG-derived complexity markers (capturing both the local complexity and inter-hemispheric coordination) for distinguishing positive and negative emotions, reaching 98% classification accuracy. Song et al. used an attention-enhanced LSTM (A-LSTM) to model multimodal physiological data from their own custom-built dataset, MPED, achieving high precision in discrete emotion classification.

Bidirectional LSTMs (BiLSTMs) enhance LSTM capabilities by processing sequences in both directions, improving context comprehension. Du et al. used a Convolutional BiLSTM to detect driver emotions from facial and heart rate data in real-time, while on another study Du together with other colleagues applied a BiLSTM-CRF (Conditional Random Fields) model for contactless emotion recognition with complexion and heart rate, achieving over 80% accuracy in detecting emotions like anger and doubt.