Facial Expression Recognition Using a Sequential Convolutional Neural Network for Multi-Class Classification
DOI:
https://doi.org/10.64021/Keywords:
Facial Emotion Recognition, Convolutional Neural Network, Deep Learning, Affective Computing, Multi-Class ClassificationAbstract
Facial Emotion Recognition (FER) has become an important area of research in affective computing, human–computer interaction, intelligent surveillance, and healthcare applications due to its ability to automatically identify and interpret human emotional states from facial expressions. This study presents a lightweight Sequential Convolutional Neural Network (S-CNN) framework for multi-class facial emotion recognition using facial expression images categorized into eight emotional classes: Anger, Contempt, Disgust, Fear, Happy, Neutral, Sad, and Surprised. The proposed framework integrates image preprocessing, data augmentation, convolutional feature extraction, and deep learning-based classification to develop an efficient and computationally lightweight emotion recognition system. The model was trained and evaluated using a publicly available facial expression dataset, with performance assessed using accuracy, precision, recall, F1-score, and confusion matrix analysis. Experimental results demonstrated strong classification performance, achieving an overall accuracy of 92%, a weighted precision of 87%, a weighted recall of 92%, and a weighted F1-score of 89%. The confusion matrix further revealed effective discrimination among most emotional categories, with only minor misclassification observed between visually similar expressions. Comparative analysis with established deep learning architectures reported in the literature, including VGG16, MobileNetV2, ResNet50, and EfficientNet-B0, highlights the potential effectiveness of the proposed lightweight architecture while maintaining lower computational complexity. The findings demonstrate that simplified CNN architectures can provide accurate and efficient facial emotion recognition, making them suitable for real-time and resource-constrained applications. Future research should explore larger benchmark datasets, cross-dataset validation, and advanced deep learning architectures to further improve generalization and robustness.References
[1] Z. Jiang, S. Harati, A. Crowell, H. S. Mayberg, S. Nemati, and G. D. Clifford, “Classifying Major Depressive Disorder and Response to Deep Brain Stimulation Over Time by Analyzing Facial Expressions,” IEEE Trans. Biomed. Eng., vol. 68, no. 2, pp. 664–672, Feb. 2021, doi: 10.1109/TBME.2020.3010472.
[2] N. B. Krishna, R. Y. Kumar, M. I. Bhanu, P. O. Reddy, R. Anjali, and P. R. Reddy, “Exploring Machine Learning Algorithms for the Detection of Depression,” in 2024 International Conference on Smart Systems for applications in Electrical Sciences (ICSSES), Tumakuru, India: IEEE, May 2024, pp. 1–6, doi: 10.1109/ICSSES62373.2024.10561447.
[3] P. Vinayagam, P. Kishore Kumar, and M. Kandhan, “FPGA-Based Automated EEG Analysis for Efficient Emotion Detection,” in 2024 Third International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE), Ballari, India: IEEE, Apr. 2024, pp. 1–6, doi: 10.1109/ICDCECE60827.2024.10548113.
[4] H. Zhang, T. Zuo, Z. Chen, X. Wang, and P. Z. H. Sun, “Evolutionary Ensemble Learning for EEG-Based Cross-Subject Emotion Recognition,” IEEE J. Biomed. Health Inform., vol. 28, no. 7, pp. 3872–3881, July 2024, doi: 10.1109/JBHI.2024.3384816.
[5] Z. Jiang et al., “Multimodal Mental Health Digital Biomarker Analysis From Remote Interviews Using Facial, Vocal, Linguistic, and Cardiovascular Patterns,” IEEE J. Biomed. Health Inform., vol. 28, no. 3, pp. 1680–1691, Mar. 2024, 10.1109/JBHI.2024.3352075.
[6] D. Yang et al., “Using Wearable and Structured Emotion-Sensing-Graphs for Assessment of Depressive Symptoms in Patients Undergoing Treatment,” IEEE Sensors J., vol. 24, no. 3, pp. 3637–3648, Feb. 2024, doi: 10.1109/JSEN.2023.3339498.
[7] H. V, S. T, and M. Siddappa, “Facial Emotion Recognition Method Based on Canny Edge Detection Using Convolutional Neural Network,” in 2023 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), Greater Noida, India: IEEE, Nov. 2023, pp. 425–430, doi: 10.1109/ICCCIS60361.2023.10425079.
[8] D.-H. Lee and J.-H. Yoo, “CNN Learning Strategy for Recognizing Facial Expressions,” IEEE Access, vol. 11, pp. 70865–70872, 2023, doi: 10.1109/ACCESS.2023.3294099.
[9] G. D. Jadhav, S. D. Babar, and P. N. Mahalle, “Exploratory Data Analysis and Algorithmic Comparison for Early Depression Detection,” in 2024 IEEE 4th International Conference on ICT in Business Industry & Government (ICTBIG), Indore, India: IEEE, Dec. 2024, pp. 1–7, doi: 10.1109/ICTBIG64922.2024.10911717.
[10] A. Sharma, A. Saxena, A. Kumar, and D. Singh, “Depression Detection Using Multimodal Analysis with Chatbot Support,” in 2024 2nd International Conference on Disruptive Technologies (ICDT), Greater Noida, India: IEEE, Mar. 2024, pp. 328–334, doi: 10.1109/ICDT61202.2024.10489080.
[11] C. Białek, A. Matiolański, and M. Grega, “An Efficient Approach to Face Emotion Recognition with Convolutional Neural Networks,” Electronics, vol. 12, no. 12, p. 2707, Jun. 2023, doi: 10.3390/electronics12122707.
[12] D. D. Mandave and L. V. Patil, “A Statistically Rigorous Comparison of MobileNetV2 and EfficientNet-B0 for Facial Expression Recognition on the FER2013 Benchmark,” Inf. Dyn. Appl., vol. 4, no. 4, pp. 189–200, Dec. 2025, doi: 10.56578/ida040401.
[13] Y. Khaireddin and Z. Chen, “Facial Emotion Recognition: State of the Art Performance on FER2013,” 2021, arXiv. doi: 10.48550/ARXIV.2105.03588.
[14] T. Lee et al., “A Deep Learning Driven Simulation Analysis of the Emotional Profiles of Depression Based on Facial Expression Dynamics,” Clin Psychopharmacol Neurosci, vol. 22, no. 1, pp. 87–94, Feb. 2024, doi: 10.9758/cpn.23.1059.
[15] V. S. Bakkialakshmi and T. Sudalaimuthu, “A Survey on Affective Computing for Psychological Emotion Recognition,” in 2021 5th International Conference on Electrical, Electronics, Communication, Computer Technologies and Optimization Techniques (ICEECCOT), Mysuru, India: IEEE, Dec. 2021, pp. 480–486, doi: 10.1109/ICEECCOT52851.2021.9707947.
[16] M. Yang et al., “LMS-VDR: Integrating Landmarks into Multi-scale Hybrid Net for Video-Based Depression Recognition,” in Pattern Recognition and Computer Vision, vol. 15040, Z. Lin, M.-M. Cheng, R. He, K. Ubul, W. Silamu, H. Zha, J. Zhou, and C.-L. Liu, Eds., in Lecture Notes in Computer Science, vol. 15040, Singapore: Springer Nature Singapore, 2025, pp. 299–312, doi: 10.1007/978-981-97-8792-0_21.
[17] M. Senthil Sivakumar, T. Gurumekala, L. Megalan Leo, and R. Thandaiah Prabu, “Expert System for Smart Virtual Facial Emotion Detection Using Convolutional Neural Network,” Wireless Pers Commun, vol. 133, no. 4, pp. 2297–2319, Dec. 2023, doi: 10.1007/s11277-024-10867-0.
[18] C. Gupta and V. Khullar, “Exploring the Role of Conventional and Non-Conventional Technologies in Identifying MDD,” in 2022 IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI), Delhi, India: IEEE, Dec. 2022, pp. 1–6, doi: 10.1109/SOLI57430.2022.10294421.
[19] Y. Guo, L. Wang, Y. Xiao, and Y. Lin, “A Personalized Spatial-Temporal Cold Pain Intensity Estimation Model Based on Facial Expression,” IEEE J. Transl. Eng. Health Med., vol. 9, pp. 1–8, 2021, doi: 10.1109/JTEHM.2021.3116867.
[20] X. Li, W. Guo, and H. Yang, “Depression severity prediction from facial expression based on the DRR_DepressionNet network,” in 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Seoul, Korea (South): IEEE, Dec. 2020, pp. 2757–2764, doi: 10.1109/BIBM49941.2020.9313597.
[21] Q. Li et al., “Machine Learning-Based Prediction of Depressive Disorders via Various Data Modalities: A Survey,” IEEE/CAA J. Autom. Sinica, vol. 12, no. 7, pp. 1320–1349, July 2025, doi: 10.1109/JAS.2025.125393.
[22] S. Zhang et al., “MTDAN: A Lightweight Multi-Scale Temporal Difference Attention Networks for Automated Video Depression Detection,” IEEE Trans. Affective Comput., vol. 15, no. 3, pp. 1078–1089, July 2024, doi: 10.1109/TAFFC.2023.3312263.
[23] M. Li, Z. Lu, Q. Cao, J. Gao, and B. Hu, “Automatic Assessment Method and Device for Depression Symptom Severity Based on Emotional Facial Expression and Pupil-Wave,” IEEE Trans. Instrum. Meas., vol. 73, pp. 1–15, 2024, doi: 10.1109/TIM.2024.3415778.
[24] M. Niu, Y. Li, J. Tao, X. Zhou, and B. W. Schuller, “DepressionMLP: A Multi-Layer Perceptron Architecture for Automatic Depression Level Prediction via Facial Keypoints and Action Units,” IEEE Trans. Circuits Syst. Video Technol., vol. 34, no. 9, pp. 8924–8938, Sept. 2024, doi: 10.1109/TCSVT.2024.3382334.
[25] J. Ye, Y. Yu, Y. Zheng, Y. Liu, and Q. Wang, “Dep-FER: Facial Expression Recognition in Depressed Patients Based on Voluntary Facial Expression Mimicry,” IEEE Trans. Affective Comput., vol. 15, no. 3, pp. 1725–1738, July 2024, doi: 10.1109/TAFFC.2024.3370103.
[26] G. Bargshady and R. Goecke, “Estimating Depression Severity from Long-Sequence Face Videos via an Ensemble Global Diverse Convolutional Model,” in 2023 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Port Macquarie, Australia: IEEE, Nov. 2023, pp. 296–303, doi: 10.1109/DICTA60407.2023.00048.
[27] G. Bargshady and R. Goecke, “Estimating Depression Severity from Long-Sequence Face Videos via an Ensemble Global Diverse Convolutional Model,” in 2023 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Port Macquarie, Australia: IEEE, Nov. 2023, pp. 296–303, doi: 10.1109/DICTA60407.2023.00048.
[28] L. He et al., “LMTformer: facial depression recognition with lightweight multi-scale transformer from videos,” Appl Intell, vol. 55, no. 3, p. 195, Feb. 2025, doi: 10.1007/s10489-024-05908-x.
[29] M. Kolahdouzi, A. Sepas-Moghaddam, and A. Etemad, “FaceTopoNet: Facial Expression Recognition Using Face Topology Learning,” IEEE Trans. Artif. Intell., vol. 4, no. 6, pp. 1526–1539, Dec. 2023, doi: 10.1109/TAI.2022.3207450.
[30] G. P. OISE, P. O. EJENARHOME, and A. S. OYEDOTUN, “Enhancing road safety through deep learning-based drowsiness detection using vision AI,” Journal of Electrical Systems and Information Technology, vol. 13, no. 1, p. 30, Mar. 2026, doi: 10.1186/s43067-026-00335-z.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Godfrey Oise, Immunhierokene Clinton OBRORINDO, Roli Lydia OSHASHA, Kevin Chinedu PIUS, Felix Oshiorenoya ULOKO (Author)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
