FACIAL EMOTION AND SENTIMENT DETECTION USING CONVOLUTIONAL NEURAL NETWORK
Keywords:
Artificial Intelligence, Deep Learning, Convolutional Neural Network, Facial Emotion Detection, Sentiment Analysis, Machine Learning, Sentiment DetectionAbstract
This paper develops a deep convolutional neural network model for facial emotion recognition. The model is trained on a large dataset of 36,000 grayscale images of faces labeled with 7 categories of emotion - happy, sad, angry, surprised, fearful, disgusted and neutral. Our methodology applies 4 convolutional layers and 2 fully connected layers to extract hierarchical visual features which are classified into probabilities over the emotion categories. Analysis shows the model attains a reasonable validation accuracy of 65% at categorizing basic emotions from subtle facial expressions. However, performance declines substantially on more diverse real-world data reflecting various poses, illumination, resolution and background conditions. The paper discusses opportunities around overcoming these challenges through techniques including data augmentation to expand diversity, personalized adaptation for individual facial variation, ensemble methods, three-dimensional modeling, graph networks and self-supervised representation learning. Facial emotion recognition promises human-computer interaction applications across domains like healthcare, education and transportation. Realizing the full potential requires moving beyond individual images to incorporate context, temporal dynamics of micro expressions, and complex affective modeling achieved intuitively by humans.
References
Antipov, G. (2017). Deep learning for semantic description of visual human traits (Doctoral dissertation, Télécom ParisTech).
Bouchra, N., Mohammed, N., Nabil, H., & Aouatif, A. (2019). Deep belief network and auto-encoder for face classification.
Li, Z. (2018). Optimizing neural network structures: faster speed, smaller size, less tuning. The University of Iowa.
Phillips, P. J., Flynn, P. J., Scruggs, T., Bowyer, K. W., Chang, J., Hoffman, K., ... & Worek, W. (2005, June). Overview of the face recognition grand challenge. In 2005, the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) (Vol. 1, pp. 947–954). IEEE.
Renda, A., Barsacchi, M., Bechini, A., & Marcelloni, F. (2019). Comparing ensemble strategies for deep learning: An application to facial expression recognition. Expert Systems with Applications, 136, 1-11.
Minetto, R., Segundo, M. P., & Sarkar, S. (2019). Hydra: An ensemble of convolutional neural networks for geospatial land classification. IEEE Transactions on Geoscience and Remote Sensing, 57(9), 6530-6541.
Ou, Y. Y., Su, B. H., Tseng, S. P., Wang, J. F., & Kuan, T. W. (2018, October). Efficient emotion recognition based on a hybrid emotion recognition neural network. In 2018 International Conference on Orange Technologies (ICOT) (pp. 1-4). IEEE.
Arriaga, O., Valdenegro-Toro, M., & Plöger, P. (2017). Real-time convolutional neural networks for emotion and gender classification. arXiv preprint arXiv:1710.07557.
Robbes, R., & Janes, A. (2019, May). Leveraging small software engineering data sets with pre-trained neural networks. In 2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) (pp. 29–32). IEEE.
Bettadapura, V. (2012). Face expression recognition and analysis: the state of the art. arXiv preprint arXiv:1203.6722.
Brodie, R. J., Ilic, A., Juric, B., & Hollebeek, L. (2013). Consumer engagement in a virtual brand community: An exploratory analysis. Journal of Business Research, 66(1), 105-114.
Kim, J., & Gupta, P. (2012). Emotional expressions in online user reviews: How they influence consumers' product evaluations. Journal of Business Research, 65(7), 985–992.
Chengeta, K. (2019). A Review of Local Feature Algorithms and Deep Learning Approaches in Facial Expression Recognition with TensorFlow and Keras. In Pattern Recognition: 11th Mexican Conference, MCPR 2019, Querétaro, Mexico, June 26–29, 2019, Proceedings 11 (pp. 127–138). Springer International Publishing.
Yang, S., Luo, P., Loy, C. C., & Tang, X. (2016). Wider face: A face detection benchmark. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp.5525-5533).
Downloads
Published
Issue
Section
License
Copyright (c) -1 Dinesh Kalla, Nathan Smith, Fnu Samaah, Kiran Polimetla (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
