Tinuk Agustin; Moch. Hari Purwidiantoro; Mochammad Luthfi Rahmadi

Deep Learning for Robust Facial Expression Recognition: A Resilient Defense Against Adversarial Attacks

PDF (930KB), PP.39-52

Views: 0 Downloads: 0

Author(s)

Tinuk Agustin ^1,* Moch. Hari Purwidiantoro ¹ Mochammad Luthfi Rahmadi ²

1. Informatics, STMIK AMIKOM Surakarta, Sukoharjo, 57163, Indonesia

2. Informatics, Universitas Siber Muhammadiyah, Yogyakarta, 55253, Indonesia

* Corresponding author.

DOI: https://doi.org/10.5815/ijisa.2024.05.04

Received: 10 Apr. 2024 / Revised: 27 Jun. 2024 / Accepted: 10 Sep. 2024 / Published: 8 Oct. 2024

Index Terms

Adversarial Training, Convolutional Neural Network, Adversarial Example, Model Robustness

Abstract

Adversarial attacks can be extremely dangerous, particularly in scenarios where the precision of facial expression identification is of utmost importance. Hiring adversarial training methods proves effective in mitigating these threats. Although effective, this technique requires large computing resources. This study aims to strengthen deep learning model resilience against adversarial attacks while optimizing performance and resource efficiency. Our proposed method uses adversarial training techniques to create adversarial examples, which are permanently stored as a separate dataset. This strategy helps the model learn and enhances its resilience to adversarial attacks. This study also evaluates models by subjecting them to adversarial attacks, such as the One Pixel Attack and the Fast Gradient Sign Method, to identify any potential vulnerabilities. Moreover, we use two different model architectures to see how well they are protected against adversarial attacks. It compared their performances to determine the best model for making systems more resistant while still maintaining good performance. The findings show that the combination of the proposed adversarial training technique and an efficient model architecture outcome in increased resistance to adversarial attacks. This also improves the reliability of the model and saves more resources for computation. This is evidenced by the high accuracy results achieved at 98.81% accuracy on the CK+ datasets. The adversarial training technique proposed in this study offers an efficient alternative to overcome the limitations of computational resources. This fortifies the model against adversarial attacks, resulting in significant increases in model resilience without loss of performance.

Cite This Paper

Tinuk Agustin, Moch. Hari Purwidiantoro, Mochammad Luthfi Rahmadi, "Deep Learning for Robust Facial Expression Recognition: A Resilient Defense Against Adversarial Attacks", International Journal of Intelligent Systems and Applications(IJISA), Vol.16, No.5, pp.39-52, 2024. DOI:10.5815/ijisa.2024.05.04

Reference

[1]W. Wei et al., “Adversarial Deception in Deep Learning: Analysis and Mitigation,” Proc. - 2020 2nd IEEE Int. Conf. Trust. Priv. Secur. Intell. Syst. Appl. TPS-ISA 2020, pp. 236–245, 2020. 10.1109/TPS-ISA50397.2020.00039.
[2]D. W. Otter, J. R. Medina, and J. K. Kalita, “A Survey of the Usages of Deep Learning for Natural Language Processing,” IEEE Trans. Neural Networks Learn. Syst., vol. 32, no. 2, pp. 604–624, Feb. 2021. doi: 10.1109/TNNLS.2020.2979670.
[3]Y. Yang and Y. Yue, “English speech sound improvement system based on deep learning from signal processing to semantic recognition,” Int. J. Speech Technol., vol. 23, no. 3, pp. 505–515, Sep. 2020. doi: 10.1007/s10772-020-09733-8.
[4]A. Ortis, G. Farinella, and S. Battiato, “An Overview on Image Sentiment Analysis: Methods, Datasets and Current Challenges,” in Proceedings of the 16th International Joint Conference on e-Business and Telecommunications, 2019, pp. 290–300. doi: 10.5220/0007909602900300.
[5]W. Wu, Z. Sun, Y. Song, J. Wang, and W. Ouyang, “Transferring Vision-Language Models for Visual Recognition: A Classifier Perspective,” Int. J. Comput. Vis., Sep. 2023. doi: 10.1007/s11263-023-01876-w.
[6]Z. Meng, M. Zhang, and H. Wang, “CNN with Pose Segmentation for Suspicious Object Detection in MMW Security Images,” Sensors, vol. 20, no. 17, p. 4974, Sep. 2020. doi: 10.3390/s20174974.
[7]M. Puttagunta and S. Ravi, “Medical image analysis based on deep learning approach,” Multimed. Tools Appl., vol. 80, no. 16, pp. 24365–24398, Jul. 2021. doi: 10.1007/s11042-021-10707-4.
[8]D. R. Sarvamangala and R. V. Kulkarni, “Convolutional neural networks in medical image understanding: a survey,” Evol. Intell., vol. 15, no. 1, pp. 1–22, Mar. 2022. doi: 10.1007/s12065-020-00540-3.
[9]M. Senthil Sivakumar, T. Gurumekala, L. Megalan Leo, and R. Thandaiah Prabu, “Expert System for Smart Virtual Facial Emotion Detection Using Convolutional Neural Network,” Wirel. Pers. Commun., vol. 133, no. 4, pp. 2297–2319, Dec. 2023. doi: 10.1007/s11277-024-10867-0.
[10]L. Alzubaidi et al., Review of deep learning: concepts, CNN architectures, challenges, applications, future directions, vol. 8, no. 1. Springer International Publishing, 2021. doi: 10.1186/s40537-021-00444-8.
[11]J. Damilola Akinyemi and O. F. Williams Onifade, “An Individualized Face Pairing Model for AgeInvariant Face Recognition,” Int. J. Math. Sci. Comput., vol. 9, no. 1, pp. 1–12, Feb. 2023, doi: 10.5815/ijmsc.2023.01.01.
[12]D. Graupe, “Deep Learning Convolutional Neural Network,” Deep Learn. Neural Networks, pp. 41–55, 2016. doi: 10.1142/9789813146464_0005.
[13]C. Szegedy et al., “Intriguing properties of neural networks,” Dec. 2013, [Online]. Available: http://arxiv.org/abs/1312.6199. Access, May 5, 2024.
[14]N. Akhtar, A. Mian, N. Kardan, and M. Shah, “Advances in Adversarial Attacks and Defenses in Computer Vision: A Survey,” IEEE Access, vol. 9, pp. 155161–155196, 2021.
[15]L. Rice, E. Wong, and J. Z. Kolter, “Overfitting in adversarially robust deep learning,” Feb. 2020, [Online]. Available: http://arxiv.org/abs/2002.11569. Access, May 5, 2024.
[16]Y. Wang et al., “Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey,” Mar. 2023, [Online]. Available: http://arxiv.org/abs/2303.06302. Access, May 5, 2024.
[17]N. Mani, M. Moh, and T.-S. Moh, “Defending Deep Learning Models Against Adversarial Attacks,” Int. J. Softw. Sci. Comput. Intell., vol. 13, no. 1, pp. 72–89, Jan. 2021. doi: 10.4018/IJSSCI.2021010105.
[18]I. Rosenberg, A. Shabtai, Y. Elovici, and L. Rokach, “Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain,” Jul. 2020, [Online]. Available: http://arxiv.org/abs/2007.02407. Access, May 5, 2024.
[19]C. Wang, J. Wang, and Q. Lin, “Adversarial Attacks and Defenses in Deep Learning: A Survey,” in Intelligent Computing Theories and Application, 2021, pp. 450–461. doi: 10.1007/978-3-030-84522-3_37.
[20]E. Nowroozi, M. Mohammadi, P. Golmohammadi, Y. Mekdad, M. Conti, and A. S. Uluagac, “Resisting Deep Learning Models Against Adversarial Attack Transferability Via Feature Randomization,” IEEE Trans. Serv. Comput., pp. 1–12, 2023. doi: 10.1109/TSC.2023.3329081.
[21]Y. Ganin et al., “Domain-Adversarial Training of Neural Networks,” May 2015, [Online]. Available: http://arxiv.org/abs/1505.07818. Access, May 5, 2024.
[22]W. Zhao, S. Alwidian, and Q. H. Mahmoud, “Adversarial Training Methods for Deep Learning: A Systematic Review,” Algorithms, vol. 15, no. 8, p. 283, Aug. 2022. doi: 10.3390/a15080283.
[23]F. V. Massoli, F. Carrara, G. Amato, and F. Falchi, “Detection of Face Recognition Adversarial Attacks,” Comput. Vis. Image Underst., vol. 202, no. September 2020, p. 103103, 2021. doi: 10.1016/j.cviu.2020.103103.
[24]S. Liu and Y. Han, “ATRA: Efficient adversarial training with high-robust area,” Vis. Comput., vol. 40, no. 5, pp. 3649–3661, May 2024. doi: 10.1007/s00371-023-03057-9.
[25]H. Ren, T. Huang, and H. Yan, “Adversarial examples: attacks and defenses in the physical world,” Int. J. Mach. Learn. Cybern., vol. 12, no. 11, pp. 3325–3336, Nov. 2021. doi: 10.1007/s13042-020-01242-z.
[26]I. Kraidia, A. Ghenai, and S. B. Belhaouari, “Defense against adversarial attacks: robust and efficient compressed optimized neural networks,” Sci. Rep., vol. 14, no. 1, p. 6420, Mar. 2024. doi: 10.1038/s41598-024-56259-z.
[27]A. Zolfi, S. Avidan, Y. Elovici, and A. Shabtai, “Adversarial Mask: Real-World Universal Adversarial Attack on Face Recognition Model,” Nov. 2021, [Online]. Available: http://arxiv.org/abs/2111.10759. Access, May 5, 2024.
[28]J. Zheng, B. Li, S. Zhang, S. Wu, L. Cao, and S. Ding, “Attack Can Benefit: An Adversarial Approach to Recognizing Facial Expressions under Noisy Annotations,” Proc. AAAI Conf. Artif. Intell., vol. 37, no. 3, pp. 3660–3668, Jun. 2023. doi: 10.1609/aaai.v37i3.25477.
[29]X. Yuan, P. He, Q. Zhu, and X. Li, “Adversarial Examples: Attacks and Defenses for Deep Learning,” IEEE Trans. Neural Networks Learn. Syst., vol. 30, no. 9, pp. 2805–2824, Sep. 2019. doi: 10.1007/978-3-030-87664-7_7.
[30]Y. Xu, K. Raja, R. Ramachandra, and C. Busch, “Adversarial Attacks on Face Recognition Systems,” 2022, pp. 139–161.
[31]H. Zheng, Z. Zhang, J. Gu, H. Lee, and A. Prakash, “Efficient Adversarial Training with Transferable Adversarial Examples,” Dec. 2019, [Online]. Available: http://arxiv.org/abs/1912.11969. Access, May 5, 2024.
[32]T. Huang et al., “Enhancing Adversarial Training via Reweighting Optimization Trajectory,” Jun. 2023, [Online]. Available: http://arxiv.org/abs/2306.14275. Access, May 5, 2024.
[33]A. Shafahi et al., “Adversarial Training for Free!” Apr. 2019, [Online]. Available: http://arxiv.org/abs/1904.12843. Access, May 5, 2024.
[34]Z. Wang, X. Li, H. Zhu, and C. Xie, “Revisiting Adversarial Training at Scale,” Jan. 2024, [Online]. Available: http://arxiv.org/abs/2401.04727. Access, May 5, 2024.
[35]X. Wei et al., “Learning Extremely Lightweight and Robust Model with Differentiable Constraints on Sparsity and Condition Number,” 2022, pp. 690–707. doi: 10.1007/978-3-031-19772-7_40.
[36]X. Yuan, P. He, Q. Zhu, and X. Li, “Adversarial Examples: Attacks and Defenses for Deep Learning,” Dec. 2017, doi: 1712.07107. Access, May 5, 2024.
[37]P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, Jun. 2010, pp. 94–101. doi: 10.1109/CVPRW.2010.5543262.
[38]J. Peck and B. Goossens, “Robust width: A lightweight and certifiable adversarial defense,” May 2024, [online]. Available: http://arxiv.org/abs/2405.15971. Access, May 5, 2024.
[39]M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” Jan. 2018, [Online]. Available: http://arxiv.org/abs/1801.04381. Access, May 5, 2024.
[40]T. Agustin, M. H. Purwidiantoro, and M. L. Rahmadi, “Enhancing Facial Expression Recognition through Ensemble Deep Learning,” in 2023 5th International Conference on Cybernetics and Intelligent System (ICORIS), Oct. 2023, pp. 1–6. doi: 10.1109/ICORIS60118.2023.10352183.
[41]I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and Harnessing Adversarial Examples,” Dec. 2014, doi: 1412.6572. Access, May 5, 2024.
[42]W. Villegas-Ch, A. Jaramillo-Alcázar, and S. Luján-Mora, “Evaluating the Robustness of Deep Learning Models against Adversarial Attacks: An Analysis with FGSM, PGD and CW,” Big Data Cogn. Comput., vol. 8, no. 1, p. 8, Jan. 2024. doi: 10.3390/bdcc8010008.
[43]Y. Liu, S. Mao, X. Mei, T. Yang, and X. Zhao, “Sensitivity of Adversarial Perturbation in Fast Gradient Sign Method,” 2019 IEEE Symp. Ser. Comput. Intell. SSCI 2019, no. 2, pp. 433–436, 2019. doi: 10.1109/TEVC.2019.2890858.
[44]J. Su, D. V. Vargas, and S. Kouichi, “One-pixel attack for fooling deep neural networks,” Oct. 2017, doi: 10.1109/TEVC.2019.2890858. Access, May 5, 2024.
[45]T. Bai, J. Luo, J. Zhao, B. Wen, and Q. Wang, “Recent Advances in Adversarial Training for Adversarial Robustness,” Feb. 2021, [Online]. Available: http://arxiv.org/abs/2102.01356. Access, May 5, 2024.
[46]S. Hussain et al., “ReFace: Real-time Adversarial Attacks on Face Recognition Systems,” Jun. 2022, [Online]. Available: http://arxiv.org/abs/2206.04783. Access, May 5, 2024.
[47]D. Chicco and G. Jurman, “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation,” BMC Genomics, vol. 21, no. 1, p. 6, Dec. 2020. doi: 10.1186/s12864-019-6413-7.
[48]I. M. Ross, “An optimal control theory for nonlinear optimization,” J. Comput. Appl. Math., vol. 354, pp. 39–51, Jul. 2019, doi: 10.1016/j.cam.2018.12.044.

International Journal of Intelligent Systems and Applications (IJISA)