IR-capsule: two-stream network for face forgery detection.

Lin, Kaihan; Han, Weihong; Li, Shudong; Gu, Zhaoquan; Zhao, Huimin; Ren, Jinchang; Zhu, Li; Lv, Jujian

Abstract

With the emergence of deep learning, generating forged images or videos has become much easier in recent years. Face forgery detection, as a way to detect forgery, is an important topic in digital media forensics. Despite previous works having made remarkable progress, the spatial relationships of each part of the face that has significant forgery clues are seldom explored. To overcome this shortcoming, a two-stream face forgery detection network that fuses Inception ResNet stream and capsule network stream (IR-Capsule) is proposed in this paper, which can learn both conventional facial features and hierarchical pose relationships and angle features between different parts of the face. Furthermore, part of the Inception ResNet V1 model pre-trained on the VGGFACE2 dataset is utilized as an initial feature extractor to reduce overfitting and training time, and a modified capsule loss is proposed for the IR-Capsule network. Experimental results on the challenging FaceForensics++ benchmark show that the proposed IR-Capsule improves accuracy by more than 3% compared with several recently published methods.

Citation

LIN, K., HAN, W., LI, S., GU, Z., ZHAO, H., REN, J., ZHU, L. and LV, J. 2023 IR-capsule: two-stream network for face forgery detection. Cognitive computation [online], 15(1), pages 13-22. Available from: https://doi.org/10.1007/s12559-022-10008-4

• Journal Article Type: Article
• Acceptance Date: Mar 6, 2022
• Online Publication Date: Jun 2, 2022
• Publication Date: Jan 31, 2023
• Deposit Date: Jun 30, 2022
• Publicly Available Date: Jun 3, 2023
• Journal: Cognitive computation
• Print ISSN: 1866-9956
• Electronic ISSN: 1866-9964
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 15
• Issue: 1
• Pages: 13-22
• DOI: https://doi.org/10.1007/s12559-022-10008-4
• Keywords: Facial recognition; Face forgery detection; Machine learning; Artificial intelligence; Two-stream network; IR-capsule; Capsule network; Inception ResNet
• Public URL: https://rgu-repository.worktribe.com/output/1688273