The widespread dissemination of forged images generated by techniques like generative adversarial networks (GANs) has posed a serious threat to the trustworthiness of digital information, which demands effective forgery detection algorithms. Key to detecting forged images is to attend to critical regions like that might contain subtle artifacts. We leverage various attention mechanisms to detect forged media data due to their ability to learn global consistencies. We have also released several large-scale deepfake datasets to stimulate this line of research.
In this paper, we introduce ObjectFormer, an end-to-end multi-modal framework for image manipulation detection and localization. We explicitly leverage learnable object prototypes as mid-level representations to model object-level consistencies and refined patch embeddings to capture patch- level consistencies.
We propose a Multi-modal Multi-scale Transformer (M2TR) for Deepfake forensics, which uses a multi-scale transformer to detect local inconsistency at different scales and leverages frequency features to improve the robustness of detection.
We provided a dataset called wildDeepfake, which contains 707 deepfake videos collected from Internet, to help researcher build applicable detection algorithms. Besides, we proposed ADDNets, generates the attention and weights the shallow features to detect the Deepfake videos in the real world.