MASTER Assignment
Deepfake detection using capsule networks with long short-term memory networks
Type : Master M-CS
Period: Jan, 2019- Aug, 2020
Student : Mehra, A. (Akul, Student M-CS)
Date Final project: August 27, 2020
Supervisors:
Abstract:
With the recent advancement of technology, particularly with graphics processing and artificial intelligence algorithms, fake media generation has become easier. Using deep learning techniques like Deepfakes and FaceSwap, anyone can generate fake videos by manipulating the face/voice of the target in the video. These deepfakes can be used for malicious purposes like phishing scams and fake news. Detecting face tampering in realistic forged videos generated has become of utmost importance. This paper provides an overview of what inconsistencies are introduced in videos due to deepfake generation and proposes a spatio-temporal hybrid model of Capsule Networks integrated with LSTM Networks. This model exploits the inconsistencies and identifies real and fake videos and is our contribution towards deepfake detection. Using visualization of the capsule’s activation, we understand what features the capsules learn and provide an explanation for identifying deepfakes and real videos. Using 3 different frame selection techniques, we also show that frame selection has a significant impact on performance. With almost comparable performance with the state-of-the-art model, in contrast to the size, our model has 1/5th the number of parameters and 1/4th the size of the state-of-the-art model and hence, is a lighter model and has reduced computational cost.