Abstract
This research examines how CNN-based deepfake detection is affected by different fixed learning rates (0.0001, 0.0002, and 0.0005). To train a lightweight CNN model, video frames were taken, modified, and normalized using the Celeb-DF (v2) dataset. The model was trained utilizing three epochs with a batch size of four, employing the Adam optimization algorithm for enhanced performance. The performance of the model was evaluated through the analysis of training accuracy, validation accuracy, training loss, and validation loss. These metrics provided a comprehensive assessment of the model's effectiveness in both learning from the training data and generalizing to the validation dataset. The results of the study indicate that a learning rate of 0.0005 leads to instability and a tendency toward overfitting, while a learning rate of 0.0001 is associated with underfitting due to slow convergence. In contrast, an optimal learning rate of 0.0002 achieves a balanced performance, yielding the highest validation accuracy at 86% and the lowest validation loss of 0.35. This highlights the importance of selecting an appropriate learning rate to enhance model performance effectively. With possible uses in enhancing GAN-based image classification systems, this study focuses on the influence of learning rate selection on deepfake detection.
References
Goodfellow, Ian J., Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. "Generative adversarial nets." Advances in neural information processing systems 27 (2014).
Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).
Kingma, Diederik P., and Jimmy Ba. "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980 (2014).
Zhang, Xu, Svebor Karaman, and Shih-Fu Chang. "Detecting and simulating artifacts in gan fake images." In 2019 IEEE international workshop on information forensics and security (WIFS), pp. 1-6. IEEE, 2019.
Heusel, Martin, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. "Gans trained by a two time-scale update rule converge to a local nash equilibrium." Advances in neural information processing systems 30 (2017).
Smith, Leslie N. "Cyclical learning rates for training neural networks." In 2017 IEEE winter conference on applications of computer vision (WACV), pp. 464-472. IEEE, 2017.
Loshchilov, Ilya, and Frank Hutter. "Sgdr: Stochastic gradient descent with warm restarts." arXiv preprint arXiv:1608.03983 (2016).
Salimans, Tim, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. "Improved techniques for training gans." Advances in neural information processing systems 29 (2016).
Mescheder, Lars, Andreas Geiger, and Sebastian Nowozin. "Which training methods for GANs do actually converge?." In International conference on machine learning, PMLR, 2018. 3481-3490.
Goyal, Priya, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. "Accurate, large minibatch sgd: Training imagenet in 1 hour." arXiv preprint arXiv:1706.02677 (2017).
Defazio, Aaron, Ashok Cutkosky, Harsh Mehta, and Konstantin Mishchenko. "When, why and how much? adaptive learning rate scheduling by refinement." (2023).
Alanazi, Meshari Huwaytim. "G-GANS for Adaptive Learning in Dynamic Network Slices." Engineering, Technology & Applied Science Research 14, no. 3 (2024): 14327-14341.
Dhar, S., Jana, N. D., & Das, S. (2022). An adaptive-learning-based generative adversarial network for one-to-one voice conversion. IEEE Transactions on artificial intelligence, 4(1), 92-106.
Kamiya, Toshiki, Fumihiko Sakaue, and Jun Sato. "Deep Automatic Control of Learning Rates for GANs." In International Workshop on Frontiers of Computer Vision, pp. 112-126. Cham: Springer International Publishing, 2022.
Blanchard, Andrew E., Christopher Stanley, and Debsindhu Bhowmik. "Using GANs with adaptive training data to search for new molecules." Journal of cheminformatics 13 (2021): 1-8.
Li, Kun, and Dae-Ki Kang. "Enhanced generative adversarial networks with restart learning rate in discriminator." Applied Sciences 12, no. 3 (2022): 1191.