Face swapping is an exciting visual effect with many potential applications in entertainment and privacy protection. This paper presents an efficient approach for real-time face swapping and facial landmark detection using only computer vision techniques. It achieves real-time performance without relying on deep learning or GPU acceleration, making it accessible on standard CPUs. This enables face swapping to be implemented on a wider range of devices. The method combines classical computer vision approaches with modern facial landmark detection, striking a balance between accuracy and speed. This hybrid approach demonstrates how traditional techniques can still be relevant alongside AI advancements. By achieving 25 FPS processing on live video streams, it opens up possibilities for interactive applications like video conferencing and live streaming with face swap effects. The research provides a detailed breakdown of the face swapping pipeline, from landmark detection to mesh generation and seamless blending. This offers valuable insights into the technical challenges of face manipulation. Comparing the method to state-of-the-art approaches shows how optimized classical techniques can sometimes match or exceed the performance of more complex AI-based solutions, especially for real-time applications. The work has potential implications for privacy protection, entertainment, and creative applications, showcasing the broader impact of computer vision research on various fields.
Tài liệu tham khảo
Nirkin, Y., Keller, Y., & Hassner, T. (2018). FSGAN: Subject agnostic face swapping and reenactment. Proceedings of the IEEE/CVF International Conference on Computer Vision, 7184-7193
Yang, H., Zhu, H., Wang, Y., Huang, M., Shen, Q., Yang, R., & Cao, X. (2020). FaceShifter: Towards high fidelity and occlusion aware face swapping. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5893-5902.
Garrido, P., Valgaerts, L., Sarmadi, H., Steiner, I., Varanasi, K., Perez, P., & Theobalt, C. (2015). VDub: Modifying face video of actors for plausible visual alignment to a dubbed audio track. In Computer Graphics Forum (Vol. 34, No. 2, pp. 193-204). Blackwell Publishing Ltd.
Pumarola, A., Agudo, A., Martinez, A. M., Sanfeliu, A., & Moreno-Noguer, F. (2018). Ganimation: Anatomically-aware facial animation from a single image. Proceedings of the European Conference on Computer Vision, 818-833.
Thies, J., Zollhöfer, M., & Nießner, M. (2019). Deferred neural rendering: Image synthesis using neural textures. ACM Transactions on Graphics, 38(4), 1-12.
Korshunova, I., Shi, W., Damien, J., & Theis, L. (2017). Fast face-swap using convolutional neural networks. Proceedings of the IEEE International Conference on Computer Vision, 3677-3685.
Xing, J., Liu, H., Xu, X., Zhou, Y., & Shen, X. (2019). Attention-based face swapping. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 0-0.
Nirkin, Y., Keller, Y., & Hassner, T. (2018). FSGAN: Subject agnostic face swapping and reenactment. Proceedings of the IEEE/CVF International Conference on Computer Vision, 7184-7193.
Yang, H., Zhu, H., Wang, Y., Huang, M., Shen, Q., Yang, R., & Cao, X. (2020). FaceShifter: Towards high fidelity and occlusion aware face swapping. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5893-5902.
Weng, C. H., Wu, T. L., Chen, Y. H., Chu, H. K., & Tsai, Y. C. (2021). Towards more natural and better face swapping. ArXiv preprint arXiv:2102.1187
