MAPPING FACE TEXTURE GENERATED BY SMARTPHONE ONTO GENERIC FACE 3D MODEL
DOI:
https://doi.org/10.24867/12SA02LazorKeywords:
Texture, mapping, face, detection, UV maps, 3DAbstract
A simple and universal solution was presented, which satisfies the needs of users and the level of quality corresponds to the device on which it is displayed, i.e. a smartphone. The basic idea is the ability to quickly create a personalized animated avatar that would be used on social networks, virtual conferences, etc. Ready-made, averaged models are used, with a good topology, as well as a set skeleton. The personalization of the model is done with the help of texture, so it must be convincing enough. The texture was obtained using a camera on a mobile phone. Mapping an image onto a model is done by recognizing facial landmarks using the Dlib library. The key algorithm in this paper is the Thin Plate Spline algorithm.
References
[1] Po-Han Lee, Jui-Wen Huang, Huei-Yung Lin, 2012, 3D Model reconstruction based on multiple view image capture, International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012).
[2] Xian-Feng Han, Hamid Laga, Mohammed Bennamoun, 2019, Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era. IEEE Transactions on Pattern Analysis and Machine Intelligence.
[3] Yasutaka Furukawa, Jean Ponce, 2007 Accurate, Dense, and Robust Multi-View Stereopsis, IEEE Transactions on Pattern Analysis and Machine Intelligence.
[4] Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, Xi Zhou, 2018, Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network, ECCV 2018.
[5] Shu Liang, Linda G. Shapiro, Ira Kemelmacher-Shlizerman., 2016, Head Reconstruction from Internet Photos, ECCV 2016.
[6] Qi Zhao, Jiajun Bu, Mingli Song, 2004, Towards talking face on portable devices - A flexible and realistic approach, TENCON 2004.
[7] J. Riviere, P. Gotardo, D. Bradley, A. Ghosh, T. Beeler, 2020, Single-Shot High-Quality Facial Geometry and Skin Appearance Capture, ACM Trans. Graph., Vol. 39, No. 4, Article 81.
[8] T. Beeler, B. Bickel, P. Beardsley, B. Sumner, M. Gross, 2010, High-Quality Single-Shot Capture of Facial Geometry, ACM Transactions on Graphics 29(4).
[9] Shu-Fan Wang, Shang-Hong Lai, 2006, Efficient 3D Face Reconstruction from a Single 2D Image by Combining Statistical and Geometrical Information, Computer Vision - ACCV 2006.
[10] Laurent Balmelli, Gabriel Taubin, Fausto Bernardin, 2002, Space-Optimized Texture Maps, EUROGRAPHICS 2002
[11] Arun Ponnusamy, 2018, CNN based face detector from dlib, Elsevier Science, https://towardsdatascience.com/cnn-based-face-detector-from-dlib-c3696195e01c (приступљено, октобар 2020.)
[12] V.Kazemi, J.Sullivan, 2014, One millisecond face alignment with an ensemble of regression trees, 2014 IEEE Conference on Computer Vision and Pattern Recognition
[13] Facial point annotations, https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/, (приступљено, октобар 2020.)
[14] Herve Lombaert, Manual Registration with Thin Plates, https://profs.etsmtl.ca/hlombaert/thinplates/, (приступљено, октобар 2020.)
[2] Xian-Feng Han, Hamid Laga, Mohammed Bennamoun, 2019, Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era. IEEE Transactions on Pattern Analysis and Machine Intelligence.
[3] Yasutaka Furukawa, Jean Ponce, 2007 Accurate, Dense, and Robust Multi-View Stereopsis, IEEE Transactions on Pattern Analysis and Machine Intelligence.
[4] Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, Xi Zhou, 2018, Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network, ECCV 2018.
[5] Shu Liang, Linda G. Shapiro, Ira Kemelmacher-Shlizerman., 2016, Head Reconstruction from Internet Photos, ECCV 2016.
[6] Qi Zhao, Jiajun Bu, Mingli Song, 2004, Towards talking face on portable devices - A flexible and realistic approach, TENCON 2004.
[7] J. Riviere, P. Gotardo, D. Bradley, A. Ghosh, T. Beeler, 2020, Single-Shot High-Quality Facial Geometry and Skin Appearance Capture, ACM Trans. Graph., Vol. 39, No. 4, Article 81.
[8] T. Beeler, B. Bickel, P. Beardsley, B. Sumner, M. Gross, 2010, High-Quality Single-Shot Capture of Facial Geometry, ACM Transactions on Graphics 29(4).
[9] Shu-Fan Wang, Shang-Hong Lai, 2006, Efficient 3D Face Reconstruction from a Single 2D Image by Combining Statistical and Geometrical Information, Computer Vision - ACCV 2006.
[10] Laurent Balmelli, Gabriel Taubin, Fausto Bernardin, 2002, Space-Optimized Texture Maps, EUROGRAPHICS 2002
[11] Arun Ponnusamy, 2018, CNN based face detector from dlib, Elsevier Science, https://towardsdatascience.com/cnn-based-face-detector-from-dlib-c3696195e01c (приступљено, октобар 2020.)
[12] V.Kazemi, J.Sullivan, 2014, One millisecond face alignment with an ensemble of regression trees, 2014 IEEE Conference on Computer Vision and Pattern Recognition
[13] Facial point annotations, https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/, (приступљено, октобар 2020.)
[14] Herve Lombaert, Manual Registration with Thin Plates, https://profs.etsmtl.ca/hlombaert/thinplates/, (приступљено, октобар 2020.)
Downloads
Published
2021-05-10
Issue
Section
Engineering Animation
