MANIFOLD GEOMETRY ANALYSIS IN AUTOENCODERS
DOI:
https://doi.org/10.24867/26BE28JovicicKeywords:
Autoencoders, machine learning, manifolds, interpolationAbstract
This paper attempts to help in understanding and clarifying some of the principles in the work of deep neural networks. Specifically, how the width of the bottleneck layer in the autoencoder, the intrinsic dimensionality of the input data and the training duration affect the geometry of the formed manifold.
References
[1] Lei, Na, et al. "A geometric understanding of deep learning." Engineering 6.3 (2020): 361-374.
[2] Rey, Luis A. Pérez, Vlado Menkovski, and Jacobus W. Portegies. "Diffusion variational autoencoders." arXiv preprint arXiv:1901.08991 (2019).
[3] Ansuini, Alessio, et al. "Intrinsic dimension of data representations in deep neural networks." Advances in Neural Information Processing Systems 32 (2019).
[4] Ozair, Sherjil, and Yoshua Bengio. "Deep directed generative autoencoders." arXiv preprint arXiv:1410.0630 (2014).
[5] Goldt, Sebastian, et al. "Modeling the influence of data structure on learning in neural networks: The hidden manifold model." Physical Review X 10.4 (2020): 041044.
[6] Ceruti, Claudio, et al. "DANCo: dimensionality from angle and norm concentration." arXiv preprint arXiv:1206.3881 (2012).
[7] Farahmand, Amir Massoud, Csaba Szepesvári, and Jean-Yves Audibert. "Manifold-adaptive dimension estimation." Proceedings of the 24th international conference on Machine learning. 2007.
[8] Facco, Elena, et al. "Estimating the intrinsic dimension of datasets by a minimal neighborhood information." Scientific reports 7.1 (2017): 1-8.
[9] Chollet, Francois. Deep Learning with Python. Manning Publications, 2017.
[2] Rey, Luis A. Pérez, Vlado Menkovski, and Jacobus W. Portegies. "Diffusion variational autoencoders." arXiv preprint arXiv:1901.08991 (2019).
[3] Ansuini, Alessio, et al. "Intrinsic dimension of data representations in deep neural networks." Advances in Neural Information Processing Systems 32 (2019).
[4] Ozair, Sherjil, and Yoshua Bengio. "Deep directed generative autoencoders." arXiv preprint arXiv:1410.0630 (2014).
[5] Goldt, Sebastian, et al. "Modeling the influence of data structure on learning in neural networks: The hidden manifold model." Physical Review X 10.4 (2020): 041044.
[6] Ceruti, Claudio, et al. "DANCo: dimensionality from angle and norm concentration." arXiv preprint arXiv:1206.3881 (2012).
[7] Farahmand, Amir Massoud, Csaba Szepesvári, and Jean-Yves Audibert. "Manifold-adaptive dimension estimation." Proceedings of the 24th international conference on Machine learning. 2007.
[8] Facco, Elena, et al. "Estimating the intrinsic dimension of datasets by a minimal neighborhood information." Scientific reports 7.1 (2017): 1-8.
[9] Chollet, Francois. Deep Learning with Python. Manning Publications, 2017.
Downloads
Published
2024-03-04
Issue
Section
Electrotechnical and Computer Engineering
