Topological Regularization for Representation Learning via Persistent Homology
Generalization is challenging in small-sample-size regimes with over-parameterized deep neural networks, and a better representation is generally beneficial for generalization. In this paper, we present a novel method for controlling the internal representation of deep neural networks from a topolog...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/4/1008 |
| _version_ | 1797619462878265344 |
|---|---|
| author | Muyi Chen Daling Wang Shi Feng Yifei Zhang |
| author_facet | Muyi Chen Daling Wang Shi Feng Yifei Zhang |
| author_sort | Muyi Chen |
| collection | DOAJ |
| description | Generalization is challenging in small-sample-size regimes with over-parameterized deep neural networks, and a better representation is generally beneficial for generalization. In this paper, we present a novel method for controlling the internal representation of deep neural networks from a topological perspective. Leveraging the power of topology data analysis (TDA), we study the push-forward probability measure induced by the feature extractor, and we formulate a notion of “separation” to characterize a property of this measure in terms of persistent homology for the first time. Moreover, we perform a theoretical analysis of this property and prove that enforcing this property leads to better generalization. To impose this property, we propose a novel weight function to extract topological information, and we introduce a new regularizer including three items to guide the representation learning in a topology-aware manner. Experimental results in the point cloud optimization task show that our method is effective and powerful. Furthermore, results in the image classification task show that our method outperforms the previous methods by a significant margin. |
| first_indexed | 2024-03-11T08:28:28Z |
| format | Article |
| id | doaj.art-81782a9f3d7444e18bb75e429a573fb1 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-11T08:28:28Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-81782a9f3d7444e18bb75e429a573fb12023-11-16T21:57:06ZengMDPI AGMathematics2227-73902023-02-01114100810.3390/math11041008Topological Regularization for Representation Learning via Persistent HomologyMuyi Chen0Daling Wang1Shi Feng2Yifei Zhang3School of Computer Science and Engineering, Northeastern University, Shenyang 110169, ChinaSchool of Computer Science and Engineering, Northeastern University, Shenyang 110169, ChinaSchool of Computer Science and Engineering, Northeastern University, Shenyang 110169, ChinaSchool of Computer Science and Engineering, Northeastern University, Shenyang 110169, ChinaGeneralization is challenging in small-sample-size regimes with over-parameterized deep neural networks, and a better representation is generally beneficial for generalization. In this paper, we present a novel method for controlling the internal representation of deep neural networks from a topological perspective. Leveraging the power of topology data analysis (TDA), we study the push-forward probability measure induced by the feature extractor, and we formulate a notion of “separation” to characterize a property of this measure in terms of persistent homology for the first time. Moreover, we perform a theoretical analysis of this property and prove that enforcing this property leads to better generalization. To impose this property, we propose a novel weight function to extract topological information, and we introduce a new regularizer including three items to guide the representation learning in a topology-aware manner. Experimental results in the point cloud optimization task show that our method is effective and powerful. Furthermore, results in the image classification task show that our method outperforms the previous methods by a significant margin.https://www.mdpi.com/2227-7390/11/4/1008deep neural networkrepresentation spacepersistent homologypush-forward probability measure |
| spellingShingle | Muyi Chen Daling Wang Shi Feng Yifei Zhang Topological Regularization for Representation Learning via Persistent Homology Mathematics deep neural network representation space persistent homology push-forward probability measure |
| title | Topological Regularization for Representation Learning via Persistent Homology |
| title_full | Topological Regularization for Representation Learning via Persistent Homology |
| title_fullStr | Topological Regularization for Representation Learning via Persistent Homology |
| title_full_unstemmed | Topological Regularization for Representation Learning via Persistent Homology |
| title_short | Topological Regularization for Representation Learning via Persistent Homology |
| title_sort | topological regularization for representation learning via persistent homology |
| topic | deep neural network representation space persistent homology push-forward probability measure |
| url | https://www.mdpi.com/2227-7390/11/4/1008 |
| work_keys_str_mv | AT muyichen topologicalregularizationforrepresentationlearningviapersistenthomology AT dalingwang topologicalregularizationforrepresentationlearningviapersistenthomology AT shifeng topologicalregularizationforrepresentationlearningviapersistenthomology AT yifeizhang topologicalregularizationforrepresentationlearningviapersistenthomology |