Learning with group invariant features: A Kernel perspective
We analyze in this paper a random feature map based on a theory of invariance (I-theory) introduced in [1]. More specifically, a group invariant signal signature is obtained through cumulative distributions of group-transformed random projections. Our analysis bridges invariant feature learning with...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Association for Computing Machinery
2017
|
| Online Access: | http://hdl.handle.net/1721.1/112309 https://orcid.org/0000-0002-3944-0455 https://orcid.org/0000-0002-5727-9941 |
| _version_ | 1826213822277353472 |
|---|---|
| author | Mroueh, Youssef Poggio, Tomaso A Voinea, Stephen Constantin |
| author2 | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences |
| author_facet | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Mroueh, Youssef Poggio, Tomaso A Voinea, Stephen Constantin |
| author_sort | Mroueh, Youssef |
| collection | MIT |
| description | We analyze in this paper a random feature map based on a theory of invariance (I-theory) introduced in [1]. More specifically, a group invariant signal signature is obtained through cumulative distributions of group-transformed random projections. Our analysis bridges invariant feature learning with kernel methods, as we show that this feature map defines an expected Haar-integration kernel that is invariant to the specified group action. We show how this non-linear random feature map approximates this group invariant kernel uniformly on a set of N points. Moreover, we show that it defines a function space that is dense in the equivalent Invariant Reproducing Kernel Hilbert Space. Finally, we quantify error rates of the convergence of the empirical risk minimization, as well as the reduction in the sample complexity of a learning algorithm using such an invariant representation for signal classification, in a classical supervised learning setting. |
| first_indexed | 2024-09-23T15:55:22Z |
| format | Article |
| id | mit-1721.1/112309 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T15:55:22Z |
| publishDate | 2017 |
| publisher | Association for Computing Machinery |
| record_format | dspace |
| spelling | mit-1721.1/1123092022-09-29T17:04:35Z Learning with group invariant features: A Kernel perspective Mroueh, Youssef Poggio, Tomaso A Voinea, Stephen Constantin Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Poggio, Tomaso A Voinea, Stephen Constantin We analyze in this paper a random feature map based on a theory of invariance (I-theory) introduced in [1]. More specifically, a group invariant signal signature is obtained through cumulative distributions of group-transformed random projections. Our analysis bridges invariant feature learning with kernel methods, as we show that this feature map defines an expected Haar-integration kernel that is invariant to the specified group action. We show how this non-linear random feature map approximates this group invariant kernel uniformly on a set of N points. Moreover, we show that it defines a function space that is dense in the equivalent Invariant Reproducing Kernel Hilbert Space. Finally, we quantify error rates of the convergence of the empirical risk minimization, as well as the reduction in the sample complexity of a learning algorithm using such an invariant representation for signal classification, in a classical supervised learning setting. 2017-11-28T19:15:40Z 2017-11-28T19:15:40Z 2015-12 2017-11-17T18:19:46Z Article http://purl.org/eprint/type/ConferencePaper http://hdl.handle.net/1721.1/112309 Mroueh, Youssef, Stephen Voinea and Tomaso Poggio. "Learning with Group Invariant Features: A Kernel Perspective." Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 (NIPS '15), December 7-12, 2015, Montreal, Canada, Association of Computing Machinery, December 2015. © 2015 Association of Computing Machinery ACM https://orcid.org/0000-0002-3944-0455 https://orcid.org/0000-0002-5727-9941 https://dl.acm.org/citation.cfm?id=2969413 Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS '15) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Association for Computing Machinery arXiv |
| spellingShingle | Mroueh, Youssef Poggio, Tomaso A Voinea, Stephen Constantin Learning with group invariant features: A Kernel perspective |
| title | Learning with group invariant features: A Kernel perspective |
| title_full | Learning with group invariant features: A Kernel perspective |
| title_fullStr | Learning with group invariant features: A Kernel perspective |
| title_full_unstemmed | Learning with group invariant features: A Kernel perspective |
| title_short | Learning with group invariant features: A Kernel perspective |
| title_sort | learning with group invariant features a kernel perspective |
| url | http://hdl.handle.net/1721.1/112309 https://orcid.org/0000-0002-3944-0455 https://orcid.org/0000-0002-5727-9941 |
| work_keys_str_mv | AT mrouehyoussef learningwithgroupinvariantfeaturesakernelperspective AT poggiotomasoa learningwithgroupinvariantfeaturesakernelperspective AT voineastephenconstantin learningwithgroupinvariantfeaturesakernelperspective |