Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks
Low-dimensional statistics of measurements play an important role in detection problems, including those encountered in sensor networks. In this work, we focus on learning low-dimensional linear statistics of high-dimensional measurement data along with decision rules defined in the low-dimensional...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute of Electrical and Electronics Engineers
2013
|
| Online Access: | http://hdl.handle.net/1721.1/81207 https://orcid.org/0000-0003-0149-5888 |
| _version_ | 1826195297192116224 |
|---|---|
| author | Varshney, Kush R. Willsky, Alan S. Willsky, Alan |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Varshney, Kush R. Willsky, Alan S. Willsky, Alan |
| author_sort | Varshney, Kush R. |
| collection | MIT |
| description | Low-dimensional statistics of measurements play an important role in detection problems, including those encountered in sensor networks. In this work, we focus on learning low-dimensional linear statistics of high-dimensional measurement data along with decision rules defined in the low-dimensional space in the case when the probability density of the measurements and class labels is not given, but a training set of samples from this distribution is given. We pose a joint optimization problem for linear dimensionality reduction and margin-based classification, and develop a coordinate descent algorithm on the Stiefel manifold for its solution. Although the coordinate descent is not guaranteed to find the globally optimal solution, crucially, its alternating structure enables us to extend it for sensor networks with a message-passing approach requiring little communication. Linear dimensionality reduction prevents overfitting when learning from finite training data. In the sensor network setting, dimensionality reduction not only prevents overfitting, but also reduces power consumption due to communication. The learned reduced-dimensional space and decision rule is shown to be consistent and its Rademacher complexity is characterized. Experimental results are presented for a variety of datasets, including those from existing sensor networks, demonstrating the potential of our methodology in comparison with other dimensionality reduction approaches. |
| first_indexed | 2024-09-23T10:10:28Z |
| format | Article |
| id | mit-1721.1/81207 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T10:10:28Z |
| publishDate | 2013 |
| publisher | Institute of Electrical and Electronics Engineers |
| record_format | dspace |
| spelling | mit-1721.1/812072022-09-30T19:22:49Z Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks Varshney, Kush R. Willsky, Alan S. Willsky, Alan Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Laboratory for Information and Decision Systems Willsky, Alan Varshney, Kush R. Low-dimensional statistics of measurements play an important role in detection problems, including those encountered in sensor networks. In this work, we focus on learning low-dimensional linear statistics of high-dimensional measurement data along with decision rules defined in the low-dimensional space in the case when the probability density of the measurements and class labels is not given, but a training set of samples from this distribution is given. We pose a joint optimization problem for linear dimensionality reduction and margin-based classification, and develop a coordinate descent algorithm on the Stiefel manifold for its solution. Although the coordinate descent is not guaranteed to find the globally optimal solution, crucially, its alternating structure enables us to extend it for sensor networks with a message-passing approach requiring little communication. Linear dimensionality reduction prevents overfitting when learning from finite training data. In the sensor network setting, dimensionality reduction not only prevents overfitting, but also reduces power consumption due to communication. The learned reduced-dimensional space and decision rule is shown to be consistent and its Rademacher complexity is characterized. Experimental results are presented for a variety of datasets, including those from existing sensor networks, demonstrating the potential of our methodology in comparison with other dimensionality reduction approaches. National Science Foundation (U.S.). Graduate Research Fellowship Program United States. Army Research Office (MURI funded through ARO Grant W911NF-06-1-0076) United States. Air Force Office of Scientific Research (Award FA9550-06-1-0324) Shell International Exploration and Production B.V. 2013-09-26T20:31:19Z 2013-09-26T20:31:19Z 2011-06 2010-12 Article http://purl.org/eprint/type/JournalArticle 1053-587X 1941-0476 http://hdl.handle.net/1721.1/81207 Varshney, Kush R., and Alan S. Willsky. Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks. IEEE Transactions on Signal Processing 59, no. 6 (June 2011): 2496-2512. https://orcid.org/0000-0003-0149-5888 en_US http://dx.doi.org/10.1109/tsp.2011.2123891 IEEE Transactions on Signal Processing Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf Institute of Electrical and Electronics Engineers Willsky via Amy Stout |
| spellingShingle | Varshney, Kush R. Willsky, Alan S. Willsky, Alan Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks |
| title | Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks |
| title_full | Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks |
| title_fullStr | Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks |
| title_full_unstemmed | Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks |
| title_short | Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks |
| title_sort | linear dimensionality reduction for margin based classification high dimensional data and sensor networks |
| url | http://hdl.handle.net/1721.1/81207 https://orcid.org/0000-0003-0149-5888 |
| work_keys_str_mv | AT varshneykushr lineardimensionalityreductionformarginbasedclassificationhighdimensionaldataandsensornetworks AT willskyalans lineardimensionalityreductionformarginbasedclassificationhighdimensionaldataandsensornetworks AT willskyalan lineardimensionalityreductionformarginbasedclassificationhighdimensionaldataandsensornetworks |