Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification
Over the past decades, there has been an increase of attention to adapting machine learning methods to fully exploit the higher order structure of tensorial data. One problem of great interest is tensor classification, and in particular the extension of linear discriminant analysis to the multilinea...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/16/2/104 |
| _version_ | 1797622844163620864 |
|---|---|
| author | Cristian Minoccheri Olivia Alge Jonathan Gryak Kayvan Najarian Harm Derksen |
| author_facet | Cristian Minoccheri Olivia Alge Jonathan Gryak Kayvan Najarian Harm Derksen |
| author_sort | Cristian Minoccheri |
| collection | DOAJ |
| description | Over the past decades, there has been an increase of attention to adapting machine learning methods to fully exploit the higher order structure of tensorial data. One problem of great interest is tensor classification, and in particular the extension of linear discriminant analysis to the multilinear setting. We propose a novel method for multilinear discriminant analysis that is radically different from the ones considered so far, and it is the first extension to tensors of quadratic discriminant analysis. Our proposed approach uses invariant theory to extend the nearest Mahalanobis distance classifier to the higher-order setting, and to formulate a well-behaved optimization problem. We extensively test our method on a variety of synthetic data, outperforming previously proposed MDA techniques. We also show how to leverage multi-lead ECG data by constructing tensors via taut string, and use our method to classify healthy signals versus unhealthy ones; our method outperforms state-of-the-art MDA methods, especially after adding significant levels of noise to the signals. Our approach reached an AUC of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.95</mn><mo>(</mo><mn>0.03</mn><mo>)</mo></mrow></semantics></math></inline-formula> on clean signals—where the second best method reached <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.91</mn><mo>(</mo><mn>0.03</mn><mo>)</mo></mrow></semantics></math></inline-formula>—and an AUC of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.89</mn><mo>(</mo><mn>0.03</mn><mo>)</mo></mrow></semantics></math></inline-formula> after adding noise to the signals (with a signal-to-noise-ratio of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mo>−</mo><mn>30</mn></mrow></semantics></math></inline-formula>)—where the second best method reached <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.85</mn><mo>(</mo><mn>0.05</mn><mo>)</mo></mrow></semantics></math></inline-formula>. Our approach is fundamentally different than previous work in this direction, and proves to be faster, more stable, and more accurate on the tests we performed. |
| first_indexed | 2024-03-11T09:16:02Z |
| format | Article |
| id | doaj.art-6bca1068951840d0ae9b3963f3c35402 |
| institution | Directory Open Access Journal |
| issn | 1999-4893 |
| language | English |
| last_indexed | 2024-03-11T09:16:02Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj.art-6bca1068951840d0ae9b3963f3c354022023-11-16T18:37:49ZengMDPI AGAlgorithms1999-48932023-02-0116210410.3390/a16020104Quadratic Multilinear Discriminant Analysis for Tensorial Data ClassificationCristian Minoccheri0Olivia Alge1Jonathan Gryak2Kayvan Najarian3Harm Derksen4Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI 48109, USADepartment of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI 48109, USAComputer Science Department, Queen’s College, CUNY, New York, NY 11367, USADepartment of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI 48109, USAMathematics Department, Northeastern University, Boston, MA 02115, USAOver the past decades, there has been an increase of attention to adapting machine learning methods to fully exploit the higher order structure of tensorial data. One problem of great interest is tensor classification, and in particular the extension of linear discriminant analysis to the multilinear setting. We propose a novel method for multilinear discriminant analysis that is radically different from the ones considered so far, and it is the first extension to tensors of quadratic discriminant analysis. Our proposed approach uses invariant theory to extend the nearest Mahalanobis distance classifier to the higher-order setting, and to formulate a well-behaved optimization problem. We extensively test our method on a variety of synthetic data, outperforming previously proposed MDA techniques. We also show how to leverage multi-lead ECG data by constructing tensors via taut string, and use our method to classify healthy signals versus unhealthy ones; our method outperforms state-of-the-art MDA methods, especially after adding significant levels of noise to the signals. Our approach reached an AUC of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.95</mn><mo>(</mo><mn>0.03</mn><mo>)</mo></mrow></semantics></math></inline-formula> on clean signals—where the second best method reached <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.91</mn><mo>(</mo><mn>0.03</mn><mo>)</mo></mrow></semantics></math></inline-formula>—and an AUC of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.89</mn><mo>(</mo><mn>0.03</mn><mo>)</mo></mrow></semantics></math></inline-formula> after adding noise to the signals (with a signal-to-noise-ratio of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mo>−</mo><mn>30</mn></mrow></semantics></math></inline-formula>)—where the second best method reached <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.85</mn><mo>(</mo><mn>0.05</mn><mo>)</mo></mrow></semantics></math></inline-formula>. Our approach is fundamentally different than previous work in this direction, and proves to be faster, more stable, and more accurate on the tests we performed.https://www.mdpi.com/1999-4893/16/2/104tensorsmultilinear discriminant analysisquadratic discriminant analysisclassification |
| spellingShingle | Cristian Minoccheri Olivia Alge Jonathan Gryak Kayvan Najarian Harm Derksen Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification Algorithms tensors multilinear discriminant analysis quadratic discriminant analysis classification |
| title | Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification |
| title_full | Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification |
| title_fullStr | Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification |
| title_full_unstemmed | Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification |
| title_short | Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification |
| title_sort | quadratic multilinear discriminant analysis for tensorial data classification |
| topic | tensors multilinear discriminant analysis quadratic discriminant analysis classification |
| url | https://www.mdpi.com/1999-4893/16/2/104 |
| work_keys_str_mv | AT cristianminoccheri quadraticmultilineardiscriminantanalysisfortensorialdataclassification AT oliviaalge quadraticmultilineardiscriminantanalysisfortensorialdataclassification AT jonathangryak quadraticmultilineardiscriminantanalysisfortensorialdataclassification AT kayvannajarian quadraticmultilineardiscriminantanalysisfortensorialdataclassification AT harmderksen quadraticmultilineardiscriminantanalysisfortensorialdataclassification |