Measurement Invariance Investigation for Performance of Deep Learning Architectures
Models can be compared against each other. Such comparison helps to find strengths and weaknesses of different approaches and gives additional possibilities for model validation. Multi-class classification is a classification task where each image is assigned to one and only one label. Confusion mat...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9832864/ |
| _version_ | 1811342043309408256 |
|---|---|
| author | Dewang Chen Yuqi Lu Chih-Yu Hsu |
| author_facet | Dewang Chen Yuqi Lu Chih-Yu Hsu |
| author_sort | Dewang Chen |
| collection | DOAJ |
| description | Models can be compared against each other. Such comparison helps to find strengths and weaknesses of different approaches and gives additional possibilities for model validation. Multi-class classification is a classification task where each image is assigned to one and only one label. Confusion matrix, Precision, Recall, and F1 Score are popular performance metrics. The common sense is that the performance of any architecture is dependent on the sizes of data set. The goodness of the architecture of deep learning models for different data sets is critical issue. This paper implements Pearson correlation coefficient and the multivariate linear regression method to assess the accuracy of deep learning architectures with five performance indicators. The five performance indicators are training loss rate, robustness, training time, number of model parameters and computation complexity. There are five image datasets used to test four deep learning models: Alexnet, GoogLeNet, ResNet, MobileNet to obtain the values of accuracy and other five indicators. The most important contribution of the article is to show that the accuracy indicator related to training loss rate and training time indicators are not dependent on the selection of the data group. According to the definition of Measurement Invariance (MI), the measurement invariance is demonstrated by the linear regression analysis and inner product of the unit normal vector of the linear regression planes. |
| first_indexed | 2024-04-13T19:04:31Z |
| format | Article |
| id | doaj.art-2b44fc1365d5491284deffd065fa7305 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-13T19:04:31Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-2b44fc1365d5491284deffd065fa73052022-12-22T02:34:01ZengIEEEIEEE Access2169-35362022-01-0110780707808710.1109/ACCESS.2022.31924689832864Measurement Invariance Investigation for Performance of Deep Learning ArchitecturesDewang Chen0https://orcid.org/0000-0002-8660-9700Yuqi Lu1Chih-Yu Hsu2https://orcid.org/0000-0003-1074-8170School of Transportation, Fujian University of Technology, Fuzhou, ChinaSchool of Computer Science and Mathematics, Fujian University of Technology, Fuzhou, ChinaSchool of Transportation, Fujian University of Technology, Fuzhou, ChinaModels can be compared against each other. Such comparison helps to find strengths and weaknesses of different approaches and gives additional possibilities for model validation. Multi-class classification is a classification task where each image is assigned to one and only one label. Confusion matrix, Precision, Recall, and F1 Score are popular performance metrics. The common sense is that the performance of any architecture is dependent on the sizes of data set. The goodness of the architecture of deep learning models for different data sets is critical issue. This paper implements Pearson correlation coefficient and the multivariate linear regression method to assess the accuracy of deep learning architectures with five performance indicators. The five performance indicators are training loss rate, robustness, training time, number of model parameters and computation complexity. There are five image datasets used to test four deep learning models: Alexnet, GoogLeNet, ResNet, MobileNet to obtain the values of accuracy and other five indicators. The most important contribution of the article is to show that the accuracy indicator related to training loss rate and training time indicators are not dependent on the selection of the data group. According to the definition of Measurement Invariance (MI), the measurement invariance is demonstrated by the linear regression analysis and inner product of the unit normal vector of the linear regression planes.https://ieeexplore.ieee.org/document/9832864/Deep learning modelmultivariate linear regressionPearson correlation coefficientmeasurement invarianceperformance indicators |
| spellingShingle | Dewang Chen Yuqi Lu Chih-Yu Hsu Measurement Invariance Investigation for Performance of Deep Learning Architectures IEEE Access Deep learning model multivariate linear regression Pearson correlation coefficient measurement invariance performance indicators |
| title | Measurement Invariance Investigation for Performance of Deep Learning Architectures |
| title_full | Measurement Invariance Investigation for Performance of Deep Learning Architectures |
| title_fullStr | Measurement Invariance Investigation for Performance of Deep Learning Architectures |
| title_full_unstemmed | Measurement Invariance Investigation for Performance of Deep Learning Architectures |
| title_short | Measurement Invariance Investigation for Performance of Deep Learning Architectures |
| title_sort | measurement invariance investigation for performance of deep learning architectures |
| topic | Deep learning model multivariate linear regression Pearson correlation coefficient measurement invariance performance indicators |
| url | https://ieeexplore.ieee.org/document/9832864/ |
| work_keys_str_mv | AT dewangchen measurementinvarianceinvestigationforperformanceofdeeplearningarchitectures AT yuqilu measurementinvarianceinvestigationforperformanceofdeeplearningarchitectures AT chihyuhsu measurementinvarianceinvestigationforperformanceofdeeplearningarchitectures |