Deep Neural Network Optimization Based on Binary Method for Handling Multi-Class Problems
In this paper, we conceive a new kind of output layer design in deep neural networks for the multi-class problems. The traditional output layer is set by the one-to-one method. For the one-to-one method, the output layer neuron number is the same as the class number. And the ideal output for the j-t...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10479493/ |
| _version_ | 1797227311965143040 |
|---|---|
| author | Yuqi Liu Sibo Yang Yuan Bao |
| author_facet | Yuqi Liu Sibo Yang Yuan Bao |
| author_sort | Yuqi Liu |
| collection | DOAJ |
| description | In this paper, we conceive a new kind of output layer design in deep neural networks for the multi-class problems. The traditional output layer is set by the one-to-one method. For the one-to-one method, the output layer neuron number is the same as the class number. And the ideal output for the j-th class sample is <inline-formula> <tex-math notation="LaTeX">$e_{j}$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$e_{j}$ </tex-math></inline-formula> is j-th unit vector. However, one-to-one method requires too many output neurons, which will increase the number of weights connecting the last-hidden and the output layers. Furthermore, during the process of network training, computation time and cost will greatly increase. We design the binary method for the output layer: Let the class number be k (<inline-formula> <tex-math notation="LaTeX">$k\geq 3$ </tex-math></inline-formula>), and <inline-formula> <tex-math notation="LaTeX">$2^{a-1} < k \le 2^{a} \,\,({a=\lceil log_{2}k \rceil })$ </tex-math></inline-formula>, then the output layer neuron number is a and the ideal output is designed by binary method. Obviously, the binary method uses less output nodes than the traditional one-to-one method. On this foundation, the number of hidden-output weights will also decrease. On the other hand, while training the deep neural network, the learning efficiency will also be significantly improved. Numerical experiments show that binary method has better classification performance and calculation speed than one-to-one method on the datasets. |
| first_indexed | 2024-04-24T14:38:48Z |
| format | Article |
| id | doaj.art-b29fbf1e34ea4c1599526fa61f2fcbbf |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-24T14:38:48Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-b29fbf1e34ea4c1599526fa61f2fcbbf2024-04-02T23:00:45ZengIEEEIEEE Access2169-35362024-01-0112468814689010.1109/ACCESS.2024.338219510479493Deep Neural Network Optimization Based on Binary Method for Handling Multi-Class ProblemsYuqi Liu0Sibo Yang1https://orcid.org/0000-0002-2301-3803Yuan Bao2https://orcid.org/0000-0002-6838-9089Marine Engineering College, Dalian Maritime University, Dalian, ChinaSchool of Science, Dalian Maritime University, Dalian, ChinaSchool of Mathematics and Statistics, Xinyang Normal University, Xinyang, ChinaIn this paper, we conceive a new kind of output layer design in deep neural networks for the multi-class problems. The traditional output layer is set by the one-to-one method. For the one-to-one method, the output layer neuron number is the same as the class number. And the ideal output for the j-th class sample is <inline-formula> <tex-math notation="LaTeX">$e_{j}$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$e_{j}$ </tex-math></inline-formula> is j-th unit vector. However, one-to-one method requires too many output neurons, which will increase the number of weights connecting the last-hidden and the output layers. Furthermore, during the process of network training, computation time and cost will greatly increase. We design the binary method for the output layer: Let the class number be k (<inline-formula> <tex-math notation="LaTeX">$k\geq 3$ </tex-math></inline-formula>), and <inline-formula> <tex-math notation="LaTeX">$2^{a-1} < k \le 2^{a} \,\,({a=\lceil log_{2}k \rceil })$ </tex-math></inline-formula>, then the output layer neuron number is a and the ideal output is designed by binary method. Obviously, the binary method uses less output nodes than the traditional one-to-one method. On this foundation, the number of hidden-output weights will also decrease. On the other hand, while training the deep neural network, the learning efficiency will also be significantly improved. Numerical experiments show that binary method has better classification performance and calculation speed than one-to-one method on the datasets.https://ieeexplore.ieee.org/document/10479493/Deep neural networksone-to-one methodbinary methodmulti-class problems |
| spellingShingle | Yuqi Liu Sibo Yang Yuan Bao Deep Neural Network Optimization Based on Binary Method for Handling Multi-Class Problems IEEE Access Deep neural networks one-to-one method binary method multi-class problems |
| title | Deep Neural Network Optimization Based on Binary Method for Handling Multi-Class Problems |
| title_full | Deep Neural Network Optimization Based on Binary Method for Handling Multi-Class Problems |
| title_fullStr | Deep Neural Network Optimization Based on Binary Method for Handling Multi-Class Problems |
| title_full_unstemmed | Deep Neural Network Optimization Based on Binary Method for Handling Multi-Class Problems |
| title_short | Deep Neural Network Optimization Based on Binary Method for Handling Multi-Class Problems |
| title_sort | deep neural network optimization based on binary method for handling multi class problems |
| topic | Deep neural networks one-to-one method binary method multi-class problems |
| url | https://ieeexplore.ieee.org/document/10479493/ |
| work_keys_str_mv | AT yuqiliu deepneuralnetworkoptimizationbasedonbinarymethodforhandlingmulticlassproblems AT siboyang deepneuralnetworkoptimizationbasedonbinarymethodforhandlingmulticlassproblems AT yuanbao deepneuralnetworkoptimizationbasedonbinarymethodforhandlingmulticlassproblems |