Toward Improving Ensemble-Based Collaborative Inference at the Edge
Ensemble-based collaborative inference systems, Edge Ensembles, are deep learning edge inference systems that enhance accuracy by aggregating predictions from models deployed on each device. They offer several advantages, including scalability based on task complexity and decentralized functionality...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10384371/ |
| _version_ | 1797243489925201920 |
|---|---|
| author | Shungo Kumazawa Jaehoon Yu Kazushi Kawamura Thiem Van Chu Masato Motomura |
| author_facet | Shungo Kumazawa Jaehoon Yu Kazushi Kawamura Thiem Van Chu Masato Motomura |
| author_sort | Shungo Kumazawa |
| collection | DOAJ |
| description | Ensemble-based collaborative inference systems, Edge Ensembles, are deep learning edge inference systems that enhance accuracy by aggregating predictions from models deployed on each device. They offer several advantages, including scalability based on task complexity and decentralized functionality without dependency on centralized servers. In general, ensemble methods effectively improve the accuracy of deep learning, and conventional research uses several model integration techniques for deep learning ensembles. Some of these existing integration methods are more effective than those used in previous Edge Ensembles. However, it remains uncertain whether these methods can be directly applied in the context of cooperative inference systems involving multiple edge devices. This study investigates the effectiveness of conventional model integration techniques, including cascade, weighted averaging, and test-time augmentation (TTA), when applied to Edge Ensembles to enhance their performance. Furthermore, we propose enhancements of these techniques tailored for Edge Ensembles. The cascade reduces the number of models required for inference but worsens latency by sequential inference processing. To address this latency issue, we propose <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula>-parallel cascade, which adjusts the number of models processed simultaneously to <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula>. We also propose learning TTA policies and weights for weighted averaging using ensemble prediction labels instead of ground truth labels. In the experiments, we verified the effectiveness of each technique for Edge Ensembles. The proposed <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula>-parallel cascade achieved a 2.8 times reduction in latency compared to the conventional cascade, even with a 1.06 times increase in computational costs. Additionally, the ensemble label-based learning demonstrated comparable effectiveness to the approach using ground truth labels. |
| first_indexed | 2024-04-24T18:55:56Z |
| format | Article |
| id | doaj.art-25f3b7022f904ef1b16610bfb7796aa4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-24T18:55:56Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-25f3b7022f904ef1b16610bfb7796aa42024-03-26T17:34:53ZengIEEEIEEE Access2169-35362024-01-01126926694010.1109/ACCESS.2024.335130810384371Toward Improving Ensemble-Based Collaborative Inference at the EdgeShungo Kumazawa0https://orcid.org/0009-0006-5608-5666Jaehoon Yu1https://orcid.org/0000-0001-6639-7694Kazushi Kawamura2https://orcid.org/0000-0002-0795-2974Thiem Van Chu3Masato Motomura4https://orcid.org/0000-0003-1543-1252Tokyo Institute of Technology, Midori-ku, Yokohama, JapanTokyo Institute of Technology, Midori-ku, Yokohama, JapanTokyo Institute of Technology, Midori-ku, Yokohama, JapanTokyo Institute of Technology, Midori-ku, Yokohama, JapanTokyo Institute of Technology, Midori-ku, Yokohama, JapanEnsemble-based collaborative inference systems, Edge Ensembles, are deep learning edge inference systems that enhance accuracy by aggregating predictions from models deployed on each device. They offer several advantages, including scalability based on task complexity and decentralized functionality without dependency on centralized servers. In general, ensemble methods effectively improve the accuracy of deep learning, and conventional research uses several model integration techniques for deep learning ensembles. Some of these existing integration methods are more effective than those used in previous Edge Ensembles. However, it remains uncertain whether these methods can be directly applied in the context of cooperative inference systems involving multiple edge devices. This study investigates the effectiveness of conventional model integration techniques, including cascade, weighted averaging, and test-time augmentation (TTA), when applied to Edge Ensembles to enhance their performance. Furthermore, we propose enhancements of these techniques tailored for Edge Ensembles. The cascade reduces the number of models required for inference but worsens latency by sequential inference processing. To address this latency issue, we propose <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula>-parallel cascade, which adjusts the number of models processed simultaneously to <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula>. We also propose learning TTA policies and weights for weighted averaging using ensemble prediction labels instead of ground truth labels. In the experiments, we verified the effectiveness of each technique for Edge Ensembles. The proposed <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula>-parallel cascade achieved a 2.8 times reduction in latency compared to the conventional cascade, even with a 1.06 times increase in computational costs. Additionally, the ensemble label-based learning demonstrated comparable effectiveness to the approach using ground truth labels.https://ieeexplore.ieee.org/document/10384371/Ensembleedge computingcollaborative inferenceneural networkscascadetest time augmentation |
| spellingShingle | Shungo Kumazawa Jaehoon Yu Kazushi Kawamura Thiem Van Chu Masato Motomura Toward Improving Ensemble-Based Collaborative Inference at the Edge IEEE Access Ensemble edge computing collaborative inference neural networks cascade test time augmentation |
| title | Toward Improving Ensemble-Based Collaborative Inference at the Edge |
| title_full | Toward Improving Ensemble-Based Collaborative Inference at the Edge |
| title_fullStr | Toward Improving Ensemble-Based Collaborative Inference at the Edge |
| title_full_unstemmed | Toward Improving Ensemble-Based Collaborative Inference at the Edge |
| title_short | Toward Improving Ensemble-Based Collaborative Inference at the Edge |
| title_sort | toward improving ensemble based collaborative inference at the edge |
| topic | Ensemble edge computing collaborative inference neural networks cascade test time augmentation |
| url | https://ieeexplore.ieee.org/document/10384371/ |
| work_keys_str_mv | AT shungokumazawa towardimprovingensemblebasedcollaborativeinferenceattheedge AT jaehoonyu towardimprovingensemblebasedcollaborativeinferenceattheedge AT kazushikawamura towardimprovingensemblebasedcollaborativeinferenceattheedge AT thiemvanchu towardimprovingensemblebasedcollaborativeinferenceattheedge AT masatomotomura towardimprovingensemblebasedcollaborativeinferenceattheedge |