A new data transmission paradigm for visual analysis in edge-cloud collaboration
Edge-cloud collaboration, where sensor data is acquired at edge end while analyses finish at cloud end, has become a new fashion for deep learning based visual analysis applications. The data communication which serves as the fundamental infrastructure is playing an important role in edge-cloud coll...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Doctor of Philosophy |
| Language: | English |
| Published: |
Nanyang Technological University
2021
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/153055 |
| _version_ | 1811681271715201024 |
|---|---|
| author | Chen, Zhuo |
| author2 | Lin Weisi |
| author_facet | Lin Weisi Chen, Zhuo |
| author_sort | Chen, Zhuo |
| collection | NTU |
| description | Edge-cloud collaboration, where sensor data is acquired at edge end while analyses finish at cloud end, has become a new fashion for deep learning based visual analysis applications. The data communication which serves as the fundamental infrastructure is playing an important role in edge-cloud collaboration. To enable better balance among computing load, bandwidth usage and generalization ability, I propose a new paradigm of transmitting intermediate deep learning features instead of visual signals or ultimately utilized features, which inspires research and standardization of compression techniques for intermediate deep learning features.
To improve the data transmission efficiency, I develop a video-codec-based coding framework for intermediate deep learning feature compression. Besides, I also provide an overview and propose new coding tools for PreQuantization and Repack modules in the coding framework, with extensive comparative experiments analyzing their pros and cons. The optimal combination of the proposed modes can achieve over 50x compression ratio with less than 1% task performance drop, where the bitstream of intermediate deep learning features can be much smaller than that of corresponding visual signals. It is also worth mentioning that the proposed coding framework and coding tools have been partially adopted into the ongoing AVS (Audio Video Coding Standard Workgroup) - Visual Feature Coding Standard, and provided evidences for MPEG Video Coding for Machine (VCM) standard.
Moreover, to train more robust and generic backbone neural networks for feature extraction at edge end, I present an image quality assessment (IQA) based label smoothing method to tune the objective functions in neural network training. To provide better task-specific models on top of the intermediate deep features for the cloud end, I also propose a deep holographic network with a holographic composition operator to improve task performance with less memory costs. Extensive evaluations demonstrate the efficiency of the proposed methods. |
| first_indexed | 2024-10-01T03:38:18Z |
| format | Thesis-Doctor of Philosophy |
| id | ntu-10356/153055 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:38:18Z |
| publishDate | 2021 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1530552023-03-05T16:32:57Z A new data transmission paradigm for visual analysis in edge-cloud collaboration Chen, Zhuo Lin Weisi Interdisciplinary Graduate School (IGS) Rapid-Rich Object Search (ROSE) Lab WSLin@ntu.edu.sg Engineering::Computer science and engineering Edge-cloud collaboration, where sensor data is acquired at edge end while analyses finish at cloud end, has become a new fashion for deep learning based visual analysis applications. The data communication which serves as the fundamental infrastructure is playing an important role in edge-cloud collaboration. To enable better balance among computing load, bandwidth usage and generalization ability, I propose a new paradigm of transmitting intermediate deep learning features instead of visual signals or ultimately utilized features, which inspires research and standardization of compression techniques for intermediate deep learning features. To improve the data transmission efficiency, I develop a video-codec-based coding framework for intermediate deep learning feature compression. Besides, I also provide an overview and propose new coding tools for PreQuantization and Repack modules in the coding framework, with extensive comparative experiments analyzing their pros and cons. The optimal combination of the proposed modes can achieve over 50x compression ratio with less than 1% task performance drop, where the bitstream of intermediate deep learning features can be much smaller than that of corresponding visual signals. It is also worth mentioning that the proposed coding framework and coding tools have been partially adopted into the ongoing AVS (Audio Video Coding Standard Workgroup) - Visual Feature Coding Standard, and provided evidences for MPEG Video Coding for Machine (VCM) standard. Moreover, to train more robust and generic backbone neural networks for feature extraction at edge end, I present an image quality assessment (IQA) based label smoothing method to tune the objective functions in neural network training. To provide better task-specific models on top of the intermediate deep features for the cloud end, I also propose a deep holographic network with a holographic composition operator to improve task performance with less memory costs. Extensive evaluations demonstrate the efficiency of the proposed methods. Doctor of Philosophy 2021-11-02T04:17:29Z 2021-11-02T04:17:29Z 2021 Thesis-Doctor of Philosophy Chen, Z. (2021). A new data transmission paradigm for visual analysis in edge-cloud collaboration. Doctoral thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/153055 https://hdl.handle.net/10356/153055 10.32657/10356/153055 en This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Computer science and engineering Chen, Zhuo A new data transmission paradigm for visual analysis in edge-cloud collaboration |
| title | A new data transmission paradigm for visual analysis in edge-cloud collaboration |
| title_full | A new data transmission paradigm for visual analysis in edge-cloud collaboration |
| title_fullStr | A new data transmission paradigm for visual analysis in edge-cloud collaboration |
| title_full_unstemmed | A new data transmission paradigm for visual analysis in edge-cloud collaboration |
| title_short | A new data transmission paradigm for visual analysis in edge-cloud collaboration |
| title_sort | new data transmission paradigm for visual analysis in edge cloud collaboration |
| topic | Engineering::Computer science and engineering |
| url | https://hdl.handle.net/10356/153055 |
| work_keys_str_mv | AT chenzhuo anewdatatransmissionparadigmforvisualanalysisinedgecloudcollaboration AT chenzhuo newdatatransmissionparadigmforvisualanalysisinedgecloudcollaboration |