Communicating Neural Network architectures for resource constrained systems
<p>The deployment of millions of embedded sensors plagued by resource constraints in sophisticated, complex and dynamic Internet of Things (IoT) environments continues to inspire the need to build novel architectures and models for automated, efficient inference and communication. In such sett...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
2022
|
| Subjects: |
| _version_ | 1826316291643801600 |
|---|---|
| author | Abudu, P |
| author2 | Markham, A |
| author_facet | Markham, A Abudu, P |
| author_sort | Abudu, P |
| collection | OXFORD |
| description | <p>The deployment of millions of embedded sensors plagued by resource constraints in sophisticated, complex and dynamic Internet of Things (IoT) environments continues to inspire the need to build novel architectures and models for automated, efficient inference and communication. In such settings, practical challenges related to energy efficiency, computational power and reliability, tedious design implementation, effective communication, optimal sampling and accurate event classification, prediction and detection exist. Sensors operating in such environments must be capable of overcoming such challenges and enable scalable monitoring of dynamic phenomena whilst respecting limitations computational capabilities and battery energy. We continue to see a shift from reliance on cloud-centric to edge-centric architectures for data processing, inference and actuation. Distributed edge inference techniques address real-time, connectivity, network bandwidth and latency challenges in spatially dis- tributed IoT applications. Achieving efficient, resource-aware communication in such systems is a longstanding challenge. Many current approaches require complex, hand-engineered communication protocols.</p>
<p>The development of Machine Learning (ML) continues to motivate a new wave of innovative solutions that intermarry embedded sensors, IoT, and ML to enable various applications. In this thesis, we explore the idea of maximizing the shared utility of multiple nodes (agents) seeking to achieve a sensing objective using a data-driven approach. Among other considerations, it is imperative that agents not only learn to communicate vital information, but also learn to know when to communicate.</p>
<p>Firstly, we describe an in-network processing computing paradigm in which modern distributed Deep Networks are employed for inference on sensed objectives. This computing approach unifies deep learning and decentralized computation.</p>
<p>Secondly, we propose a distributed communicating architecture based on Recurrent Neural Networks (RNNs) that can be instantiated on resource- constrained devices observing unique data and performing automated dis- tributed inference via hidden-state communication. We prove that our framework is able to use a data-driven approach to collectively solve various distributed objectives using sensor-like data, as evidenced by a series of systematic analyses we present.</p>
<p>Thirdly, we present a novel scalable, data-driven and communication- efficient Convolutional Recurrent Neural Network (C-RNN) framework for distributed tasks. Our framework can extract relevant features on visual data and automatically learns communication protocols to solve tasks in distributed visual settings.
Finally, we demonstrate that our framework can be extended to visual tracking scenarios. In doing this, we introduce novel and scalable methods to setup, train, and deploy communication-efficient distributed nodes using Deep Networks.</p>
<p>Our empirical and systematic analyses of model convergence, node scalability, energy-efficiency, computational complexity and communication- cost on varying node topologies demonstrate that our proposed frame- works are able to employ Deep Networks to discover and optimize communication protocols on data with different modalities and can be extended to many distributed applications in resource-constrained systems.</p>
|
| first_indexed | 2024-03-07T07:19:28Z |
| format | Thesis |
| id | oxford-uuid:3669176b-3907-412c-9a4d-7bacac2367f4 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-12-09T03:42:16Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | oxford-uuid:3669176b-3907-412c-9a4d-7bacac2367f42024-12-07T13:26:57ZCommunicating Neural Network architectures for resource constrained systemsThesishttp://purl.org/coar/resource_type/c_db06uuid:3669176b-3907-412c-9a4d-7bacac2367f4Internet of thingsDistributed artificial intelligenceMachine learningDeep learning (Machine learning)EnglishHyrax Deposit2022Abudu, PMarkham, AMartinovic, IMcCann, J<p>The deployment of millions of embedded sensors plagued by resource constraints in sophisticated, complex and dynamic Internet of Things (IoT) environments continues to inspire the need to build novel architectures and models for automated, efficient inference and communication. In such settings, practical challenges related to energy efficiency, computational power and reliability, tedious design implementation, effective communication, optimal sampling and accurate event classification, prediction and detection exist. Sensors operating in such environments must be capable of overcoming such challenges and enable scalable monitoring of dynamic phenomena whilst respecting limitations computational capabilities and battery energy. We continue to see a shift from reliance on cloud-centric to edge-centric architectures for data processing, inference and actuation. Distributed edge inference techniques address real-time, connectivity, network bandwidth and latency challenges in spatially dis- tributed IoT applications. Achieving efficient, resource-aware communication in such systems is a longstanding challenge. Many current approaches require complex, hand-engineered communication protocols.</p> <p>The development of Machine Learning (ML) continues to motivate a new wave of innovative solutions that intermarry embedded sensors, IoT, and ML to enable various applications. In this thesis, we explore the idea of maximizing the shared utility of multiple nodes (agents) seeking to achieve a sensing objective using a data-driven approach. Among other considerations, it is imperative that agents not only learn to communicate vital information, but also learn to know when to communicate.</p> <p>Firstly, we describe an in-network processing computing paradigm in which modern distributed Deep Networks are employed for inference on sensed objectives. This computing approach unifies deep learning and decentralized computation.</p> <p>Secondly, we propose a distributed communicating architecture based on Recurrent Neural Networks (RNNs) that can be instantiated on resource- constrained devices observing unique data and performing automated dis- tributed inference via hidden-state communication. We prove that our framework is able to use a data-driven approach to collectively solve various distributed objectives using sensor-like data, as evidenced by a series of systematic analyses we present.</p> <p>Thirdly, we present a novel scalable, data-driven and communication- efficient Convolutional Recurrent Neural Network (C-RNN) framework for distributed tasks. Our framework can extract relevant features on visual data and automatically learns communication protocols to solve tasks in distributed visual settings. Finally, we demonstrate that our framework can be extended to visual tracking scenarios. In doing this, we introduce novel and scalable methods to setup, train, and deploy communication-efficient distributed nodes using Deep Networks.</p> <p>Our empirical and systematic analyses of model convergence, node scalability, energy-efficiency, computational complexity and communication- cost on varying node topologies demonstrate that our proposed frame- works are able to employ Deep Networks to discover and optimize communication protocols on data with different modalities and can be extended to many distributed applications in resource-constrained systems.</p> |
| spellingShingle | Internet of things Distributed artificial intelligence Machine learning Deep learning (Machine learning) Abudu, P Communicating Neural Network architectures for resource constrained systems |
| title | Communicating Neural Network architectures for resource constrained systems |
| title_full | Communicating Neural Network architectures for resource constrained systems |
| title_fullStr | Communicating Neural Network architectures for resource constrained systems |
| title_full_unstemmed | Communicating Neural Network architectures for resource constrained systems |
| title_short | Communicating Neural Network architectures for resource constrained systems |
| title_sort | communicating neural network architectures for resource constrained systems |
| topic | Internet of things Distributed artificial intelligence Machine learning Deep learning (Machine learning) |
| work_keys_str_mv | AT abudup communicatingneuralnetworkarchitecturesforresourceconstrainedsystems |