GenSynth: a generative synthesis approach to learning generative machines for generate efficient neural networks
The tremendous potential exhibited by deep learning is often offset by architectural and computational complexity, making widespread deployment a challenge for edge scenarios such as mobile and other consumer devices. To tackle this challenge, we explore the following idea: Can we learn generative m...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2019-09-01
|
| Series: | Electronics Letters |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/el.2019.1719 |
| _version_ | 1798042711790452736 |
|---|---|
| author | Alexander Wong Mohammad Javad Shafiee Brendan Chwyl Francis Li |
| author_facet | Alexander Wong Mohammad Javad Shafiee Brendan Chwyl Francis Li |
| author_sort | Alexander Wong |
| collection | DOAJ |
| description | The tremendous potential exhibited by deep learning is often offset by architectural and computational complexity, making widespread deployment a challenge for edge scenarios such as mobile and other consumer devices. To tackle this challenge, we explore the following idea: Can we learn generative machines to automatically generate deep neural networks with efficient network architectures? In this study, we introduce the idea of generative synthesis, which is premised on the intricate interplay between a generator‐inquisitor pair that work in tandem to garner insights and learn to generate highly efficient deep neural networks that best satisfies operational requirements. Experimental results for image classification, semantic segmentation, and object detection tasks illustrate the efficacy of generative synthesis (GenSynth) in producing generators that automatically generate highly efficient deep neural networks (which we nickname FermiNets with higher model efficiency and lower computational costs (reaching >10× more efficient and fewer multiply‐accumulate operations than several tested state‐of‐the‐art networks), as well as higher energy efficiency (reaching >4× improvements in image inferences per joule consumed on a Nvidia Tegra X2 mobile processor). As such, GenSynth can be a powerful, generalised approach for accelerating and improving the building of deep neural networks for on‐device edge scenarios. |
| first_indexed | 2024-04-11T22:39:16Z |
| format | Article |
| id | doaj.art-4158bd7f31554c148cc9b2788361ad21 |
| institution | Directory Open Access Journal |
| issn | 0013-5194 1350-911X |
| language | English |
| last_indexed | 2024-04-11T22:39:16Z |
| publishDate | 2019-09-01 |
| publisher | Wiley |
| record_format | Article |
| series | Electronics Letters |
| spelling | doaj.art-4158bd7f31554c148cc9b2788361ad212022-12-22T03:59:05ZengWileyElectronics Letters0013-51941350-911X2019-09-01551898698910.1049/el.2019.1719GenSynth: a generative synthesis approach to learning generative machines for generate efficient neural networksAlexander Wong0Mohammad Javad Shafiee1Brendan Chwyl2Francis Li3Waterloo Artificial Intelligence InstituteWaterlooCanadaWaterloo Artificial Intelligence InstituteWaterlooCanadaDarwinAI Corp.WaterlooCanadaDarwinAI Corp.WaterlooCanadaThe tremendous potential exhibited by deep learning is often offset by architectural and computational complexity, making widespread deployment a challenge for edge scenarios such as mobile and other consumer devices. To tackle this challenge, we explore the following idea: Can we learn generative machines to automatically generate deep neural networks with efficient network architectures? In this study, we introduce the idea of generative synthesis, which is premised on the intricate interplay between a generator‐inquisitor pair that work in tandem to garner insights and learn to generate highly efficient deep neural networks that best satisfies operational requirements. Experimental results for image classification, semantic segmentation, and object detection tasks illustrate the efficacy of generative synthesis (GenSynth) in producing generators that automatically generate highly efficient deep neural networks (which we nickname FermiNets with higher model efficiency and lower computational costs (reaching >10× more efficient and fewer multiply‐accumulate operations than several tested state‐of‐the‐art networks), as well as higher energy efficiency (reaching >4× improvements in image inferences per joule consumed on a Nvidia Tegra X2 mobile processor). As such, GenSynth can be a powerful, generalised approach for accelerating and improving the building of deep neural networks for on‐device edge scenarios.https://doi.org/10.1049/el.2019.1719computational complexitygenerative machinesefficient network architecturesGenSynthgenerator‐inquisitor pairunique highly efficient deep neural networks |
| spellingShingle | Alexander Wong Mohammad Javad Shafiee Brendan Chwyl Francis Li GenSynth: a generative synthesis approach to learning generative machines for generate efficient neural networks Electronics Letters computational complexity generative machines efficient network architectures GenSynth generator‐inquisitor pair unique highly efficient deep neural networks |
| title | GenSynth: a generative synthesis approach to learning generative machines for generate efficient neural networks |
| title_full | GenSynth: a generative synthesis approach to learning generative machines for generate efficient neural networks |
| title_fullStr | GenSynth: a generative synthesis approach to learning generative machines for generate efficient neural networks |
| title_full_unstemmed | GenSynth: a generative synthesis approach to learning generative machines for generate efficient neural networks |
| title_short | GenSynth: a generative synthesis approach to learning generative machines for generate efficient neural networks |
| title_sort | gensynth a generative synthesis approach to learning generative machines for generate efficient neural networks |
| topic | computational complexity generative machines efficient network architectures GenSynth generator‐inquisitor pair unique highly efficient deep neural networks |
| url | https://doi.org/10.1049/el.2019.1719 |
| work_keys_str_mv | AT alexanderwong gensynthagenerativesynthesisapproachtolearninggenerativemachinesforgenerateefficientneuralnetworks AT mohammadjavadshafiee gensynthagenerativesynthesisapproachtolearninggenerativemachinesforgenerateefficientneuralnetworks AT brendanchwyl gensynthagenerativesynthesisapproachtolearninggenerativemachinesforgenerateefficientneuralnetworks AT francisli gensynthagenerativesynthesisapproachtolearninggenerativemachinesforgenerateefficientneuralnetworks |