A Generalist Reinforcement Learning Agent for Compressing Convolutional Neural Networks

Over the years, researchers have proposed multiple approaches to reduce the number of parameters Deep Learning models have. Due to the complexity of compressing models, some authors have opted to train Reinforcement Learning agents that learn how to compress a particular model without losing conside...

Full description

Bibliographic Details
Main Authors: Gabriel Gonzalez-Sahagun, Santiago Enrique Conant-Pablos, Jose Carlos Ortiz-Bayliss, Jorge M. Cruz-Duarte
Format: Article
Language:English
Published: IEEE 2024-01-01
Series:IEEE Access
Subjects:
Online Access:https://ieeexplore.ieee.org/document/10494337/
_version_ 1797206087556923392
author Gabriel Gonzalez-Sahagun
Santiago Enrique Conant-Pablos
Jose Carlos Ortiz-Bayliss
Jorge M. Cruz-Duarte
author_facet Gabriel Gonzalez-Sahagun
Santiago Enrique Conant-Pablos
Jose Carlos Ortiz-Bayliss
Jorge M. Cruz-Duarte
author_sort Gabriel Gonzalez-Sahagun
collection DOAJ
description Over the years, researchers have proposed multiple approaches to reduce the number of parameters Deep Learning models have. Due to the complexity of compressing models, some authors have opted to train Reinforcement Learning agents that learn how to compress a particular model without losing considerable accuracy. Nonetheless, training an agent for each model can be time-consuming. We propose a methodology for training a generalist agent capable of compressing other convolutional neural networks that it was not trained to compress. Our generalist agent uses feature maps to select which compression technique to apply to convolutional and dense layers. Since the shape of the feature maps is reduced as it goes deeper into the network, we implemented a Dueling Deep Q-Network with a Region of Interest layer, allowing it to generate features of a fixed size for feature maps of various heights and widths. Our generalist agent trained to compress two LeNet models, one trained with fashion MNIST and the other with Kuzushiji-MNIST, compressed the same architecture trained on MNIST to less than 15% of its original size with an accuracy loss of less than 2.5%.
first_indexed 2024-04-24T09:01:26Z
format Article
id doaj.art-1429d0ceed844ca6820ced92ff25d756
institution Directory Open Access Journal
issn 2169-3536
language English
last_indexed 2024-04-24T09:01:26Z
publishDate 2024-01-01
publisher IEEE
record_format Article
series IEEE Access
spelling doaj.art-1429d0ceed844ca6820ced92ff25d7562024-04-15T23:00:35ZengIEEEIEEE Access2169-35362024-01-0112511005111410.1109/ACCESS.2024.338585710494337A Generalist Reinforcement Learning Agent for Compressing Convolutional Neural NetworksGabriel Gonzalez-Sahagun0https://orcid.org/0009-0009-7931-4654Santiago Enrique Conant-Pablos1https://orcid.org/0000-0001-6270-3164Jose Carlos Ortiz-Bayliss2https://orcid.org/0000-0003-3408-2166Jorge M. Cruz-Duarte3https://orcid.org/0000-0003-4494-7864School of Engineering and Sciences, Tecnológico de Monterrey, Monterrey, Nuevo Leon, MexicoSchool of Engineering and Sciences, Tecnológico de Monterrey, Monterrey, Nuevo Leon, MexicoSchool of Engineering and Sciences, Tecnológico de Monterrey, Monterrey, Nuevo Leon, MexicoSchool of Engineering and Sciences, Tecnológico de Monterrey, Monterrey, Nuevo Leon, MexicoOver the years, researchers have proposed multiple approaches to reduce the number of parameters Deep Learning models have. Due to the complexity of compressing models, some authors have opted to train Reinforcement Learning agents that learn how to compress a particular model without losing considerable accuracy. Nonetheless, training an agent for each model can be time-consuming. We propose a methodology for training a generalist agent capable of compressing other convolutional neural networks that it was not trained to compress. Our generalist agent uses feature maps to select which compression technique to apply to convolutional and dense layers. Since the shape of the feature maps is reduced as it goes deeper into the network, we implemented a Dueling Deep Q-Network with a Region of Interest layer, allowing it to generate features of a fixed size for feature maps of various heights and widths. Our generalist agent trained to compress two LeNet models, one trained with fashion MNIST and the other with Kuzushiji-MNIST, compressed the same architecture trained on MNIST to less than 15% of its original size with an accuracy loss of less than 2.5%.https://ieeexplore.ieee.org/document/10494337/Computer visiondeep learningmodel compressionmodel optimizationreinforcement learning
spellingShingle Gabriel Gonzalez-Sahagun
Santiago Enrique Conant-Pablos
Jose Carlos Ortiz-Bayliss
Jorge M. Cruz-Duarte
A Generalist Reinforcement Learning Agent for Compressing Convolutional Neural Networks
IEEE Access
Computer vision
deep learning
model compression
model optimization
reinforcement learning
title A Generalist Reinforcement Learning Agent for Compressing Convolutional Neural Networks
title_full A Generalist Reinforcement Learning Agent for Compressing Convolutional Neural Networks
title_fullStr A Generalist Reinforcement Learning Agent for Compressing Convolutional Neural Networks
title_full_unstemmed A Generalist Reinforcement Learning Agent for Compressing Convolutional Neural Networks
title_short A Generalist Reinforcement Learning Agent for Compressing Convolutional Neural Networks
title_sort generalist reinforcement learning agent for compressing convolutional neural networks
topic Computer vision
deep learning
model compression
model optimization
reinforcement learning
url https://ieeexplore.ieee.org/document/10494337/
work_keys_str_mv AT gabrielgonzalezsahagun ageneralistreinforcementlearningagentforcompressingconvolutionalneuralnetworks
AT santiagoenriqueconantpablos ageneralistreinforcementlearningagentforcompressingconvolutionalneuralnetworks
AT josecarlosortizbayliss ageneralistreinforcementlearningagentforcompressingconvolutionalneuralnetworks
AT jorgemcruzduarte ageneralistreinforcementlearningagentforcompressingconvolutionalneuralnetworks
AT gabrielgonzalezsahagun generalistreinforcementlearningagentforcompressingconvolutionalneuralnetworks
AT santiagoenriqueconantpablos generalistreinforcementlearningagentforcompressingconvolutionalneuralnetworks
AT josecarlosortizbayliss generalistreinforcementlearningagentforcompressingconvolutionalneuralnetworks
AT jorgemcruzduarte generalistreinforcementlearningagentforcompressingconvolutionalneuralnetworks