The Impact of Analog-to-Digital Converter Architecture and Variability on Analog Neural Network Accuracy
The analog-to-digital converter (ADC) is not only a key component in analog in-memory computing (IMC) accelerators but also a bottleneck for the efficiency and accuracy of these systems. While the tradeoffs between power consumption, latency, and area in ADC design are well studied, it is relatively...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Journal on Exploratory Solid-State Computational Devices and Circuits |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10250846/ |
| _version_ | 1797338469852250112 |
|---|---|
| author | Matthew Spear Joshua E. Kim Christopher H. Bennett Sapan Agarwal Matthew J. Marinella T. Patrick Xiao |
| author_facet | Matthew Spear Joshua E. Kim Christopher H. Bennett Sapan Agarwal Matthew J. Marinella T. Patrick Xiao |
| author_sort | Matthew Spear |
| collection | DOAJ |
| description | The analog-to-digital converter (ADC) is not only a key component in analog in-memory computing (IMC) accelerators but also a bottleneck for the efficiency and accuracy of these systems. While the tradeoffs between power consumption, latency, and area in ADC design are well studied, it is relatively unknown which ADC implementations are optimal for algorithmic accuracy, particularly for neural network inference. We explore the design space of the ADC with a focus on accuracy, investigating the sensitivity of neural network outputs to component variability inside the ADC and how this sensitivity depends on the ADC architecture. The compact models of the pipeline, cyclic, successive-approximation-register (SAR) and ramp ADCs are developed, and these models are used in a system-level accuracy simulation of analog neural network inference. Our results show how the accuracy on a complex image recognition benchmark (ResNet50 on ImageNet) depends on the capacitance mismatch, comparator offset, and effective number of bits (ENOB) for each of the four ADC architectures. We find that robustness to component variations depends strongly on the ADC design and that inference accuracy is particularly sensitive to the value-dependent error characteristics of the ADC, which cannot be captured by the conventional ENOB precision metric. |
| first_indexed | 2024-03-08T09:31:41Z |
| format | Article |
| id | doaj.art-2fea95d8692c448d91928b5cba991cb8 |
| institution | Directory Open Access Journal |
| issn | 2329-9231 |
| language | English |
| last_indexed | 2024-03-08T09:31:41Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal on Exploratory Solid-State Computational Devices and Circuits |
| spelling | doaj.art-2fea95d8692c448d91928b5cba991cb82024-01-31T00:01:51ZengIEEEIEEE Journal on Exploratory Solid-State Computational Devices and Circuits2329-92312023-01-019217618410.1109/JXCDC.2023.331513410250846The Impact of Analog-to-Digital Converter Architecture and Variability on Analog Neural Network AccuracyMatthew Spear0https://orcid.org/0000-0002-0452-1765Joshua E. Kim1https://orcid.org/0009-0004-8898-694XChristopher H. Bennett2https://orcid.org/0000-0002-6989-292XSapan Agarwal3https://orcid.org/0000-0002-3676-6986Matthew J. Marinella4https://orcid.org/0000-0002-6537-1836T. Patrick Xiao5https://orcid.org/0000-0001-9066-2961School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, USASchool of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, USASandia National Laboratories, Albuquerque, NM, USASandia National Laboratories, Albuquerque, NM, USASchool of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ, USASandia National Laboratories, Albuquerque, NM, USAThe analog-to-digital converter (ADC) is not only a key component in analog in-memory computing (IMC) accelerators but also a bottleneck for the efficiency and accuracy of these systems. While the tradeoffs between power consumption, latency, and area in ADC design are well studied, it is relatively unknown which ADC implementations are optimal for algorithmic accuracy, particularly for neural network inference. We explore the design space of the ADC with a focus on accuracy, investigating the sensitivity of neural network outputs to component variability inside the ADC and how this sensitivity depends on the ADC architecture. The compact models of the pipeline, cyclic, successive-approximation-register (SAR) and ramp ADCs are developed, and these models are used in a system-level accuracy simulation of analog neural network inference. Our results show how the accuracy on a complex image recognition benchmark (ResNet50 on ImageNet) depends on the capacitance mismatch, comparator offset, and effective number of bits (ENOB) for each of the four ADC architectures. We find that robustness to component variations depends strongly on the ADC design and that inference accuracy is particularly sensitive to the value-dependent error characteristics of the ADC, which cannot be captured by the conventional ENOB precision metric.https://ieeexplore.ieee.org/document/10250846/Analog computinganalog-to-digital conversionin-memory computing (IMC)machine learningneural networkprocess variations |
| spellingShingle | Matthew Spear Joshua E. Kim Christopher H. Bennett Sapan Agarwal Matthew J. Marinella T. Patrick Xiao The Impact of Analog-to-Digital Converter Architecture and Variability on Analog Neural Network Accuracy IEEE Journal on Exploratory Solid-State Computational Devices and Circuits Analog computing analog-to-digital conversion in-memory computing (IMC) machine learning neural network process variations |
| title | The Impact of Analog-to-Digital Converter Architecture and Variability on Analog Neural Network Accuracy |
| title_full | The Impact of Analog-to-Digital Converter Architecture and Variability on Analog Neural Network Accuracy |
| title_fullStr | The Impact of Analog-to-Digital Converter Architecture and Variability on Analog Neural Network Accuracy |
| title_full_unstemmed | The Impact of Analog-to-Digital Converter Architecture and Variability on Analog Neural Network Accuracy |
| title_short | The Impact of Analog-to-Digital Converter Architecture and Variability on Analog Neural Network Accuracy |
| title_sort | impact of analog to digital converter architecture and variability on analog neural network accuracy |
| topic | Analog computing analog-to-digital conversion in-memory computing (IMC) machine learning neural network process variations |
| url | https://ieeexplore.ieee.org/document/10250846/ |
| work_keys_str_mv | AT matthewspear theimpactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT joshuaekim theimpactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT christopherhbennett theimpactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT sapanagarwal theimpactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT matthewjmarinella theimpactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT tpatrickxiao theimpactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT matthewspear impactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT joshuaekim impactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT christopherhbennett impactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT sapanagarwal impactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT matthewjmarinella impactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy AT tpatrickxiao impactofanalogtodigitalconverterarchitectureandvariabilityonanalogneuralnetworkaccuracy |