IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar Arrays
Resistive Crossbar Arrays present an elegant implementation solution for Deep Neural Networks acceleration. The Matrix-Vector Multiplication, which is the corner-stone of DNNs, is carried out in O(1) compared to O(N<sup>2</sup>) steps for digital realizations of O(log<sub>2</sub...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9293271/ |
| _version_ | 1818853470540660736 |
|---|---|
| author | Mohammed E. Fouda Sugil Lee Jongeun Lee Gun Hwan Kim Fadi Kurdahi Ahmed M. Eltawi |
| author_facet | Mohammed E. Fouda Sugil Lee Jongeun Lee Gun Hwan Kim Fadi Kurdahi Ahmed M. Eltawi |
| author_sort | Mohammed E. Fouda |
| collection | DOAJ |
| description | Resistive Crossbar Arrays present an elegant implementation solution for Deep Neural Networks acceleration. The Matrix-Vector Multiplication, which is the corner-stone of DNNs, is carried out in O(1) compared to O(N<sup>2</sup>) steps for digital realizations of O(log<sub>2</sub>(N)) steps for in-memory associative processors. However, the IR drop problem, caused by the inevitable interconnect wire resistance in RCAs remains a daunting challenge. In this article, we propose a fast and efficient training and validation framework to incorporate the wire resistance in Quantized DNNs, without the need for computationally extensive SPICE simulations during the training process. A fabricated four-bit Au/Al<sub>2</sub>O<sub>3</sub>/HfO<sub>2</sub>/TiN device is modelled and used within the framework with two-mapping schemes to realize the quantized weights. Efficient system-level IR-drop estimation methods are used to accelerate training. SPICE validation results show the effectiveness of the proposed method to capture the IR drop problem achieving the baseline accuracy with a 2% and 4% drop in the worst-case scenario for MNIST dataset on multilayer perceptron network and CIFAR 10 dataset on modified VGG and AlexNet networks, respectively. Other nonidealities, such as stuck-at fault defects, variability, and aging, are studied. Finally, the design considerations of the neuronal and the driver circuits are discussed. |
| first_indexed | 2024-12-19T07:37:20Z |
| format | Article |
| id | doaj.art-82cd359f03304aea973f73ac901d4ea1 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T07:37:20Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-82cd359f03304aea973f73ac901d4ea12022-12-21T20:30:33ZengIEEEIEEE Access2169-35362020-01-01822839222840810.1109/ACCESS.2020.30446529293271IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar ArraysMohammed E. Fouda0https://orcid.org/0000-0001-7139-3428Sugil Lee1https://orcid.org/0000-0003-3092-6501Jongeun Lee2https://orcid.org/0000-0003-1523-2974Gun Hwan Kim3Fadi Kurdahi4https://orcid.org/0000-0002-6982-365XAhmed M. Eltawi5https://orcid.org/0000-0003-1849-083XCenter for Embedded and Cyber-Physical Systems, University of California at Irvine, Irvine, CA, USASchool of ECE, UNIST, Ulsan, South KoreaSchool of ECE, UNIST, Ulsan, South KoreaDivision of Advanced Materials, Korea Research Institute of Chemical Technology (KRICT), Daejeon, South KoreaCenter for Embedded and Cyber-Physical Systems, University of California at Irvine, Irvine, CA, USACenter for Embedded and Cyber-Physical Systems, University of California at Irvine, Irvine, CA, USAResistive Crossbar Arrays present an elegant implementation solution for Deep Neural Networks acceleration. The Matrix-Vector Multiplication, which is the corner-stone of DNNs, is carried out in O(1) compared to O(N<sup>2</sup>) steps for digital realizations of O(log<sub>2</sub>(N)) steps for in-memory associative processors. However, the IR drop problem, caused by the inevitable interconnect wire resistance in RCAs remains a daunting challenge. In this article, we propose a fast and efficient training and validation framework to incorporate the wire resistance in Quantized DNNs, without the need for computationally extensive SPICE simulations during the training process. A fabricated four-bit Au/Al<sub>2</sub>O<sub>3</sub>/HfO<sub>2</sub>/TiN device is modelled and used within the framework with two-mapping schemes to realize the quantized weights. Efficient system-level IR-drop estimation methods are used to accelerate training. SPICE validation results show the effectiveness of the proposed method to capture the IR drop problem achieving the baseline accuracy with a 2% and 4% drop in the worst-case scenario for MNIST dataset on multilayer perceptron network and CIFAR 10 dataset on modified VGG and AlexNet networks, respectively. Other nonidealities, such as stuck-at fault defects, variability, and aging, are studied. Finally, the design considerations of the neuronal and the driver circuits are discussed.https://ieeexplore.ieee.org/document/9293271/RRAMmemristordeep neural networksquantized neural networksIR dropnonidealities |
| spellingShingle | Mohammed E. Fouda Sugil Lee Jongeun Lee Gun Hwan Kim Fadi Kurdahi Ahmed M. Eltawi IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar Arrays IEEE Access RRAM memristor deep neural networks quantized neural networks IR drop nonidealities |
| title | IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar Arrays |
| title_full | IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar Arrays |
| title_fullStr | IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar Arrays |
| title_full_unstemmed | IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar Arrays |
| title_short | IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar Arrays |
| title_sort | ir qnn framework an ir drop aware offline training of quantized crossbar arrays |
| topic | RRAM memristor deep neural networks quantized neural networks IR drop nonidealities |
| url | https://ieeexplore.ieee.org/document/9293271/ |
| work_keys_str_mv | AT mohammedefouda irqnnframeworkanirdropawareofflinetrainingofquantizedcrossbararrays AT sugillee irqnnframeworkanirdropawareofflinetrainingofquantizedcrossbararrays AT jongeunlee irqnnframeworkanirdropawareofflinetrainingofquantizedcrossbararrays AT gunhwankim irqnnframeworkanirdropawareofflinetrainingofquantizedcrossbararrays AT fadikurdahi irqnnframeworkanirdropawareofflinetrainingofquantizedcrossbararrays AT ahmedmeltawi irqnnframeworkanirdropawareofflinetrainingofquantizedcrossbararrays |