Synapse-Neuron-Aware Training Scheme of Defect-Tolerant Neural Networks with Defective Memristor Crossbars
To overcome the limitations of CMOS digital systems, emerging computing circuits such as memristor crossbars have been investigated as potential candidates for significantly increasing the speed and energy efficiency of next-generation computing systems, which are required for implementing future AI...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-02-01
|
| Series: | Micromachines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-666X/13/2/273 |
| _version_ | 1797477896065908736 |
|---|---|
| author | Jiyong An Seokjin Oh Tien Van Nguyen Kyeong-Sik Min |
| author_facet | Jiyong An Seokjin Oh Tien Van Nguyen Kyeong-Sik Min |
| author_sort | Jiyong An |
| collection | DOAJ |
| description | To overcome the limitations of CMOS digital systems, emerging computing circuits such as memristor crossbars have been investigated as potential candidates for significantly increasing the speed and energy efficiency of next-generation computing systems, which are required for implementing future AI hardware. Unfortunately, manufacturing yield still remains a serious challenge in adopting memristor-based computing systems due to the limitations of immature fabrication technology. To compensate for malfunction of neural networks caused from the fabrication-related defects, a new crossbar training scheme combining the synapse-aware with the neuron-aware together is proposed in this paper, for optimizing the defect map size and the neural network’s performance simultaneously. In the proposed scheme, the memristor crossbar’s columns are divided into 3 groups, which are the severely-defective, moderately-defective, and normal columns, respectively. Here, each group is trained according to the trade-off relationship between the neural network’s performance and the hardware overhead of defect-tolerant training. As a result of this group-based training method combining the neuron-aware with the synapse-aware, in this paper, the new scheme can be successful in improving the network’s performance better than both the synapse-aware and the neuron-aware while minimizing its hardware burden. For example, when testing the defect percentage = 10% with MNIST dataset, the proposed scheme outperforms the synapse-aware and the neuron-aware by 3.8% and 3.4% for the number of crossbar’s columns trained for synapse defects = 10 and 138 among 310, respectively, while maintaining the smaller memory size than the synapse-aware. When the trained columns = 138, the normalized memory size of the synapse-neuron-aware scheme can be smaller by 3.1% than the synapse-aware. |
| first_indexed | 2024-03-09T21:24:16Z |
| format | Article |
| id | doaj.art-02c1e47783d44d7fad305177d49df23f |
| institution | Directory Open Access Journal |
| issn | 2072-666X |
| language | English |
| last_indexed | 2024-03-09T21:24:16Z |
| publishDate | 2022-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Micromachines |
| spelling | doaj.art-02c1e47783d44d7fad305177d49df23f2023-11-23T21:11:19ZengMDPI AGMicromachines2072-666X2022-02-0113227310.3390/mi13020273Synapse-Neuron-Aware Training Scheme of Defect-Tolerant Neural Networks with Defective Memristor CrossbarsJiyong An0Seokjin Oh1Tien Van Nguyen2Kyeong-Sik Min3School of Electrical Engineering, Kookmin University, Seoul 02707, KoreaSchool of Electrical Engineering, Kookmin University, Seoul 02707, KoreaSchool of Electrical Engineering, Kookmin University, Seoul 02707, KoreaSchool of Electrical Engineering, Kookmin University, Seoul 02707, KoreaTo overcome the limitations of CMOS digital systems, emerging computing circuits such as memristor crossbars have been investigated as potential candidates for significantly increasing the speed and energy efficiency of next-generation computing systems, which are required for implementing future AI hardware. Unfortunately, manufacturing yield still remains a serious challenge in adopting memristor-based computing systems due to the limitations of immature fabrication technology. To compensate for malfunction of neural networks caused from the fabrication-related defects, a new crossbar training scheme combining the synapse-aware with the neuron-aware together is proposed in this paper, for optimizing the defect map size and the neural network’s performance simultaneously. In the proposed scheme, the memristor crossbar’s columns are divided into 3 groups, which are the severely-defective, moderately-defective, and normal columns, respectively. Here, each group is trained according to the trade-off relationship between the neural network’s performance and the hardware overhead of defect-tolerant training. As a result of this group-based training method combining the neuron-aware with the synapse-aware, in this paper, the new scheme can be successful in improving the network’s performance better than both the synapse-aware and the neuron-aware while minimizing its hardware burden. For example, when testing the defect percentage = 10% with MNIST dataset, the proposed scheme outperforms the synapse-aware and the neuron-aware by 3.8% and 3.4% for the number of crossbar’s columns trained for synapse defects = 10 and 138 among 310, respectively, while maintaining the smaller memory size than the synapse-aware. When the trained columns = 138, the normalized memory size of the synapse-neuron-aware scheme can be smaller by 3.1% than the synapse-aware.https://www.mdpi.com/2072-666X/13/2/273synapse-neuron-aware trainingdefect-tolerant neural networksdefective memristor crossbarsmemristor defectsneuromorphic |
| spellingShingle | Jiyong An Seokjin Oh Tien Van Nguyen Kyeong-Sik Min Synapse-Neuron-Aware Training Scheme of Defect-Tolerant Neural Networks with Defective Memristor Crossbars Micromachines synapse-neuron-aware training defect-tolerant neural networks defective memristor crossbars memristor defects neuromorphic |
| title | Synapse-Neuron-Aware Training Scheme of Defect-Tolerant Neural Networks with Defective Memristor Crossbars |
| title_full | Synapse-Neuron-Aware Training Scheme of Defect-Tolerant Neural Networks with Defective Memristor Crossbars |
| title_fullStr | Synapse-Neuron-Aware Training Scheme of Defect-Tolerant Neural Networks with Defective Memristor Crossbars |
| title_full_unstemmed | Synapse-Neuron-Aware Training Scheme of Defect-Tolerant Neural Networks with Defective Memristor Crossbars |
| title_short | Synapse-Neuron-Aware Training Scheme of Defect-Tolerant Neural Networks with Defective Memristor Crossbars |
| title_sort | synapse neuron aware training scheme of defect tolerant neural networks with defective memristor crossbars |
| topic | synapse-neuron-aware training defect-tolerant neural networks defective memristor crossbars memristor defects neuromorphic |
| url | https://www.mdpi.com/2072-666X/13/2/273 |
| work_keys_str_mv | AT jiyongan synapseneuronawaretrainingschemeofdefecttolerantneuralnetworkswithdefectivememristorcrossbars AT seokjinoh synapseneuronawaretrainingschemeofdefecttolerantneuralnetworkswithdefectivememristorcrossbars AT tienvannguyen synapseneuronawaretrainingschemeofdefecttolerantneuralnetworkswithdefectivememristorcrossbars AT kyeongsikmin synapseneuronawaretrainingschemeofdefecttolerantneuralnetworkswithdefectivememristorcrossbars |