Backpropagation through nonlinear units for the all-optical training of neural networks
We propose a practical scheme for end-to-end optical backpropagation in neural networks. Using saturable absorption for the nonlinear units, we find that the backward-propagating gradients required to train the network can be approximated in a surprisingly simple pump-probe scheme that requires only...
| Main Authors: | , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Optical Society of America
2021
|
| _version_ | 1826262505119285248 |
|---|---|
| author | Guo, X Barrett, TD Wang, ZM Lvovsky, AI |
| author_facet | Guo, X Barrett, TD Wang, ZM Lvovsky, AI |
| author_sort | Guo, X |
| collection | OXFORD |
| description | We propose a practical scheme for end-to-end optical backpropagation in neural networks. Using saturable absorption for the nonlinear units, we find that the backward-propagating gradients required to train the network can be approximated in a surprisingly simple pump-probe scheme that requires only simple passive optical elements. Simulations show that, with readily obtainable optical depths, our approach can achieve equivalent performance to state-of-the-art computational networks on image classification benchmarks, even in deep networks with multiple sequential gradient approximation. With backpropagation through nonlinear units being an outstanding challenge to the field, this work provides a feasible path toward truly all-optical neural networks. |
| first_indexed | 2024-03-06T19:37:14Z |
| format | Journal article |
| id | oxford-uuid:1f733859-f95b-4bf0-b71c-1eb605ea9917 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T19:37:14Z |
| publishDate | 2021 |
| publisher | Optical Society of America |
| record_format | dspace |
| spelling | oxford-uuid:1f733859-f95b-4bf0-b71c-1eb605ea99172022-03-26T11:21:57ZBackpropagation through nonlinear units for the all-optical training of neural networksJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:1f733859-f95b-4bf0-b71c-1eb605ea9917EnglishSymplectic ElementsOptical Society of America2021Guo, XBarrett, TDWang, ZMLvovsky, AIWe propose a practical scheme for end-to-end optical backpropagation in neural networks. Using saturable absorption for the nonlinear units, we find that the backward-propagating gradients required to train the network can be approximated in a surprisingly simple pump-probe scheme that requires only simple passive optical elements. Simulations show that, with readily obtainable optical depths, our approach can achieve equivalent performance to state-of-the-art computational networks on image classification benchmarks, even in deep networks with multiple sequential gradient approximation. With backpropagation through nonlinear units being an outstanding challenge to the field, this work provides a feasible path toward truly all-optical neural networks. |
| spellingShingle | Guo, X Barrett, TD Wang, ZM Lvovsky, AI Backpropagation through nonlinear units for the all-optical training of neural networks |
| title | Backpropagation through nonlinear units for the all-optical training of neural networks |
| title_full | Backpropagation through nonlinear units for the all-optical training of neural networks |
| title_fullStr | Backpropagation through nonlinear units for the all-optical training of neural networks |
| title_full_unstemmed | Backpropagation through nonlinear units for the all-optical training of neural networks |
| title_short | Backpropagation through nonlinear units for the all-optical training of neural networks |
| title_sort | backpropagation through nonlinear units for the all optical training of neural networks |
| work_keys_str_mv | AT guox backpropagationthroughnonlinearunitsfortheallopticaltrainingofneuralnetworks AT barretttd backpropagationthroughnonlinearunitsfortheallopticaltrainingofneuralnetworks AT wangzm backpropagationthroughnonlinearunitsfortheallopticaltrainingofneuralnetworks AT lvovskyai backpropagationthroughnonlinearunitsfortheallopticaltrainingofneuralnetworks |