On the use of deep learning for computational imaging
Since their inception in the 1930–1960s, the research disciplines of computational imaging and machine learning have followed parallel tracks and, during the last two decades, experienced explosive growth drawing on similar progress in mathematical optimization and computing hardware. While these de...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
The Optical Society
2020
|
| Online Access: | https://hdl.handle.net/1721.1/126023 |
| _version_ | 1826202748117319680 |
|---|---|
| author | Barbastathis, George Ozcan, Aydogan Situ, Guohai |
| author2 | Massachusetts Institute of Technology. Department of Mechanical Engineering |
| author_facet | Massachusetts Institute of Technology. Department of Mechanical Engineering Barbastathis, George Ozcan, Aydogan Situ, Guohai |
| author_sort | Barbastathis, George |
| collection | MIT |
| description | Since their inception in the 1930–1960s, the research disciplines of computational imaging and machine learning have followed parallel tracks and, during the last two decades, experienced explosive growth drawing on similar progress in mathematical optimization and computing hardware. While these developments have always been to the benefit of image interpretation and machine vision, only recently has it become evident that machine learning architectures, and deep neural networks in particular, can be effective for computational image formation, aside from interpretation. The deep learning approach has proven to be especially attractive when the measurement is noisy and the measurement operator ill posed or uncertain. Examples reviewed here are: super-resolution; lensless retrieval of phase and complex amplitude from intensity; photon-limited scenes, including ghost imaging; and imaging through scatter. In this paper, we cast these works in a common framework. We relate the deep-learning-inspired solutions to the original computational imaging formulation and use the relationship to derive design insights, principles, and caveats of more general applicability. We also explore how the machine learning process is aided by the physics of imaging when ill posedness and uncertainties become particularly severe. It is hoped that the present unifying exposition will stimulate further progress in this promising field of research. ©2019 Optical Society of America. |
| first_indexed | 2024-09-23T12:17:08Z |
| format | Article |
| id | mit-1721.1/126023 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T12:17:08Z |
| publishDate | 2020 |
| publisher | The Optical Society |
| record_format | dspace |
| spelling | mit-1721.1/1260232022-09-28T00:55:45Z On the use of deep learning for computational imaging Barbastathis, George Ozcan, Aydogan Situ, Guohai Massachusetts Institute of Technology. Department of Mechanical Engineering Singapore-MIT Alliance in Research and Technology (SMART) Since their inception in the 1930–1960s, the research disciplines of computational imaging and machine learning have followed parallel tracks and, during the last two decades, experienced explosive growth drawing on similar progress in mathematical optimization and computing hardware. While these developments have always been to the benefit of image interpretation and machine vision, only recently has it become evident that machine learning architectures, and deep neural networks in particular, can be effective for computational image formation, aside from interpretation. The deep learning approach has proven to be especially attractive when the measurement is noisy and the measurement operator ill posed or uncertain. Examples reviewed here are: super-resolution; lensless retrieval of phase and complex amplitude from intensity; photon-limited scenes, including ghost imaging; and imaging through scatter. In this paper, we cast these works in a common framework. We relate the deep-learning-inspired solutions to the original computational imaging formulation and use the relationship to derive design insights, principles, and caveats of more general applicability. We also explore how the machine learning process is aided by the physics of imaging when ill posedness and uncertainties become particularly severe. It is hoped that the present unifying exposition will stimulate further progress in this promising field of research. ©2019 Optical Society of America. Intelligence Advanced Research Projects Activity - IARPA (FA8650-17-C-9113) Chinese Academy of Sciences- CAS (QYZDB-SSW-JSC002) Chinesisch-Deutsche Zentrum für Wissenschaftsförderung - CDZ (GZ1931) National Research Foundation Singapore - NRF (SMART Centre). 2020-06-30T13:34:42Z 2020-06-30T13:34:42Z 2019-07 2019-03 2020-06-22T18:53:42Z Article http://purl.org/eprint/type/JournalArticle 2334-2536 https://hdl.handle.net/1721.1/126023 Barbastathis, George et al., "On the use of deep learning for computational imaging." Optica 6, 8 (July 2019): p. 921-943 doi. 10.1364/OPTICA.6.000921 ©2019 Authors en https://dx.doi.org/10.1364/OPTICA.6.000921 Optica Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf The Optical Society OSA Publishing |
| spellingShingle | Barbastathis, George Ozcan, Aydogan Situ, Guohai On the use of deep learning for computational imaging |
| title | On the use of deep learning for computational imaging |
| title_full | On the use of deep learning for computational imaging |
| title_fullStr | On the use of deep learning for computational imaging |
| title_full_unstemmed | On the use of deep learning for computational imaging |
| title_short | On the use of deep learning for computational imaging |
| title_sort | on the use of deep learning for computational imaging |
| url | https://hdl.handle.net/1721.1/126023 |
| work_keys_str_mv | AT barbastathisgeorge ontheuseofdeeplearningforcomputationalimaging AT ozcanaydogan ontheuseofdeeplearningforcomputationalimaging AT situguohai ontheuseofdeeplearningforcomputationalimaging |