Universal adaptive optics for microscopy through embedded neural network control
The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutio...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Springer Nature
2023
|
| _version_ | 1797111607709401088 |
|---|---|
| author | Hu, Q Hailstone, M Wang, J Wincott, M Stoychev, D Atilgan, H Gala, D Chaiamarit, T Parton, RM Antonello, J Packer, AM Davis, I Booth, MJ |
| author_facet | Hu, Q Hailstone, M Wang, J Wincott, M Stoychev, D Atilgan, H Gala, D Chaiamarit, T Parton, RM Antonello, J Packer, AM Davis, I Booth, MJ |
| author_sort | Hu, Q |
| collection | OXFORD |
| description | The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution – one that can be readily transferred between microscope modalities – has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a “black box”, but provided physical insights on internal workings, which could influence future designs. |
| first_indexed | 2024-03-07T08:12:48Z |
| format | Journal article |
| id | oxford-uuid:87c299fc-9a68-4374-a3bd-0d270a149ab4 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T08:12:48Z |
| publishDate | 2023 |
| publisher | Springer Nature |
| record_format | dspace |
| spelling | oxford-uuid:87c299fc-9a68-4374-a3bd-0d270a149ab42023-12-04T11:04:12ZUniversal adaptive optics for microscopy through embedded neural network controlJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:87c299fc-9a68-4374-a3bd-0d270a149ab4EnglishSymplectic ElementsSpringer Nature2023Hu, QHailstone, MWang, JWincott, MStoychev, DAtilgan, HGala, DChaiamarit, TParton, RMAntonello, JPacker, AMDavis, IBooth, MJThe resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution – one that can be readily transferred between microscope modalities – has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a “black box”, but provided physical insights on internal workings, which could influence future designs. |
| spellingShingle | Hu, Q Hailstone, M Wang, J Wincott, M Stoychev, D Atilgan, H Gala, D Chaiamarit, T Parton, RM Antonello, J Packer, AM Davis, I Booth, MJ Universal adaptive optics for microscopy through embedded neural network control |
| title | Universal adaptive optics for microscopy through embedded neural network control |
| title_full | Universal adaptive optics for microscopy through embedded neural network control |
| title_fullStr | Universal adaptive optics for microscopy through embedded neural network control |
| title_full_unstemmed | Universal adaptive optics for microscopy through embedded neural network control |
| title_short | Universal adaptive optics for microscopy through embedded neural network control |
| title_sort | universal adaptive optics for microscopy through embedded neural network control |
| work_keys_str_mv | AT huq universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT hailstonem universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT wangj universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT wincottm universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT stoychevd universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT atilganh universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT galad universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT chaiamaritt universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT partonrm universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT antonelloj universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT packeram universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT davisi universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol AT boothmj universaladaptiveopticsformicroscopythroughembeddedneuralnetworkcontrol |