Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time
Abstract Noninvasive X-ray imaging of nanoscale three-dimensional objects, such as integrated circuits (ICs), generally requires two types of scanning: ptychographic, which is translational and returns estimates of the complex electromagnetic field through the IC; combined with a tomographic scan, w...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Publishing Group
2023-05-01
|
| Series: | Light: Science & Applications |
| Online Access: | https://doi.org/10.1038/s41377-023-01181-8 |
| _version_ | 1797811263441469440 |
|---|---|
| author | Iksung Kang Ziling Wu Yi Jiang Yudong Yao Junjing Deng Jeffrey Klug Stefan Vogt George Barbastathis |
| author_facet | Iksung Kang Ziling Wu Yi Jiang Yudong Yao Junjing Deng Jeffrey Klug Stefan Vogt George Barbastathis |
| author_sort | Iksung Kang |
| collection | DOAJ |
| description | Abstract Noninvasive X-ray imaging of nanoscale three-dimensional objects, such as integrated circuits (ICs), generally requires two types of scanning: ptychographic, which is translational and returns estimates of the complex electromagnetic field through the IC; combined with a tomographic scan, which collects these complex field projections from multiple angles. Here, we present Attentional Ptycho-Tomography (APT), an approach to drastically reduce the amount of angular scanning, and thus the total acquisition time. APT is machine learning-based, utilizing axial self-Attention for Ptycho-Tomographic reconstruction. APT is trained to obtain accurate reconstructions of the ICs, despite the incompleteness of the measurements. The training process includes regularizing priors in the form of typical patterns found in IC interiors, and the physics of X-ray propagation through the IC. We show that APT with ×12 reduced angles achieves fidelity comparable to the gold standard Simultaneous Algebraic Reconstruction Technique (SART) with the original set of angles. When using the same set of reduced angles, then APT also outperforms Filtered Back Projection (FBP), Simultaneous Iterative Reconstruction Technique (SIRT) and SART. The time needed to compute the reconstruction is also reduced, because the trained neural network is a forward operation, unlike the iterative nature of these alternatives. Our experiments show that, without loss in quality, for a 4.48 × 93.2 × 3.92 µm3 IC (≃6 × 108 voxels), APT reduces the total data acquisition and computation time from 67.96 h to 38 min. We expect our physics-assisted and attention-utilizing machine learning framework to be applicable to other branches of nanoscale imaging, including materials science and biological imaging. |
| first_indexed | 2024-03-13T07:21:07Z |
| format | Article |
| id | doaj.art-55ea2e92ba8a4e1bb6f4bf7f7ec8b1d3 |
| institution | Directory Open Access Journal |
| issn | 2047-7538 |
| language | English |
| last_indexed | 2024-03-13T07:21:07Z |
| publishDate | 2023-05-01 |
| publisher | Nature Publishing Group |
| record_format | Article |
| series | Light: Science & Applications |
| spelling | doaj.art-55ea2e92ba8a4e1bb6f4bf7f7ec8b1d32023-06-04T11:38:53ZengNature Publishing GroupLight: Science & Applications2047-75382023-05-0112111410.1038/s41377-023-01181-8Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation timeIksung Kang0Ziling Wu1Yi Jiang2Yudong Yao3Junjing Deng4Jeffrey Klug5Stefan Vogt6George Barbastathis7Department of Electrical Engineering and Computer Science, Massachusetts Institute of TechnologyDepartment of Mechanical Engineering, Massachusetts Institute of TechnologyArgonne National LaboratoryArgonne National LaboratoryArgonne National LaboratoryArgonne National LaboratoryArgonne National LaboratoryDepartment of Mechanical Engineering, Massachusetts Institute of TechnologyAbstract Noninvasive X-ray imaging of nanoscale three-dimensional objects, such as integrated circuits (ICs), generally requires two types of scanning: ptychographic, which is translational and returns estimates of the complex electromagnetic field through the IC; combined with a tomographic scan, which collects these complex field projections from multiple angles. Here, we present Attentional Ptycho-Tomography (APT), an approach to drastically reduce the amount of angular scanning, and thus the total acquisition time. APT is machine learning-based, utilizing axial self-Attention for Ptycho-Tomographic reconstruction. APT is trained to obtain accurate reconstructions of the ICs, despite the incompleteness of the measurements. The training process includes regularizing priors in the form of typical patterns found in IC interiors, and the physics of X-ray propagation through the IC. We show that APT with ×12 reduced angles achieves fidelity comparable to the gold standard Simultaneous Algebraic Reconstruction Technique (SART) with the original set of angles. When using the same set of reduced angles, then APT also outperforms Filtered Back Projection (FBP), Simultaneous Iterative Reconstruction Technique (SIRT) and SART. The time needed to compute the reconstruction is also reduced, because the trained neural network is a forward operation, unlike the iterative nature of these alternatives. Our experiments show that, without loss in quality, for a 4.48 × 93.2 × 3.92 µm3 IC (≃6 × 108 voxels), APT reduces the total data acquisition and computation time from 67.96 h to 38 min. We expect our physics-assisted and attention-utilizing machine learning framework to be applicable to other branches of nanoscale imaging, including materials science and biological imaging.https://doi.org/10.1038/s41377-023-01181-8 |
| spellingShingle | Iksung Kang Ziling Wu Yi Jiang Yudong Yao Junjing Deng Jeffrey Klug Stefan Vogt George Barbastathis Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time Light: Science & Applications |
| title | Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time |
| title_full | Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time |
| title_fullStr | Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time |
| title_full_unstemmed | Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time |
| title_short | Attentional Ptycho-Tomography (APT) for three-dimensional nanoscale X-ray imaging with minimal data acquisition and computation time |
| title_sort | attentional ptycho tomography apt for three dimensional nanoscale x ray imaging with minimal data acquisition and computation time |
| url | https://doi.org/10.1038/s41377-023-01181-8 |
| work_keys_str_mv | AT iksungkang attentionalptychotomographyaptforthreedimensionalnanoscalexrayimagingwithminimaldataacquisitionandcomputationtime AT zilingwu attentionalptychotomographyaptforthreedimensionalnanoscalexrayimagingwithminimaldataacquisitionandcomputationtime AT yijiang attentionalptychotomographyaptforthreedimensionalnanoscalexrayimagingwithminimaldataacquisitionandcomputationtime AT yudongyao attentionalptychotomographyaptforthreedimensionalnanoscalexrayimagingwithminimaldataacquisitionandcomputationtime AT junjingdeng attentionalptychotomographyaptforthreedimensionalnanoscalexrayimagingwithminimaldataacquisitionandcomputationtime AT jeffreyklug attentionalptychotomographyaptforthreedimensionalnanoscalexrayimagingwithminimaldataacquisitionandcomputationtime AT stefanvogt attentionalptychotomographyaptforthreedimensionalnanoscalexrayimagingwithminimaldataacquisitionandcomputationtime AT georgebarbastathis attentionalptychotomographyaptforthreedimensionalnanoscalexrayimagingwithminimaldataacquisitionandcomputationtime |