DETransMVSnet: Research on Terahertz 3D Reconstruction of Multi-View Stereo Network With Deep Equilibrium Transformers
Terahertz waves, positioned between microwaves and infrared in the electromagnetic spectrum, are distinguished by their exceptional penetration capabilities, minimal energy requirements, and consistent absorption profiles for specific substances. Their versatility in applications such as non-destruc...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10360123/ |
| _version_ | 1797370517327446016 |
|---|---|
| author | Fan Bai Lun Li Wencheng Wang Xiaojin Wu |
| author_facet | Fan Bai Lun Li Wencheng Wang Xiaojin Wu |
| author_sort | Fan Bai |
| collection | DOAJ |
| description | Terahertz waves, positioned between microwaves and infrared in the electromagnetic spectrum, are distinguished by their exceptional penetration capabilities, minimal energy requirements, and consistent absorption profiles for specific substances. Their versatility in applications such as non-destructive evaluation, human security scans, and biological diagnostics has propelled them to the forefront of scientific inquiry. However, existing terahertz equipment poses limitations in terms of compromised resolution, diffraction-induced blurring, and degradation of clarity due to texture overlaps. Consequently, numerous multi-view 3D reconstruction algorithms struggle to produce high-quality results with terahertz imagery. To address these challenges—particularly the scarcity of terahertz datasets and texture conflation—we integrated X-ray images from the DTU dataset with our collected terahertz projections. By observing inconsistent texture projections in multi-view terahertz images resulting from angle shifts, we developed DETransMVSnet—a state-of-the-art multi-view 3D reconstruction approach based on the Multi-Scale Deep Equilibrium Layer (MDEQ) paradigm. Leveraging equilibrium layers within homography-projected feature maps enables us to extract masks that differentiate different layers within a scene. The Intra-Attention and Mask-Attention Blocks further refine feature selection by preserving relevant terahertz details while suppressing disruptive background elements. As evidence of its effectiveness, DETransMVSnet achieves comparable performance to conventional algorithms on the DTU dataset but notably outperforms them when applied to terahertz datasets by successfully reconstructing images where previous methods have failed. |
| first_indexed | 2024-03-08T18:03:40Z |
| format | Article |
| id | doaj.art-c218d5aa89284e30a8ecd5cdc78c50f5 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T18:03:40Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c218d5aa89284e30a8ecd5cdc78c50f52024-01-02T00:01:56ZengIEEEIEEE Access2169-35362023-01-011114604214605310.1109/ACCESS.2023.334284710360123DETransMVSnet: Research on Terahertz 3D Reconstruction of Multi-View Stereo Network With Deep Equilibrium TransformersFan Bai0Lun Li1https://orcid.org/0000-0001-6027-4423Wencheng Wang2https://orcid.org/0000-0002-0888-9225Xiaojin Wu3College of Equipment Engineering, Shenyang Ligong University, Shenyang, ChinaInstitute of Machinery and Automation, Weifang University, Weifang, ChinaUniversity Engineering Research Center for Robot Vision Perception and Control, Weifang, ChinaInstitute of Machinery and Automation, Weifang University, Weifang, ChinaTerahertz waves, positioned between microwaves and infrared in the electromagnetic spectrum, are distinguished by their exceptional penetration capabilities, minimal energy requirements, and consistent absorption profiles for specific substances. Their versatility in applications such as non-destructive evaluation, human security scans, and biological diagnostics has propelled them to the forefront of scientific inquiry. However, existing terahertz equipment poses limitations in terms of compromised resolution, diffraction-induced blurring, and degradation of clarity due to texture overlaps. Consequently, numerous multi-view 3D reconstruction algorithms struggle to produce high-quality results with terahertz imagery. To address these challenges—particularly the scarcity of terahertz datasets and texture conflation—we integrated X-ray images from the DTU dataset with our collected terahertz projections. By observing inconsistent texture projections in multi-view terahertz images resulting from angle shifts, we developed DETransMVSnet—a state-of-the-art multi-view 3D reconstruction approach based on the Multi-Scale Deep Equilibrium Layer (MDEQ) paradigm. Leveraging equilibrium layers within homography-projected feature maps enables us to extract masks that differentiate different layers within a scene. The Intra-Attention and Mask-Attention Blocks further refine feature selection by preserving relevant terahertz details while suppressing disruptive background elements. As evidence of its effectiveness, DETransMVSnet achieves comparable performance to conventional algorithms on the DTU dataset but notably outperforms them when applied to terahertz datasets by successfully reconstructing images where previous methods have failed.https://ieeexplore.ieee.org/document/10360123/Terahertz imagingtransmission typeDETransMVSnetMDEQthree-dimensional reconstruction |
| spellingShingle | Fan Bai Lun Li Wencheng Wang Xiaojin Wu DETransMVSnet: Research on Terahertz 3D Reconstruction of Multi-View Stereo Network With Deep Equilibrium Transformers IEEE Access Terahertz imaging transmission type DETransMVSnet MDEQ three-dimensional reconstruction |
| title | DETransMVSnet: Research on Terahertz 3D Reconstruction of Multi-View Stereo Network With Deep Equilibrium Transformers |
| title_full | DETransMVSnet: Research on Terahertz 3D Reconstruction of Multi-View Stereo Network With Deep Equilibrium Transformers |
| title_fullStr | DETransMVSnet: Research on Terahertz 3D Reconstruction of Multi-View Stereo Network With Deep Equilibrium Transformers |
| title_full_unstemmed | DETransMVSnet: Research on Terahertz 3D Reconstruction of Multi-View Stereo Network With Deep Equilibrium Transformers |
| title_short | DETransMVSnet: Research on Terahertz 3D Reconstruction of Multi-View Stereo Network With Deep Equilibrium Transformers |
| title_sort | detransmvsnet research on terahertz 3d reconstruction of multi view stereo network with deep equilibrium transformers |
| topic | Terahertz imaging transmission type DETransMVSnet MDEQ three-dimensional reconstruction |
| url | https://ieeexplore.ieee.org/document/10360123/ |
| work_keys_str_mv | AT fanbai detransmvsnetresearchonterahertz3dreconstructionofmultiviewstereonetworkwithdeepequilibriumtransformers AT lunli detransmvsnetresearchonterahertz3dreconstructionofmultiviewstereonetworkwithdeepequilibriumtransformers AT wenchengwang detransmvsnetresearchonterahertz3dreconstructionofmultiviewstereonetworkwithdeepequilibriumtransformers AT xiaojinwu detransmvsnetresearchonterahertz3dreconstructionofmultiviewstereonetworkwithdeepequilibriumtransformers |