3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video
This study aims to achieve accurate three-dimensional (3D) localization of multiple objects in a complicated scene using passive imaging. It is challenging, as it requires accurate localization of the objects in all three dimensions given recorded 2D images. An integral imaging system captures the s...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/9/4191 |
| _version_ | 1797601743405580288 |
|---|---|
| author | Michael Kadosh Yitzhak Yitzhaky |
| author_facet | Michael Kadosh Yitzhak Yitzhaky |
| author_sort | Michael Kadosh |
| collection | DOAJ |
| description | This study aims to achieve accurate three-dimensional (3D) localization of multiple objects in a complicated scene using passive imaging. It is challenging, as it requires accurate localization of the objects in all three dimensions given recorded 2D images. An integral imaging system captures the scene from multiple angles and is able to computationally produce blur-based depth information about the objects in the scene. We propose a method to detect and segment objects in a 3D space using integral-imaging data obtained by a video camera array. Using objects’ two-dimensional regions detected via deep learning, we employ local computational integral imaging in detected objects’ depth tubes to estimate the depth positions of the objects along the viewing axis. This method analyzes object-based blurring characteristics in the 3D environment efficiently. Our camera array produces an array of multiple-view videos of the scene, called elemental videos. Thus, the proposed 3D object detection applied to the video frames allows for 3D tracking of the objects with knowledge of their depth positions along the video. Results show successful 3D object detection with depth localization in a real-life scene based on passive integral imaging. Such outcomes have not been obtained in previous studies using integral imaging; mainly, the proposed method outperforms them in its ability to detect the depth locations of objects that are in close proximity to each other, regardless of the object size. This study may contribute when robust 3D object localization is desired with passive imaging, but it requires a camera or lens array imaging apparatus. |
| first_indexed | 2024-03-11T04:07:57Z |
| format | Article |
| id | doaj.art-701818a6daca4f36ab1339fa553faf2b |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T04:07:57Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-701818a6daca4f36ab1339fa553faf2b2023-11-17T23:40:57ZengMDPI AGSensors1424-82202023-04-01239419110.3390/s230941913D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real VideoMichael Kadosh0Yitzhak Yitzhaky1Department of Electro-Optics and Photonics Engineering, School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer Sheva 84105, IsraelDepartment of Electro-Optics and Photonics Engineering, School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer Sheva 84105, IsraelThis study aims to achieve accurate three-dimensional (3D) localization of multiple objects in a complicated scene using passive imaging. It is challenging, as it requires accurate localization of the objects in all three dimensions given recorded 2D images. An integral imaging system captures the scene from multiple angles and is able to computationally produce blur-based depth information about the objects in the scene. We propose a method to detect and segment objects in a 3D space using integral-imaging data obtained by a video camera array. Using objects’ two-dimensional regions detected via deep learning, we employ local computational integral imaging in detected objects’ depth tubes to estimate the depth positions of the objects along the viewing axis. This method analyzes object-based blurring characteristics in the 3D environment efficiently. Our camera array produces an array of multiple-view videos of the scene, called elemental videos. Thus, the proposed 3D object detection applied to the video frames allows for 3D tracking of the objects with knowledge of their depth positions along the video. Results show successful 3D object detection with depth localization in a real-life scene based on passive integral imaging. Such outcomes have not been obtained in previous studies using integral imaging; mainly, the proposed method outperforms them in its ability to detect the depth locations of objects that are in close proximity to each other, regardless of the object size. This study may contribute when robust 3D object localization is desired with passive imaging, but it requires a camera or lens array imaging apparatus.https://www.mdpi.com/1424-8220/23/9/4191computational integral imaging3D objects detectioninstance segmentation3D imagingdepth estimation |
| spellingShingle | Michael Kadosh Yitzhak Yitzhaky 3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video Sensors computational integral imaging 3D objects detection instance segmentation 3D imaging depth estimation |
| title | 3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video |
| title_full | 3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video |
| title_fullStr | 3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video |
| title_full_unstemmed | 3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video |
| title_short | 3D Object Detection via 2D Segmentation-Based Computational Integral Imaging Applied to a Real Video |
| title_sort | 3d object detection via 2d segmentation based computational integral imaging applied to a real video |
| topic | computational integral imaging 3D objects detection instance segmentation 3D imaging depth estimation |
| url | https://www.mdpi.com/1424-8220/23/9/4191 |
| work_keys_str_mv | AT michaelkadosh 3dobjectdetectionvia2dsegmentationbasedcomputationalintegralimagingappliedtoarealvideo AT yitzhakyitzhaky 3dobjectdetectionvia2dsegmentationbasedcomputationalintegralimagingappliedtoarealvideo |