Automatic Multiview Alignment of RGB-D Range Maps of Upper Limb Anatomy
Digital representations of anatomical parts are crucial for various biomedical applications. This paper presents an automatic alignment procedure for creating accurate 3D models of upper limb anatomy using a low-cost handheld 3D scanner. The goal is to overcome the challenges associated with forearm...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/18/7841 |
| _version_ | 1797577027374546944 |
|---|---|
| author | Luca Di Angelo Paolo Di Stefano Emanuele Guardiani Paolo Neri Alessandro Paoli Armando Viviano Razionale |
| author_facet | Luca Di Angelo Paolo Di Stefano Emanuele Guardiani Paolo Neri Alessandro Paoli Armando Viviano Razionale |
| author_sort | Luca Di Angelo |
| collection | DOAJ |
| description | Digital representations of anatomical parts are crucial for various biomedical applications. This paper presents an automatic alignment procedure for creating accurate 3D models of upper limb anatomy using a low-cost handheld 3D scanner. The goal is to overcome the challenges associated with forearm 3D scanning, such as needing multiple views, stability requirements, and optical undercuts. While bulky and expensive multi-camera systems have been used in previous research, this study explores the feasibility of using multiple consumer RGB-D sensors for scanning human anatomies. The proposed scanner comprises three Intel<sup>®</sup> RealSenseTM D415 depth cameras assembled on a lightweight circular jig, enabling simultaneous acquisition from three viewpoints. To achieve automatic alignment, the paper introduces a procedure that extracts common key points between acquisitions deriving from different scanner poses. Relevant hand key points are detected using a neural network, which works on the RGB images captured by the depth cameras. A set of forearm key points is meanwhile identified by processing the acquired data through a specifically developed algorithm that seeks the forearm’s skeleton line. The alignment process involves automatic, rough 3D alignment and fine registration using an iterative-closest-point (ICP) algorithm expressly developed for this application. The proposed method was tested on forearm scans and compared the results obtained by a manual coarse alignment followed by an ICP algorithm for fine registration using commercial software. Deviations below 5 mm, with a mean value of 1.5 mm, were found. The obtained results are critically discussed and compared with the available implementations of published methods. The results demonstrate significant improvements to the state of the art and the potential of the proposed approach to accelerate the acquisition process and automatically register point clouds from different scanner poses without the intervention of skilled operators. This study contributes to developing effective upper limb rehabilitation frameworks and personalized biomedical applications by addressing these critical challenges. |
| first_indexed | 2024-03-10T22:02:06Z |
| format | Article |
| id | doaj.art-308a11a592e8468abc70094e8e01487b |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T22:02:06Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-308a11a592e8468abc70094e8e01487b2023-11-19T12:54:57ZengMDPI AGSensors1424-82202023-09-012318784110.3390/s23187841Automatic Multiview Alignment of RGB-D Range Maps of Upper Limb AnatomyLuca Di Angelo0Paolo Di Stefano1Emanuele Guardiani2Paolo Neri3Alessandro Paoli4Armando Viviano Razionale5Department of Industrial and Information Engineering and Economics, University of L’Aquila, 67100 L’Aquila, ItalyDepartment of Industrial and Information Engineering and Economics, University of L’Aquila, 67100 L’Aquila, ItalyDepartment of Industrial and Information Engineering and Economics, University of L’Aquila, 67100 L’Aquila, ItalyDepartment of Civil and Industrial Engineering, University of Pisa, 56122 Pisa, ItalyDepartment of Civil and Industrial Engineering, University of Pisa, 56122 Pisa, ItalyDepartment of Civil and Industrial Engineering, University of Pisa, 56122 Pisa, ItalyDigital representations of anatomical parts are crucial for various biomedical applications. This paper presents an automatic alignment procedure for creating accurate 3D models of upper limb anatomy using a low-cost handheld 3D scanner. The goal is to overcome the challenges associated with forearm 3D scanning, such as needing multiple views, stability requirements, and optical undercuts. While bulky and expensive multi-camera systems have been used in previous research, this study explores the feasibility of using multiple consumer RGB-D sensors for scanning human anatomies. The proposed scanner comprises three Intel<sup>®</sup> RealSenseTM D415 depth cameras assembled on a lightweight circular jig, enabling simultaneous acquisition from three viewpoints. To achieve automatic alignment, the paper introduces a procedure that extracts common key points between acquisitions deriving from different scanner poses. Relevant hand key points are detected using a neural network, which works on the RGB images captured by the depth cameras. A set of forearm key points is meanwhile identified by processing the acquired data through a specifically developed algorithm that seeks the forearm’s skeleton line. The alignment process involves automatic, rough 3D alignment and fine registration using an iterative-closest-point (ICP) algorithm expressly developed for this application. The proposed method was tested on forearm scans and compared the results obtained by a manual coarse alignment followed by an ICP algorithm for fine registration using commercial software. Deviations below 5 mm, with a mean value of 1.5 mm, were found. The obtained results are critically discussed and compared with the available implementations of published methods. The results demonstrate significant improvements to the state of the art and the potential of the proposed approach to accelerate the acquisition process and automatically register point clouds from different scanner poses without the intervention of skilled operators. This study contributes to developing effective upper limb rehabilitation frameworks and personalized biomedical applications by addressing these critical challenges.https://www.mdpi.com/1424-8220/23/18/7841depth cameras3D optical scanningupper limb anatomyautomatic point cloud alignmentneural network |
| spellingShingle | Luca Di Angelo Paolo Di Stefano Emanuele Guardiani Paolo Neri Alessandro Paoli Armando Viviano Razionale Automatic Multiview Alignment of RGB-D Range Maps of Upper Limb Anatomy Sensors depth cameras 3D optical scanning upper limb anatomy automatic point cloud alignment neural network |
| title | Automatic Multiview Alignment of RGB-D Range Maps of Upper Limb Anatomy |
| title_full | Automatic Multiview Alignment of RGB-D Range Maps of Upper Limb Anatomy |
| title_fullStr | Automatic Multiview Alignment of RGB-D Range Maps of Upper Limb Anatomy |
| title_full_unstemmed | Automatic Multiview Alignment of RGB-D Range Maps of Upper Limb Anatomy |
| title_short | Automatic Multiview Alignment of RGB-D Range Maps of Upper Limb Anatomy |
| title_sort | automatic multiview alignment of rgb d range maps of upper limb anatomy |
| topic | depth cameras 3D optical scanning upper limb anatomy automatic point cloud alignment neural network |
| url | https://www.mdpi.com/1424-8220/23/18/7841 |
| work_keys_str_mv | AT lucadiangelo automaticmultiviewalignmentofrgbdrangemapsofupperlimbanatomy AT paolodistefano automaticmultiviewalignmentofrgbdrangemapsofupperlimbanatomy AT emanueleguardiani automaticmultiviewalignmentofrgbdrangemapsofupperlimbanatomy AT paoloneri automaticmultiviewalignmentofrgbdrangemapsofupperlimbanatomy AT alessandropaoli automaticmultiviewalignmentofrgbdrangemapsofupperlimbanatomy AT armandovivianorazionale automaticmultiviewalignmentofrgbdrangemapsofupperlimbanatomy |