3D convolutional neural networks for efficient and robust hand pose estimation from single depth images
We propose a simple, yet effective approach for real-time hand pose estimation from single depth images using three-dimensional Convolutional Neural Networks (3D CNNs). Image based features extracted by 2D CNNs are not directly suitable for 3D hand pose estimation due to the lack of 3D spatial infor...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Language: | English |
| Published: |
2019
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/82836 http://hdl.handle.net/10220/50409 |
| _version_ | 1826116589430243328 |
|---|---|
| author | Ge, Liuhao Liang, Hui Yuan, Junsong Thalmann, Daniel |
| author2 | Interdisciplinary Graduate School (IGS) |
| author_facet | Interdisciplinary Graduate School (IGS) Ge, Liuhao Liang, Hui Yuan, Junsong Thalmann, Daniel |
| author_sort | Ge, Liuhao |
| collection | NTU |
| description | We propose a simple, yet effective approach for real-time hand pose estimation from single depth images using three-dimensional Convolutional Neural Networks (3D CNNs). Image based features extracted by 2D CNNs are not directly suitable for 3D hand pose estimation due to the lack of 3D spatial information. Our proposed 3D CNN taking a 3D volumetric representation of the hand depth image as input can capture the 3D spatial structure of the input and accurately regress full 3D hand pose in a single pass. In order to make the 3D CNN robust to variations in hand sizes and global orientations, we perform 3D data augmentation on the training data. Experiments show that our proposed 3D CNN based approach outperforms state-of-the-art methods on two challenging hand pose datasets, and is very efficient as our implementation runs at over 215 fps on a standard computer with a single GPU. |
| first_indexed | 2024-10-01T04:14:12Z |
| format | Conference Paper |
| id | ntu-10356/82836 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T04:14:12Z |
| publishDate | 2019 |
| record_format | dspace |
| spelling | ntu-10356/828362020-11-01T04:43:18Z 3D convolutional neural networks for efficient and robust hand pose estimation from single depth images Ge, Liuhao Liang, Hui Yuan, Junsong Thalmann, Daniel Interdisciplinary Graduate School (IGS) 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Three-dimensional Displays Pose Estimation Engineering::Electrical and electronic engineering We propose a simple, yet effective approach for real-time hand pose estimation from single depth images using three-dimensional Convolutional Neural Networks (3D CNNs). Image based features extracted by 2D CNNs are not directly suitable for 3D hand pose estimation due to the lack of 3D spatial information. Our proposed 3D CNN taking a 3D volumetric representation of the hand depth image as input can capture the 3D spatial structure of the input and accurately regress full 3D hand pose in a single pass. In order to make the 3D CNN robust to variations in hand sizes and global orientations, we perform 3D data augmentation on the training data. Experiments show that our proposed 3D CNN based approach outperforms state-of-the-art methods on two challenging hand pose datasets, and is very efficient as our implementation runs at over 215 fps on a standard computer with a single GPU. NRF (Natl Research Foundation, S’pore) ASTAR (Agency for Sci., Tech. and Research, S’pore) MOE (Min. of Education, S’pore) Accepted version 2019-11-14T01:44:46Z 2019-12-06T15:06:35Z 2019-11-14T01:44:46Z 2019-12-06T15:06:35Z 2017 Conference Paper Ge, L., Liang, H., Yuan, J., & Thalmann, D. (2017). 3D convolutional neural networks for efficient and robust hand pose estimation from single depth images. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/CVPR.2017.602 https://hdl.handle.net/10356/82836 http://hdl.handle.net/10220/50409 10.1109/CVPR.2017.602 en © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: https://doi.org/10.1109/CVPR.2017.602 10 p. application/pdf |
| spellingShingle | Three-dimensional Displays Pose Estimation Engineering::Electrical and electronic engineering Ge, Liuhao Liang, Hui Yuan, Junsong Thalmann, Daniel 3D convolutional neural networks for efficient and robust hand pose estimation from single depth images |
| title | 3D convolutional neural networks for efficient and robust hand pose estimation from single depth images |
| title_full | 3D convolutional neural networks for efficient and robust hand pose estimation from single depth images |
| title_fullStr | 3D convolutional neural networks for efficient and robust hand pose estimation from single depth images |
| title_full_unstemmed | 3D convolutional neural networks for efficient and robust hand pose estimation from single depth images |
| title_short | 3D convolutional neural networks for efficient and robust hand pose estimation from single depth images |
| title_sort | 3d convolutional neural networks for efficient and robust hand pose estimation from single depth images |
| topic | Three-dimensional Displays Pose Estimation Engineering::Electrical and electronic engineering |
| url | https://hdl.handle.net/10356/82836 http://hdl.handle.net/10220/50409 |
| work_keys_str_mv | AT geliuhao 3dconvolutionalneuralnetworksforefficientandrobusthandposeestimationfromsingledepthimages AT lianghui 3dconvolutionalneuralnetworksforefficientandrobusthandposeestimationfromsingledepthimages AT yuanjunsong 3dconvolutionalneuralnetworksforefficientandrobusthandposeestimationfromsingledepthimages AT thalmanndaniel 3dconvolutionalneuralnetworksforefficientandrobusthandposeestimationfromsingledepthimages |