3D pose estimation and segmentation using specular cues
We present a system for fast model-based segmentation and 3D pose estimation of specular objects using appearance based specular features. We use observed (a) specular reflection and (b) specular flow as cues, which are matched against similar cues generated from a CAD model of the object in various...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute of Electrical and Electronics Engineers
2010
|
| Online Access: | http://hdl.handle.net/1721.1/54707 https://orcid.org/0000-0002-3254-3224 |
| _version_ | 1826209902830288896 |
|---|---|
| author | Raskar, Ramesh Agrawal, Amit Chang, Ju Yong |
| author2 | Program in Media Arts and Sciences (Massachusetts Institute of Technology) |
| author_facet | Program in Media Arts and Sciences (Massachusetts Institute of Technology) Raskar, Ramesh Agrawal, Amit Chang, Ju Yong |
| author_sort | Raskar, Ramesh |
| collection | MIT |
| description | We present a system for fast model-based segmentation and 3D pose estimation of specular objects using appearance based specular features. We use observed (a) specular reflection and (b) specular flow as cues, which are matched against similar cues generated from a CAD model of the object in various poses. We avoid estimating 3D geometry or depths, which is difficult and unreliable for specular scenes. In the first method, the environment map of the scene is utilized to generate a database containing synthesized specular reflections of the object for densely sampled 3D poses. This database is compared with captured images of the scene at run time to locate and estimate the 3D pose of the object. In the second method, specular flows are generated for dense 3D poses as illumination invariant features and are matched to the specular flow of the scene. We incorporate several practical heuristics such as use of saturated/highlight pixels for fast matching and normal selection to minimize the effects of inter-reflections and cluttered backgrounds. Despite its simplicity, our approach is effective in scenes with multiple specular objects, partial occlusions, inter-reflections, cluttered backgrounds and changes in ambient illumination. Experimental results demonstrate the effectiveness of our method for various synthetic and real objects. |
| first_indexed | 2024-09-23T14:34:40Z |
| format | Article |
| id | mit-1721.1/54707 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T14:34:40Z |
| publishDate | 2010 |
| publisher | Institute of Electrical and Electronics Engineers |
| record_format | dspace |
| spelling | mit-1721.1/547072022-09-29T09:53:47Z 3D pose estimation and segmentation using specular cues Raskar, Ramesh Agrawal, Amit Chang, Ju Yong Program in Media Arts and Sciences (Massachusetts Institute of Technology) Raskar, Ramesh Raskar, Ramesh We present a system for fast model-based segmentation and 3D pose estimation of specular objects using appearance based specular features. We use observed (a) specular reflection and (b) specular flow as cues, which are matched against similar cues generated from a CAD model of the object in various poses. We avoid estimating 3D geometry or depths, which is difficult and unreliable for specular scenes. In the first method, the environment map of the scene is utilized to generate a database containing synthesized specular reflections of the object for densely sampled 3D poses. This database is compared with captured images of the scene at run time to locate and estimate the 3D pose of the object. In the second method, specular flows are generated for dense 3D poses as illumination invariant features and are matched to the specular flow of the scene. We incorporate several practical heuristics such as use of saturated/highlight pixels for fast matching and normal selection to minimize the effects of inter-reflections and cluttered backgrounds. Despite its simplicity, our approach is effective in scenes with multiple specular objects, partial occlusions, inter-reflections, cluttered backgrounds and changes in ambient illumination. Experimental results demonstrate the effectiveness of our method for various synthetic and real objects. 2010-05-05T13:41:36Z 2010-05-05T13:41:36Z 2009-08 Article http://purl.org/eprint/type/ConferencePaper 978-1-4244-3992-8 1063-6919 http://hdl.handle.net/1721.1/54707 Ju Yong Chang, R. Raskar, and A. Agrawal. “3D pose estimation and segmentation using specular cues.” Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. 2009. 1706-1713. © 2009 IEEE https://orcid.org/0000-0002-3254-3224 en_US http://dx.doi.org/10.1109/CVPRW.2009.5206820 IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf Institute of Electrical and Electronics Engineers IEEE |
| spellingShingle | Raskar, Ramesh Agrawal, Amit Chang, Ju Yong 3D pose estimation and segmentation using specular cues |
| title | 3D pose estimation and segmentation using specular cues |
| title_full | 3D pose estimation and segmentation using specular cues |
| title_fullStr | 3D pose estimation and segmentation using specular cues |
| title_full_unstemmed | 3D pose estimation and segmentation using specular cues |
| title_short | 3D pose estimation and segmentation using specular cues |
| title_sort | 3d pose estimation and segmentation using specular cues |
| url | http://hdl.handle.net/1721.1/54707 https://orcid.org/0000-0002-3254-3224 |
| work_keys_str_mv | AT raskarramesh 3dposeestimationandsegmentationusingspecularcues AT agrawalamit 3dposeestimationandsegmentationusingspecularcues AT changjuyong 3dposeestimationandsegmentationusingspecularcues |