Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection
Cluttered environments with partial object occlusions pose significant challenges to robot manipulation. In settings composed of one dominant object type and various undesirable contaminants, occlusions make it difficult to both recognize and isolate undesirable objects. Spatial features alone are n...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-10-01
|
| Series: | Frontiers in Robotics and AI |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frobt.2022.982131/full |
| _version_ | 1811182445645529088 |
|---|---|
| author | Nathaniel Hanson Nathaniel Hanson Gary Lvov Gary Lvov Taşkın Padir Taşkın Padir |
| author_facet | Nathaniel Hanson Nathaniel Hanson Gary Lvov Gary Lvov Taşkın Padir Taşkın Padir |
| author_sort | Nathaniel Hanson |
| collection | DOAJ |
| description | Cluttered environments with partial object occlusions pose significant challenges to robot manipulation. In settings composed of one dominant object type and various undesirable contaminants, occlusions make it difficult to both recognize and isolate undesirable objects. Spatial features alone are not always sufficiently distinct to reliably identify anomalies under multiple layers of clutter, with only a fractional part of the object exposed. We create a multi-modal data representation of cluttered object scenes pairing depth data with a registered hyperspectral data cube. Hyperspectral imaging provides pixel-wise Visible Near-Infrared (VNIR) reflectance spectral curves which are invariant in similar material types. Spectral reflectance data is grounded in the chemical-physical properties of an object, making spectral curves an excellent modality to differentiate inter-class material types. Our approach proposes a new automated method to perform hyperspectral anomaly detection in cluttered workspaces with the goal of improving robot manipulation. We first assume the dominance of a single material class, and coarsely identify the dominant, non-anomalous class. Next these labels are used to train an unsupervised autoencoder to identify anomalous pixels through reconstruction error. To tie our anomaly detection to robot actions, we then apply a set of heuristically-evaluated motion primitives to perturb and further expose local areas containing anomalies. The utility of this approach is demonstrated in numerous cluttered environments including organic and inorganic materials. In each of our four constructed scenarios, our proposed anomaly detection method is able to consistently increase the exposed surface area of anomalies. Our work advances robot perception for cluttered environments by incorporating multi-modal anomaly detection aided by hyperspectral sensing into detecting fractional object presence without need for laboriously curated labels. |
| first_indexed | 2024-04-11T09:32:33Z |
| format | Article |
| id | doaj.art-aba5d52145d1497aac240eab153234f2 |
| institution | Directory Open Access Journal |
| issn | 2296-9144 |
| language | English |
| last_indexed | 2024-04-11T09:32:33Z |
| publishDate | 2022-10-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Robotics and AI |
| spelling | doaj.art-aba5d52145d1497aac240eab153234f22022-12-22T04:31:49ZengFrontiers Media S.A.Frontiers in Robotics and AI2296-91442022-10-01910.3389/frobt.2022.982131982131Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detectionNathaniel Hanson0Nathaniel Hanson1Gary Lvov2Gary Lvov3Taşkın Padir4Taşkın Padir5Institute for Experiential Robotics, Northeastern University, Boston, MA, United StatesDepartment of Electrical and Computer Engineering, Northeastern University, Boston, MA, United StatesInstitute for Experiential Robotics, Northeastern University, Boston, MA, United StatesDepartment of Electrical and Computer Engineering, Northeastern University, Boston, MA, United StatesInstitute for Experiential Robotics, Northeastern University, Boston, MA, United StatesDepartment of Electrical and Computer Engineering, Northeastern University, Boston, MA, United StatesCluttered environments with partial object occlusions pose significant challenges to robot manipulation. In settings composed of one dominant object type and various undesirable contaminants, occlusions make it difficult to both recognize and isolate undesirable objects. Spatial features alone are not always sufficiently distinct to reliably identify anomalies under multiple layers of clutter, with only a fractional part of the object exposed. We create a multi-modal data representation of cluttered object scenes pairing depth data with a registered hyperspectral data cube. Hyperspectral imaging provides pixel-wise Visible Near-Infrared (VNIR) reflectance spectral curves which are invariant in similar material types. Spectral reflectance data is grounded in the chemical-physical properties of an object, making spectral curves an excellent modality to differentiate inter-class material types. Our approach proposes a new automated method to perform hyperspectral anomaly detection in cluttered workspaces with the goal of improving robot manipulation. We first assume the dominance of a single material class, and coarsely identify the dominant, non-anomalous class. Next these labels are used to train an unsupervised autoencoder to identify anomalous pixels through reconstruction error. To tie our anomaly detection to robot actions, we then apply a set of heuristically-evaluated motion primitives to perturb and further expose local areas containing anomalies. The utility of this approach is demonstrated in numerous cluttered environments including organic and inorganic materials. In each of our four constructed scenarios, our proposed anomaly detection method is able to consistently increase the exposed surface area of anomalies. Our work advances robot perception for cluttered environments by incorporating multi-modal anomaly detection aided by hyperspectral sensing into detecting fractional object presence without need for laboriously curated labels.https://www.frontiersin.org/articles/10.3389/frobt.2022.982131/fullcluttered environmenthyperspectral imagingmulti-modal scene segmentationsystem architectureautomated machine learning |
| spellingShingle | Nathaniel Hanson Nathaniel Hanson Gary Lvov Gary Lvov Taşkın Padir Taşkın Padir Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection Frontiers in Robotics and AI cluttered environment hyperspectral imaging multi-modal scene segmentation system architecture automated machine learning |
| title | Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection |
| title_full | Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection |
| title_fullStr | Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection |
| title_full_unstemmed | Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection |
| title_short | Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection |
| title_sort | occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection |
| topic | cluttered environment hyperspectral imaging multi-modal scene segmentation system architecture automated machine learning |
| url | https://www.frontiersin.org/articles/10.3389/frobt.2022.982131/full |
| work_keys_str_mv | AT nathanielhanson occludedobjectdetectionandexposureinclutteredenvironmentswithautomatedhyperspectralanomalydetection AT nathanielhanson occludedobjectdetectionandexposureinclutteredenvironmentswithautomatedhyperspectralanomalydetection AT garylvov occludedobjectdetectionandexposureinclutteredenvironmentswithautomatedhyperspectralanomalydetection AT garylvov occludedobjectdetectionandexposureinclutteredenvironmentswithautomatedhyperspectralanomalydetection AT taskınpadir occludedobjectdetectionandexposureinclutteredenvironmentswithautomatedhyperspectralanomalydetection AT taskınpadir occludedobjectdetectionandexposureinclutteredenvironmentswithautomatedhyperspectralanomalydetection |