Multi‐modal broad learning for material recognition
Abstract Material recognition plays an important role in the interaction between robots and the external environment. For example, household service robots need to replace humans in the home environment to complete housework, so they need to interact with daily necessities and obtain their material...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021-06-01
|
| Series: | Cognitive Computation and Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ccs2.12004 |
| _version_ | 1828142334241406976 |
|---|---|
| author | Zhaoxin Wang Huaping Liu Xinying Xu Fuchun Sun |
| author_facet | Zhaoxin Wang Huaping Liu Xinying Xu Fuchun Sun |
| author_sort | Zhaoxin Wang |
| collection | DOAJ |
| description | Abstract Material recognition plays an important role in the interaction between robots and the external environment. For example, household service robots need to replace humans in the home environment to complete housework, so they need to interact with daily necessities and obtain their material performance. Images provide rich visual information about objects; however, it is often difficult to apply when objects are not visually distinct. In addition, tactile signals can be used to capture multiple characteristics of objects, such as texture, roughness, softness, and friction, which provides another crucial way for perception. How to effectively integrate multi‐modal information is an urgent problem to be addressed. Therefore, a multi‐modal material recognition framework CFBRL‐KCCA for target recognition tasks is proposed in the paper. The preliminary features of each model are extracted by cascading broad learning, which is combined with the kernel canonical correlation learning, considering the differences among different models of heterogeneous data. Finally, the open dataset of household objects is evaluated. The results demonstrate that the proposed fusion algorithm provides an effective strategy for material recognition. |
| first_indexed | 2024-04-11T19:40:12Z |
| format | Article |
| id | doaj.art-00553a6cd9b7484c88fadd543afb48e9 |
| institution | Directory Open Access Journal |
| issn | 2517-7567 |
| language | English |
| last_indexed | 2024-04-11T19:40:12Z |
| publishDate | 2021-06-01 |
| publisher | Wiley |
| record_format | Article |
| series | Cognitive Computation and Systems |
| spelling | doaj.art-00553a6cd9b7484c88fadd543afb48e92022-12-22T04:06:44ZengWileyCognitive Computation and Systems2517-75672021-06-013212313010.1049/ccs2.12004Multi‐modal broad learning for material recognitionZhaoxin Wang0Huaping Liu1Xinying Xu2Fuchun Sun3Department of Computer Science and Technology Tsinghua University Beijing ChinaDepartment of Computer Science and Technology Tsinghua University Beijing ChinaDepartment of Computer Science and Technology Tsinghua University Beijing ChinaDepartment of Computer Science and Technology Tsinghua University Beijing ChinaAbstract Material recognition plays an important role in the interaction between robots and the external environment. For example, household service robots need to replace humans in the home environment to complete housework, so they need to interact with daily necessities and obtain their material performance. Images provide rich visual information about objects; however, it is often difficult to apply when objects are not visually distinct. In addition, tactile signals can be used to capture multiple characteristics of objects, such as texture, roughness, softness, and friction, which provides another crucial way for perception. How to effectively integrate multi‐modal information is an urgent problem to be addressed. Therefore, a multi‐modal material recognition framework CFBRL‐KCCA for target recognition tasks is proposed in the paper. The preliminary features of each model are extracted by cascading broad learning, which is combined with the kernel canonical correlation learning, considering the differences among different models of heterogeneous data. Finally, the open dataset of household objects is evaluated. The results demonstrate that the proposed fusion algorithm provides an effective strategy for material recognition.https://doi.org/10.1049/ccs2.12004correlation methodsfeature extractionimage fusionimage recognitionimage representationlearning (artificial intelligence) |
| spellingShingle | Zhaoxin Wang Huaping Liu Xinying Xu Fuchun Sun Multi‐modal broad learning for material recognition Cognitive Computation and Systems correlation methods feature extraction image fusion image recognition image representation learning (artificial intelligence) |
| title | Multi‐modal broad learning for material recognition |
| title_full | Multi‐modal broad learning for material recognition |
| title_fullStr | Multi‐modal broad learning for material recognition |
| title_full_unstemmed | Multi‐modal broad learning for material recognition |
| title_short | Multi‐modal broad learning for material recognition |
| title_sort | multi modal broad learning for material recognition |
| topic | correlation methods feature extraction image fusion image recognition image representation learning (artificial intelligence) |
| url | https://doi.org/10.1049/ccs2.12004 |
| work_keys_str_mv | AT zhaoxinwang multimodalbroadlearningformaterialrecognition AT huapingliu multimodalbroadlearningformaterialrecognition AT xinyingxu multimodalbroadlearningformaterialrecognition AT fuchunsun multimodalbroadlearningformaterialrecognition |