Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot
In aspect of the natural language processing field, previous studies have generally analyzed sound signals and provided related responses. However, in various conversation scenarios, image information is still vital. Without the image information, misunderstanding may occur, and lead to wrong respon...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/15/5132 |
| _version_ | 1797525108362838016 |
|---|---|
| author | Ping-Huan Kuo Ssu-Ting Lin Jun Hu Chiou-Jye Huang |
| author_facet | Ping-Huan Kuo Ssu-Ting Lin Jun Hu Chiou-Jye Huang |
| author_sort | Ping-Huan Kuo |
| collection | DOAJ |
| description | In aspect of the natural language processing field, previous studies have generally analyzed sound signals and provided related responses. However, in various conversation scenarios, image information is still vital. Without the image information, misunderstanding may occur, and lead to wrong responses. In order to address this problem, this study proposes a recurrent neural network (RNNs) based multi-sensor context-aware chatbot technology. The proposed chatbot model incorporates image information with sound signals and gives appropriate responses to the user. In order to improve the performance of the proposed model, the long short-term memory (LSTM) structure is replaced by gated recurrent unit (GRU). Moreover, a VGG16 model is also chosen for a feature extractor for the image information. The experimental results demonstrate that the integrative technology of sound and image information, which are obtained by the image sensor and sound sensor in a companion robot, is helpful for the chatbot model proposed in this study. The feasibility of the proposed technology was also confirmed in the experiment. |
| first_indexed | 2024-03-10T09:09:01Z |
| format | Article |
| id | doaj.art-42d6fdad22c74d42a97203cd8334b12f |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T09:09:01Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-42d6fdad22c74d42a97203cd8334b12f2023-11-22T06:10:56ZengMDPI AGSensors1424-82202021-07-012115513210.3390/s21155132Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion RobotPing-Huan Kuo0Ssu-Ting Lin1Jun Hu2Chiou-Jye Huang3Department of Mechanical Engineering, National Chung Cheng University, Chiayi 62102, TaiwanDepartment of Intelligent Robotics, National Pingtung University, Pingtung 90004, TaiwanDepartment of Intelligent Robotics, National Pingtung University, Pingtung 90004, TaiwanDepartment of Data Science and Big Data Analytics, Providence University, Taichung 43301, TaiwanIn aspect of the natural language processing field, previous studies have generally analyzed sound signals and provided related responses. However, in various conversation scenarios, image information is still vital. Without the image information, misunderstanding may occur, and lead to wrong responses. In order to address this problem, this study proposes a recurrent neural network (RNNs) based multi-sensor context-aware chatbot technology. The proposed chatbot model incorporates image information with sound signals and gives appropriate responses to the user. In order to improve the performance of the proposed model, the long short-term memory (LSTM) structure is replaced by gated recurrent unit (GRU). Moreover, a VGG16 model is also chosen for a feature extractor for the image information. The experimental results demonstrate that the integrative technology of sound and image information, which are obtained by the image sensor and sound sensor in a companion robot, is helpful for the chatbot model proposed in this study. The feasibility of the proposed technology was also confirmed in the experiment.https://www.mdpi.com/1424-8220/21/15/5132multi-sensor fusionnatural languageprocessingcontext-aware computingchatbotcompanion robot |
| spellingShingle | Ping-Huan Kuo Ssu-Ting Lin Jun Hu Chiou-Jye Huang Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot Sensors multi-sensor fusion natural language processing context-aware computing chatbot companion robot |
| title | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
| title_full | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
| title_fullStr | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
| title_full_unstemmed | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
| title_short | Multi-Sensor Context-Aware Based Chatbot Model: An Application of Humanoid Companion Robot |
| title_sort | multi sensor context aware based chatbot model an application of humanoid companion robot |
| topic | multi-sensor fusion natural language processing context-aware computing chatbot companion robot |
| url | https://www.mdpi.com/1424-8220/21/15/5132 |
| work_keys_str_mv | AT pinghuankuo multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot AT ssutinglin multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot AT junhu multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot AT chioujyehuang multisensorcontextawarebasedchatbotmodelanapplicationofhumanoidcompanionrobot |