Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion
The comprehension of spoken language is a crucial aspect of dialogue systems, encompassing two fundamental tasks: intent classification and slot filling. Currently, the joint modeling approach for these two tasks has emerged as the dominant method in spoken language understanding modeling. However,...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/5/2848 |
| _version_ | 1827752069762646016 |
|---|---|
| author | Yan Chen Zhenghang Luo |
| author_facet | Yan Chen Zhenghang Luo |
| author_sort | Yan Chen |
| collection | DOAJ |
| description | The comprehension of spoken language is a crucial aspect of dialogue systems, encompassing two fundamental tasks: intent classification and slot filling. Currently, the joint modeling approach for these two tasks has emerged as the dominant method in spoken language understanding modeling. However, the existing joint models have limitations in terms of their relevancy and utilization of contextual semantic features between the multiple tasks. To address these limitations, a joint model based on BERT and semantic fusion (JMBSF) is proposed. The model employs pre-trained BERT to extract semantic features and utilizes semantic fusion to associate and integrate this information. The results of experiments on two benchmark datasets, ATIS and Snips, in spoken language comprehension demonstrate that the proposed JMBSF model attains 98.80% and 99.71% intent classification accuracy, 98.25% and 97.24% slot-filling F1-score, and 93.40% and 93.57% sentence accuracy, respectively. These results reveal a significant improvement compared to other joint models. Furthermore, comprehensive ablation studies affirm the effectiveness of each component in the design of JMBSF. |
| first_indexed | 2024-03-11T07:09:36Z |
| format | Article |
| id | doaj.art-7e770ca651f84ebab860766df462542e |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T07:09:36Z |
| publishDate | 2023-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-7e770ca651f84ebab860766df462542e2023-11-17T08:40:33ZengMDPI AGSensors1424-82202023-03-01235284810.3390/s23052848Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature FusionYan Chen0Zhenghang Luo1School of Computer and Electronic Information, Guangxi University, Nanning 530004, ChinaSchool of Computer and Electronic Information, Guangxi University, Nanning 530004, ChinaThe comprehension of spoken language is a crucial aspect of dialogue systems, encompassing two fundamental tasks: intent classification and slot filling. Currently, the joint modeling approach for these two tasks has emerged as the dominant method in spoken language understanding modeling. However, the existing joint models have limitations in terms of their relevancy and utilization of contextual semantic features between the multiple tasks. To address these limitations, a joint model based on BERT and semantic fusion (JMBSF) is proposed. The model employs pre-trained BERT to extract semantic features and utilizes semantic fusion to associate and integrate this information. The results of experiments on two benchmark datasets, ATIS and Snips, in spoken language comprehension demonstrate that the proposed JMBSF model attains 98.80% and 99.71% intent classification accuracy, 98.25% and 97.24% slot-filling F1-score, and 93.40% and 93.57% sentence accuracy, respectively. These results reveal a significant improvement compared to other joint models. Furthermore, comprehensive ablation studies affirm the effectiveness of each component in the design of JMBSF.https://www.mdpi.com/1424-8220/23/5/2848spoken language understandingintent classificationslot fillingjoint model |
| spellingShingle | Yan Chen Zhenghang Luo Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion Sensors spoken language understanding intent classification slot filling joint model |
| title | Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion |
| title_full | Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion |
| title_fullStr | Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion |
| title_full_unstemmed | Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion |
| title_short | Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion |
| title_sort | pre trained joint model for intent classification and slot filling with semantic feature fusion |
| topic | spoken language understanding intent classification slot filling joint model |
| url | https://www.mdpi.com/1424-8220/23/5/2848 |
| work_keys_str_mv | AT yanchen pretrainedjointmodelforintentclassificationandslotfillingwithsemanticfeaturefusion AT zhenghangluo pretrainedjointmodelforintentclassificationandslotfillingwithsemanticfeaturefusion |