An Interactive Image Segmentation Method Based on Multi-Level Semantic Fusion
Understanding and analyzing 2D/3D sensor data is crucial for a wide range of machine learning-based applications, including object detection, scene segmentation, and salient object detection. In this context, interactive object segmentation is a vital task in image editing and medical diagnosis, inv...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/14/6394 |
| _version_ | 1797587501435584512 |
|---|---|
| author | Ruirui Zou Qinghui Wang Falin Wen Yang Chen Jiale Liu Shaoyi Du Chengzhi Yuan |
| author_facet | Ruirui Zou Qinghui Wang Falin Wen Yang Chen Jiale Liu Shaoyi Du Chengzhi Yuan |
| author_sort | Ruirui Zou |
| collection | DOAJ |
| description | Understanding and analyzing 2D/3D sensor data is crucial for a wide range of machine learning-based applications, including object detection, scene segmentation, and salient object detection. In this context, interactive object segmentation is a vital task in image editing and medical diagnosis, involving the accurate separation of the target object from its background based on user annotation information. However, existing interactive object segmentation methods struggle to effectively leverage such information to guide object-segmentation models. To address these challenges, this paper proposes an interactive image-segmentation technique for static images based on multi-level semantic fusion. Our method utilizes user-guidance information both inside and outside the target object to segment it from the static image, making it applicable to both 2D and 3D sensor data. The proposed method introduces a cross-stage feature aggregation module, enabling the effective propagation of multi-scale features from previous stages to the current stage. This mechanism prevents the loss of semantic information caused by multiple upsampling and downsampling of the network, allowing the current stage to make better use of semantic information from the previous stage. Additionally, we incorporate a feature channel attention mechanism to address the issue of rough network segmentation edges. This mechanism captures richer feature details from the feature channel level, leading to finer segmentation edges. In the experimental evaluation conducted on the PASCAL Visual Object Classes (VOC) 2012 dataset, our proposed interactive image segmentation method based on multi-level semantic fusion demonstrates an intersection over union (IOU) accuracy approximately 2.1% higher than the currently popular interactive image segmentation method in static images. The comparative analysis highlights the improved performance and effectiveness of our method. Furthermore, our method exhibits potential applications in various fields, including medical imaging and robotics. Its compatibility with other machine learning methods for visual semantic analysis allows for integration into existing workflows. These aspects emphasize the significance of our contributions in advancing interactive image-segmentation techniques and their practical utility in real-world applications. |
| first_indexed | 2024-03-11T00:39:50Z |
| format | Article |
| id | doaj.art-349d5a9096144391980a16c125c6b2fd |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T00:39:50Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-349d5a9096144391980a16c125c6b2fd2023-11-18T21:17:07ZengMDPI AGSensors1424-82202023-07-012314639410.3390/s23146394An Interactive Image Segmentation Method Based on Multi-Level Semantic FusionRuirui Zou0Qinghui Wang1Falin Wen2Yang Chen3Jiale Liu4Shaoyi Du5Chengzhi Yuan6School of Physics and Mechanical and Electrical Engineering, Longyan University, Longyan 364012, ChinaSchool of Physics and Mechanical and Electrical Engineering, Longyan University, Longyan 364012, ChinaSchool of Physics and Mechanical and Electrical Engineering, Longyan University, Longyan 364012, ChinaSchool of Physics and Mechanical and Electrical Engineering, Longyan University, Longyan 364012, ChinaSchool of Software Engineering, Xi’an Jiaotong University, Xi’an 710049, ChinaInstitute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an 710049, ChinaDepartment of Mechanical, Industrial and Systems Engineering, University of Rhode Island, Kingston, RI 02881, USAUnderstanding and analyzing 2D/3D sensor data is crucial for a wide range of machine learning-based applications, including object detection, scene segmentation, and salient object detection. In this context, interactive object segmentation is a vital task in image editing and medical diagnosis, involving the accurate separation of the target object from its background based on user annotation information. However, existing interactive object segmentation methods struggle to effectively leverage such information to guide object-segmentation models. To address these challenges, this paper proposes an interactive image-segmentation technique for static images based on multi-level semantic fusion. Our method utilizes user-guidance information both inside and outside the target object to segment it from the static image, making it applicable to both 2D and 3D sensor data. The proposed method introduces a cross-stage feature aggregation module, enabling the effective propagation of multi-scale features from previous stages to the current stage. This mechanism prevents the loss of semantic information caused by multiple upsampling and downsampling of the network, allowing the current stage to make better use of semantic information from the previous stage. Additionally, we incorporate a feature channel attention mechanism to address the issue of rough network segmentation edges. This mechanism captures richer feature details from the feature channel level, leading to finer segmentation edges. In the experimental evaluation conducted on the PASCAL Visual Object Classes (VOC) 2012 dataset, our proposed interactive image segmentation method based on multi-level semantic fusion demonstrates an intersection over union (IOU) accuracy approximately 2.1% higher than the currently popular interactive image segmentation method in static images. The comparative analysis highlights the improved performance and effectiveness of our method. Furthermore, our method exhibits potential applications in various fields, including medical imaging and robotics. Its compatibility with other machine learning methods for visual semantic analysis allows for integration into existing workflows. These aspects emphasize the significance of our contributions in advancing interactive image-segmentation techniques and their practical utility in real-world applications.https://www.mdpi.com/1424-8220/23/14/6394interactive image segmentationattentioncross-stage feature aggregationmodel complexity |
| spellingShingle | Ruirui Zou Qinghui Wang Falin Wen Yang Chen Jiale Liu Shaoyi Du Chengzhi Yuan An Interactive Image Segmentation Method Based on Multi-Level Semantic Fusion Sensors interactive image segmentation attention cross-stage feature aggregation model complexity |
| title | An Interactive Image Segmentation Method Based on Multi-Level Semantic Fusion |
| title_full | An Interactive Image Segmentation Method Based on Multi-Level Semantic Fusion |
| title_fullStr | An Interactive Image Segmentation Method Based on Multi-Level Semantic Fusion |
| title_full_unstemmed | An Interactive Image Segmentation Method Based on Multi-Level Semantic Fusion |
| title_short | An Interactive Image Segmentation Method Based on Multi-Level Semantic Fusion |
| title_sort | interactive image segmentation method based on multi level semantic fusion |
| topic | interactive image segmentation attention cross-stage feature aggregation model complexity |
| url | https://www.mdpi.com/1424-8220/23/14/6394 |
| work_keys_str_mv | AT ruiruizou aninteractiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT qinghuiwang aninteractiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT falinwen aninteractiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT yangchen aninteractiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT jialeliu aninteractiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT shaoyidu aninteractiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT chengzhiyuan aninteractiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT ruiruizou interactiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT qinghuiwang interactiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT falinwen interactiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT yangchen interactiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT jialeliu interactiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT shaoyidu interactiveimagesegmentationmethodbasedonmultilevelsemanticfusion AT chengzhiyuan interactiveimagesegmentationmethodbasedonmultilevelsemanticfusion |