FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy Videos
Wireless capsule endoscopy (WCE) is a recently developed tool that allows for the painless and non-invasive examination of the entire gastrointestinal (GI) tract. The microcamera captures a large number of redundant frames for each WCE examination such that a video summarization technique is needed...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10032547/ |
| _version_ | 1811170556531179520 |
|---|---|
| author | Weijie Xie Zefeiyun Chen Qingyuan Li Qingfei Ma Yusi Wang Tianbao Liu Yuxin Fang Zhanpeng Zhao Side Liu Wei Yang |
| author_facet | Weijie Xie Zefeiyun Chen Qingyuan Li Qingfei Ma Yusi Wang Tianbao Liu Yuxin Fang Zhanpeng Zhao Side Liu Wei Yang |
| author_sort | Weijie Xie |
| collection | DOAJ |
| description | Wireless capsule endoscopy (WCE) is a recently developed tool that allows for the painless and non-invasive examination of the entire gastrointestinal (GI) tract. The microcamera captures a large number of redundant frames for each WCE examination such that a video summarization technique is needed to assist in diagnosis. However, prevalent methods of summarizing WCE videos focus only on the representativeness of the frames owing to a lack of high-level information on their importance. This paper develops a Frame Importance-Assisted Sparse Subset Selection model, called FIAS3, to integrate the high-level frame importance from networks into a sparse subset selection model. The FIAS3 is optimized under three constraints: 1) a frame importance matrix to help pay more attention to important frames, 2) a sparsity constraint to make video summaries more compact, and 3) a similarity-inhibiting constraint to reduce redundancy. The results of experiments on a public dataset demonstrated that our FIAS3 outperforms other methods of summarizing WCE videos. Specifically, its coverage and video reconstruction error were 92% and 0.143, respectively, at a 90% compression ratio, recording respective at least 16.9% and 0.031 improvements over other methods. The results of generalization experiments showed that FIAS3 also achieves competitive results on private datasets. |
| first_indexed | 2024-04-10T16:59:38Z |
| format | Article |
| id | doaj.art-71823ad81b564464b7fe88aec4e79260 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T16:59:38Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-71823ad81b564464b7fe88aec4e792602023-02-07T00:00:55ZengIEEEIEEE Access2169-35362023-01-0111108501086310.1109/ACCESS.2023.324099910032547FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy VideosWeijie Xie0https://orcid.org/0000-0002-4024-3619Zefeiyun Chen1Qingyuan Li2Qingfei Ma3Yusi Wang4Tianbao Liu5Yuxin Fang6Zhanpeng Zhao7Side Liu8Wei Yang9https://orcid.org/0000-0002-2161-3231School of Biomedical Engineering, Southern Medical University, Guangzhou, ChinaSchool of Biomedical Engineering, Southern Medical University, Guangzhou, ChinaDepartment of Gastroenterology, Nanfang Hospital, Southern Medical University, Guangzhou, ChinaGuangzhou SiDe MedTech Company Ltd., Guangzhou, ChinaDepartment of Gastroenterology, Nanfang Hospital, Southern Medical University, Guangzhou, ChinaSchool of Biomedical Engineering, Southern Medical University, Guangzhou, ChinaDepartment of Gastroenterology, Nanfang Hospital, Southern Medical University, Guangzhou, ChinaGuangzhou SiDe MedTech Company Ltd., Guangzhou, ChinaDepartment of Gastroenterology, Nanfang Hospital, Southern Medical University, Guangzhou, ChinaSchool of Biomedical Engineering, Southern Medical University, Guangzhou, ChinaWireless capsule endoscopy (WCE) is a recently developed tool that allows for the painless and non-invasive examination of the entire gastrointestinal (GI) tract. The microcamera captures a large number of redundant frames for each WCE examination such that a video summarization technique is needed to assist in diagnosis. However, prevalent methods of summarizing WCE videos focus only on the representativeness of the frames owing to a lack of high-level information on their importance. This paper develops a Frame Importance-Assisted Sparse Subset Selection model, called FIAS3, to integrate the high-level frame importance from networks into a sparse subset selection model. The FIAS3 is optimized under three constraints: 1) a frame importance matrix to help pay more attention to important frames, 2) a sparsity constraint to make video summaries more compact, and 3) a similarity-inhibiting constraint to reduce redundancy. The results of experiments on a public dataset demonstrated that our FIAS3 outperforms other methods of summarizing WCE videos. Specifically, its coverage and video reconstruction error were 92% and 0.143, respectively, at a 90% compression ratio, recording respective at least 16.9% and 0.031 improvements over other methods. The results of generalization experiments showed that FIAS3 also achieves competitive results on private datasets.https://ieeexplore.ieee.org/document/10032547/Computer-aided diagnosisdeep learningkeyframe extractionvideo summarizationwireless capsule endoscopy (WCE) |
| spellingShingle | Weijie Xie Zefeiyun Chen Qingyuan Li Qingfei Ma Yusi Wang Tianbao Liu Yuxin Fang Zhanpeng Zhao Side Liu Wei Yang FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy Videos IEEE Access Computer-aided diagnosis deep learning keyframe extraction video summarization wireless capsule endoscopy (WCE) |
| title | FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy Videos |
| title_full | FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy Videos |
| title_fullStr | FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy Videos |
| title_full_unstemmed | FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy Videos |
| title_short | FIAS3: Frame Importance-Assisted Sparse Subset Selection to Summarize Wireless Capsule Endoscopy Videos |
| title_sort | fias3 frame importance assisted sparse subset selection to summarize wireless capsule endoscopy videos |
| topic | Computer-aided diagnosis deep learning keyframe extraction video summarization wireless capsule endoscopy (WCE) |
| url | https://ieeexplore.ieee.org/document/10032547/ |
| work_keys_str_mv | AT weijiexie fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos AT zefeiyunchen fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos AT qingyuanli fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos AT qingfeima fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos AT yusiwang fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos AT tianbaoliu fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos AT yuxinfang fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos AT zhanpengzhao fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos AT sideliu fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos AT weiyang fias3frameimportanceassistedsparsesubsetselectiontosummarizewirelesscapsuleendoscopyvideos |