Background Subtraction With Real-Time Semantic Segmentation
Accurate and fast foreground (FG) object extraction is very important for object tracking and recognition in video surveillance. Although many background subtraction (BGS) methods have been proposed in the recent past, it is still regarded as a tough problem due to the variety of challenging situati...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8645635/ |
| _version_ | 1823926100102742016 |
|---|---|
| author | Dongdong Zeng Xiang Chen Ming Zhu Michael Goesele Arjan Kuijper |
| author_facet | Dongdong Zeng Xiang Chen Ming Zhu Michael Goesele Arjan Kuijper |
| author_sort | Dongdong Zeng |
| collection | DOAJ |
| description | Accurate and fast foreground (FG) object extraction is very important for object tracking and recognition in video surveillance. Although many background subtraction (BGS) methods have been proposed in the recent past, it is still regarded as a tough problem due to the variety of challenging situations that occur in real-world scenarios. In this paper, we explore this problem from a new perspective and propose a novel BGS framework with the real-time semantic segmentation. Our proposed framework consists of two components, a traditional BGS segmenter B and a real-time semantic segmenter S. The BGS segmenter B aims to construct background (BG) models and segments FG objects. The real-time semantic segmenter S is used to refine the FG segmentation outputs as feedbacks for improving the model updating accuracy. B and S work in parallel on two threads. For each input frame It, the BGS segmenter B computes a preliminary FG/BG mask B<sub>t</sub>. At the same time, the real-time semantic segmenter S extracts the object-level semantics S<sub>t</sub>. Then, some specific rules are applied on B<sub>t</sub> and S<sub>t</sub> to generate the final detection D<sub>t</sub>. Finally, the refined FG/BG mask D<sub>t</sub> is fed back to update the BG model. The comprehensive experiments evaluated on the CDnet 2014 dataset demonstrate that our proposed method achieves the state-of-the-art performance among all unsupervised BGS methods while operating at the real-time and even performs better than some deep learning-based supervised algorithms. In addition, our proposed framework is very flexible and has the potential for generalization. |
| first_indexed | 2024-12-16T20:19:06Z |
| format | Article |
| id | doaj.art-5e194dddf8914a27b5b26fae216481ef |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-16T20:19:06Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-5e194dddf8914a27b5b26fae216481ef2022-12-21T22:17:50ZengIEEEIEEE Access2169-35362019-01-01715386915388410.1109/ACCESS.2019.28993488645635Background Subtraction With Real-Time Semantic SegmentationDongdong Zeng0https://orcid.org/0000-0002-9990-5162Xiang Chen1Ming Zhu2Michael Goesele3Arjan Kuijper4Chinese Academy of Sciences, Changchun Institute of OpticsFine Mechanics and Physics, Changchun, ChinaGraphics, Capture and Massively Parallel Computing Group, Technische Universität Darmstadt, Darmstadt, GermanyChinese Academy of Sciences, Changchun Institute of OpticsFine Mechanics and Physics, Changchun, ChinaGraphics, Capture and Massively Parallel Computing Group, Technische Universität Darmstadt, Darmstadt, GermanyFraunhofer IGD, Darmstadt, GermanyAccurate and fast foreground (FG) object extraction is very important for object tracking and recognition in video surveillance. Although many background subtraction (BGS) methods have been proposed in the recent past, it is still regarded as a tough problem due to the variety of challenging situations that occur in real-world scenarios. In this paper, we explore this problem from a new perspective and propose a novel BGS framework with the real-time semantic segmentation. Our proposed framework consists of two components, a traditional BGS segmenter B and a real-time semantic segmenter S. The BGS segmenter B aims to construct background (BG) models and segments FG objects. The real-time semantic segmenter S is used to refine the FG segmentation outputs as feedbacks for improving the model updating accuracy. B and S work in parallel on two threads. For each input frame It, the BGS segmenter B computes a preliminary FG/BG mask B<sub>t</sub>. At the same time, the real-time semantic segmenter S extracts the object-level semantics S<sub>t</sub>. Then, some specific rules are applied on B<sub>t</sub> and S<sub>t</sub> to generate the final detection D<sub>t</sub>. Finally, the refined FG/BG mask D<sub>t</sub> is fed back to update the BG model. The comprehensive experiments evaluated on the CDnet 2014 dataset demonstrate that our proposed method achieves the state-of-the-art performance among all unsupervised BGS methods while operating at the real-time and even performs better than some deep learning-based supervised algorithms. In addition, our proposed framework is very flexible and has the potential for generalization.https://ieeexplore.ieee.org/document/8645635/Background subtractionforeground object detectionsemantic segmentationvideo surveillance |
| spellingShingle | Dongdong Zeng Xiang Chen Ming Zhu Michael Goesele Arjan Kuijper Background Subtraction With Real-Time Semantic Segmentation IEEE Access Background subtraction foreground object detection semantic segmentation video surveillance |
| title | Background Subtraction With Real-Time Semantic Segmentation |
| title_full | Background Subtraction With Real-Time Semantic Segmentation |
| title_fullStr | Background Subtraction With Real-Time Semantic Segmentation |
| title_full_unstemmed | Background Subtraction With Real-Time Semantic Segmentation |
| title_short | Background Subtraction With Real-Time Semantic Segmentation |
| title_sort | background subtraction with real time semantic segmentation |
| topic | Background subtraction foreground object detection semantic segmentation video surveillance |
| url | https://ieeexplore.ieee.org/document/8645635/ |
| work_keys_str_mv | AT dongdongzeng backgroundsubtractionwithrealtimesemanticsegmentation AT xiangchen backgroundsubtractionwithrealtimesemanticsegmentation AT mingzhu backgroundsubtractionwithrealtimesemanticsegmentation AT michaelgoesele backgroundsubtractionwithrealtimesemanticsegmentation AT arjankuijper backgroundsubtractionwithrealtimesemanticsegmentation |