Moving Car Recognition and Removal for 3D Urban Modelling Using Oblique Images
With the progress of photogrammetry and computer vision technology, three-dimensional (3D) reconstruction using aerial oblique images has been widely applied in urban modelling and smart city applications. However, state-of-the-art image-based automatic 3D reconstruction methods cannot effectively h...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-08-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/17/3458 |
| _version_ | 1797520903510163456 |
|---|---|
| author | Chong Yang Fan Zhang Yunlong Gao Zhu Mao Liang Li Xianfeng Huang |
| author_facet | Chong Yang Fan Zhang Yunlong Gao Zhu Mao Liang Li Xianfeng Huang |
| author_sort | Chong Yang |
| collection | DOAJ |
| description | With the progress of photogrammetry and computer vision technology, three-dimensional (3D) reconstruction using aerial oblique images has been widely applied in urban modelling and smart city applications. However, state-of-the-art image-based automatic 3D reconstruction methods cannot effectively handle the unavoidable geometric deformation and incorrect texture mapping problems caused by moving cars in a city. This paper proposes a method to address this situation and prevent the influence of moving cars on 3D modelling by recognizing moving cars and combining the recognition results with a photogrammetric 3D modelling procedure. Through car detection using a deep learning method and multiview geometry constraints, we can analyse the state of a car’s movement and apply a proper preprocessing method to the geometrically model generation and texture mapping steps of 3D reconstruction pipelines. First, we apply the traditional Mask R-CNN object detection method to detect cars from oblique images. Then, a detected car and its corresponding image patch calculated by the geometry constraints in the other view images are used to identify the moving state of the car. Finally, the geometry and texture information corresponding to the moving car will be processed according to its moving state. Experiments on three different urban datasets demonstrate that the proposed method is effective in recognizing and removing moving cars and can repair the geometric deformation and error texture mapping problems caused by moving cars. In addition, the methods proposed in this paper can be applied to eliminate other moving objects in 3D modelling applications. |
| first_indexed | 2024-03-10T08:05:07Z |
| format | Article |
| id | doaj.art-94c063465dc44fc799376636bddcf6b6 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T08:05:07Z |
| publishDate | 2021-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-94c063465dc44fc799376636bddcf6b62023-11-22T11:09:12ZengMDPI AGRemote Sensing2072-42922021-08-011317345810.3390/rs13173458Moving Car Recognition and Removal for 3D Urban Modelling Using Oblique ImagesChong Yang0Fan Zhang1Yunlong Gao2Zhu Mao3Liang Li4Xianfeng Huang5School of Remote Sensing and Information Engineering, Wuhan University, 129 Luoyu Road, Wuhan 430079, ChinaThe State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, ChinaThe State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, ChinaThe State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, ChinaThe State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, ChinaThe State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, ChinaWith the progress of photogrammetry and computer vision technology, three-dimensional (3D) reconstruction using aerial oblique images has been widely applied in urban modelling and smart city applications. However, state-of-the-art image-based automatic 3D reconstruction methods cannot effectively handle the unavoidable geometric deformation and incorrect texture mapping problems caused by moving cars in a city. This paper proposes a method to address this situation and prevent the influence of moving cars on 3D modelling by recognizing moving cars and combining the recognition results with a photogrammetric 3D modelling procedure. Through car detection using a deep learning method and multiview geometry constraints, we can analyse the state of a car’s movement and apply a proper preprocessing method to the geometrically model generation and texture mapping steps of 3D reconstruction pipelines. First, we apply the traditional Mask R-CNN object detection method to detect cars from oblique images. Then, a detected car and its corresponding image patch calculated by the geometry constraints in the other view images are used to identify the moving state of the car. Finally, the geometry and texture information corresponding to the moving car will be processed according to its moving state. Experiments on three different urban datasets demonstrate that the proposed method is effective in recognizing and removing moving cars and can repair the geometric deformation and error texture mapping problems caused by moving cars. In addition, the methods proposed in this paper can be applied to eliminate other moving objects in 3D modelling applications.https://www.mdpi.com/2072-4292/13/17/3458oblique imagesimage-based 3D reconstructionobject detectionclean model |
| spellingShingle | Chong Yang Fan Zhang Yunlong Gao Zhu Mao Liang Li Xianfeng Huang Moving Car Recognition and Removal for 3D Urban Modelling Using Oblique Images Remote Sensing oblique images image-based 3D reconstruction object detection clean model |
| title | Moving Car Recognition and Removal for 3D Urban Modelling Using Oblique Images |
| title_full | Moving Car Recognition and Removal for 3D Urban Modelling Using Oblique Images |
| title_fullStr | Moving Car Recognition and Removal for 3D Urban Modelling Using Oblique Images |
| title_full_unstemmed | Moving Car Recognition and Removal for 3D Urban Modelling Using Oblique Images |
| title_short | Moving Car Recognition and Removal for 3D Urban Modelling Using Oblique Images |
| title_sort | moving car recognition and removal for 3d urban modelling using oblique images |
| topic | oblique images image-based 3D reconstruction object detection clean model |
| url | https://www.mdpi.com/2072-4292/13/17/3458 |
| work_keys_str_mv | AT chongyang movingcarrecognitionandremovalfor3durbanmodellingusingobliqueimages AT fanzhang movingcarrecognitionandremovalfor3durbanmodellingusingobliqueimages AT yunlonggao movingcarrecognitionandremovalfor3durbanmodellingusingobliqueimages AT zhumao movingcarrecognitionandremovalfor3durbanmodellingusingobliqueimages AT liangli movingcarrecognitionandremovalfor3durbanmodellingusingobliqueimages AT xianfenghuang movingcarrecognitionandremovalfor3durbanmodellingusingobliqueimages |