Detecting Object-Level Scene Changes in Images with Viewpoint Differences Using Graph Matching
We developed a robust <i>object-level</i> change detection method that could capture distinct scene changes in an image pair with viewpoint differences. To achieve this, we designed a network that could detect object-level changes in an image pair. In contrast to previous studies, we con...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-08-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/17/4225 |
| _version_ | 1797493394095734784 |
|---|---|
| author | Kento Doi Ryuhei Hamaguchi Yusuke Iwasawa Masaki Onishi Yutaka Matsuo Ken Sakurada |
| author_facet | Kento Doi Ryuhei Hamaguchi Yusuke Iwasawa Masaki Onishi Yutaka Matsuo Ken Sakurada |
| author_sort | Kento Doi |
| collection | DOAJ |
| description | We developed a robust <i>object-level</i> change detection method that could capture distinct scene changes in an image pair with viewpoint differences. To achieve this, we designed a network that could detect object-level changes in an image pair. In contrast to previous studies, we considered the change detection task as a graph matching problem for two object graphs that were extracted from each image. By virtue of this, the proposed network more robustly detected object-level changes with viewpoint differences than existing pixel-level approaches. In addition, the network did not require pixel-level change annotations, which have been required in previous studies. Specifically, the proposed network extracted the objects in each image using an object detection module and then constructed correspondences between the objects using an object matching module. Finally, the network detected objects that appeared or disappeared in a scene using the correspondences that were obtained between the objects. To verify the effectiveness of the proposed network, we created a synthetic dataset of images that contained object-level changes. In experiments on the created dataset, the proposed method improved the <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi mathvariant="normal">F</mi><mrow><mn>1</mn></mrow></msub></semantics></math></inline-formula> score of conventional methods by more than 40%. Our synthetic dataset will be available publicly online. |
| first_indexed | 2024-03-10T01:19:25Z |
| format | Article |
| id | doaj.art-d2b926d0af5b47429f3e9e0c65726cd0 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T01:19:25Z |
| publishDate | 2022-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-d2b926d0af5b47429f3e9e0c65726cd02023-11-23T14:03:03ZengMDPI AGRemote Sensing2072-42922022-08-011417422510.3390/rs14174225Detecting Object-Level Scene Changes in Images with Viewpoint Differences Using Graph MatchingKento Doi0Ryuhei Hamaguchi1Yusuke Iwasawa2Masaki Onishi3Yutaka Matsuo4Ken Sakurada5School of Engineering, The University of Tokyo, Tokyo 113-0033, JapanArtificial Intelligence Research Center (AIRC), The National Institute of Advanced Industrial Science and Technology (AIST), Tokyo 135-0064, JapanSchool of Engineering, The University of Tokyo, Tokyo 113-0033, JapanArtificial Intelligence Research Center (AIRC), The National Institute of Advanced Industrial Science and Technology (AIST), Tokyo 135-0064, JapanSchool of Engineering, The University of Tokyo, Tokyo 113-0033, JapanArtificial Intelligence Research Center (AIRC), The National Institute of Advanced Industrial Science and Technology (AIST), Tokyo 135-0064, JapanWe developed a robust <i>object-level</i> change detection method that could capture distinct scene changes in an image pair with viewpoint differences. To achieve this, we designed a network that could detect object-level changes in an image pair. In contrast to previous studies, we considered the change detection task as a graph matching problem for two object graphs that were extracted from each image. By virtue of this, the proposed network more robustly detected object-level changes with viewpoint differences than existing pixel-level approaches. In addition, the network did not require pixel-level change annotations, which have been required in previous studies. Specifically, the proposed network extracted the objects in each image using an object detection module and then constructed correspondences between the objects using an object matching module. Finally, the network detected objects that appeared or disappeared in a scene using the correspondences that were obtained between the objects. To verify the effectiveness of the proposed network, we created a synthetic dataset of images that contained object-level changes. In experiments on the created dataset, the proposed method improved the <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi mathvariant="normal">F</mi><mrow><mn>1</mn></mrow></msub></semantics></math></inline-formula> score of conventional methods by more than 40%. Our synthetic dataset will be available publicly online.https://www.mdpi.com/2072-4292/14/17/4225change detectionobject detectiongraph matching |
| spellingShingle | Kento Doi Ryuhei Hamaguchi Yusuke Iwasawa Masaki Onishi Yutaka Matsuo Ken Sakurada Detecting Object-Level Scene Changes in Images with Viewpoint Differences Using Graph Matching Remote Sensing change detection object detection graph matching |
| title | Detecting Object-Level Scene Changes in Images with Viewpoint Differences Using Graph Matching |
| title_full | Detecting Object-Level Scene Changes in Images with Viewpoint Differences Using Graph Matching |
| title_fullStr | Detecting Object-Level Scene Changes in Images with Viewpoint Differences Using Graph Matching |
| title_full_unstemmed | Detecting Object-Level Scene Changes in Images with Viewpoint Differences Using Graph Matching |
| title_short | Detecting Object-Level Scene Changes in Images with Viewpoint Differences Using Graph Matching |
| title_sort | detecting object level scene changes in images with viewpoint differences using graph matching |
| topic | change detection object detection graph matching |
| url | https://www.mdpi.com/2072-4292/14/17/4225 |
| work_keys_str_mv | AT kentodoi detectingobjectlevelscenechangesinimageswithviewpointdifferencesusinggraphmatching AT ryuheihamaguchi detectingobjectlevelscenechangesinimageswithviewpointdifferencesusinggraphmatching AT yusukeiwasawa detectingobjectlevelscenechangesinimageswithviewpointdifferencesusinggraphmatching AT masakionishi detectingobjectlevelscenechangesinimageswithviewpointdifferencesusinggraphmatching AT yutakamatsuo detectingobjectlevelscenechangesinimageswithviewpointdifferencesusinggraphmatching AT kensakurada detectingobjectlevelscenechangesinimageswithviewpointdifferencesusinggraphmatching |