COTS: A Multipurpose RGB-D Dataset for Saliency and Image Manipulation Applications
Many modern computer vision systems include several modules that perform different processing operations packaged as a single pipeline architecture. This generally introduces a challenge in the evaluation process since most datasets provide evaluation data for just one of the operations. In this pap...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9340352/ |
| _version_ | 1818433417328459776 |
|---|---|
| author | Dylan Seychell Carl James Debono Mark Bugeja Jeremy Borg Matthew Sacco |
| author_facet | Dylan Seychell Carl James Debono Mark Bugeja Jeremy Borg Matthew Sacco |
| author_sort | Dylan Seychell |
| collection | DOAJ |
| description | Many modern computer vision systems include several modules that perform different processing operations packaged as a single pipeline architecture. This generally introduces a challenge in the evaluation process since most datasets provide evaluation data for just one of the operations. In this paper, we present an RGB-D dataset that was designed from first principles to cater for applications that involve salient object detection, segmentation, inpainting and blending techniques. This addresses a gap in the evaluation of image inpainting and blending applications that generally rely on subjective evaluation due to the lack of availability of comparative data. A set of experiments were carried out to demonstrate how the COTS dataset can be used to evaluate these different applications. This dataset includes a variety of scenes, where each scene is captured multiple times, each time adding a new object to the previous scene. This allows for a comparative analysis at pixel level in image inpainting and blending applications. Moreover, all objects were manually labeled in order to offer the possibility of salient object detection even in scenes that contain multiple objects. An online test with 1267 participants was also carried out, and this dataset also includes the click coordinates of users' selection for every image, introducing a user interaction dimension in the same RGB-D dataset. This dataset was also validated using state of the art techniques, obtaining an $F_\beta $ of 0.957 in salient object detection and a mean (Intersection over Union) IoU of 0.942 in Segmentation. Results demonstrate that the COTS dataset introduces novel possibilities for the evaluation of computer vision applications. |
| first_indexed | 2024-12-14T16:20:46Z |
| format | Article |
| id | doaj.art-3c327fa3dec74ddeb5d81f1dea42b314 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T16:20:46Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-3c327fa3dec74ddeb5d81f1dea42b3142022-12-21T22:54:48ZengIEEEIEEE Access2169-35362021-01-019214812149710.1109/ACCESS.2021.30556479340352COTS: A Multipurpose RGB-D Dataset for Saliency and Image Manipulation ApplicationsDylan Seychell0https://orcid.org/0000-0002-2377-9833Carl James Debono1https://orcid.org/0000-0003-2659-8752Mark Bugeja2Jeremy Borg3Matthew Sacco4Department of Computer and Communications Engineering, University of Malta, Msida, MaltaDepartment of Computer and Communications Engineering, University of Malta, Msida, MaltaDepartment of Artificial Intelligence, University of Malta, Msida, MaltaDepartment of Artificial Intelligence, University of Malta, Msida, MaltaDepartment of Computer and Communications Engineering, University of Malta, Msida, MaltaMany modern computer vision systems include several modules that perform different processing operations packaged as a single pipeline architecture. This generally introduces a challenge in the evaluation process since most datasets provide evaluation data for just one of the operations. In this paper, we present an RGB-D dataset that was designed from first principles to cater for applications that involve salient object detection, segmentation, inpainting and blending techniques. This addresses a gap in the evaluation of image inpainting and blending applications that generally rely on subjective evaluation due to the lack of availability of comparative data. A set of experiments were carried out to demonstrate how the COTS dataset can be used to evaluate these different applications. This dataset includes a variety of scenes, where each scene is captured multiple times, each time adding a new object to the previous scene. This allows for a comparative analysis at pixel level in image inpainting and blending applications. Moreover, all objects were manually labeled in order to offer the possibility of salient object detection even in scenes that contain multiple objects. An online test with 1267 participants was also carried out, and this dataset also includes the click coordinates of users' selection for every image, introducing a user interaction dimension in the same RGB-D dataset. This dataset was also validated using state of the art techniques, obtaining an $F_\beta $ of 0.957 in salient object detection and a mean (Intersection over Union) IoU of 0.942 in Segmentation. Results demonstrate that the COTS dataset introduces novel possibilities for the evaluation of computer vision applications.https://ieeexplore.ieee.org/document/9340352/DatasetRGB-Dsalient object detectioninpaintingblendingsegmentation |
| spellingShingle | Dylan Seychell Carl James Debono Mark Bugeja Jeremy Borg Matthew Sacco COTS: A Multipurpose RGB-D Dataset for Saliency and Image Manipulation Applications IEEE Access Dataset RGB-D salient object detection inpainting blending segmentation |
| title | COTS: A Multipurpose RGB-D Dataset for Saliency and Image Manipulation Applications |
| title_full | COTS: A Multipurpose RGB-D Dataset for Saliency and Image Manipulation Applications |
| title_fullStr | COTS: A Multipurpose RGB-D Dataset for Saliency and Image Manipulation Applications |
| title_full_unstemmed | COTS: A Multipurpose RGB-D Dataset for Saliency and Image Manipulation Applications |
| title_short | COTS: A Multipurpose RGB-D Dataset for Saliency and Image Manipulation Applications |
| title_sort | cots a multipurpose rgb d dataset for saliency and image manipulation applications |
| topic | Dataset RGB-D salient object detection inpainting blending segmentation |
| url | https://ieeexplore.ieee.org/document/9340352/ |
| work_keys_str_mv | AT dylanseychell cotsamultipurposergbddatasetforsaliencyandimagemanipulationapplications AT carljamesdebono cotsamultipurposergbddatasetforsaliencyandimagemanipulationapplications AT markbugeja cotsamultipurposergbddatasetforsaliencyandimagemanipulationapplications AT jeremyborg cotsamultipurposergbddatasetforsaliencyandimagemanipulationapplications AT matthewsacco cotsamultipurposergbddatasetforsaliencyandimagemanipulationapplications |