Virtual to Real Adaptation of Pedestrian Detectors
Pedestrian detection through Computer Vision is a building block for a multitude of applications. Recently, there has been an increasing interest in convolutional neural network-based architectures to execute such a task. One of these supervised networks’ critical goals is to generalize the knowledg...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/18/5250 |
| _version_ | 1797553734176210944 |
|---|---|
| author | Luca Ciampi Nicola Messina Fabrizio Falchi Claudio Gennaro Giuseppe Amato |
| author_facet | Luca Ciampi Nicola Messina Fabrizio Falchi Claudio Gennaro Giuseppe Amato |
| author_sort | Luca Ciampi |
| collection | DOAJ |
| description | Pedestrian detection through Computer Vision is a building block for a multitude of applications. Recently, there has been an increasing interest in convolutional neural network-based architectures to execute such a task. One of these supervised networks’ critical goals is to generalize the knowledge learned during the training phase to new scenarios with different characteristics. A suitably labeled dataset is essential to achieve this purpose. The main problem is that manually annotating a dataset usually requires a lot of human effort, and it is costly. To this end, we introduce ViPeD (Virtual Pedestrian Dataset), a new synthetically generated set of images collected with the highly photo-realistic graphical engine of the video game GTA V (Grand Theft Auto V), where annotations are automatically acquired. However, when training solely on the synthetic dataset, the model experiences a Synthetic2Real domain shift leading to a performance drop when applied to real-world images. To mitigate this gap, we propose two different domain adaptation techniques suitable for the pedestrian detection task, but possibly applicable to general object detection. Experiments show that the network trained with ViPeD can generalize over unseen real-world scenarios better than the detector trained over real-world data, exploiting the variety of our synthetic dataset. Furthermore, we demonstrate that with our domain adaptation techniques, we can reduce the Synthetic2Real domain shift, making the two domains closer and obtaining a performance improvement when testing the network over the real-world images. |
| first_indexed | 2024-03-10T16:20:48Z |
| format | Article |
| id | doaj.art-a8cdda8d5db1410c8091c3e99dee014f |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T16:20:48Z |
| publishDate | 2020-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-a8cdda8d5db1410c8091c3e99dee014f2023-11-20T13:43:19ZengMDPI AGSensors1424-82202020-09-012018525010.3390/s20185250Virtual to Real Adaptation of Pedestrian DetectorsLuca Ciampi0Nicola Messina1Fabrizio Falchi2Claudio Gennaro3Giuseppe Amato4Institute of Information Science and Technologies, National Research Council, Via G. Moruzzi 1, 56124 Pisa, ItalyInstitute of Information Science and Technologies, National Research Council, Via G. Moruzzi 1, 56124 Pisa, ItalyInstitute of Information Science and Technologies, National Research Council, Via G. Moruzzi 1, 56124 Pisa, ItalyInstitute of Information Science and Technologies, National Research Council, Via G. Moruzzi 1, 56124 Pisa, ItalyInstitute of Information Science and Technologies, National Research Council, Via G. Moruzzi 1, 56124 Pisa, ItalyPedestrian detection through Computer Vision is a building block for a multitude of applications. Recently, there has been an increasing interest in convolutional neural network-based architectures to execute such a task. One of these supervised networks’ critical goals is to generalize the knowledge learned during the training phase to new scenarios with different characteristics. A suitably labeled dataset is essential to achieve this purpose. The main problem is that manually annotating a dataset usually requires a lot of human effort, and it is costly. To this end, we introduce ViPeD (Virtual Pedestrian Dataset), a new synthetically generated set of images collected with the highly photo-realistic graphical engine of the video game GTA V (Grand Theft Auto V), where annotations are automatically acquired. However, when training solely on the synthetic dataset, the model experiences a Synthetic2Real domain shift leading to a performance drop when applied to real-world images. To mitigate this gap, we propose two different domain adaptation techniques suitable for the pedestrian detection task, but possibly applicable to general object detection. Experiments show that the network trained with ViPeD can generalize over unseen real-world scenarios better than the detector trained over real-world data, exploiting the variety of our synthetic dataset. Furthermore, we demonstrate that with our domain adaptation techniques, we can reduce the Synthetic2Real domain shift, making the two domains closer and obtaining a performance improvement when testing the network over the real-world images.https://www.mdpi.com/1424-8220/20/18/5250pedestrian detectiondomain adaptationsynthetic datasetsconvolutional neural networksdeep learning |
| spellingShingle | Luca Ciampi Nicola Messina Fabrizio Falchi Claudio Gennaro Giuseppe Amato Virtual to Real Adaptation of Pedestrian Detectors Sensors pedestrian detection domain adaptation synthetic datasets convolutional neural networks deep learning |
| title | Virtual to Real Adaptation of Pedestrian Detectors |
| title_full | Virtual to Real Adaptation of Pedestrian Detectors |
| title_fullStr | Virtual to Real Adaptation of Pedestrian Detectors |
| title_full_unstemmed | Virtual to Real Adaptation of Pedestrian Detectors |
| title_short | Virtual to Real Adaptation of Pedestrian Detectors |
| title_sort | virtual to real adaptation of pedestrian detectors |
| topic | pedestrian detection domain adaptation synthetic datasets convolutional neural networks deep learning |
| url | https://www.mdpi.com/1424-8220/20/18/5250 |
| work_keys_str_mv | AT lucaciampi virtualtorealadaptationofpedestriandetectors AT nicolamessina virtualtorealadaptationofpedestriandetectors AT fabriziofalchi virtualtorealadaptationofpedestriandetectors AT claudiogennaro virtualtorealadaptationofpedestriandetectors AT giuseppeamato virtualtorealadaptationofpedestriandetectors |