Replay: multi-modal multi-view acted videos for casual holography
We introduce Replay, a collection of multi-view, multimodal videos of humans interacting socially. Each scene is filmed in high production quality, from different viewpoints with several static cameras, as well as wearable action cameras, and recorded with a large array of microphones at different p...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
IEEE
2024
|
| _version_ | 1797112421867847680 |
|---|---|
| author | Shapovalov, R Kleiman, Y Rocco, I Novotny, D Vedaldi, A Chen, C Kokkinos, F Graham, B Neverova, N |
| author_facet | Shapovalov, R Kleiman, Y Rocco, I Novotny, D Vedaldi, A Chen, C Kokkinos, F Graham, B Neverova, N |
| author_sort | Shapovalov, R |
| collection | OXFORD |
| description | We introduce Replay, a collection of multi-view, multimodal videos of humans interacting socially. Each scene
is filmed in high production quality, from different viewpoints with several static cameras, as well as wearable
action cameras, and recorded with a large array of microphones at different positions in the room. Overall, the dataset contains over 4000 minutes of footage and over 7 million timestamped high-resolution frames annotated with camera poses and partially with foreground masks. The Replay dataset has many potential applications, such as novelview synthesis, 3D reconstruction, novel-view acoustic synthesis, human body and face analysis, and training generative models. We provide a benchmark for training and evaluating novel-view synthesis, with two scenarios of different difficulty. Finally, we evaluate several baseline state-of-theart methods on the new benchmark. |
| first_indexed | 2024-03-07T08:25:31Z |
| format | Conference item |
| id | oxford-uuid:fb8af908-03cb-485c-8483-1b79245f6ed4 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T08:25:31Z |
| publishDate | 2024 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:fb8af908-03cb-485c-8483-1b79245f6ed42024-02-12T09:49:47ZReplay: multi-modal multi-view acted videos for casual holographyConference itemhttp://purl.org/coar/resource_type/c_5794uuid:fb8af908-03cb-485c-8483-1b79245f6ed4EnglishSymplectic ElementsIEEE2024Shapovalov, RKleiman, YRocco, INovotny, DVedaldi, AChen, CKokkinos, FGraham, BNeverova, NWe introduce Replay, a collection of multi-view, multimodal videos of humans interacting socially. Each scene is filmed in high production quality, from different viewpoints with several static cameras, as well as wearable action cameras, and recorded with a large array of microphones at different positions in the room. Overall, the dataset contains over 4000 minutes of footage and over 7 million timestamped high-resolution frames annotated with camera poses and partially with foreground masks. The Replay dataset has many potential applications, such as novelview synthesis, 3D reconstruction, novel-view acoustic synthesis, human body and face analysis, and training generative models. We provide a benchmark for training and evaluating novel-view synthesis, with two scenarios of different difficulty. Finally, we evaluate several baseline state-of-theart methods on the new benchmark. |
| spellingShingle | Shapovalov, R Kleiman, Y Rocco, I Novotny, D Vedaldi, A Chen, C Kokkinos, F Graham, B Neverova, N Replay: multi-modal multi-view acted videos for casual holography |
| title | Replay: multi-modal multi-view acted videos for casual holography |
| title_full | Replay: multi-modal multi-view acted videos for casual holography |
| title_fullStr | Replay: multi-modal multi-view acted videos for casual holography |
| title_full_unstemmed | Replay: multi-modal multi-view acted videos for casual holography |
| title_short | Replay: multi-modal multi-view acted videos for casual holography |
| title_sort | replay multi modal multi view acted videos for casual holography |
| work_keys_str_mv | AT shapovalovr replaymultimodalmultiviewactedvideosforcasualholography AT kleimany replaymultimodalmultiviewactedvideosforcasualholography AT roccoi replaymultimodalmultiviewactedvideosforcasualholography AT novotnyd replaymultimodalmultiviewactedvideosforcasualholography AT vedaldia replaymultimodalmultiviewactedvideosforcasualholography AT chenc replaymultimodalmultiviewactedvideosforcasualholography AT kokkinosf replaymultimodalmultiviewactedvideosforcasualholography AT grahamb replaymultimodalmultiviewactedvideosforcasualholography AT neverovan replaymultimodalmultiviewactedvideosforcasualholography |