Automatic Camera Selection, Shot Size, and Video Editing in Theater Multi-Camera Recordings
In a non-professional environment, multi-camera recordings of theater performances or other stage shows are difficult to realize, because amateurs are usually untrained in camera work and in using a vision mixing desk that mixes multiple cameras. This can be remedied by a production process with hig...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10237191/ |
| _version_ | 1797688557207289856 |
|---|---|
| author | Eckhard Stoll Stephan Breide Steve Goring Alexander Raake |
| author_facet | Eckhard Stoll Stephan Breide Steve Goring Alexander Raake |
| author_sort | Eckhard Stoll |
| collection | DOAJ |
| description | In a non-professional environment, multi-camera recordings of theater performances or other stage shows are difficult to realize, because amateurs are usually untrained in camera work and in using a vision mixing desk that mixes multiple cameras. This can be remedied by a production process with high-resolution cameras where recordings of image sections from long shots or medium-long shots are manually or automatically cropped in post-production. For this purpose, Gandhi et al. presented a single-camera system (referred to as Gandhi Recording System in the paper) that obtains close-ups from a high-resolution recording from the central perspective. The proposed system in this paper referred to as “Proposed Recording System” extends the method to four perspectives based on a Reference Recording System from professional TV theater recordings from the Ohnsorg Theater. Rules for camera selection, image cropping, and montage are derived from the Reference Recording System in this paper. For this purpose, body and pose recognition software is used and the stage action is reconstructed from the recordings into the stage set. Speakers are recognized by detecting lip movements and speaker changes are identified using audio diarization software. The Proposed Recording System proposed in this paper is practically instantiated on a school theater recording made by laymen using four 4K cameras. An automatic editing script is generated that outputs a montage of a scene. The principles can also be adapted for other recording situations with an audience, such as lectures, interviews, discussions, talk shows, gala events, award ceremonies, and the like. More than 70 % of test persons confirm in an online study the added value of the perspective diversity of four cameras of the Proposed Recording System versus the single-camera method of Gandhi et al. |
| first_indexed | 2024-03-12T01:32:36Z |
| format | Article |
| id | doaj.art-c5ce9a20044c4836940b8cb5fb1d5b84 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-12T01:32:36Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c5ce9a20044c4836940b8cb5fb1d5b842023-09-11T23:00:40ZengIEEEIEEE Access2169-35362023-01-0111966739669210.1109/ACCESS.2023.331125610237191Automatic Camera Selection, Shot Size, and Video Editing in Theater Multi-Camera RecordingsEckhard Stoll0https://orcid.org/0000-0002-0437-0522Stephan Breide1https://orcid.org/0000-0003-3410-4134Steve Goring2https://orcid.org/0000-0001-6810-6969Alexander Raake3https://orcid.org/0000-0002-9357-1763Audiovisual Technology Group, Technische Universität Ilmenau, Ilmenau, GermanyAudio Visual Media Center, South Westphalia University of Applied Sciences, Meschede, GermanyAudiovisual Technology Group, Technische Universität Ilmenau, Ilmenau, GermanyAudiovisual Technology Group, Technische Universität Ilmenau, Ilmenau, GermanyIn a non-professional environment, multi-camera recordings of theater performances or other stage shows are difficult to realize, because amateurs are usually untrained in camera work and in using a vision mixing desk that mixes multiple cameras. This can be remedied by a production process with high-resolution cameras where recordings of image sections from long shots or medium-long shots are manually or automatically cropped in post-production. For this purpose, Gandhi et al. presented a single-camera system (referred to as Gandhi Recording System in the paper) that obtains close-ups from a high-resolution recording from the central perspective. The proposed system in this paper referred to as “Proposed Recording System” extends the method to four perspectives based on a Reference Recording System from professional TV theater recordings from the Ohnsorg Theater. Rules for camera selection, image cropping, and montage are derived from the Reference Recording System in this paper. For this purpose, body and pose recognition software is used and the stage action is reconstructed from the recordings into the stage set. Speakers are recognized by detecting lip movements and speaker changes are identified using audio diarization software. The Proposed Recording System proposed in this paper is practically instantiated on a school theater recording made by laymen using four 4K cameras. An automatic editing script is generated that outputs a montage of a scene. The principles can also be adapted for other recording situations with an audience, such as lectures, interviews, discussions, talk shows, gala events, award ceremonies, and the like. More than 70 % of test persons confirm in an online study the added value of the perspective diversity of four cameras of the Proposed Recording System versus the single-camera method of Gandhi et al.https://ieeexplore.ieee.org/document/10237191/Multi-camera theater recordingscroppingautomatic montage4Kautomatic video editing |
| spellingShingle | Eckhard Stoll Stephan Breide Steve Goring Alexander Raake Automatic Camera Selection, Shot Size, and Video Editing in Theater Multi-Camera Recordings IEEE Access Multi-camera theater recordings cropping automatic montage 4K automatic video editing |
| title | Automatic Camera Selection, Shot Size, and Video Editing in Theater Multi-Camera Recordings |
| title_full | Automatic Camera Selection, Shot Size, and Video Editing in Theater Multi-Camera Recordings |
| title_fullStr | Automatic Camera Selection, Shot Size, and Video Editing in Theater Multi-Camera Recordings |
| title_full_unstemmed | Automatic Camera Selection, Shot Size, and Video Editing in Theater Multi-Camera Recordings |
| title_short | Automatic Camera Selection, Shot Size, and Video Editing in Theater Multi-Camera Recordings |
| title_sort | automatic camera selection shot size and video editing in theater multi camera recordings |
| topic | Multi-camera theater recordings cropping automatic montage 4K automatic video editing |
| url | https://ieeexplore.ieee.org/document/10237191/ |
| work_keys_str_mv | AT eckhardstoll automaticcameraselectionshotsizeandvideoeditingintheatermulticamerarecordings AT stephanbreide automaticcameraselectionshotsizeandvideoeditingintheatermulticamerarecordings AT stevegoring automaticcameraselectionshotsizeandvideoeditingintheatermulticamerarecordings AT alexanderraake automaticcameraselectionshotsizeandvideoeditingintheatermulticamerarecordings |