Light-Field Raw Data Synthesis From RGB-D Images: Pushing to the Extreme
Light-field raw data captured by a state-of-the-art light-field camera is limited in its spatial and angular resolutions due to the camera's optical hardware. In this paper, we propose an all-software algorithm to synthesize light-field raw data from a single RGB-D input image, which is driven...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8999544/ |
| _version_ | 1818624274594791424 |
|---|---|
| author | Yiqun Wu Shuaicheng Liu Chao Sun Bing Zeng |
| author_facet | Yiqun Wu Shuaicheng Liu Chao Sun Bing Zeng |
| author_sort | Yiqun Wu |
| collection | DOAJ |
| description | Light-field raw data captured by a state-of-the-art light-field camera is limited in its spatial and angular resolutions due to the camera's optical hardware. In this paper, we propose an all-software algorithm to synthesize light-field raw data from a single RGB-D input image, which is driven largely by the need in the research area of light-field data compression. Our synthesis algorithm consists of three key steps: (1) each pixel of the input image is regarded as a spot lighting source that emits directional light rays with an equal strength; (2) the optical path of each directional light ray through the camera's main lens as well as the corresponding micro lens is considered as accurately as possible; and (3) the occlusion of light rays among objects at different distances within the input image is handled with the depth information. The spatial and angular resolutions of our synthesized light-field data can be scaled up when the input RGB-D image has a higher and higher spatial resolution. Meanwhile, for a given input image with a fixed size, we pay a special attention to what would be the extreme we can push the parameters involved in our synthesis algorithm, such as the number of rays emitted from each pixel, the number of micro lenses, and the number of sensors associated with each micro lens. The usefulness of our synthesized data is validated by refocusing, all-in-focus, and sub-aperture reconstructions. In particular, all-in-focus images are evaluated objectively by computing the structural similarity (SSIM) index, which allows us to reach the goal of pushing to the extreme through selecting various parameters mentioned above. |
| first_indexed | 2024-12-16T18:54:21Z |
| format | Article |
| id | doaj.art-770ad4233c9e4cc69a7d6d22ec5b1c4d |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-16T18:54:21Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-770ad4233c9e4cc69a7d6d22ec5b1c4d2022-12-21T22:20:35ZengIEEEIEEE Access2169-35362020-01-018333913340510.1109/ACCESS.2020.29740638999544Light-Field Raw Data Synthesis From RGB-D Images: Pushing to the ExtremeYiqun Wu0https://orcid.org/0000-0001-6516-2754Shuaicheng Liu1https://orcid.org/0000-0002-8815-5335Chao Sun2Bing Zeng3https://orcid.org/0000-0002-4491-7967School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaSchool of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, ChinaLight-field raw data captured by a state-of-the-art light-field camera is limited in its spatial and angular resolutions due to the camera's optical hardware. In this paper, we propose an all-software algorithm to synthesize light-field raw data from a single RGB-D input image, which is driven largely by the need in the research area of light-field data compression. Our synthesis algorithm consists of three key steps: (1) each pixel of the input image is regarded as a spot lighting source that emits directional light rays with an equal strength; (2) the optical path of each directional light ray through the camera's main lens as well as the corresponding micro lens is considered as accurately as possible; and (3) the occlusion of light rays among objects at different distances within the input image is handled with the depth information. The spatial and angular resolutions of our synthesized light-field data can be scaled up when the input RGB-D image has a higher and higher spatial resolution. Meanwhile, for a given input image with a fixed size, we pay a special attention to what would be the extreme we can push the parameters involved in our synthesis algorithm, such as the number of rays emitted from each pixel, the number of micro lenses, and the number of sensors associated with each micro lens. The usefulness of our synthesized data is validated by refocusing, all-in-focus, and sub-aperture reconstructions. In particular, all-in-focus images are evaluated objectively by computing the structural similarity (SSIM) index, which allows us to reach the goal of pushing to the extreme through selecting various parameters mentioned above.https://ieeexplore.ieee.org/document/8999544/Light-fieldRGB-D imagesmicro lensrefocusingsub-aperture |
| spellingShingle | Yiqun Wu Shuaicheng Liu Chao Sun Bing Zeng Light-Field Raw Data Synthesis From RGB-D Images: Pushing to the Extreme IEEE Access Light-field RGB-D images micro lens refocusing sub-aperture |
| title | Light-Field Raw Data Synthesis From RGB-D Images: Pushing to the Extreme |
| title_full | Light-Field Raw Data Synthesis From RGB-D Images: Pushing to the Extreme |
| title_fullStr | Light-Field Raw Data Synthesis From RGB-D Images: Pushing to the Extreme |
| title_full_unstemmed | Light-Field Raw Data Synthesis From RGB-D Images: Pushing to the Extreme |
| title_short | Light-Field Raw Data Synthesis From RGB-D Images: Pushing to the Extreme |
| title_sort | light field raw data synthesis from rgb d images pushing to the extreme |
| topic | Light-field RGB-D images micro lens refocusing sub-aperture |
| url | https://ieeexplore.ieee.org/document/8999544/ |
| work_keys_str_mv | AT yiqunwu lightfieldrawdatasynthesisfromrgbdimagespushingtotheextreme AT shuaichengliu lightfieldrawdatasynthesisfromrgbdimagespushingtotheextreme AT chaosun lightfieldrawdatasynthesisfromrgbdimagespushingtotheextreme AT bingzeng lightfieldrawdatasynthesisfromrgbdimagespushingtotheextreme |