Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep Learning
Light field (LF) can capture the spatial and angular information of the light in one single exposure. And the LF images are widely used in various fields, especially in immersive media. The rich imaging information in the LF poses great challenges for transmission. However, LF images are sparse and...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9145540/ |
| _version_ | 1818917701256478720 |
|---|---|
| author | Linhui Wei Yumei Wang Yu Liu |
| author_facet | Linhui Wei Yumei Wang Yu Liu |
| author_sort | Linhui Wei |
| collection | DOAJ |
| description | Light field (LF) can capture the spatial and angular information of the light in one single exposure. And the LF images are widely used in various fields, especially in immersive media. The rich imaging information in the LF poses great challenges for transmission. However, LF images are sparse and redundant to some extent, which makes LF compression possible. Besides, the compressed sensing (CS) theory shows that images can be recovered from a small number of measurements when they are sparse. In this paper, we propose a Tensor-based Compressed Sensing method to compress images and Epipolar Plane Images for reconstruction (TCSEPI). This method divides the viewpoints of LF images into several regions and stacks the images in each region into a 4D tensor, which conduct CS together and yields measurements with common characteristics. Subsequently, the epipolar plane images are used to reconstruct the LF images and restore the geometric consistency information. To achieve better reconstruction results, we design two cascaded convolutional neural networks to implement the measurement matrix optimization and LF images reconstruction sequentially. Experimental results show the superior performance of TCSEPI, which achieves at least 3dB gain in PSNR and outperforms state-of-the-art in the reconstruction quality. |
| first_indexed | 2024-12-20T00:38:15Z |
| format | Article |
| id | doaj.art-da80ddc27cc043f9a5dd2004ce0bf403 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-20T00:38:15Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-da80ddc27cc043f9a5dd2004ce0bf4032022-12-21T19:59:41ZengIEEEIEEE Access2169-35362020-01-01813489813491010.1109/ACCESS.2020.30109729145540Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep LearningLinhui Wei0https://orcid.org/0000-0002-6264-7040Yumei Wang1Yu Liu2School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, ChinaSchool of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, ChinaSchool of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, ChinaLight field (LF) can capture the spatial and angular information of the light in one single exposure. And the LF images are widely used in various fields, especially in immersive media. The rich imaging information in the LF poses great challenges for transmission. However, LF images are sparse and redundant to some extent, which makes LF compression possible. Besides, the compressed sensing (CS) theory shows that images can be recovered from a small number of measurements when they are sparse. In this paper, we propose a Tensor-based Compressed Sensing method to compress images and Epipolar Plane Images for reconstruction (TCSEPI). This method divides the viewpoints of LF images into several regions and stacks the images in each region into a 4D tensor, which conduct CS together and yields measurements with common characteristics. Subsequently, the epipolar plane images are used to reconstruct the LF images and restore the geometric consistency information. To achieve better reconstruction results, we design two cascaded convolutional neural networks to implement the measurement matrix optimization and LF images reconstruction sequentially. Experimental results show the superior performance of TCSEPI, which achieves at least 3dB gain in PSNR and outperforms state-of-the-art in the reconstruction quality.https://ieeexplore.ieee.org/document/9145540/Light fieldcompressed sensingepipolar plane imagesconvolutional neural networkimage reconstruction |
| spellingShingle | Linhui Wei Yumei Wang Yu Liu Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep Learning IEEE Access Light field compressed sensing epipolar plane images convolutional neural network image reconstruction |
| title | Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep Learning |
| title_full | Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep Learning |
| title_fullStr | Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep Learning |
| title_full_unstemmed | Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep Learning |
| title_short | Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep Learning |
| title_sort | tensor based light field compressed sensing and epipolar plane images reconstruction via deep learning |
| topic | Light field compressed sensing epipolar plane images convolutional neural network image reconstruction |
| url | https://ieeexplore.ieee.org/document/9145540/ |
| work_keys_str_mv | AT linhuiwei tensorbasedlightfieldcompressedsensingandepipolarplaneimagesreconstructionviadeeplearning AT yumeiwang tensorbasedlightfieldcompressedsensingandepipolarplaneimagesreconstructionviadeeplearning AT yuliu tensorbasedlightfieldcompressedsensingandepipolarplaneimagesreconstructionviadeeplearning |