Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs
RGBN cameras that can capture visible light and near-infrared (NIR) light simultaneously produce better color image quality in low-light-level conditions. However, these RGBN cameras introduce additional color bias caused by the mixing of visible information and NIR information. The color correction...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Photonics Journal |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9200786/ |
| _version_ | 1818410269529866240 |
|---|---|
| author | Zhenghao Han Li Li Weiqi Jin Xia Wang Gangcheng Jiao Hailin Wang |
| author_facet | Zhenghao Han Li Li Weiqi Jin Xia Wang Gangcheng Jiao Hailin Wang |
| author_sort | Zhenghao Han |
| collection | DOAJ |
| description | RGBN cameras that can capture visible light and near-infrared (NIR) light simultaneously produce better color image quality in low-light-level conditions. However, these RGBN cameras introduce additional color bias caused by the mixing of visible information and NIR information. The color correction matrix model widely used in current commercial color digital cameras cannot handle the complicated mapping function between biased color and ground truth color. Convolutional neural networks (CNNs) are good at fitting such complicated relationships, but they require a large quantity of training image pairs of different scenes. In order to achieve satisfactory training results, large amounts of data must be captured manually, even when data augmentation techniques are applied, requiring significant time and effort. Hence, a data generation method for training pairs that are consistent with target RGBN camera parameters, based on an open access RGB-NIR dataset, is proposed. The proposed method is verified by training an RGBN camera color restoration CNN model with generated data. The results show that the CNN model trained with the generated data can achieve satisfactory RGBN color restoration performance with different RGBN sensors. |
| first_indexed | 2024-12-14T10:12:50Z |
| format | Article |
| id | doaj.art-5df80739b29040ebad426e4789f1db75 |
| institution | Directory Open Access Journal |
| issn | 1943-0655 |
| language | English |
| last_indexed | 2024-12-14T10:12:50Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Photonics Journal |
| spelling | doaj.art-5df80739b29040ebad426e4789f1db752022-12-21T23:06:57ZengIEEEIEEE Photonics Journal1943-06552020-01-0112511510.1109/JPHOT.2020.30250889200786Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image PairsZhenghao Han0https://orcid.org/0000-0002-4477-2331Li Li1https://orcid.org/0000-0001-9674-3447Weiqi Jin2https://orcid.org/0000-0002-1147-5242Xia Wang3https://orcid.org/0000-0003-0951-4844Gangcheng Jiao4Hailin Wang5https://orcid.org/0000-0002-0796-1926Key Laboratory of Photo-Electronic Imaging Technology and Systems, School of Optics and Photonics, Ministry of Education of China, Beijing Institute of Technology, Beijing, ChinaKey Laboratory of Photo-Electronic Imaging Technology and Systems, School of Optics and Photonics, Ministry of Education of China, Beijing Institute of Technology, Beijing, ChinaKey Laboratory of Photo-Electronic Imaging Technology and Systems, School of Optics and Photonics, Ministry of Education of China, Beijing Institute of Technology, Beijing, ChinaKey Laboratory of Photo-Electronic Imaging Technology and Systems, School of Optics and Photonics, Ministry of Education of China, Beijing Institute of Technology, Beijing, ChinaScience and Technology of Low-Light-Level Night Vision Laboratory, Xi'an, ChinaKey Laboratory of Photo-Electronic Imaging Technology and Systems, School of Optics and Photonics, Ministry of Education of China, Beijing Institute of Technology, Beijing, ChinaRGBN cameras that can capture visible light and near-infrared (NIR) light simultaneously produce better color image quality in low-light-level conditions. However, these RGBN cameras introduce additional color bias caused by the mixing of visible information and NIR information. The color correction matrix model widely used in current commercial color digital cameras cannot handle the complicated mapping function between biased color and ground truth color. Convolutional neural networks (CNNs) are good at fitting such complicated relationships, but they require a large quantity of training image pairs of different scenes. In order to achieve satisfactory training results, large amounts of data must be captured manually, even when data augmentation techniques are applied, requiring significant time and effort. Hence, a data generation method for training pairs that are consistent with target RGBN camera parameters, based on an open access RGB-NIR dataset, is proposed. The proposed method is verified by training an RGBN camera color restoration CNN model with generated data. The results show that the CNN model trained with the generated data can achieve satisfactory RGBN color restoration performance with different RGBN sensors.https://ieeexplore.ieee.org/document/9200786/Color restorationnear-infraredRGB-NIR cameraconvolutional neural networksimage generationcolor bias model |
| spellingShingle | Zhenghao Han Li Li Weiqi Jin Xia Wang Gangcheng Jiao Hailin Wang Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs IEEE Photonics Journal Color restoration near-infrared RGB-NIR camera convolutional neural networks image generation color bias model |
| title | Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs |
| title_full | Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs |
| title_fullStr | Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs |
| title_full_unstemmed | Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs |
| title_short | Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs |
| title_sort | convolutional neural network training for rgbn camera color restoration using generated image pairs |
| topic | Color restoration near-infrared RGB-NIR camera convolutional neural networks image generation color bias model |
| url | https://ieeexplore.ieee.org/document/9200786/ |
| work_keys_str_mv | AT zhenghaohan convolutionalneuralnetworktrainingforrgbncameracolorrestorationusinggeneratedimagepairs AT lili convolutionalneuralnetworktrainingforrgbncameracolorrestorationusinggeneratedimagepairs AT weiqijin convolutionalneuralnetworktrainingforrgbncameracolorrestorationusinggeneratedimagepairs AT xiawang convolutionalneuralnetworktrainingforrgbncameracolorrestorationusinggeneratedimagepairs AT gangchengjiao convolutionalneuralnetworktrainingforrgbncameracolorrestorationusinggeneratedimagepairs AT hailinwang convolutionalneuralnetworktrainingforrgbncameracolorrestorationusinggeneratedimagepairs |