TMO-Net: A Parameter-Free Tone Mapping Operator Using Generative Adversarial Network, and Performance Benchmarking on Large Scale HDR Dataset
Currently published tone mapping operators (TMO) are often evaluated on a very limited test set of high dynamic range (HDR) images. Thus, the resulting performance index is highly subject to extensive hyperparameter tuning, and many TMOs exhibit sub-optimal performance when tested on a broader spect...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9371686/ |
| _version_ | 1818480054082994176 |
|---|---|
| author | Karen Panetta Landry Kezebou Victor Oludare Sos Agaian Zehua Xia |
| author_facet | Karen Panetta Landry Kezebou Victor Oludare Sos Agaian Zehua Xia |
| author_sort | Karen Panetta |
| collection | DOAJ |
| description | Currently published tone mapping operators (TMO) are often evaluated on a very limited test set of high dynamic range (HDR) images. Thus, the resulting performance index is highly subject to extensive hyperparameter tuning, and many TMOs exhibit sub-optimal performance when tested on a broader spectrum of HDR images. This indicates that there are deficiencies in the generalizable applicability of these techniques. Finally, it is a challenge developing parameter-free tone mapping operators using data-hungry advanced deep learning methods due to the paucity of large scale HDR datasets. In this paper, these issues are addressed through the following contributions: a) a large scale HDR image benchmark dataset (LVZ-HDR dataset) with multiple variations in sceneries and lighting conditions is created to enable performance evaluation of TMOs across a diverse conditions and scenes that will also contribute to facilitate the development of more robust TMOs using state-of-the-art deep learning methods; b) a deep learning-based tone mapping operator (TMO-Net) is presented, which offers an efficient and parameter-free method capable of generalizing effectively across a wider spectrum of HDR content; c) finally, a comparative analysis, and performance benchmarking of 19 state-of-the-art TMOs on the new LVZ-HDR dataset are presented. Standard metrics including the Tone Mapping Quality Index (TMQI), Feature Similarity Index for Tone Mapped images (FSITM), and Natural Image Quality Evaluator (NIQE) are used to qualitatively evaluate the performance index of the benchmarked TMOs. Experimental results demonstrate that the proposed TMO-Net qualitatively and quantitatively outperforms current state-of-the-art TMOs. |
| first_indexed | 2024-12-10T11:18:01Z |
| format | Article |
| id | doaj.art-6f495f8b1bec42e790350d9e26ac8c71 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-10T11:18:01Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-6f495f8b1bec42e790350d9e26ac8c712022-12-22T01:51:05ZengIEEEIEEE Access2169-35362021-01-019395003951710.1109/ACCESS.2021.30642959371686TMO-Net: A Parameter-Free Tone Mapping Operator Using Generative Adversarial Network, and Performance Benchmarking on Large Scale HDR DatasetKaren Panetta0Landry Kezebou1https://orcid.org/0000-0003-0027-0767Victor Oludare2Sos Agaian3https://orcid.org/0000-0003-4601-4507Zehua Xia4https://orcid.org/0000-0003-2073-2340Department of Electrical and Computer Engineering, Tufts University, Medford, MA, USADepartment of Electrical and Computer Engineering, Tufts University, Medford, MA, USADepartment of Electrical and Computer Engineering, Tufts University, Medford, MA, USADepartment of Computer Science, City University New~York (CUNY), New York, NY, USADepartment of Electrical and Computer Engineering, Tufts University, Medford, MA, USACurrently published tone mapping operators (TMO) are often evaluated on a very limited test set of high dynamic range (HDR) images. Thus, the resulting performance index is highly subject to extensive hyperparameter tuning, and many TMOs exhibit sub-optimal performance when tested on a broader spectrum of HDR images. This indicates that there are deficiencies in the generalizable applicability of these techniques. Finally, it is a challenge developing parameter-free tone mapping operators using data-hungry advanced deep learning methods due to the paucity of large scale HDR datasets. In this paper, these issues are addressed through the following contributions: a) a large scale HDR image benchmark dataset (LVZ-HDR dataset) with multiple variations in sceneries and lighting conditions is created to enable performance evaluation of TMOs across a diverse conditions and scenes that will also contribute to facilitate the development of more robust TMOs using state-of-the-art deep learning methods; b) a deep learning-based tone mapping operator (TMO-Net) is presented, which offers an efficient and parameter-free method capable of generalizing effectively across a wider spectrum of HDR content; c) finally, a comparative analysis, and performance benchmarking of 19 state-of-the-art TMOs on the new LVZ-HDR dataset are presented. Standard metrics including the Tone Mapping Quality Index (TMQI), Feature Similarity Index for Tone Mapped images (FSITM), and Natural Image Quality Evaluator (NIQE) are used to qualitatively evaluate the performance index of the benchmarked TMOs. Experimental results demonstrate that the proposed TMO-Net qualitatively and quantitatively outperforms current state-of-the-art TMOs.https://ieeexplore.ieee.org/document/9371686/Deep learningGAN-based tone mappingTMO benchmarkingtone mappingTMO-Net HDR datasetparameter-free tone mapping |
| spellingShingle | Karen Panetta Landry Kezebou Victor Oludare Sos Agaian Zehua Xia TMO-Net: A Parameter-Free Tone Mapping Operator Using Generative Adversarial Network, and Performance Benchmarking on Large Scale HDR Dataset IEEE Access Deep learning GAN-based tone mapping TMO benchmarking tone mapping TMO-Net HDR dataset parameter-free tone mapping |
| title | TMO-Net: A Parameter-Free Tone Mapping Operator Using Generative Adversarial Network, and Performance Benchmarking on Large Scale HDR Dataset |
| title_full | TMO-Net: A Parameter-Free Tone Mapping Operator Using Generative Adversarial Network, and Performance Benchmarking on Large Scale HDR Dataset |
| title_fullStr | TMO-Net: A Parameter-Free Tone Mapping Operator Using Generative Adversarial Network, and Performance Benchmarking on Large Scale HDR Dataset |
| title_full_unstemmed | TMO-Net: A Parameter-Free Tone Mapping Operator Using Generative Adversarial Network, and Performance Benchmarking on Large Scale HDR Dataset |
| title_short | TMO-Net: A Parameter-Free Tone Mapping Operator Using Generative Adversarial Network, and Performance Benchmarking on Large Scale HDR Dataset |
| title_sort | tmo net a parameter free tone mapping operator using generative adversarial network and performance benchmarking on large scale hdr dataset |
| topic | Deep learning GAN-based tone mapping TMO benchmarking tone mapping TMO-Net HDR dataset parameter-free tone mapping |
| url | https://ieeexplore.ieee.org/document/9371686/ |
| work_keys_str_mv | AT karenpanetta tmonetaparameterfreetonemappingoperatorusinggenerativeadversarialnetworkandperformancebenchmarkingonlargescalehdrdataset AT landrykezebou tmonetaparameterfreetonemappingoperatorusinggenerativeadversarialnetworkandperformancebenchmarkingonlargescalehdrdataset AT victoroludare tmonetaparameterfreetonemappingoperatorusinggenerativeadversarialnetworkandperformancebenchmarkingonlargescalehdrdataset AT sosagaian tmonetaparameterfreetonemappingoperatorusinggenerativeadversarialnetworkandperformancebenchmarkingonlargescalehdrdataset AT zehuaxia tmonetaparameterfreetonemappingoperatorusinggenerativeadversarialnetworkandperformancebenchmarkingonlargescalehdrdataset |