A pluggable single-image super-resolution algorithm based on second-order gradient loss
Convolutional neural networks for single-image super-resolution have been widely used with great success. However, most of these methods use L1 loss to guide network optimization, resulting in blurry restored images with sharp edges smoothed. This is because L1 loss limits the optimization goal of t...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co. Ltd.
2023-12-01
|
| Series: | BenchCouncil Transactions on Benchmarks, Standards and Evaluations |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2772485923000650 |
| _version_ | 1797287590960824320 |
|---|---|
| author | Shuran Lin Chunjie Zhang Yanwu Yang |
| author_facet | Shuran Lin Chunjie Zhang Yanwu Yang |
| author_sort | Shuran Lin |
| collection | DOAJ |
| description | Convolutional neural networks for single-image super-resolution have been widely used with great success. However, most of these methods use L1 loss to guide network optimization, resulting in blurry restored images with sharp edges smoothed. This is because L1 loss limits the optimization goal of the network to the statistical average of all solutions within the solution space of that task. To go beyond the L1 loss, this paper designs an image super-resolution algorithm based on second-order gradient loss. We impose additional constraints by considering the high-order gradient level of the image so that the network can focus on the recovery of fine details such as texture during the learning process. This helps to alleviate the problem of restored image texture over-smoothing to some extent. During network training, we extract the second-order gradient map of the generated image and the target image of the network by minimizing the distance between them, this guides the network to pay attention to the high-frequency detail information in the image and generate a high-resolution image with clearer edge and texture. Besides, the proposed loss function has good embeddability and can be easily integrated with existing image super-resolution networks. Experimental results show that the second-order gradient loss can significantly improve both Learned Perceptual Image Patch Similarity (LPIPS) and Frechet Inception Distance score (FID) performance over other image super-resolution deep learning models. |
| first_indexed | 2024-03-07T18:35:33Z |
| format | Article |
| id | doaj.art-b2d2757381ac451fb261dc1754786e7b |
| institution | Directory Open Access Journal |
| issn | 2772-4859 |
| language | English |
| last_indexed | 2024-03-07T18:35:33Z |
| publishDate | 2023-12-01 |
| publisher | KeAi Communications Co. Ltd. |
| record_format | Article |
| series | BenchCouncil Transactions on Benchmarks, Standards and Evaluations |
| spelling | doaj.art-b2d2757381ac451fb261dc1754786e7b2024-03-02T04:55:24ZengKeAi Communications Co. Ltd.BenchCouncil Transactions on Benchmarks, Standards and Evaluations2772-48592023-12-0134100148A pluggable single-image super-resolution algorithm based on second-order gradient lossShuran Lin0Chunjie Zhang1Yanwu Yang2Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University, Beijing, 100044, China; Institute of Information Science, Beijing Jiaotong University, Beijing, 100044, ChinaBeijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University, Beijing, 100044, China; Institute of Information Science, Beijing Jiaotong University, Beijing, 100044, China; Correspondence to: Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University, No.3 Shangyuancun, Haidian District, Beijing, China.School of Management, Huazhong University of Science and Technology, Wuhan, 430074, ChinaConvolutional neural networks for single-image super-resolution have been widely used with great success. However, most of these methods use L1 loss to guide network optimization, resulting in blurry restored images with sharp edges smoothed. This is because L1 loss limits the optimization goal of the network to the statistical average of all solutions within the solution space of that task. To go beyond the L1 loss, this paper designs an image super-resolution algorithm based on second-order gradient loss. We impose additional constraints by considering the high-order gradient level of the image so that the network can focus on the recovery of fine details such as texture during the learning process. This helps to alleviate the problem of restored image texture over-smoothing to some extent. During network training, we extract the second-order gradient map of the generated image and the target image of the network by minimizing the distance between them, this guides the network to pay attention to the high-frequency detail information in the image and generate a high-resolution image with clearer edge and texture. Besides, the proposed loss function has good embeddability and can be easily integrated with existing image super-resolution networks. Experimental results show that the second-order gradient loss can significantly improve both Learned Perceptual Image Patch Similarity (LPIPS) and Frechet Inception Distance score (FID) performance over other image super-resolution deep learning models.http://www.sciencedirect.com/science/article/pii/S2772485923000650Single-image super-resolutionGradientTextureLoss function |
| spellingShingle | Shuran Lin Chunjie Zhang Yanwu Yang A pluggable single-image super-resolution algorithm based on second-order gradient loss BenchCouncil Transactions on Benchmarks, Standards and Evaluations Single-image super-resolution Gradient Texture Loss function |
| title | A pluggable single-image super-resolution algorithm based on second-order gradient loss |
| title_full | A pluggable single-image super-resolution algorithm based on second-order gradient loss |
| title_fullStr | A pluggable single-image super-resolution algorithm based on second-order gradient loss |
| title_full_unstemmed | A pluggable single-image super-resolution algorithm based on second-order gradient loss |
| title_short | A pluggable single-image super-resolution algorithm based on second-order gradient loss |
| title_sort | pluggable single image super resolution algorithm based on second order gradient loss |
| topic | Single-image super-resolution Gradient Texture Loss function |
| url | http://www.sciencedirect.com/science/article/pii/S2772485923000650 |
| work_keys_str_mv | AT shuranlin apluggablesingleimagesuperresolutionalgorithmbasedonsecondordergradientloss AT chunjiezhang apluggablesingleimagesuperresolutionalgorithmbasedonsecondordergradientloss AT yanwuyang apluggablesingleimagesuperresolutionalgorithmbasedonsecondordergradientloss AT shuranlin pluggablesingleimagesuperresolutionalgorithmbasedonsecondordergradientloss AT chunjiezhang pluggablesingleimagesuperresolutionalgorithmbasedonsecondordergradientloss AT yanwuyang pluggablesingleimagesuperresolutionalgorithmbasedonsecondordergradientloss |