Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K Conversion
We propose a new hardware-friendly super-resolution (SR) algorithm using computationally simple feature extraction and regression methods, i.e., local binary pattern (LBP) and linear mapping, respectively. The proposed method pre-trains dedicated linear mapping kernels for different texture types of...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9252878/ |
| _version_ | 1818349976265162752 |
|---|---|
| author | Sung-Ho Bae Jae-Hyeong Bae Abdul Muqeet Mst Sirazam Monira Lokwon Kim |
| author_facet | Sung-Ho Bae Jae-Hyeong Bae Abdul Muqeet Mst Sirazam Monira Lokwon Kim |
| author_sort | Sung-Ho Bae |
| collection | DOAJ |
| description | We propose a new hardware-friendly super-resolution (SR) algorithm using computationally simple feature extraction and regression methods, i.e., local binary pattern (LBP) and linear mapping, respectively. The proposed method pre-trains dedicated linear mapping kernels for different texture types of low-resolution (LR) image patches where the texture type is classified based on LBP features. On inference operation, a high-resolution (HR) image patch is reconstructed by multiplying an LR image patch with a linear mapping kernel, which is inferred by the LBP feature class of the corresponding LR patch. Since, the LBP is a highly efficient feature extraction operator for local texture classification, our method is extremely fast and power-efficient while showing competitive reconstruction quality to the latest machine learning-based SR techniques. We also present a fully pipe-lined hardware architecture and its implementation for real-time operations of the proposed SR method. The proposed SR algorithm has been implemented on a field-programmable-gate-array (FPGA) platform including Xilinx KCU105 that can process 63 frames-per-second (fps) while converting full-high-definition (FHD) images to 4K ultra-high-definition (UHD) images. Extensive experimental results show that the proposed proposed algorithm and its hardware implementation can achieve high reconstruction performance compared to the latest machine-learning-based SR methods while utilizing minimum hardware resources, thereby having remarkably less computational complexity. Sometimes, the latest deep-learning-based SR approaches offer slightly higher reconstruction quality, but they require significantly larger amount of hardware resources than the proposed method. |
| first_indexed | 2024-12-13T18:14:30Z |
| format | Article |
| id | doaj.art-284dcf37513045438d4fd3214d975122 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-13T18:14:30Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-284dcf37513045438d4fd3214d9751222022-12-21T23:35:53ZengIEEEIEEE Access2169-35362020-01-01822438322439310.1109/ACCESS.2020.30368289252878Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K ConversionSung-Ho Bae0https://orcid.org/0000-0003-2677-3186Jae-Hyeong Bae1https://orcid.org/0000-0002-4152-3030Abdul Muqeet2https://orcid.org/0000-0002-4803-7000Mst Sirazam Monira3https://orcid.org/0000-0001-6932-5557Lokwon Kim4https://orcid.org/0000-0002-7405-6985Department of Computer Science and Engineering, Kyung Hee University, Yongin-si, South KoreaDepartment of Computer Science and Engineering, Kyung Hee University, Yongin-si, South KoreaDepartment of Computer Science and Engineering, Kyung Hee University, Yongin-si, South KoreaDepartment of Computer Science and Engineering, Kyung Hee University, Yongin-si, South KoreaDepartment of Computer Science and Engineering, Kyung Hee University, Yongin-si, South KoreaWe propose a new hardware-friendly super-resolution (SR) algorithm using computationally simple feature extraction and regression methods, i.e., local binary pattern (LBP) and linear mapping, respectively. The proposed method pre-trains dedicated linear mapping kernels for different texture types of low-resolution (LR) image patches where the texture type is classified based on LBP features. On inference operation, a high-resolution (HR) image patch is reconstructed by multiplying an LR image patch with a linear mapping kernel, which is inferred by the LBP feature class of the corresponding LR patch. Since, the LBP is a highly efficient feature extraction operator for local texture classification, our method is extremely fast and power-efficient while showing competitive reconstruction quality to the latest machine learning-based SR techniques. We also present a fully pipe-lined hardware architecture and its implementation for real-time operations of the proposed SR method. The proposed SR algorithm has been implemented on a field-programmable-gate-array (FPGA) platform including Xilinx KCU105 that can process 63 frames-per-second (fps) while converting full-high-definition (FHD) images to 4K ultra-high-definition (UHD) images. Extensive experimental results show that the proposed proposed algorithm and its hardware implementation can achieve high reconstruction performance compared to the latest machine-learning-based SR methods while utilizing minimum hardware resources, thereby having remarkably less computational complexity. Sometimes, the latest deep-learning-based SR approaches offer slightly higher reconstruction quality, but they require significantly larger amount of hardware resources than the proposed method.https://ieeexplore.ieee.org/document/9252878/Local binary patternreal-timesuper-resolutionUHD |
| spellingShingle | Sung-Ho Bae Jae-Hyeong Bae Abdul Muqeet Mst Sirazam Monira Lokwon Kim Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K Conversion IEEE Access Local binary pattern real-time super-resolution UHD |
| title | Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K Conversion |
| title_full | Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K Conversion |
| title_fullStr | Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K Conversion |
| title_full_unstemmed | Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K Conversion |
| title_short | Cost-Efficient Super-Resolution Hardware Using Local Binary Pattern Classification and Linear Mapping for Real-Time 4K Conversion |
| title_sort | cost efficient super resolution hardware using local binary pattern classification and linear mapping for real time 4k conversion |
| topic | Local binary pattern real-time super-resolution UHD |
| url | https://ieeexplore.ieee.org/document/9252878/ |
| work_keys_str_mv | AT sunghobae costefficientsuperresolutionhardwareusinglocalbinarypatternclassificationandlinearmappingforrealtime4kconversion AT jaehyeongbae costefficientsuperresolutionhardwareusinglocalbinarypatternclassificationandlinearmappingforrealtime4kconversion AT abdulmuqeet costefficientsuperresolutionhardwareusinglocalbinarypatternclassificationandlinearmappingforrealtime4kconversion AT mstsirazammonira costefficientsuperresolutionhardwareusinglocalbinarypatternclassificationandlinearmappingforrealtime4kconversion AT lokwonkim costefficientsuperresolutionhardwareusinglocalbinarypatternclassificationandlinearmappingforrealtime4kconversion |