Learning Non-Parametric Surrogate Losses With Correlated Gradients
Training models by minimizing surrogate loss functions with gradient-based algorithms is a standard approach in various vision tasks. This strategy often leads to suboptimal solutions due to the gap between the target evaluation metrics and surrogate loss functions. In this paper, we propose a frame...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9570322/ |
| _version_ | 1818682075744567296 |
|---|---|
| author | Seungdong Yoa Jinyoung Park Hyunwoo J. Kim |
| author_facet | Seungdong Yoa Jinyoung Park Hyunwoo J. Kim |
| author_sort | Seungdong Yoa |
| collection | DOAJ |
| description | Training models by minimizing surrogate loss functions with gradient-based algorithms is a standard approach in various vision tasks. This strategy often leads to suboptimal solutions due to the gap between the target evaluation metrics and surrogate loss functions. In this paper, we propose a framework to learn a surrogate loss function that approximates the evaluation metric with correlated gradients. We observe that the correlated gradients significantly benefit the gradient-based algorithms to improve the quality of solutions. We verify the effectiveness of our method in various tasks such as multi-class classification, ordinal regression, and pose estimation with three evaluation metrics and five datasets. Our extensive experiments showed that our method outperforms conventional loss functions and surrogate loss learning methods. |
| first_indexed | 2024-12-17T10:13:05Z |
| format | Article |
| id | doaj.art-ca7f6616a3d54055ac07358cd8af428b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-17T10:13:05Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-ca7f6616a3d54055ac07358cd8af428b2022-12-21T21:52:59ZengIEEEIEEE Access2169-35362021-01-01914119914120910.1109/ACCESS.2021.31200929570322Learning Non-Parametric Surrogate Losses With Correlated GradientsSeungdong Yoa0https://orcid.org/0000-0002-2982-0884Jinyoung Park1https://orcid.org/0000-0001-6913-7556Hyunwoo J. Kim2https://orcid.org/0000-0002-2181-9264Department of Computer Science, Korea University, Seoul, Republic of KoreaDepartment of Computer Science, Korea University, Seoul, Republic of KoreaDepartment of Computer Science, Korea University, Seoul, Republic of KoreaTraining models by minimizing surrogate loss functions with gradient-based algorithms is a standard approach in various vision tasks. This strategy often leads to suboptimal solutions due to the gap between the target evaluation metrics and surrogate loss functions. In this paper, we propose a framework to learn a surrogate loss function that approximates the evaluation metric with correlated gradients. We observe that the correlated gradients significantly benefit the gradient-based algorithms to improve the quality of solutions. We verify the effectiveness of our method in various tasks such as multi-class classification, ordinal regression, and pose estimation with three evaluation metrics and five datasets. Our extensive experiments showed that our method outperforms conventional loss functions and surrogate loss learning methods.https://ieeexplore.ieee.org/document/9570322/Learning lossdeep learningmachine learningcomputer vision |
| spellingShingle | Seungdong Yoa Jinyoung Park Hyunwoo J. Kim Learning Non-Parametric Surrogate Losses With Correlated Gradients IEEE Access Learning loss deep learning machine learning computer vision |
| title | Learning Non-Parametric Surrogate Losses With Correlated Gradients |
| title_full | Learning Non-Parametric Surrogate Losses With Correlated Gradients |
| title_fullStr | Learning Non-Parametric Surrogate Losses With Correlated Gradients |
| title_full_unstemmed | Learning Non-Parametric Surrogate Losses With Correlated Gradients |
| title_short | Learning Non-Parametric Surrogate Losses With Correlated Gradients |
| title_sort | learning non parametric surrogate losses with correlated gradients |
| topic | Learning loss deep learning machine learning computer vision |
| url | https://ieeexplore.ieee.org/document/9570322/ |
| work_keys_str_mv | AT seungdongyoa learningnonparametricsurrogatelosseswithcorrelatedgradients AT jinyoungpark learningnonparametricsurrogatelosseswithcorrelatedgradients AT hyunwoojkim learningnonparametricsurrogatelosseswithcorrelatedgradients |