Gradient Learning under Tilted Empirical Risk Minimization

Gradient Learning (GL), aiming to estimate the gradient of target function, has attracted much attention in variable selection problems due to its mild structure requirements and wide applicability. Despite rapid progress, the majority of the existing GL works are based on the empirical risk minimiz...

Full description

Bibliographic Details
Main Authors: Liyuan Liu, Biqin Song, Zhibin Pan, Chuanwu Yang, Chi Xiao, Weifu Li
Format: Article
Language:English
Published: MDPI AG 2022-07-01
Series:Entropy
Subjects:
Online Access:https://www.mdpi.com/1099-4300/24/7/956
Description
Summary:Gradient Learning (GL), aiming to estimate the gradient of target function, has attracted much attention in variable selection problems due to its mild structure requirements and wide applicability. Despite rapid progress, the majority of the existing GL works are based on the empirical risk minimization (ERM) principle, which may face the degraded performance under complex data environment, e.g., non-Gaussian noise. To alleviate this sensitiveness, we propose a new GL model with the help of the tilted ERM criterion, and establish its theoretical support from the function approximation viewpoint. Specifically, the operator approximation technique plays the crucial role in our analysis. To solve the proposed learning objective, a gradient descent method is proposed, and the convergence analysis is provided. Finally, simulated experimental results validate the effectiveness of our approach when the input variables are correlated.
ISSN:1099-4300