Cross-Validation, Information Theory, or Maximum Likelihood? A Comparison of Tuning Methods for Penalized Splines
Functional data analysis techniques, such as penalized splines, have become common tools used in a variety of applied research settings. Penalized spline estimators are frequently used in applied research to estimate unknown functions from noisy data. The success of these estimators depends on choos...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-09-01
|
| Series: | Stats |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2571-905X/4/3/42 |
| _version_ | 1797517165538050048 |
|---|---|
| author | Lauren N. Berry Nathaniel E. Helwig |
| author_facet | Lauren N. Berry Nathaniel E. Helwig |
| author_sort | Lauren N. Berry |
| collection | DOAJ |
| description | Functional data analysis techniques, such as penalized splines, have become common tools used in a variety of applied research settings. Penalized spline estimators are frequently used in applied research to estimate unknown functions from noisy data. The success of these estimators depends on choosing a tuning parameter that provides the correct balance between fitting and smoothing the data. Several different smoothing parameter selection methods have been proposed for choosing a reasonable tuning parameter. The proposed methods generally fall into one of three categories: cross-validation methods, information theoretic methods, or maximum likelihood methods. Despite the well-known importance of selecting an ideal smoothing parameter, there is little agreement in the literature regarding which method(s) should be considered when analyzing real data. In this paper, we address this issue by exploring the practical performance of six popular tuning methods under a variety of simulated and real data situations. Our results reveal that maximum likelihood methods outperform the popular cross-validation methods in most situations—especially in the presence of correlated errors. Furthermore, our results reveal that the maximum likelihood methods perform well even when the errors are non-Gaussian and/or heteroscedastic. For real data applications, we recommend comparing results using cross-validation and maximum likelihood tuning methods, given that these methods tend to perform similarly (differently) when the model is correctly (incorrectly) specified. |
| first_indexed | 2024-03-10T07:12:55Z |
| format | Article |
| id | doaj.art-ddc807f7d5304b60b23d2e30f9134478 |
| institution | Directory Open Access Journal |
| issn | 2571-905X |
| language | English |
| last_indexed | 2024-03-10T07:12:55Z |
| publishDate | 2021-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Stats |
| spelling | doaj.art-ddc807f7d5304b60b23d2e30f91344782023-11-22T15:18:25ZengMDPI AGStats2571-905X2021-09-014370172410.3390/stats4030042Cross-Validation, Information Theory, or Maximum Likelihood? A Comparison of Tuning Methods for Penalized SplinesLauren N. Berry0Nathaniel E. Helwig1Department of Psychology, University of Minnesota, Minneapolis, MN 55455, USADepartment of Psychology, University of Minnesota, Minneapolis, MN 55455, USAFunctional data analysis techniques, such as penalized splines, have become common tools used in a variety of applied research settings. Penalized spline estimators are frequently used in applied research to estimate unknown functions from noisy data. The success of these estimators depends on choosing a tuning parameter that provides the correct balance between fitting and smoothing the data. Several different smoothing parameter selection methods have been proposed for choosing a reasonable tuning parameter. The proposed methods generally fall into one of three categories: cross-validation methods, information theoretic methods, or maximum likelihood methods. Despite the well-known importance of selecting an ideal smoothing parameter, there is little agreement in the literature regarding which method(s) should be considered when analyzing real data. In this paper, we address this issue by exploring the practical performance of six popular tuning methods under a variety of simulated and real data situations. Our results reveal that maximum likelihood methods outperform the popular cross-validation methods in most situations—especially in the presence of correlated errors. Furthermore, our results reveal that the maximum likelihood methods perform well even when the errors are non-Gaussian and/or heteroscedastic. For real data applications, we recommend comparing results using cross-validation and maximum likelihood tuning methods, given that these methods tend to perform similarly (differently) when the model is correctly (incorrectly) specified.https://www.mdpi.com/2571-905X/4/3/42functional data analysisnonparametric regressionregularizationsmoothing |
| spellingShingle | Lauren N. Berry Nathaniel E. Helwig Cross-Validation, Information Theory, or Maximum Likelihood? A Comparison of Tuning Methods for Penalized Splines Stats functional data analysis nonparametric regression regularization smoothing |
| title | Cross-Validation, Information Theory, or Maximum Likelihood? A Comparison of Tuning Methods for Penalized Splines |
| title_full | Cross-Validation, Information Theory, or Maximum Likelihood? A Comparison of Tuning Methods for Penalized Splines |
| title_fullStr | Cross-Validation, Information Theory, or Maximum Likelihood? A Comparison of Tuning Methods for Penalized Splines |
| title_full_unstemmed | Cross-Validation, Information Theory, or Maximum Likelihood? A Comparison of Tuning Methods for Penalized Splines |
| title_short | Cross-Validation, Information Theory, or Maximum Likelihood? A Comparison of Tuning Methods for Penalized Splines |
| title_sort | cross validation information theory or maximum likelihood a comparison of tuning methods for penalized splines |
| topic | functional data analysis nonparametric regression regularization smoothing |
| url | https://www.mdpi.com/2571-905X/4/3/42 |
| work_keys_str_mv | AT laurennberry crossvalidationinformationtheoryormaximumlikelihoodacomparisonoftuningmethodsforpenalizedsplines AT nathanielehelwig crossvalidationinformationtheoryormaximumlikelihoodacomparisonoftuningmethodsforpenalizedsplines |