Using Akaike's information theoretic criterion in mixed-effects modeling of pharmacokinetic data: a simulation study [version 3; referees: 2 approved, 1 approved with reservations]
Akaike's information theoretic criterion for model discrimination (AIC) is often stated to "overfit", i.e., it selects models with a higher dimension than the dimension of the model that generated the data. However, with experimental pharmacokinetic data it may not be possible to iden...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
F1000 Research Ltd
2015-07-01
|
| Series: | F1000Research |
| Subjects: | |
| Online Access: | http://f1000research.com/articles/2-71/v3 |
| _version_ | 1818535305327673344 |
|---|---|
| author | Erik Olofsen Albert Dahan |
| author_facet | Erik Olofsen Albert Dahan |
| author_sort | Erik Olofsen |
| collection | DOAJ |
| description | Akaike's information theoretic criterion for model discrimination (AIC) is often stated to "overfit", i.e., it selects models with a higher dimension than the dimension of the model that generated the data. However, with experimental pharmacokinetic data it may not be possible to identify the correct model, because of the complexity of the processes governing drug disposition. Instead of trying to find the correct model, a more useful objective might be to minimize the prediction error of drug concentrations in subjects with unknown disposition characteristics. In that case, the AIC might be the selection criterion of choice. We performed Monte Carlo simulations using a model of pharmacokinetic data (a power function of time) with the property that fits with common multi-exponential models can never be perfect - thus resembling the situation with real data. Prespecified models were fitted to simulated data sets, and AIC and AICc (the criterion with a correction for small sample sizes) values were calculated and averaged. The average predictive performances of the models, quantified using simulated validation sets, were compared to the means of the AICs. The data for fits and validation consisted of 11 concentration measurements each obtained in 5 individuals, with three degrees of interindividual variability in the pharmacokinetic volume of distribution. Mean AICc corresponded very well, and better than mean AIC, with mean predictive performance. With increasing interindividual variability, there was a trend towards larger optimal models, but with respect to both lowest AICc and best predictive performance. Furthermore, it was observed that the mean square prediction error itself became less suitable as a validation criterion, and that a predictive performance measure should incorporate interindividual variability. This simulation study showed that, at least in a relatively simple mixed-effects modelling context with a set of prespecified models, minimal mean AICc corresponded to best predictive performance even in the presence of relatively large interindividual variability. |
| first_indexed | 2024-12-11T18:23:04Z |
| format | Article |
| id | doaj.art-0ede7ef444d6410da8a618ebecce561f |
| institution | Directory Open Access Journal |
| issn | 2046-1402 |
| language | English |
| last_indexed | 2024-12-11T18:23:04Z |
| publishDate | 2015-07-01 |
| publisher | F1000 Research Ltd |
| record_format | Article |
| series | F1000Research |
| spelling | doaj.art-0ede7ef444d6410da8a618ebecce561f2022-12-22T00:55:11ZengF1000 Research LtdF1000Research2046-14022015-07-01210.12688/f1000research.2-71.v37312Using Akaike's information theoretic criterion in mixed-effects modeling of pharmacokinetic data: a simulation study [version 3; referees: 2 approved, 1 approved with reservations]Erik Olofsen0Albert Dahan1Department of Anesthesiology, Leiden University Medical Center, PO Box 9600, 2300 RC Leiden, NetherlandsDepartment of Anesthesiology, Leiden University Medical Center, PO Box 9600, 2300 RC Leiden, NetherlandsAkaike's information theoretic criterion for model discrimination (AIC) is often stated to "overfit", i.e., it selects models with a higher dimension than the dimension of the model that generated the data. However, with experimental pharmacokinetic data it may not be possible to identify the correct model, because of the complexity of the processes governing drug disposition. Instead of trying to find the correct model, a more useful objective might be to minimize the prediction error of drug concentrations in subjects with unknown disposition characteristics. In that case, the AIC might be the selection criterion of choice. We performed Monte Carlo simulations using a model of pharmacokinetic data (a power function of time) with the property that fits with common multi-exponential models can never be perfect - thus resembling the situation with real data. Prespecified models were fitted to simulated data sets, and AIC and AICc (the criterion with a correction for small sample sizes) values were calculated and averaged. The average predictive performances of the models, quantified using simulated validation sets, were compared to the means of the AICs. The data for fits and validation consisted of 11 concentration measurements each obtained in 5 individuals, with three degrees of interindividual variability in the pharmacokinetic volume of distribution. Mean AICc corresponded very well, and better than mean AIC, with mean predictive performance. With increasing interindividual variability, there was a trend towards larger optimal models, but with respect to both lowest AICc and best predictive performance. Furthermore, it was observed that the mean square prediction error itself became less suitable as a validation criterion, and that a predictive performance measure should incorporate interindividual variability. This simulation study showed that, at least in a relatively simple mixed-effects modelling context with a set of prespecified models, minimal mean AICc corresponded to best predictive performance even in the presence of relatively large interindividual variability.http://f1000research.com/articles/2-71/v3Pharmacokinetics & Drug Delivery |
| spellingShingle | Erik Olofsen Albert Dahan Using Akaike's information theoretic criterion in mixed-effects modeling of pharmacokinetic data: a simulation study [version 3; referees: 2 approved, 1 approved with reservations] F1000Research Pharmacokinetics & Drug Delivery |
| title | Using Akaike's information theoretic criterion in mixed-effects modeling of pharmacokinetic data: a simulation study [version 3; referees: 2 approved, 1 approved with reservations] |
| title_full | Using Akaike's information theoretic criterion in mixed-effects modeling of pharmacokinetic data: a simulation study [version 3; referees: 2 approved, 1 approved with reservations] |
| title_fullStr | Using Akaike's information theoretic criterion in mixed-effects modeling of pharmacokinetic data: a simulation study [version 3; referees: 2 approved, 1 approved with reservations] |
| title_full_unstemmed | Using Akaike's information theoretic criterion in mixed-effects modeling of pharmacokinetic data: a simulation study [version 3; referees: 2 approved, 1 approved with reservations] |
| title_short | Using Akaike's information theoretic criterion in mixed-effects modeling of pharmacokinetic data: a simulation study [version 3; referees: 2 approved, 1 approved with reservations] |
| title_sort | using akaike s information theoretic criterion in mixed effects modeling of pharmacokinetic data a simulation study version 3 referees 2 approved 1 approved with reservations |
| topic | Pharmacokinetics & Drug Delivery |
| url | http://f1000research.com/articles/2-71/v3 |
| work_keys_str_mv | AT erikolofsen usingakaikesinformationtheoreticcriterioninmixedeffectsmodelingofpharmacokineticdataasimulationstudyversion3referees2approved1approvedwithreservations AT albertdahan usingakaikesinformationtheoreticcriterioninmixedeffectsmodelingofpharmacokineticdataasimulationstudyversion3referees2approved1approvedwithreservations |