Development and Validation of Multivariable Prediction Algorithms to Estimate Future Walking Behavior in Adults: Retrospective Cohort Study
BackgroundPhysical inactivity is associated with numerous health risks, including cancer, cardiovascular disease, type 2 diabetes, increased health care expenditure, and preventable, premature deaths. The majority of Americans fall short of clinical guideline goals (ie, 8000-...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2023-01-01
|
| Series: | JMIR mHealth and uHealth |
| Online Access: | https://mhealth.jmir.org/2023/1/e44296 |
| _version_ | 1797734466613936128 |
|---|---|
| author | Junghwan Park Gregory J Norman Predrag Klasnja Daniel E Rivera Eric Hekler |
| author_facet | Junghwan Park Gregory J Norman Predrag Klasnja Daniel E Rivera Eric Hekler |
| author_sort | Junghwan Park |
| collection | DOAJ |
| description |
BackgroundPhysical inactivity is associated with numerous health risks, including cancer, cardiovascular disease, type 2 diabetes, increased health care expenditure, and preventable, premature deaths. The majority of Americans fall short of clinical guideline goals (ie, 8000-10,000 steps per day). Behavior prediction algorithms could enable efficacious interventions to promote physical activity by facilitating delivery of nudges at appropriate times.
ObjectiveThe aim of this paper is to develop and validate algorithms that predict walking (ie, >5 min) within the next 3 hours, predicted from the participants’ previous 5 weeks’ steps-per-minute data.
MethodsWe conducted a retrospective, closed cohort, secondary analysis of a 6-week microrandomized trial of the HeartSteps mobile health physical-activity intervention conducted in 2015. The prediction performance of 6 algorithms was evaluated, as follows: logistic regression, radial-basis function support vector machine, eXtreme Gradient Boosting (XGBoost), multilayered perceptron (MLP), decision tree, and random forest. For the MLP, 90 random layer architectures were tested for optimization. Prior 5-week hourly walking data, including missingness, were used for predictors. Whether the participant walked during the next 3 hours was used as the outcome. K-fold cross-validation (K=10) was used for the internal validation. The primary outcome measures are classification accuracy, the Mathew correlation coefficient, sensitivity, and specificity.
ResultsThe total sample size included 6 weeks of data among 44 participants. Of the 44 participants, 31 (71%) were female, 26 (59%) were White, 36 (82%) had a college degree or more, and 15 (34%) were married. The mean age was 35.9 (SD 14.7) years. Participants (n=3, 7%) who did not have enough data (number of days <10) were excluded, resulting in 41 (93%) participants. MLP with optimized layer architecture showed the best performance in accuracy (82.0%, SD 1.1), whereas XGBoost (76.3%, SD 1.5), random forest (69.5%, SD 1.0), support vector machine (69.3%, SD 1.0), and decision tree (63.6%, SD 1.5) algorithms showed lower performance than logistic regression (77.2%, SD 1.2). MLP also showed superior overall performance to all other tried algorithms in Mathew correlation coefficient (0.643, SD 0.021), sensitivity (86.1%, SD 3.0), and specificity (77.8%, SD 3.3).
ConclusionsWalking behavior prediction models were developed and validated. MLP showed the highest overall performance of all attempted algorithms. A random search for optimal layer structure is a promising approach for prediction engine development. Future studies can test the real-world application of this algorithm in a “smart” intervention for promoting physical activity. |
| first_indexed | 2024-03-12T12:44:37Z |
| format | Article |
| id | doaj.art-8f8d1c0643d34fb5a165fe2e5c11329e |
| institution | Directory Open Access Journal |
| issn | 2291-5222 |
| language | English |
| last_indexed | 2024-03-12T12:44:37Z |
| publishDate | 2023-01-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR mHealth and uHealth |
| spelling | doaj.art-8f8d1c0643d34fb5a165fe2e5c11329e2023-08-28T23:31:07ZengJMIR PublicationsJMIR mHealth and uHealth2291-52222023-01-0111e4429610.2196/44296Development and Validation of Multivariable Prediction Algorithms to Estimate Future Walking Behavior in Adults: Retrospective Cohort StudyJunghwan Parkhttps://orcid.org/0000-0003-0638-0457Gregory J Normanhttps://orcid.org/0000-0001-7989-9597Predrag Klasnjahttps://orcid.org/0000-0002-4570-703XDaniel E Riverahttps://orcid.org/0000-0002-3141-0577Eric Heklerhttps://orcid.org/0000-0002-7434-0775 BackgroundPhysical inactivity is associated with numerous health risks, including cancer, cardiovascular disease, type 2 diabetes, increased health care expenditure, and preventable, premature deaths. The majority of Americans fall short of clinical guideline goals (ie, 8000-10,000 steps per day). Behavior prediction algorithms could enable efficacious interventions to promote physical activity by facilitating delivery of nudges at appropriate times. ObjectiveThe aim of this paper is to develop and validate algorithms that predict walking (ie, >5 min) within the next 3 hours, predicted from the participants’ previous 5 weeks’ steps-per-minute data. MethodsWe conducted a retrospective, closed cohort, secondary analysis of a 6-week microrandomized trial of the HeartSteps mobile health physical-activity intervention conducted in 2015. The prediction performance of 6 algorithms was evaluated, as follows: logistic regression, radial-basis function support vector machine, eXtreme Gradient Boosting (XGBoost), multilayered perceptron (MLP), decision tree, and random forest. For the MLP, 90 random layer architectures were tested for optimization. Prior 5-week hourly walking data, including missingness, were used for predictors. Whether the participant walked during the next 3 hours was used as the outcome. K-fold cross-validation (K=10) was used for the internal validation. The primary outcome measures are classification accuracy, the Mathew correlation coefficient, sensitivity, and specificity. ResultsThe total sample size included 6 weeks of data among 44 participants. Of the 44 participants, 31 (71%) were female, 26 (59%) were White, 36 (82%) had a college degree or more, and 15 (34%) were married. The mean age was 35.9 (SD 14.7) years. Participants (n=3, 7%) who did not have enough data (number of days <10) were excluded, resulting in 41 (93%) participants. MLP with optimized layer architecture showed the best performance in accuracy (82.0%, SD 1.1), whereas XGBoost (76.3%, SD 1.5), random forest (69.5%, SD 1.0), support vector machine (69.3%, SD 1.0), and decision tree (63.6%, SD 1.5) algorithms showed lower performance than logistic regression (77.2%, SD 1.2). MLP also showed superior overall performance to all other tried algorithms in Mathew correlation coefficient (0.643, SD 0.021), sensitivity (86.1%, SD 3.0), and specificity (77.8%, SD 3.3). ConclusionsWalking behavior prediction models were developed and validated. MLP showed the highest overall performance of all attempted algorithms. A random search for optimal layer structure is a promising approach for prediction engine development. Future studies can test the real-world application of this algorithm in a “smart” intervention for promoting physical activity.https://mhealth.jmir.org/2023/1/e44296 |
| spellingShingle | Junghwan Park Gregory J Norman Predrag Klasnja Daniel E Rivera Eric Hekler Development and Validation of Multivariable Prediction Algorithms to Estimate Future Walking Behavior in Adults: Retrospective Cohort Study JMIR mHealth and uHealth |
| title | Development and Validation of Multivariable Prediction Algorithms to Estimate Future Walking Behavior in Adults: Retrospective Cohort Study |
| title_full | Development and Validation of Multivariable Prediction Algorithms to Estimate Future Walking Behavior in Adults: Retrospective Cohort Study |
| title_fullStr | Development and Validation of Multivariable Prediction Algorithms to Estimate Future Walking Behavior in Adults: Retrospective Cohort Study |
| title_full_unstemmed | Development and Validation of Multivariable Prediction Algorithms to Estimate Future Walking Behavior in Adults: Retrospective Cohort Study |
| title_short | Development and Validation of Multivariable Prediction Algorithms to Estimate Future Walking Behavior in Adults: Retrospective Cohort Study |
| title_sort | development and validation of multivariable prediction algorithms to estimate future walking behavior in adults retrospective cohort study |
| url | https://mhealth.jmir.org/2023/1/e44296 |
| work_keys_str_mv | AT junghwanpark developmentandvalidationofmultivariablepredictionalgorithmstoestimatefuturewalkingbehaviorinadultsretrospectivecohortstudy AT gregoryjnorman developmentandvalidationofmultivariablepredictionalgorithmstoestimatefuturewalkingbehaviorinadultsretrospectivecohortstudy AT predragklasnja developmentandvalidationofmultivariablepredictionalgorithmstoestimatefuturewalkingbehaviorinadultsretrospectivecohortstudy AT danielerivera developmentandvalidationofmultivariablepredictionalgorithmstoestimatefuturewalkingbehaviorinadultsretrospectivecohortstudy AT erichekler developmentandvalidationofmultivariablepredictionalgorithmstoestimatefuturewalkingbehaviorinadultsretrospectivecohortstudy |