From raw data to a score: comparing quantitative methods that construct multi-level composite implementation strength scores of family planning programs in Malawi
Abstract Background Data that capture implementation strength can be combined in multiple ways across content and health system levels to create a summary measure that can help us to explore and compare program implementation across facility catchment areas. Summary indices can make it easier for na...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-09-01
|
| Series: | Population Health Metrics |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12963-022-00295-2 |
| _version_ | 1811297479588577280 |
|---|---|
| author | Anooj Pattnaik Diwakar Mohan Scott Zeger Mercy Kanyuka Fannie Kachale Melissa A. Marx |
| author_facet | Anooj Pattnaik Diwakar Mohan Scott Zeger Mercy Kanyuka Fannie Kachale Melissa A. Marx |
| author_sort | Anooj Pattnaik |
| collection | DOAJ |
| description | Abstract Background Data that capture implementation strength can be combined in multiple ways across content and health system levels to create a summary measure that can help us to explore and compare program implementation across facility catchment areas. Summary indices can make it easier for national policymakers to understand and address variation in strength of program implementation across jurisdictions. In this paper, we describe the development of an index that we used to describe the district-level strength of implementation of Malawi’s national family planning program. Methods To develop the index, we used data collected during a 2017 national, health facility and community health worker Implementation Strength Assessment survey in Malawi to test different methods to combine indicators within and then across domains (4 methods—simple additive, weighted additive, principal components analysis, exploratory factor analysis) and combine scores across health facility and community health worker levels (2 methods—simple average and mixed effects model) to create a catchment area-level summary score for each health facility in Malawi. We explored how well each model captures variation and predicts couple-years protection and how feasible it is to conduct each type of analysis and the resulting interpretability. Results We found little difference in how the four methods combined indicator data at the individual and combined levels of the health system. However, there were major differences when combining scores across health system levels to obtain a score at the health facility catchment area level. The scores resulting from the mixed effects model were able to better discriminate differences between catchment area scores compared to the simple average method. The scores using the mixed effects combination method also demonstrated more of a dose–response relationship with couple-years protection. Conclusions The summary measure that was calculated from the mixed effects combination method captured the variation of strength of implementation of Malawi’s national family planning program at the health facility catchment area level. However, the best method for creating an index should be based on the pros and cons listed, not least, analyst capacity and ease of interpretability of findings. Ultimately, the resulting summary measure can aid decision-makers in understanding the combined effect of multiple aspects of programs being implemented in their health system and comparing the strengths of programs across geographies. |
| first_indexed | 2024-04-13T06:05:53Z |
| format | Article |
| id | doaj.art-270265abe5ec4460a2b537f6e82152d2 |
| institution | Directory Open Access Journal |
| issn | 1478-7954 |
| language | English |
| last_indexed | 2024-04-13T06:05:53Z |
| publishDate | 2022-09-01 |
| publisher | BMC |
| record_format | Article |
| series | Population Health Metrics |
| spelling | doaj.art-270265abe5ec4460a2b537f6e82152d22022-12-22T02:59:16ZengBMCPopulation Health Metrics1478-79542022-09-0120111210.1186/s12963-022-00295-2From raw data to a score: comparing quantitative methods that construct multi-level composite implementation strength scores of family planning programs in MalawiAnooj Pattnaik0Diwakar Mohan1Scott Zeger2Mercy Kanyuka3Fannie Kachale4Melissa A. Marx5Department of International Health, Johns Hopkins Bloomberg School of Public HealthDepartment of International Health, Johns Hopkins Bloomberg School of Public HealthDepartment of International Health, Johns Hopkins Bloomberg School of Public HealthNational Statistical OfficeReproductive Health Directorate, Ministry of HealthDepartment of International Health, Johns Hopkins Bloomberg School of Public HealthAbstract Background Data that capture implementation strength can be combined in multiple ways across content and health system levels to create a summary measure that can help us to explore and compare program implementation across facility catchment areas. Summary indices can make it easier for national policymakers to understand and address variation in strength of program implementation across jurisdictions. In this paper, we describe the development of an index that we used to describe the district-level strength of implementation of Malawi’s national family planning program. Methods To develop the index, we used data collected during a 2017 national, health facility and community health worker Implementation Strength Assessment survey in Malawi to test different methods to combine indicators within and then across domains (4 methods—simple additive, weighted additive, principal components analysis, exploratory factor analysis) and combine scores across health facility and community health worker levels (2 methods—simple average and mixed effects model) to create a catchment area-level summary score for each health facility in Malawi. We explored how well each model captures variation and predicts couple-years protection and how feasible it is to conduct each type of analysis and the resulting interpretability. Results We found little difference in how the four methods combined indicator data at the individual and combined levels of the health system. However, there were major differences when combining scores across health system levels to obtain a score at the health facility catchment area level. The scores resulting from the mixed effects model were able to better discriminate differences between catchment area scores compared to the simple average method. The scores using the mixed effects combination method also demonstrated more of a dose–response relationship with couple-years protection. Conclusions The summary measure that was calculated from the mixed effects combination method captured the variation of strength of implementation of Malawi’s national family planning program at the health facility catchment area level. However, the best method for creating an index should be based on the pros and cons listed, not least, analyst capacity and ease of interpretability of findings. Ultimately, the resulting summary measure can aid decision-makers in understanding the combined effect of multiple aspects of programs being implemented in their health system and comparing the strengths of programs across geographies.https://doi.org/10.1186/s12963-022-00295-2MalawiImplementation strengthQualityHealth workerFamily planningSummary measure |
| spellingShingle | Anooj Pattnaik Diwakar Mohan Scott Zeger Mercy Kanyuka Fannie Kachale Melissa A. Marx From raw data to a score: comparing quantitative methods that construct multi-level composite implementation strength scores of family planning programs in Malawi Population Health Metrics Malawi Implementation strength Quality Health worker Family planning Summary measure |
| title | From raw data to a score: comparing quantitative methods that construct multi-level composite implementation strength scores of family planning programs in Malawi |
| title_full | From raw data to a score: comparing quantitative methods that construct multi-level composite implementation strength scores of family planning programs in Malawi |
| title_fullStr | From raw data to a score: comparing quantitative methods that construct multi-level composite implementation strength scores of family planning programs in Malawi |
| title_full_unstemmed | From raw data to a score: comparing quantitative methods that construct multi-level composite implementation strength scores of family planning programs in Malawi |
| title_short | From raw data to a score: comparing quantitative methods that construct multi-level composite implementation strength scores of family planning programs in Malawi |
| title_sort | from raw data to a score comparing quantitative methods that construct multi level composite implementation strength scores of family planning programs in malawi |
| topic | Malawi Implementation strength Quality Health worker Family planning Summary measure |
| url | https://doi.org/10.1186/s12963-022-00295-2 |
| work_keys_str_mv | AT anoojpattnaik fromrawdatatoascorecomparingquantitativemethodsthatconstructmultilevelcompositeimplementationstrengthscoresoffamilyplanningprogramsinmalawi AT diwakarmohan fromrawdatatoascorecomparingquantitativemethodsthatconstructmultilevelcompositeimplementationstrengthscoresoffamilyplanningprogramsinmalawi AT scottzeger fromrawdatatoascorecomparingquantitativemethodsthatconstructmultilevelcompositeimplementationstrengthscoresoffamilyplanningprogramsinmalawi AT mercykanyuka fromrawdatatoascorecomparingquantitativemethodsthatconstructmultilevelcompositeimplementationstrengthscoresoffamilyplanningprogramsinmalawi AT fanniekachale fromrawdatatoascorecomparingquantitativemethodsthatconstructmultilevelcompositeimplementationstrengthscoresoffamilyplanningprogramsinmalawi AT melissaamarx fromrawdatatoascorecomparingquantitativemethodsthatconstructmultilevelcompositeimplementationstrengthscoresoffamilyplanningprogramsinmalawi |