The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis
Background: As part of the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process, independent Evidence Review Groups (ERGs) critically appraise the company submission. During the critical appraisal process the ERG may undertake analyses to explore uncerta...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
NIHR Journals Library
2016-04-01
|
| Series: | Health Technology Assessment |
| Subjects: | |
| Online Access: | https://doi.org/10.3310/hta20260 |
| _version_ | 1818147222816030720 |
|---|---|
| author | Eva Kaltenthaler Christopher Carroll Daniel Hill-McManus Alison Scope Michael Holmes Stephen Rice Micah Rose Paul Tappenden Nerys Woolacott |
| author_facet | Eva Kaltenthaler Christopher Carroll Daniel Hill-McManus Alison Scope Michael Holmes Stephen Rice Micah Rose Paul Tappenden Nerys Woolacott |
| author_sort | Eva Kaltenthaler |
| collection | DOAJ |
| description | Background: As part of the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process, independent Evidence Review Groups (ERGs) critically appraise the company submission. During the critical appraisal process the ERG may undertake analyses to explore uncertainties around the company’s model and their implications for decision-making. The ERG reports are a central component of the evidence considered by the NICE Technology Appraisal Committees (ACs) in their deliberations. Objective: The aim of this research was to develop an understanding of the number and type of exploratory analyses undertaken by the ERGs within the STA process and to understand how these analyses are used by the NICE ACs in their decision-making. Methods: The 100 most recently completed STAs with published guidance were selected for inclusion in the analysis. The documents considered were ERG reports, clarification letters, the first appraisal consultation document and the final appraisal determination. Over 400 documents were assessed in this study. The categories of types of exploratory analyses included fixing errors, fixing violations, addressing matters of judgement and the ERG-preferred base case. A content analysis of documents (documentary analysis) was undertaken to identify and extract relevant data, and narrative synthesis was then used to rationalise and present these data. Results: The level and type of detail in ERG reports and clarification letters varied considerably. The vast majority (93%) of ERG reports reported one or more exploratory analyses. The most frequently reported type of analysis in these 93 ERG reports related to the category ‘matters of judgement’, which was reported in 83 (89%) reports. The category ‘ERG base-case/preferred analysis’ was reported in 45 (48%) reports, the category ‘fixing errors’ was reported in 33 (35%) reports and the category ‘fixing violations’ was reported in 17 (18%) reports. The exploratory analyses performed were the result of issues raised by an ERG in its critique of the submitted economic evidence. These analyses had more influence on recommendations earlier in the STA process than later on in the process. Limitations: The descriptions of analyses undertaken were often highly specific to a particular STA and could be inconsistent across ERG reports and thus difficult to interpret. Conclusions: Evidence Review Groups frequently conduct exploratory analyses to test or improve the economic evaluations submitted by companies as part of the STA process. ERG exploratory analyses often have an influence on the recommendations produced by the ACs. Future work: More in-depth analysis is needed to understand how ERGs make decisions regarding which exploratory analyses should be undertaken. More research is also needed to fully understand which types of exploratory analyses are most useful to ACs in their decision-making. Funding: The National Institute for Health Research Health Technology Assessment programme. |
| first_indexed | 2024-12-11T12:31:49Z |
| format | Article |
| id | doaj.art-c7fa781d00ea4a80927327a25eec33a4 |
| institution | Directory Open Access Journal |
| issn | 1366-5278 2046-4924 |
| language | English |
| last_indexed | 2024-12-11T12:31:49Z |
| publishDate | 2016-04-01 |
| publisher | NIHR Journals Library |
| record_format | Article |
| series | Health Technology Assessment |
| spelling | doaj.art-c7fa781d00ea4a80927327a25eec33a42022-12-22T01:07:13ZengNIHR Journals LibraryHealth Technology Assessment1366-52782046-49242016-04-01202610.3310/hta2026014/151/04The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysisEva Kaltenthaler0Christopher Carroll1Daniel Hill-McManus2Alison Scope3Michael Holmes4Stephen Rice5Micah Rose6Paul Tappenden7Nerys Woolacott8School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UKSchool of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UKSchool of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UKSchool of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UKSchool of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UKHealth Economics Group, Newcastle University, Newcastle upon Tyne, UKSouthampton Health Technology Assessments Centre (SHTAC), University of Southampton, Southampton, UKSchool of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UKCentre for Reviews and Dissemination (CRD), University of York, York, UKBackground: As part of the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process, independent Evidence Review Groups (ERGs) critically appraise the company submission. During the critical appraisal process the ERG may undertake analyses to explore uncertainties around the company’s model and their implications for decision-making. The ERG reports are a central component of the evidence considered by the NICE Technology Appraisal Committees (ACs) in their deliberations. Objective: The aim of this research was to develop an understanding of the number and type of exploratory analyses undertaken by the ERGs within the STA process and to understand how these analyses are used by the NICE ACs in their decision-making. Methods: The 100 most recently completed STAs with published guidance were selected for inclusion in the analysis. The documents considered were ERG reports, clarification letters, the first appraisal consultation document and the final appraisal determination. Over 400 documents were assessed in this study. The categories of types of exploratory analyses included fixing errors, fixing violations, addressing matters of judgement and the ERG-preferred base case. A content analysis of documents (documentary analysis) was undertaken to identify and extract relevant data, and narrative synthesis was then used to rationalise and present these data. Results: The level and type of detail in ERG reports and clarification letters varied considerably. The vast majority (93%) of ERG reports reported one or more exploratory analyses. The most frequently reported type of analysis in these 93 ERG reports related to the category ‘matters of judgement’, which was reported in 83 (89%) reports. The category ‘ERG base-case/preferred analysis’ was reported in 45 (48%) reports, the category ‘fixing errors’ was reported in 33 (35%) reports and the category ‘fixing violations’ was reported in 17 (18%) reports. The exploratory analyses performed were the result of issues raised by an ERG in its critique of the submitted economic evidence. These analyses had more influence on recommendations earlier in the STA process than later on in the process. Limitations: The descriptions of analyses undertaken were often highly specific to a particular STA and could be inconsistent across ERG reports and thus difficult to interpret. Conclusions: Evidence Review Groups frequently conduct exploratory analyses to test or improve the economic evaluations submitted by companies as part of the STA process. ERG exploratory analyses often have an influence on the recommendations produced by the ACs. Future work: More in-depth analysis is needed to understand how ERGs make decisions regarding which exploratory analyses should be undertaken. More research is also needed to fully understand which types of exploratory analyses are most useful to ACs in their decision-making. Funding: The National Institute for Health Research Health Technology Assessment programme.https://doi.org/10.3310/hta20260single technology appraisalnational institute for health and care excellenceniceexploratory analysisappraisal committeeevidence review group |
| spellingShingle | Eva Kaltenthaler Christopher Carroll Daniel Hill-McManus Alison Scope Michael Holmes Stephen Rice Micah Rose Paul Tappenden Nerys Woolacott The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis Health Technology Assessment single technology appraisal national institute for health and care excellence nice exploratory analysis appraisal committee evidence review group |
| title | The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis |
| title_full | The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis |
| title_fullStr | The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis |
| title_full_unstemmed | The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis |
| title_short | The use of exploratory analyses within the National Institute for Health and Care Excellence single technology appraisal process: an evaluation and qualitative analysis |
| title_sort | use of exploratory analyses within the national institute for health and care excellence single technology appraisal process an evaluation and qualitative analysis |
| topic | single technology appraisal national institute for health and care excellence nice exploratory analysis appraisal committee evidence review group |
| url | https://doi.org/10.3310/hta20260 |
| work_keys_str_mv | AT evakaltenthaler theuseofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT christophercarroll theuseofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT danielhillmcmanus theuseofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT alisonscope theuseofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT michaelholmes theuseofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT stephenrice theuseofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT micahrose theuseofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT paultappenden theuseofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT neryswoolacott theuseofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT evakaltenthaler useofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT christophercarroll useofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT danielhillmcmanus useofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT alisonscope useofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT michaelholmes useofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT stephenrice useofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT micahrose useofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT paultappenden useofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis AT neryswoolacott useofexploratoryanalyseswithinthenationalinstituteforhealthandcareexcellencesingletechnologyappraisalprocessanevaluationandqualitativeanalysis |