The Role of Prediction in Evaluating Econometric Models [and Discussion]
Many statistical and economic criteria must influence the overall evaluation of econometric systems, but success when 'predictive testing' seems to establish most credibility, perhaps because the data were unavailable to the modellers. Yet the ability of prediction tests to detect even gro...
| Main Authors: | , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Royal Society
1986
|
| _version_ | 1826287798405038080 |
|---|---|
| author | Hendry, D Mason, J Sen, A |
| author_facet | Hendry, D Mason, J Sen, A |
| author_sort | Hendry, D |
| collection | OXFORD |
| description | Many statistical and economic criteria must influence the overall evaluation of econometric systems, but success when 'predictive testing' seems to establish most credibility, perhaps because the data were unavailable to the modellers. Yet the ability of prediction tests to detect even gross mis-specifications depends both on the properties of the data process and on the structure of the selected tests. To study the issues involved, a mixed analytical-Monte Carlo approach was adopted, based on asymptotically valid approximations to forecast confidence intervals. Tests were calibrated for finite samples by simulation experiments on a Distributed Array Processor to yield numerical power function response surfaces. The main results of analysing prediction tests are that H successive one-step ahead tests are equivalent to one H-step test; that conditional multi-step forecast confidence bands are not necessarily monotonically increasing in H, so that 'intercept corrections' potentially have an objective justification; that tests based on averages (e.g. within-years) may be more useful than either one-step or H-step tests for detecting predictive failure; that the need to pool forecasts indicates non-encompassing models; and that a forecast-encompassing test should have some diagnostic power when applied to macroeconometric systems. Forecasting of the level of uncertainty is briefly considered. |
| first_indexed | 2024-03-07T02:04:03Z |
| format | Journal article |
| id | oxford-uuid:9e5a13b1-ace5-4647-8925-bfeb634abbf5 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T02:04:03Z |
| publishDate | 1986 |
| publisher | Royal Society |
| record_format | dspace |
| spelling | oxford-uuid:9e5a13b1-ace5-4647-8925-bfeb634abbf52022-03-27T00:49:31ZThe Role of Prediction in Evaluating Econometric Models [and Discussion]Journal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:9e5a13b1-ace5-4647-8925-bfeb634abbf5EnglishDepartment of Economics - ePrintsRoyal Society1986Hendry, DMason, JSen, AMany statistical and economic criteria must influence the overall evaluation of econometric systems, but success when 'predictive testing' seems to establish most credibility, perhaps because the data were unavailable to the modellers. Yet the ability of prediction tests to detect even gross mis-specifications depends both on the properties of the data process and on the structure of the selected tests. To study the issues involved, a mixed analytical-Monte Carlo approach was adopted, based on asymptotically valid approximations to forecast confidence intervals. Tests were calibrated for finite samples by simulation experiments on a Distributed Array Processor to yield numerical power function response surfaces. The main results of analysing prediction tests are that H successive one-step ahead tests are equivalent to one H-step test; that conditional multi-step forecast confidence bands are not necessarily monotonically increasing in H, so that 'intercept corrections' potentially have an objective justification; that tests based on averages (e.g. within-years) may be more useful than either one-step or H-step tests for detecting predictive failure; that the need to pool forecasts indicates non-encompassing models; and that a forecast-encompassing test should have some diagnostic power when applied to macroeconometric systems. Forecasting of the level of uncertainty is briefly considered. |
| spellingShingle | Hendry, D Mason, J Sen, A The Role of Prediction in Evaluating Econometric Models [and Discussion] |
| title | The Role of Prediction in Evaluating Econometric Models [and Discussion] |
| title_full | The Role of Prediction in Evaluating Econometric Models [and Discussion] |
| title_fullStr | The Role of Prediction in Evaluating Econometric Models [and Discussion] |
| title_full_unstemmed | The Role of Prediction in Evaluating Econometric Models [and Discussion] |
| title_short | The Role of Prediction in Evaluating Econometric Models [and Discussion] |
| title_sort | role of prediction in evaluating econometric models and discussion |
| work_keys_str_mv | AT hendryd theroleofpredictioninevaluatingeconometricmodelsanddiscussion AT masonj theroleofpredictioninevaluatingeconometricmodelsanddiscussion AT sena theroleofpredictioninevaluatingeconometricmodelsanddiscussion AT hendryd roleofpredictioninevaluatingeconometricmodelsanddiscussion AT masonj roleofpredictioninevaluatingeconometricmodelsanddiscussion AT sena roleofpredictioninevaluatingeconometricmodelsanddiscussion |