Addressing an old issue from a new methodological perspective: a proposition on how to deal with bias due to multilevel measurement error in the estimation of the effects of school composition
<p>With educational effectiveness studies, school-level aggregates of students' characteristics (e.g. achievement) are often used to assess the impact of school composition on students' outcomes – school compositional effects. Empirical findings on the magnitude and direction of scho...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
2014
|
| Subjects: |
| _version_ | 1797055328388382720 |
|---|---|
| author | Televantou, I |
| author2 | Marsh, H |
| author_facet | Marsh, H Televantou, I |
| author_sort | Televantou, I |
| collection | OXFORD |
| description | <p>With educational effectiveness studies, school-level aggregates of students' characteristics (e.g. achievement) are often used to assess the impact of school composition on students' outcomes – school compositional effects. Empirical findings on the magnitude and direction of school compositional effects have not been consistent. Relevant methodological studies raise the issue of under-specification at level 1 in compositional models - evident when the student-level indicator on which the aggregation is based is mis-measured. This phenomenon has been shown to bias compositional effect estimates, leading to misleading effects of the aggregated variables – phantom compositional effects. My thesis, consisted of three separate studies, presents an advanced methodological framework that can be used to investigate the effect of school composition net of measurement error bias.</p> <p>In Study 1, I quantify the impact of failing to account for measurement error on school compositional effects as used in value added models of educational effectiveness to explain relative school effects. Building on previous studies, multilevel structural equation models are incorporated to control for measurement error and/or sampling error. Study 1a, a large sample of English primary students in years one and four (9,059 students from 593 schools) reveals a small, significant and negative compositional effect on students' subsequent mathematics achievement that becomes more negative after controlling for measurement error. Study 1b, a large study of Cyprus primary students in year four (1694 students in 59 schools) shows a small, positive but statistically significant effect that becomes non-significant after controlling for measurement error.</p> <p>Further analyses with the English data (Study 2), demonstrates a negative compositional effect of school average mathematics achievement on subsequent mathematics self-concept – a Big Fish Little Pond Effect (BFLPE). Adjustments for measurement and sampling error result in more negative BFLPEs. The originality of Study 2 lies in verifying BFLPEs for students as young as five to eight/nine years old. Bridging the findings related to students' mathematics self-concept (Study 2) and the findings on students’ mathematics achievements (Study 1a), I demonstrate that the prevalence of BFLPEs with the English data partly explains the negative compositional effect of school average mathematics achievement on students' subsequent mathematics achievement.</p> <p>Lastly, in Study 3 I consider an alternative approach to school accountability to conventional value added models, namely the Regression Discontinuity approach. Specifically, I use the English TIMSS 1995 primary (years four and five) and secondary (years eight and nine) data to investigate the effect of one extra year of schooling on students' mathematics achievement and the variability across schools in their absolute effects. The extent to which school composition, as given by school average achievement, correlates with schools' added-year effects is addressed. Importantly the robustness of the RD estimates to measurement error bias is demonstrated.</p> <p>My findings have important methodological, substantive and theoretical implications for on-going debates on the school compositional effects on students' outcomes, because nearly all previous research has been based on traditional approaches to multilevel models, which are positively biased due to the failure to control for measurement error.</p> |
| first_indexed | 2024-03-06T19:09:08Z |
| format | Thesis |
| id | oxford-uuid:162c7ee3-2fe4-4bed-8e44-10018e85dc68 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T19:09:08Z |
| publishDate | 2014 |
| record_format | dspace |
| spelling | oxford-uuid:162c7ee3-2fe4-4bed-8e44-10018e85dc682022-03-26T10:29:44ZAddressing an old issue from a new methodological perspective: a proposition on how to deal with bias due to multilevel measurement error in the estimation of the effects of school compositionThesishttp://purl.org/coar/resource_type/c_db06uuid:162c7ee3-2fe4-4bed-8e44-10018e85dc68Self-concept enhancementEducationStatistics (social sciences)EnglishOxford University Research Archive - Valet2014Televantou, IMarsh, H<p>With educational effectiveness studies, school-level aggregates of students' characteristics (e.g. achievement) are often used to assess the impact of school composition on students' outcomes – school compositional effects. Empirical findings on the magnitude and direction of school compositional effects have not been consistent. Relevant methodological studies raise the issue of under-specification at level 1 in compositional models - evident when the student-level indicator on which the aggregation is based is mis-measured. This phenomenon has been shown to bias compositional effect estimates, leading to misleading effects of the aggregated variables – phantom compositional effects. My thesis, consisted of three separate studies, presents an advanced methodological framework that can be used to investigate the effect of school composition net of measurement error bias.</p> <p>In Study 1, I quantify the impact of failing to account for measurement error on school compositional effects as used in value added models of educational effectiveness to explain relative school effects. Building on previous studies, multilevel structural equation models are incorporated to control for measurement error and/or sampling error. Study 1a, a large sample of English primary students in years one and four (9,059 students from 593 schools) reveals a small, significant and negative compositional effect on students' subsequent mathematics achievement that becomes more negative after controlling for measurement error. Study 1b, a large study of Cyprus primary students in year four (1694 students in 59 schools) shows a small, positive but statistically significant effect that becomes non-significant after controlling for measurement error.</p> <p>Further analyses with the English data (Study 2), demonstrates a negative compositional effect of school average mathematics achievement on subsequent mathematics self-concept – a Big Fish Little Pond Effect (BFLPE). Adjustments for measurement and sampling error result in more negative BFLPEs. The originality of Study 2 lies in verifying BFLPEs for students as young as five to eight/nine years old. Bridging the findings related to students' mathematics self-concept (Study 2) and the findings on students’ mathematics achievements (Study 1a), I demonstrate that the prevalence of BFLPEs with the English data partly explains the negative compositional effect of school average mathematics achievement on students' subsequent mathematics achievement.</p> <p>Lastly, in Study 3 I consider an alternative approach to school accountability to conventional value added models, namely the Regression Discontinuity approach. Specifically, I use the English TIMSS 1995 primary (years four and five) and secondary (years eight and nine) data to investigate the effect of one extra year of schooling on students' mathematics achievement and the variability across schools in their absolute effects. The extent to which school composition, as given by school average achievement, correlates with schools' added-year effects is addressed. Importantly the robustness of the RD estimates to measurement error bias is demonstrated.</p> <p>My findings have important methodological, substantive and theoretical implications for on-going debates on the school compositional effects on students' outcomes, because nearly all previous research has been based on traditional approaches to multilevel models, which are positively biased due to the failure to control for measurement error.</p> |
| spellingShingle | Self-concept enhancement Education Statistics (social sciences) Televantou, I Addressing an old issue from a new methodological perspective: a proposition on how to deal with bias due to multilevel measurement error in the estimation of the effects of school composition |
| title | Addressing an old issue from a new methodological perspective: a proposition on how to deal with bias due to multilevel measurement error in the estimation of the effects of school composition |
| title_full | Addressing an old issue from a new methodological perspective: a proposition on how to deal with bias due to multilevel measurement error in the estimation of the effects of school composition |
| title_fullStr | Addressing an old issue from a new methodological perspective: a proposition on how to deal with bias due to multilevel measurement error in the estimation of the effects of school composition |
| title_full_unstemmed | Addressing an old issue from a new methodological perspective: a proposition on how to deal with bias due to multilevel measurement error in the estimation of the effects of school composition |
| title_short | Addressing an old issue from a new methodological perspective: a proposition on how to deal with bias due to multilevel measurement error in the estimation of the effects of school composition |
| title_sort | addressing an old issue from a new methodological perspective a proposition on how to deal with bias due to multilevel measurement error in the estimation of the effects of school composition |
| topic | Self-concept enhancement Education Statistics (social sciences) |
| work_keys_str_mv | AT televantoui addressinganoldissuefromanewmethodologicalperspectiveapropositiononhowtodealwithbiasduetomultilevelmeasurementerrorintheestimationoftheeffectsofschoolcomposition |