Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy
Quantitative literacy has been identified as an important student learning outcome (SLO) by both the higher education and workforce communities. This paper aims to provide preliminary evidence of the psychometric quality of the pilot forms for HEIghten quantitative literacy, a next-generation SLO as...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
National Numeracy Network
2017-07-01
|
| Series: | Numeracy |
| Subjects: | |
| Online Access: | http://scholarcommons.usf.edu/numeracy/vol10/iss2/art3/ |
| _version_ | 1818236799636471808 |
|---|---|
| author | Katrina C. Roohr HyeSun Lee Jun Xu Ou Lydia Liu Zhen Wang |
| author_facet | Katrina C. Roohr HyeSun Lee Jun Xu Ou Lydia Liu Zhen Wang |
| author_sort | Katrina C. Roohr |
| collection | DOAJ |
| description | Quantitative literacy has been identified as an important student learning outcome (SLO) by both the higher education and workforce communities. This paper aims to provide preliminary evidence of the psychometric quality of the pilot forms for HEIghten quantitative literacy, a next-generation SLO assessment for students in higher education. We evaluated the psychometric quality of the test items (e.g., item analyses), individual- and group-level reliability, the relationship with student performance and related variables (e.g., grade point average) as well as student perceptions, and differences across college-related and demographic subgroups. Our study used data from a pilot test administered to over 1,500 students at 23 higher education institutions in the United States. Results showed that (a) overall, items were functioning well, but a small portion of items should be dropped due to unsatisfactory performance; (b) correlations across sub-areas of the assessment were very high indicating that the assessment may be unidimensional; (c) reliability estimates similar to existing SLO assessments were found at both individual and group levels; (d) assessment scores correlated positively with high school and college GPA, number of math college courses, self-rated quantitative literacy skills, and college admissions scores; (e) students had positive perceptions about the assessment; and (f) performance differences were found across institution type, college majors, gender, racial/ethnic groups, and language groups, but not across credit-hour categories. Implications for operational test development and understanding of quantitative literacy performance are discussed. |
| first_indexed | 2024-12-12T12:15:36Z |
| format | Article |
| id | doaj.art-fbb92dc43b214aa7b8cdcde12881bc44 |
| institution | Directory Open Access Journal |
| issn | 1936-4660 1936-4660 |
| language | English |
| last_indexed | 2024-12-12T12:15:36Z |
| publishDate | 2017-07-01 |
| publisher | National Numeracy Network |
| record_format | Article |
| series | Numeracy |
| spelling | doaj.art-fbb92dc43b214aa7b8cdcde12881bc442022-12-22T00:24:46ZengNational Numeracy NetworkNumeracy1936-46601936-46602017-07-01102310.5038/1936-4660.10.2.3Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative LiteracyKatrina C. Roohr0HyeSun Lee1Jun Xu2Ou Lydia Liu3Zhen Wang4Educational Testing ServiceCalifornia State University Channel IslandsEducational Testing ServiceEducational Testing ServiceEducational Testing ServiceQuantitative literacy has been identified as an important student learning outcome (SLO) by both the higher education and workforce communities. This paper aims to provide preliminary evidence of the psychometric quality of the pilot forms for HEIghten quantitative literacy, a next-generation SLO assessment for students in higher education. We evaluated the psychometric quality of the test items (e.g., item analyses), individual- and group-level reliability, the relationship with student performance and related variables (e.g., grade point average) as well as student perceptions, and differences across college-related and demographic subgroups. Our study used data from a pilot test administered to over 1,500 students at 23 higher education institutions in the United States. Results showed that (a) overall, items were functioning well, but a small portion of items should be dropped due to unsatisfactory performance; (b) correlations across sub-areas of the assessment were very high indicating that the assessment may be unidimensional; (c) reliability estimates similar to existing SLO assessments were found at both individual and group levels; (d) assessment scores correlated positively with high school and college GPA, number of math college courses, self-rated quantitative literacy skills, and college admissions scores; (e) students had positive perceptions about the assessment; and (f) performance differences were found across institution type, college majors, gender, racial/ethnic groups, and language groups, but not across credit-hour categories. Implications for operational test development and understanding of quantitative literacy performance are discussed.http://scholarcommons.usf.edu/numeracy/vol10/iss2/art3/quantitative literacyassessmenthigher educationstudent learning outcomesreliabilityvalidity |
| spellingShingle | Katrina C. Roohr HyeSun Lee Jun Xu Ou Lydia Liu Zhen Wang Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy Numeracy quantitative literacy assessment higher education student learning outcomes reliability validity |
| title | Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy |
| title_full | Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy |
| title_fullStr | Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy |
| title_full_unstemmed | Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy |
| title_short | Preliminary Evaluation of the Psychometric Quality of HEIghtenTM Quantitative Literacy |
| title_sort | preliminary evaluation of the psychometric quality of heightentm quantitative literacy |
| topic | quantitative literacy assessment higher education student learning outcomes reliability validity |
| url | http://scholarcommons.usf.edu/numeracy/vol10/iss2/art3/ |
| work_keys_str_mv | AT katrinacroohr preliminaryevaluationofthepsychometricqualityofheightentmquantitativeliteracy AT hyesunlee preliminaryevaluationofthepsychometricqualityofheightentmquantitativeliteracy AT junxu preliminaryevaluationofthepsychometricqualityofheightentmquantitativeliteracy AT oulydialiu preliminaryevaluationofthepsychometricqualityofheightentmquantitativeliteracy AT zhenwang preliminaryevaluationofthepsychometricqualityofheightentmquantitativeliteracy |