Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory
Background: Evaluative thinking has emerged as a key construct in evaluation, especially for evaluation practitioners and researchers interested in evaluation capacity building (ECB). Yet, despite increasing calls for more research on evaluation and, more specifically, for more research on ECB, unt...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
The Evaluation Center at Western Michigan University
2020-04-01
|
| Series: | Journal of MultiDisciplinary Evaluation |
| Online Access: | https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/591 |
| _version_ | 1797812704480591872 |
|---|---|
| author | Jason S. McIntosh Jane Buckley Thomas Archibald |
| author_facet | Jason S. McIntosh Jane Buckley Thomas Archibald |
| author_sort | Jason S. McIntosh |
| collection | DOAJ |
| description |
Background: Evaluative thinking has emerged as a key construct in evaluation, especially for evaluation practitioners and researchers interested in evaluation capacity building (ECB). Yet, despite increasing calls for more research on evaluation and, more specifically, for more research on ECB, until recently little empirical inquiry on the dimensions of evaluative thinking has been conducted.
Purpose: To address that lack, the purpose of the study presented in this paper is to refine the construct of evaluative thinking by exploring its underlying dimensions and to ascertain the internal consistency of an instrument developed to measure evaluative thinking, the Evaluative Thinking Inventory (ETI).
Setting: The ETI was developed as part of an ECB initiative focused on non-formal science, engineering, technology, and math (STEM) education in the United States, and was tested as part of a study focused on evaluating gifted education programs, also in the United States.
Intervention: Not applicable.
Research design: Survey research and exploratory factor analysis (EFA).
Data collection & analysis: The ETI was administered to participants in a study measuring the effectiveness of a tool used to conduct internal evaluations of gifted education programs. SPSS was used to conduct an EFA on 96 completed ETIs. Cronbach’s alpha was used to estimate the internal consistency of the instrument.
Findings: The analysis of the ETI revealed a two-factor model of evaluative thinking (i.e., believe in and practice evaluation and pose thoughtful questions and seek alternatives). This study also provided internal consistency evidence for the ETI showing alpha reliabilities for the two factors ranging from 0.80 to 0.82. The ETI has potentially wide applicability in research and practice in ECB and in the field of evaluation more generally.
Keywords: evaluative thinking; evaluation capacity building; research on evaluation; exploratory factor analysis.
|
| first_indexed | 2024-03-13T07:41:20Z |
| format | Article |
| id | doaj.art-0d5041aed0364b81a1a59eefbc057b68 |
| institution | Directory Open Access Journal |
| issn | 1556-8180 |
| language | English |
| last_indexed | 2024-03-13T07:41:20Z |
| publishDate | 2020-04-01 |
| publisher | The Evaluation Center at Western Michigan University |
| record_format | Article |
| series | Journal of MultiDisciplinary Evaluation |
| spelling | doaj.art-0d5041aed0364b81a1a59eefbc057b682023-06-03T06:57:39ZengThe Evaluation Center at Western Michigan UniversityJournal of MultiDisciplinary Evaluation1556-81802020-04-01163410.56645/jmde.v16i34.591Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking InventoryJason S. McIntosh0Jane Buckley1Thomas Archibald2Washington Elementary School DistrictJCB ConsultingVirginia Tech Background: Evaluative thinking has emerged as a key construct in evaluation, especially for evaluation practitioners and researchers interested in evaluation capacity building (ECB). Yet, despite increasing calls for more research on evaluation and, more specifically, for more research on ECB, until recently little empirical inquiry on the dimensions of evaluative thinking has been conducted. Purpose: To address that lack, the purpose of the study presented in this paper is to refine the construct of evaluative thinking by exploring its underlying dimensions and to ascertain the internal consistency of an instrument developed to measure evaluative thinking, the Evaluative Thinking Inventory (ETI). Setting: The ETI was developed as part of an ECB initiative focused on non-formal science, engineering, technology, and math (STEM) education in the United States, and was tested as part of a study focused on evaluating gifted education programs, also in the United States. Intervention: Not applicable. Research design: Survey research and exploratory factor analysis (EFA). Data collection & analysis: The ETI was administered to participants in a study measuring the effectiveness of a tool used to conduct internal evaluations of gifted education programs. SPSS was used to conduct an EFA on 96 completed ETIs. Cronbach’s alpha was used to estimate the internal consistency of the instrument. Findings: The analysis of the ETI revealed a two-factor model of evaluative thinking (i.e., believe in and practice evaluation and pose thoughtful questions and seek alternatives). This study also provided internal consistency evidence for the ETI showing alpha reliabilities for the two factors ranging from 0.80 to 0.82. The ETI has potentially wide applicability in research and practice in ECB and in the field of evaluation more generally. Keywords: evaluative thinking; evaluation capacity building; research on evaluation; exploratory factor analysis. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/591 |
| spellingShingle | Jason S. McIntosh Jane Buckley Thomas Archibald Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory Journal of MultiDisciplinary Evaluation |
| title | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_full | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_fullStr | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_full_unstemmed | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_short | Refining and Measuring the Construct of Evaluative Thinking: An Exploratory Factor Analysis of the Evaluative Thinking Inventory |
| title_sort | refining and measuring the construct of evaluative thinking an exploratory factor analysis of the evaluative thinking inventory |
| url | https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/591 |
| work_keys_str_mv | AT jasonsmcintosh refiningandmeasuringtheconstructofevaluativethinkinganexploratoryfactoranalysisoftheevaluativethinkinginventory AT janebuckley refiningandmeasuringtheconstructofevaluativethinkinganexploratoryfactoranalysisoftheevaluativethinkinginventory AT thomasarchibald refiningandmeasuringtheconstructofevaluativethinkinganexploratoryfactoranalysisoftheevaluativethinkinginventory |