Development and Validation of the Engineering Computational Thinking Diagnostic for Undergraduate Students
Computational thinking is one barrier to enculturating as a professional engineer. We created the Engineering Computational Thinking Diagnostic (ECTD) as an instructional tool that can identify at-risk first-year engineering students. The purpose of this study is to provide construct validity, inter...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10327734/ |
| _version_ | 1797321043803635712 |
|---|---|
| author | Noemi V. Mendoza Diaz So Yoon Yoon Deborah A. Trytten Russ Meier |
| author_facet | Noemi V. Mendoza Diaz So Yoon Yoon Deborah A. Trytten Russ Meier |
| author_sort | Noemi V. Mendoza Diaz |
| collection | DOAJ |
| description | Computational thinking is one barrier to enculturating as a professional engineer. We created the Engineering Computational Thinking Diagnostic (ECTD) as an instructional tool that can identify at-risk first-year engineering students. The purpose of this study is to provide construct validity, internal consistency reliability, item characteristics, and criterion validity evidence for this diagnostic. From fall 2020 to fall 2021, 469 students from three institutions in the United States took the diagnostic. The data from 152 students at one institution was used to provide evidence of predictive validity. Exploratory and confirmatory factor analyses resulted in 20 items loading onto one factor in a good model fit range, with the internal consistency reliability coefficient, Cronbach <inline-formula> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula> of 0.86. From item analyses based on classical test theory, the diagnostic items on average tended to be slightly easy but had sufficient discrimination power. The correlation matrix for criterion validity evidence indicated that the diagnostic functions well to differentiate students’ computational thinking ability by prior computer science course experience as well as by first-generation status. Predictive validity evidence from regression analyses revealed the statistically significant effect of students’ diagnostic scores assessed at the beginning of the first semester on predicting their end of semester course grades. The ECTD can have a broad impact because it provides a tool to gauge the entry-level skills of students, enabling early curriculum interventions to help retention and persistence to graduation. We make the case that the ECTD could contribute to the development of a more diverse workforce in engineering. |
| first_indexed | 2024-03-08T04:52:45Z |
| format | Article |
| id | doaj.art-9b4fd34ea4a947628741a2fe1e8af23e |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T04:52:45Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-9b4fd34ea4a947628741a2fe1e8af23e2024-02-08T00:01:50ZengIEEEIEEE Access2169-35362023-01-011113309913311410.1109/ACCESS.2023.333593110327734Development and Validation of the Engineering Computational Thinking Diagnostic for Undergraduate StudentsNoemi V. Mendoza Diaz0https://orcid.org/0000-0003-1215-1554So Yoon Yoon1https://orcid.org/0000-0003-1868-1054Deborah A. Trytten2https://orcid.org/0000-0002-3676-661XRuss Meier3https://orcid.org/0000-0002-9380-9049Department of Engineering Technology and Industrial Distribution, Texas A&M University, College Station, TX, USADepartment of Engineering and Computing Education, University of Cincinnati, Cincinnati, OH, USASchool of Computer Science, The University of Oklahoma, Norman, OK, USADepartment of Electrical Engineering and Computer Science, Milwaukee School of Engineering, Milwaukee, WI, USAComputational thinking is one barrier to enculturating as a professional engineer. We created the Engineering Computational Thinking Diagnostic (ECTD) as an instructional tool that can identify at-risk first-year engineering students. The purpose of this study is to provide construct validity, internal consistency reliability, item characteristics, and criterion validity evidence for this diagnostic. From fall 2020 to fall 2021, 469 students from three institutions in the United States took the diagnostic. The data from 152 students at one institution was used to provide evidence of predictive validity. Exploratory and confirmatory factor analyses resulted in 20 items loading onto one factor in a good model fit range, with the internal consistency reliability coefficient, Cronbach <inline-formula> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula> of 0.86. From item analyses based on classical test theory, the diagnostic items on average tended to be slightly easy but had sufficient discrimination power. The correlation matrix for criterion validity evidence indicated that the diagnostic functions well to differentiate students’ computational thinking ability by prior computer science course experience as well as by first-generation status. Predictive validity evidence from regression analyses revealed the statistically significant effect of students’ diagnostic scores assessed at the beginning of the first semester on predicting their end of semester course grades. The ECTD can have a broad impact because it provides a tool to gauge the entry-level skills of students, enabling early curriculum interventions to help retention and persistence to graduation. We make the case that the ECTD could contribute to the development of a more diverse workforce in engineering.https://ieeexplore.ieee.org/document/10327734/Computer science educationcomputational thinkingdiagnostic methodstudent assessment toolsengineering educationfirst-year engineering student experience |
| spellingShingle | Noemi V. Mendoza Diaz So Yoon Yoon Deborah A. Trytten Russ Meier Development and Validation of the Engineering Computational Thinking Diagnostic for Undergraduate Students IEEE Access Computer science education computational thinking diagnostic method student assessment tools engineering education first-year engineering student experience |
| title | Development and Validation of the Engineering Computational Thinking Diagnostic for Undergraduate Students |
| title_full | Development and Validation of the Engineering Computational Thinking Diagnostic for Undergraduate Students |
| title_fullStr | Development and Validation of the Engineering Computational Thinking Diagnostic for Undergraduate Students |
| title_full_unstemmed | Development and Validation of the Engineering Computational Thinking Diagnostic for Undergraduate Students |
| title_short | Development and Validation of the Engineering Computational Thinking Diagnostic for Undergraduate Students |
| title_sort | development and validation of the engineering computational thinking diagnostic for undergraduate students |
| topic | Computer science education computational thinking diagnostic method student assessment tools engineering education first-year engineering student experience |
| url | https://ieeexplore.ieee.org/document/10327734/ |
| work_keys_str_mv | AT noemivmendozadiaz developmentandvalidationoftheengineeringcomputationalthinkingdiagnosticforundergraduatestudents AT soyoonyoon developmentandvalidationoftheengineeringcomputationalthinkingdiagnosticforundergraduatestudents AT deborahatrytten developmentandvalidationoftheengineeringcomputationalthinkingdiagnosticforundergraduatestudents AT russmeier developmentandvalidationoftheengineeringcomputationalthinkingdiagnosticforundergraduatestudents |