On the Design and Validation of Assessing Tools for Measuring the Impact of Programs Promoting STEM Vocations
This paper presents the design and validation process of a set of instruments to evaluate the impact of an informal learning initiative to promote Science, Technology, Engineering, and Mathematics (STEM) vocations in students, their families (parents), and teachers. The proposed set of instruments,...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-06-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyg.2022.937058/full |
| _version_ | 1811326523185037312 |
|---|---|
| author | María Pilar Herce-Palomares Carmen Botella-Mascarell Esther de Ves Emilia López-Iñesta Anabel Forte Xaro Benavent Silvia Rueda |
| author_facet | María Pilar Herce-Palomares Carmen Botella-Mascarell Esther de Ves Emilia López-Iñesta Anabel Forte Xaro Benavent Silvia Rueda |
| author_sort | María Pilar Herce-Palomares |
| collection | DOAJ |
| description | This paper presents the design and validation process of a set of instruments to evaluate the impact of an informal learning initiative to promote Science, Technology, Engineering, and Mathematics (STEM) vocations in students, their families (parents), and teachers. The proposed set of instruments, beyond assessing the satisfaction of the public involved, allow collecting data to evaluate the impact in terms of changes in the consideration of the role of women in STEM areas and STEM vocations. The procedure followed to develop the set of instruments consisted of two phases. In the first phase, a preliminary version (v1) of the questionnaires was designed based on the objectives of the Girls4STEM initiative, an inclusive project promoting STEM vocations between 6 and 18 years old boys and girls. Five specific questionnaires were designed, one for the families (post activity), two for the students (pre and post activity) and two for the teachers (pre and post avitivity). A refined version (v2) of each questionnaire was obtained with evidence of content validity after undergoing an expert judgment process. The second phase was the refinement of the (v2) instruments, to ascertain the evidence of reliability and validity so that a final version (v3) was derived. In the paper, a high-quality set of good practices focused on promoting diversity and gender equality in the STEM sector are presented from a Higher Education Institution perspective, the University of Valencia. The main contribution of this work is the achievement of a set of instruments, rigorously designed for the evaluation of the implementation and effectiveness of a STEM promoting program, with sufficient validity evidence. Moreover, the proposed instruments can be a reference for the evaluation of other projects aimed at diversifying the STEM sector. |
| first_indexed | 2024-04-13T14:51:36Z |
| format | Article |
| id | doaj.art-7d188de82701468996cf2c30c949406f |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-04-13T14:51:36Z |
| publishDate | 2022-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-7d188de82701468996cf2c30c949406f2022-12-22T02:42:35ZengFrontiers Media S.A.Frontiers in Psychology1664-10782022-06-011310.3389/fpsyg.2022.937058937058On the Design and Validation of Assessing Tools for Measuring the Impact of Programs Promoting STEM VocationsMaría Pilar Herce-Palomares0Carmen Botella-Mascarell1Esther de Ves2Emilia López-Iñesta3Anabel Forte4Xaro Benavent5Silvia Rueda6International PhD School (EIDUNED), National Distance Education University (UNED), Madrid, SpainDepartment of Computer Science, Universitat de València, Burjassot, SpainDepartment of Computer Science, Universitat de València, Burjassot, SpainDepartment of Didactics of Mathematics, Universitat de València, València, SpainDepartment of Statistics and Operational Research, Universitat de València, Burjassot, SpainDepartment of Computer Science, Universitat de València, Burjassot, SpainDepartment of Computer Science, Universitat de València, Burjassot, SpainThis paper presents the design and validation process of a set of instruments to evaluate the impact of an informal learning initiative to promote Science, Technology, Engineering, and Mathematics (STEM) vocations in students, their families (parents), and teachers. The proposed set of instruments, beyond assessing the satisfaction of the public involved, allow collecting data to evaluate the impact in terms of changes in the consideration of the role of women in STEM areas and STEM vocations. The procedure followed to develop the set of instruments consisted of two phases. In the first phase, a preliminary version (v1) of the questionnaires was designed based on the objectives of the Girls4STEM initiative, an inclusive project promoting STEM vocations between 6 and 18 years old boys and girls. Five specific questionnaires were designed, one for the families (post activity), two for the students (pre and post activity) and two for the teachers (pre and post avitivity). A refined version (v2) of each questionnaire was obtained with evidence of content validity after undergoing an expert judgment process. The second phase was the refinement of the (v2) instruments, to ascertain the evidence of reliability and validity so that a final version (v3) was derived. In the paper, a high-quality set of good practices focused on promoting diversity and gender equality in the STEM sector are presented from a Higher Education Institution perspective, the University of Valencia. The main contribution of this work is the achievement of a set of instruments, rigorously designed for the evaluation of the implementation and effectiveness of a STEM promoting program, with sufficient validity evidence. Moreover, the proposed instruments can be a reference for the evaluation of other projects aimed at diversifying the STEM sector.https://www.frontiersin.org/articles/10.3389/fpsyg.2022.937058/fulldiversity in STEMgender stereotypesinformal educationself-efficacyquestionnaire validationmixed methods |
| spellingShingle | María Pilar Herce-Palomares Carmen Botella-Mascarell Esther de Ves Emilia López-Iñesta Anabel Forte Xaro Benavent Silvia Rueda On the Design and Validation of Assessing Tools for Measuring the Impact of Programs Promoting STEM Vocations Frontiers in Psychology diversity in STEM gender stereotypes informal education self-efficacy questionnaire validation mixed methods |
| title | On the Design and Validation of Assessing Tools for Measuring the Impact of Programs Promoting STEM Vocations |
| title_full | On the Design and Validation of Assessing Tools for Measuring the Impact of Programs Promoting STEM Vocations |
| title_fullStr | On the Design and Validation of Assessing Tools for Measuring the Impact of Programs Promoting STEM Vocations |
| title_full_unstemmed | On the Design and Validation of Assessing Tools for Measuring the Impact of Programs Promoting STEM Vocations |
| title_short | On the Design and Validation of Assessing Tools for Measuring the Impact of Programs Promoting STEM Vocations |
| title_sort | on the design and validation of assessing tools for measuring the impact of programs promoting stem vocations |
| topic | diversity in STEM gender stereotypes informal education self-efficacy questionnaire validation mixed methods |
| url | https://www.frontiersin.org/articles/10.3389/fpsyg.2022.937058/full |
| work_keys_str_mv | AT mariapilarhercepalomares onthedesignandvalidationofassessingtoolsformeasuringtheimpactofprogramspromotingstemvocations AT carmenbotellamascarell onthedesignandvalidationofassessingtoolsformeasuringtheimpactofprogramspromotingstemvocations AT estherdeves onthedesignandvalidationofassessingtoolsformeasuringtheimpactofprogramspromotingstemvocations AT emilialopezinesta onthedesignandvalidationofassessingtoolsformeasuringtheimpactofprogramspromotingstemvocations AT anabelforte onthedesignandvalidationofassessingtoolsformeasuringtheimpactofprogramspromotingstemvocations AT xarobenavent onthedesignandvalidationofassessingtoolsformeasuringtheimpactofprogramspromotingstemvocations AT silviarueda onthedesignandvalidationofassessingtoolsformeasuringtheimpactofprogramspromotingstemvocations |