Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart Speakers
The concept of digital literacy has been introduced as a new cultural technique, which is regarded as essential for successful participation in a (future) digitized world. Regarding the increasing importance of AI, literacy concepts need to be extended to account for AI-related specifics. The easy h...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-08-01
|
| Series: | Frontiers in Computer Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fcomp.2021.685277/full |
| _version_ | 1818741468237398016 |
|---|---|
| author | Carolin Wienrich Astrid Carolus |
| author_facet | Carolin Wienrich Astrid Carolus |
| author_sort | Carolin Wienrich |
| collection | DOAJ |
| description | The concept of digital literacy has been introduced as a new cultural technique, which is regarded as essential for successful participation in a (future) digitized world. Regarding the increasing importance of AI, literacy concepts need to be extended to account for AI-related specifics. The easy handling of the systems results in increased usage, contrasting limited conceptualizations (e.g., imagination of future importance) and competencies (e.g., knowledge about functional principles). In reference to voice-based conversational agents as a concrete application of AI, the present paper aims for the development of a measurement to assess the conceptualizations and competencies about conversational agents. In a first step, a theoretical framework of “AI literacy” is transferred to the context of conversational agent literacy. Second, the “conversational agent literacy scale” (short CALS) is developed, constituting the first attempt to measure interindividual differences in the “(il) literate” usage of conversational agents. 29 items were derived, of which 170 participants answered. An explanatory factor analysis identified five factors leading to five subscales to assess CAL: storage and transfer of the smart speaker’s data input; smart speaker’s functional principles; smart speaker’s intelligent functions, learning abilities; smart speaker’s reach and potential; smart speaker’s technological (surrounding) infrastructure. Preliminary insights into construct validity and reliability of CALS showed satisfying results. Third, using the newly developed instrument, a student sample’s CAL was assessed, revealing intermediated values. Remarkably, owning a smart speaker did not lead to higher CAL scores, confirming our basic assumption that usage of systems does not guarantee enlightened conceptualizations and competencies. In sum, the paper contributes to the first insights into the operationalization and understanding of CAL as a specific subdomain of AI-related competencies. |
| first_indexed | 2024-12-18T01:57:06Z |
| format | Article |
| id | doaj.art-8f5a54096e58482e96ea3f1dff81a48c |
| institution | Directory Open Access Journal |
| issn | 2624-9898 |
| language | English |
| last_indexed | 2024-12-18T01:57:06Z |
| publishDate | 2021-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computer Science |
| spelling | doaj.art-8f5a54096e58482e96ea3f1dff81a48c2022-12-21T21:24:54ZengFrontiers Media S.A.Frontiers in Computer Science2624-98982021-08-01310.3389/fcomp.2021.685277685277Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart SpeakersCarolin Wienrich0Astrid Carolus1Institute Human-Computer-Media, Human-Technique-Systems, Julius-Maximilians-University, Wuerzburg, GermanyInstitute Human-Computer-Media, Media Psychology, Julius-Maximilians-University, Wuerzburg, GermanyThe concept of digital literacy has been introduced as a new cultural technique, which is regarded as essential for successful participation in a (future) digitized world. Regarding the increasing importance of AI, literacy concepts need to be extended to account for AI-related specifics. The easy handling of the systems results in increased usage, contrasting limited conceptualizations (e.g., imagination of future importance) and competencies (e.g., knowledge about functional principles). In reference to voice-based conversational agents as a concrete application of AI, the present paper aims for the development of a measurement to assess the conceptualizations and competencies about conversational agents. In a first step, a theoretical framework of “AI literacy” is transferred to the context of conversational agent literacy. Second, the “conversational agent literacy scale” (short CALS) is developed, constituting the first attempt to measure interindividual differences in the “(il) literate” usage of conversational agents. 29 items were derived, of which 170 participants answered. An explanatory factor analysis identified five factors leading to five subscales to assess CAL: storage and transfer of the smart speaker’s data input; smart speaker’s functional principles; smart speaker’s intelligent functions, learning abilities; smart speaker’s reach and potential; smart speaker’s technological (surrounding) infrastructure. Preliminary insights into construct validity and reliability of CALS showed satisfying results. Third, using the newly developed instrument, a student sample’s CAL was assessed, revealing intermediated values. Remarkably, owning a smart speaker did not lead to higher CAL scores, confirming our basic assumption that usage of systems does not guarantee enlightened conceptualizations and competencies. In sum, the paper contributes to the first insights into the operationalization and understanding of CAL as a specific subdomain of AI-related competencies.https://www.frontiersin.org/articles/10.3389/fcomp.2021.685277/fullartificial intelligence literacyartificial intelligence educationvoice-based artificial intelligenceconversational agentsmeasurement |
| spellingShingle | Carolin Wienrich Astrid Carolus Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart Speakers Frontiers in Computer Science artificial intelligence literacy artificial intelligence education voice-based artificial intelligence conversational agents measurement |
| title | Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart Speakers |
| title_full | Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart Speakers |
| title_fullStr | Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart Speakers |
| title_full_unstemmed | Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart Speakers |
| title_short | Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart Speakers |
| title_sort | development of an instrument to measure conceptualizations and competencies about conversational agents on the example of smart speakers |
| topic | artificial intelligence literacy artificial intelligence education voice-based artificial intelligence conversational agents measurement |
| url | https://www.frontiersin.org/articles/10.3389/fcomp.2021.685277/full |
| work_keys_str_mv | AT carolinwienrich developmentofaninstrumenttomeasureconceptualizationsandcompetenciesaboutconversationalagentsontheexampleofsmartspeakers AT astridcarolus developmentofaninstrumenttomeasureconceptualizationsandcompetenciesaboutconversationalagentsontheexampleofsmartspeakers |