Correction of Threshold Determination in Rapid-Guessing Behaviour Detection
Assessment is one benchmark in measuring students’ abilities. However, assessment results cannot necessarily be trusted, because students sometimes cheat or even guess in answering the questions. Therefore, to obtain valid results, it is necessary to separate valid and invalid answers by considering...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2078-2489/14/7/422 |
| _version_ | 1827732871303921664 |
|---|---|
| author | Muhammad Alfian Umi Laili Yuhana Eric Pardede Akbar Noto Ponco Bimantoro |
| author_facet | Muhammad Alfian Umi Laili Yuhana Eric Pardede Akbar Noto Ponco Bimantoro |
| author_sort | Muhammad Alfian |
| collection | DOAJ |
| description | Assessment is one benchmark in measuring students’ abilities. However, assessment results cannot necessarily be trusted, because students sometimes cheat or even guess in answering the questions. Therefore, to obtain valid results, it is necessary to separate valid and invalid answers by considering rapid-guessing behaviour. We conducted a test to record exam log data from undergraduate and postgraduate students to model rapid-guessing behaviour by determining the threshold response time. Rapid-guessing behaviour detection is inspired by the common k-second method. However, the method flattens the application of the threshold, thus allowing misclassification. The modified method considers item difficulty in determining the threshold. The evaluation results show that the system can identify students’ rapid-guessing behaviour with a success rate of 71%, which is superior to the previous method. We also analysed various aggregation techniques of response time and compared them to see the effect of selecting the aggregation technique. |
| first_indexed | 2024-03-11T00:59:07Z |
| format | Article |
| id | doaj.art-d45eeb7163a746deb232d58e84df7f55 |
| institution | Directory Open Access Journal |
| issn | 2078-2489 |
| language | English |
| last_indexed | 2024-03-11T00:59:07Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Information |
| spelling | doaj.art-d45eeb7163a746deb232d58e84df7f552023-11-18T19:47:26ZengMDPI AGInformation2078-24892023-07-0114742210.3390/info14070422Correction of Threshold Determination in Rapid-Guessing Behaviour DetectionMuhammad Alfian0Umi Laili Yuhana1Eric Pardede2Akbar Noto Ponco Bimantoro3Institut Teknologi Sepuluh Nopember, Surabaya 60111, IndonesiaInstitut Teknologi Sepuluh Nopember, Surabaya 60111, IndonesiaDepartment of Computer Science and Information Technology, La Trobe University, Melbourne, VIC 3000, AustraliaInstitut Teknologi Sepuluh Nopember, Surabaya 60111, IndonesiaAssessment is one benchmark in measuring students’ abilities. However, assessment results cannot necessarily be trusted, because students sometimes cheat or even guess in answering the questions. Therefore, to obtain valid results, it is necessary to separate valid and invalid answers by considering rapid-guessing behaviour. We conducted a test to record exam log data from undergraduate and postgraduate students to model rapid-guessing behaviour by determining the threshold response time. Rapid-guessing behaviour detection is inspired by the common k-second method. However, the method flattens the application of the threshold, thus allowing misclassification. The modified method considers item difficulty in determining the threshold. The evaluation results show that the system can identify students’ rapid-guessing behaviour with a success rate of 71%, which is superior to the previous method. We also analysed various aggregation techniques of response time and compared them to see the effect of selecting the aggregation technique.https://www.mdpi.com/2078-2489/14/7/422rapid-guessing behaviourthreshold determinationresponse time |
| spellingShingle | Muhammad Alfian Umi Laili Yuhana Eric Pardede Akbar Noto Ponco Bimantoro Correction of Threshold Determination in Rapid-Guessing Behaviour Detection Information rapid-guessing behaviour threshold determination response time |
| title | Correction of Threshold Determination in Rapid-Guessing Behaviour Detection |
| title_full | Correction of Threshold Determination in Rapid-Guessing Behaviour Detection |
| title_fullStr | Correction of Threshold Determination in Rapid-Guessing Behaviour Detection |
| title_full_unstemmed | Correction of Threshold Determination in Rapid-Guessing Behaviour Detection |
| title_short | Correction of Threshold Determination in Rapid-Guessing Behaviour Detection |
| title_sort | correction of threshold determination in rapid guessing behaviour detection |
| topic | rapid-guessing behaviour threshold determination response time |
| url | https://www.mdpi.com/2078-2489/14/7/422 |
| work_keys_str_mv | AT muhammadalfian correctionofthresholddeterminationinrapidguessingbehaviourdetection AT umilailiyuhana correctionofthresholddeterminationinrapidguessingbehaviourdetection AT ericpardede correctionofthresholddeterminationinrapidguessingbehaviourdetection AT akbarnotoponcobimantoro correctionofthresholddeterminationinrapidguessingbehaviourdetection |