ACCOUNTING FOR USERS' INFLATED ASSESSMENTS OF ONLINE CATALOG SEARCH PERFORMANCE AND USEFULNESS: AN EXPERIMENTAL STUDY
User-oriented approaches to information retrieval (IR) system performance evaluation assign a major role to user satisfaction with search results and overall system performance. Expressed satisfaction with search results is often used as a measure of utility. Many research studies indicate that user...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
University of Borås
2001-01-01
|
| Series: | Information Research: An International Electronic Journal |
| Subjects: | |
| Online Access: | http://informationr.net/ir/6-2/paper101.html |
| _version_ | 1818833783490609152 |
|---|---|
| author | Charles R. Hildreth |
| author_facet | Charles R. Hildreth |
| author_sort | Charles R. Hildreth |
| collection | DOAJ |
| description | User-oriented approaches to information retrieval (IR) system performance evaluation assign a major role to user satisfaction with search results and overall system performance. Expressed satisfaction with search results is often used as a measure of utility. Many research studies indicate that users of on-line library catalogs (OPACs) and other IR systems often express satisfaction with poor search results. This phenomenon of "false positives," inflated assessments of search results and system performance, has not been adequately explained. Non-performance factors such as interface style and ease of use may have an affect on a searcher's satisfaction with search results. The research described in this report investigates this phenomenon. This paper presents the findings of an experimental study which compared users' search performance and assessments of ease of use, system usefulness, and satisfaction with search results after use of a Web OPAC or its conventional counterpart. The primary questions addressed by this research center on the influence of two experimental factors, OPAC search interface style and search task level of difficulty, on the dependent variables: actual search performance, perceptions of ease of use and system usefulness, and assessments of satisfaction with search results. The study also investigated associations between perceived ease of use, system usefulness, and satisfaction with search results. Lastly, the study looked for associations between the dependent variables and personal characteristics. No association was found between satisfaction with search results and actual search performance. Web OPAC searchers outperformed Text OPAC searchers, but search task level of difficulty is a major determinant of search success. A strong positive correlation was found between perceptions of system ease of use and assessments of search results. |
| first_indexed | 2024-12-19T02:24:25Z |
| format | Article |
| id | doaj.art-2e09abdef7cd460596b4c3911918673f |
| institution | Directory Open Access Journal |
| issn | 1368-1613 |
| language | English |
| last_indexed | 2024-12-19T02:24:25Z |
| publishDate | 2001-01-01 |
| publisher | University of Borås |
| record_format | Article |
| series | Information Research: An International Electronic Journal |
| spelling | doaj.art-2e09abdef7cd460596b4c3911918673f2022-12-21T20:40:01ZengUniversity of BoråsInformation Research: An International Electronic Journal1368-16132001-01-0162101ACCOUNTING FOR USERS' INFLATED ASSESSMENTS OF ONLINE CATALOG SEARCH PERFORMANCE AND USEFULNESS: AN EXPERIMENTAL STUDYCharles R. HildrethUser-oriented approaches to information retrieval (IR) system performance evaluation assign a major role to user satisfaction with search results and overall system performance. Expressed satisfaction with search results is often used as a measure of utility. Many research studies indicate that users of on-line library catalogs (OPACs) and other IR systems often express satisfaction with poor search results. This phenomenon of "false positives," inflated assessments of search results and system performance, has not been adequately explained. Non-performance factors such as interface style and ease of use may have an affect on a searcher's satisfaction with search results. The research described in this report investigates this phenomenon. This paper presents the findings of an experimental study which compared users' search performance and assessments of ease of use, system usefulness, and satisfaction with search results after use of a Web OPAC or its conventional counterpart. The primary questions addressed by this research center on the influence of two experimental factors, OPAC search interface style and search task level of difficulty, on the dependent variables: actual search performance, perceptions of ease of use and system usefulness, and assessments of satisfaction with search results. The study also investigated associations between perceived ease of use, system usefulness, and satisfaction with search results. Lastly, the study looked for associations between the dependent variables and personal characteristics. No association was found between satisfaction with search results and actual search performance. Web OPAC searchers outperformed Text OPAC searchers, but search task level of difficulty is a major determinant of search success. A strong positive correlation was found between perceptions of system ease of use and assessments of search results.http://informationr.net/ir/6-2/paper101.htmluser-orientationusersinformation retrievalIRperformanceevaluationuser satisfactionsearch resultsutilityon-line library cataloguescatalogsOPACsfalse positivesinterfaceease of usesystemusefulnesssatisfactionWebsearch taskdifficultypersonal characteristics |
| spellingShingle | Charles R. Hildreth ACCOUNTING FOR USERS' INFLATED ASSESSMENTS OF ONLINE CATALOG SEARCH PERFORMANCE AND USEFULNESS: AN EXPERIMENTAL STUDY Information Research: An International Electronic Journal user-orientation users information retrieval IR performance evaluation user satisfaction search results utility on-line library catalogues catalogs OPACs false positives interface ease of use system usefulness satisfaction Web search task difficulty personal characteristics |
| title | ACCOUNTING FOR USERS' INFLATED ASSESSMENTS OF ONLINE CATALOG SEARCH PERFORMANCE AND USEFULNESS: AN EXPERIMENTAL STUDY |
| title_full | ACCOUNTING FOR USERS' INFLATED ASSESSMENTS OF ONLINE CATALOG SEARCH PERFORMANCE AND USEFULNESS: AN EXPERIMENTAL STUDY |
| title_fullStr | ACCOUNTING FOR USERS' INFLATED ASSESSMENTS OF ONLINE CATALOG SEARCH PERFORMANCE AND USEFULNESS: AN EXPERIMENTAL STUDY |
| title_full_unstemmed | ACCOUNTING FOR USERS' INFLATED ASSESSMENTS OF ONLINE CATALOG SEARCH PERFORMANCE AND USEFULNESS: AN EXPERIMENTAL STUDY |
| title_short | ACCOUNTING FOR USERS' INFLATED ASSESSMENTS OF ONLINE CATALOG SEARCH PERFORMANCE AND USEFULNESS: AN EXPERIMENTAL STUDY |
| title_sort | accounting for users inflated assessments of online catalog search performance and usefulness an experimental study |
| topic | user-orientation users information retrieval IR performance evaluation user satisfaction search results utility on-line library catalogues catalogs OPACs false positives interface ease of use system usefulness satisfaction Web search task difficulty personal characteristics |
| url | http://informationr.net/ir/6-2/paper101.html |
| work_keys_str_mv | AT charlesrhildreth accountingforusersinflatedassessmentsofonlinecatalogsearchperformanceandusefulnessanexperimentalstudy |