Limitations on robust ratings and predictions
Predictions are a well-studied form of ratings. Their objective nature allows a rigourous analysis. A problem is that there are attacks on prediction systems and rating systems. These attacks decrease the usefulness of the predictions. Attackers may ignore the incentives in the system, so we may not...
| Main Authors: | , , |
|---|---|
| Format: | Conference item |
| Published: |
Springer
2016
|
| _version_ | 1826296504248172544 |
|---|---|
| author | Muller, T Liu, Y Zhang, J |
| author_facet | Muller, T Liu, Y Zhang, J |
| author_sort | Muller, T |
| collection | OXFORD |
| description | Predictions are a well-studied form of ratings. Their objective nature allows a rigourous analysis. A problem is that there are attacks on prediction systems and rating systems. These attacks decrease the usefulness of the predictions. Attackers may ignore the incentives in the system, so we may not rely on these to protect ourselves. The user must block attackers, ideally before the attackers introduce too much misinformation.We formally axiomatically define robustness as the prop- erty that no rater can introduce too much misinformation. We formally prove that notions of robustness come at the expense of other desirable properties, such as the lack of bias or effectiveness. We also show that there do exist trade-offs between the different properties, allowing a prediction system with limited robustness, limited bias and limited effectiveness. |
| first_indexed | 2024-03-07T04:17:21Z |
| format | Conference item |
| id | oxford-uuid:c9d727c7-d242-4ef5-83a4-0c6c863f2fdf |
| institution | University of Oxford |
| last_indexed | 2024-03-07T04:17:21Z |
| publishDate | 2016 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:c9d727c7-d242-4ef5-83a4-0c6c863f2fdf2022-03-27T07:02:41ZLimitations on robust ratings and predictionsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:c9d727c7-d242-4ef5-83a4-0c6c863f2fdfSymplectic Elements at OxfordSpringer2016Muller, TLiu, YZhang, JPredictions are a well-studied form of ratings. Their objective nature allows a rigourous analysis. A problem is that there are attacks on prediction systems and rating systems. These attacks decrease the usefulness of the predictions. Attackers may ignore the incentives in the system, so we may not rely on these to protect ourselves. The user must block attackers, ideally before the attackers introduce too much misinformation.We formally axiomatically define robustness as the prop- erty that no rater can introduce too much misinformation. We formally prove that notions of robustness come at the expense of other desirable properties, such as the lack of bias or effectiveness. We also show that there do exist trade-offs between the different properties, allowing a prediction system with limited robustness, limited bias and limited effectiveness. |
| spellingShingle | Muller, T Liu, Y Zhang, J Limitations on robust ratings and predictions |
| title | Limitations on robust ratings and predictions |
| title_full | Limitations on robust ratings and predictions |
| title_fullStr | Limitations on robust ratings and predictions |
| title_full_unstemmed | Limitations on robust ratings and predictions |
| title_short | Limitations on robust ratings and predictions |
| title_sort | limitations on robust ratings and predictions |
| work_keys_str_mv | AT mullert limitationsonrobustratingsandpredictions AT liuy limitationsonrobustratingsandpredictions AT zhangj limitationsonrobustratingsandpredictions |