On the expected size of conformal prediction sets
While conformal predictors reap the benefits of rigorous statistical guarantees on their error frequency, the size of their corresponding prediction sets is critical to their practical utility. Unfortunately, there is currently a lack of finite-sample analysis and guarantees for their prediction set...
| Main Authors: | , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Journal of Machine Learning Research
2023
|
| _version_ | 1811139667704152064 |
|---|---|
| author | Dhillon, GS Deligiannidis, G Rainforth, T |
| author_facet | Dhillon, GS Deligiannidis, G Rainforth, T |
| author_sort | Dhillon, GS |
| collection | OXFORD |
| description | While conformal predictors reap the benefits of rigorous statistical
guarantees on their error frequency, the size of their corresponding prediction
sets is critical to their practical utility. Unfortunately, there is currently
a lack of finite-sample analysis and guarantees for their prediction set sizes.
To address this shortfall, we theoretically quantify the expected size of the
prediction sets under the split conformal prediction framework. As this precise
formulation cannot usually be calculated directly, we further derive point
estimates and high-probability interval bounds that can be empirically
computed, providing a practical method for characterizing the expected set
size. We corroborate the efficacy of our results with experiments on real-world
datasets for both regression and classification problems. |
| first_indexed | 2024-03-07T08:23:55Z |
| format | Conference item |
| id | oxford-uuid:eda92b14-9fe8-410d-abc9-87ed1c61b30c |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:09:44Z |
| publishDate | 2023 |
| publisher | Journal of Machine Learning Research |
| record_format | dspace |
| spelling | oxford-uuid:eda92b14-9fe8-410d-abc9-87ed1c61b30c2024-06-07T09:37:19ZOn the expected size of conformal prediction setsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:eda92b14-9fe8-410d-abc9-87ed1c61b30cEnglishSymplectic ElementsJournal of Machine Learning Research2023Dhillon, GSDeligiannidis, GRainforth, TWhile conformal predictors reap the benefits of rigorous statistical guarantees on their error frequency, the size of their corresponding prediction sets is critical to their practical utility. Unfortunately, there is currently a lack of finite-sample analysis and guarantees for their prediction set sizes. To address this shortfall, we theoretically quantify the expected size of the prediction sets under the split conformal prediction framework. As this precise formulation cannot usually be calculated directly, we further derive point estimates and high-probability interval bounds that can be empirically computed, providing a practical method for characterizing the expected set size. We corroborate the efficacy of our results with experiments on real-world datasets for both regression and classification problems. |
| spellingShingle | Dhillon, GS Deligiannidis, G Rainforth, T On the expected size of conformal prediction sets |
| title | On the expected size of conformal prediction sets |
| title_full | On the expected size of conformal prediction sets |
| title_fullStr | On the expected size of conformal prediction sets |
| title_full_unstemmed | On the expected size of conformal prediction sets |
| title_short | On the expected size of conformal prediction sets |
| title_sort | on the expected size of conformal prediction sets |
| work_keys_str_mv | AT dhillongs ontheexpectedsizeofconformalpredictionsets AT deligiannidisg ontheexpectedsizeofconformalpredictionsets AT rainfortht ontheexpectedsizeofconformalpredictionsets |