Intra-Class Mixup for Out-of-Distribution Detection
Deep neural networks (DNNs) have found widespread adoption in solving image recognition and natural language processing tasks. However, they make confident mispredictions when presented with data that does not belong to the training distribution, i.e. out-of-distribution (OoD) samples. Research has...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10064300/ |
| _version_ | 1797866960356114432 |
|---|---|
| author | Deepak Ravikumar Sangamesh Kodge Isha Garg Kaushik Roy |
| author_facet | Deepak Ravikumar Sangamesh Kodge Isha Garg Kaushik Roy |
| author_sort | Deepak Ravikumar |
| collection | DOAJ |
| description | Deep neural networks (DNNs) have found widespread adoption in solving image recognition and natural language processing tasks. However, they make confident mispredictions when presented with data that does not belong to the training distribution, i.e. out-of-distribution (OoD) samples. Research has shown that angular representations can be useful to address the curse of dimensionality and improve OoD detection performance. However, when evaluating the angular separability using Fisher’s criterion we find that empirical risk minimization and inter-class mixup trained DNNs have low angular separability between in-distribution data and OoD data. To improve angular separability, we propose intra-class mixup. We provide mathematical reasoning that shows that intra-class mixup results in reduced angular spread because of reduced variance at the input during training. Further, to take full advantage of improved angular separability from intra-class mixup we propose supplementing the separation metric with the cosine of angular margin to improve OoD detection. Angular margin is the angle between the final layer weight vector and the sample representation. The proposed intra-class mixup when applied to various existing OoD detection techniques shows an improvement of 4.21% and 6.21% in AUROC performance over empirical risk minimization and inter-class mixup, respectively. Further, intra-class mixup aided with the cosine of angular margin improves AUROC performance by 6.71% and 8.75% over empirical risk minimization and inter-class mixup, respectively. |
| first_indexed | 2024-04-09T23:33:44Z |
| format | Article |
| id | doaj.art-286cd5c4316f4bcf8f72a6fad313fe74 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-09T23:33:44Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-286cd5c4316f4bcf8f72a6fad313fe742023-03-20T23:00:14ZengIEEEIEEE Access2169-35362023-01-0111259682598110.1109/ACCESS.2023.325492010064300Intra-Class Mixup for Out-of-Distribution DetectionDeepak Ravikumar0https://orcid.org/0000-0001-6736-3250Sangamesh Kodge1https://orcid.org/0000-0001-9713-5400Isha Garg2https://orcid.org/0000-0003-4702-9444Kaushik Roy3https://orcid.org/0000-0002-0735-9695School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USASchool of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USASchool of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USASchool of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USADeep neural networks (DNNs) have found widespread adoption in solving image recognition and natural language processing tasks. However, they make confident mispredictions when presented with data that does not belong to the training distribution, i.e. out-of-distribution (OoD) samples. Research has shown that angular representations can be useful to address the curse of dimensionality and improve OoD detection performance. However, when evaluating the angular separability using Fisher’s criterion we find that empirical risk minimization and inter-class mixup trained DNNs have low angular separability between in-distribution data and OoD data. To improve angular separability, we propose intra-class mixup. We provide mathematical reasoning that shows that intra-class mixup results in reduced angular spread because of reduced variance at the input during training. Further, to take full advantage of improved angular separability from intra-class mixup we propose supplementing the separation metric with the cosine of angular margin to improve OoD detection. Angular margin is the angle between the final layer weight vector and the sample representation. The proposed intra-class mixup when applied to various existing OoD detection techniques shows an improvement of 4.21% and 6.21% in AUROC performance over empirical risk minimization and inter-class mixup, respectively. Further, intra-class mixup aided with the cosine of angular margin improves AUROC performance by 6.71% and 8.75% over empirical risk minimization and inter-class mixup, respectively.https://ieeexplore.ieee.org/document/10064300/Deep learningdeep neural netsmixupintra-class mixupout-of-distribution detectionangular margin |
| spellingShingle | Deepak Ravikumar Sangamesh Kodge Isha Garg Kaushik Roy Intra-Class Mixup for Out-of-Distribution Detection IEEE Access Deep learning deep neural nets mixup intra-class mixup out-of-distribution detection angular margin |
| title | Intra-Class Mixup for Out-of-Distribution Detection |
| title_full | Intra-Class Mixup for Out-of-Distribution Detection |
| title_fullStr | Intra-Class Mixup for Out-of-Distribution Detection |
| title_full_unstemmed | Intra-Class Mixup for Out-of-Distribution Detection |
| title_short | Intra-Class Mixup for Out-of-Distribution Detection |
| title_sort | intra class mixup for out of distribution detection |
| topic | Deep learning deep neural nets mixup intra-class mixup out-of-distribution detection angular margin |
| url | https://ieeexplore.ieee.org/document/10064300/ |
| work_keys_str_mv | AT deepakravikumar intraclassmixupforoutofdistributiondetection AT sangameshkodge intraclassmixupforoutofdistributiondetection AT ishagarg intraclassmixupforoutofdistributiondetection AT kaushikroy intraclassmixupforoutofdistributiondetection |