Understanding Robustness and Generalization of Artificial Neural Networks Through Fourier Masks
Despite the enormous success of artificial neural networks (ANNs) in many disciplines, the characterization of their computations and the origin of key properties such as generalization and robustness remain open questions. Recent literature suggests that robust networks with good generalization pro...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-07-01
|
| Series: | Frontiers in Artificial Intelligence |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frai.2022.890016/full |
| _version_ | 1818108390855933952 |
|---|---|
| author | Nikos Karantzas Nikos Karantzas Emma Besier Emma Besier Josue Ortega Caro Josue Ortega Caro Xaq Pitkow Xaq Pitkow Xaq Pitkow Andreas S. Tolias Andreas S. Tolias Andreas S. Tolias Ankit B. Patel Ankit B. Patel Ankit B. Patel Fabio Anselmi Fabio Anselmi |
| author_facet | Nikos Karantzas Nikos Karantzas Emma Besier Emma Besier Josue Ortega Caro Josue Ortega Caro Xaq Pitkow Xaq Pitkow Xaq Pitkow Andreas S. Tolias Andreas S. Tolias Andreas S. Tolias Ankit B. Patel Ankit B. Patel Ankit B. Patel Fabio Anselmi Fabio Anselmi |
| author_sort | Nikos Karantzas |
| collection | DOAJ |
| description | Despite the enormous success of artificial neural networks (ANNs) in many disciplines, the characterization of their computations and the origin of key properties such as generalization and robustness remain open questions. Recent literature suggests that robust networks with good generalization properties tend to be biased toward processing low frequencies in images. To explore the frequency bias hypothesis further, we develop an algorithm that allows us to learn modulatory masks highlighting the essential input frequencies needed for preserving a trained network's performance. We achieve this by imposing invariance in the loss with respect to such modulations in the input frequencies. We first use our method to test the low-frequency preference hypothesis of adversarially trained or data-augmented networks. Our results suggest that adversarially robust networks indeed exhibit a low-frequency bias but we find this bias is also dependent on directions in frequency space. However, this is not necessarily true for other types of data augmentation. Our results also indicate that the essential frequencies in question are effectively the ones used to achieve generalization in the first place. Surprisingly, images seen through these modulatory masks are not recognizable and resemble texture-like patterns. |
| first_indexed | 2024-12-11T02:14:36Z |
| format | Article |
| id | doaj.art-7ea7bb52abb44d12a7344e304625acda |
| institution | Directory Open Access Journal |
| issn | 2624-8212 |
| language | English |
| last_indexed | 2024-12-11T02:14:36Z |
| publishDate | 2022-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Artificial Intelligence |
| spelling | doaj.art-7ea7bb52abb44d12a7344e304625acda2022-12-22T01:24:12ZengFrontiers Media S.A.Frontiers in Artificial Intelligence2624-82122022-07-01510.3389/frai.2022.890016890016Understanding Robustness and Generalization of Artificial Neural Networks Through Fourier MasksNikos Karantzas0Nikos Karantzas1Emma Besier2Emma Besier3Josue Ortega Caro4Josue Ortega Caro5Xaq Pitkow6Xaq Pitkow7Xaq Pitkow8Andreas S. Tolias9Andreas S. Tolias10Andreas S. Tolias11Ankit B. Patel12Ankit B. Patel13Ankit B. Patel14Fabio Anselmi15Fabio Anselmi16Department of Neuroscience, Baylor College of Medicine, Houston, TX, United StatesCenter for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, United StatesDepartment of Neuroscience, Baylor College of Medicine, Houston, TX, United StatesCenter for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, United StatesDepartment of Neuroscience, Baylor College of Medicine, Houston, TX, United StatesCenter for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, United StatesDepartment of Neuroscience, Baylor College of Medicine, Houston, TX, United StatesCenter for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, United StatesDepartment of Electrical and Computer Engineering, Rice University, Houston, TX, United StatesDepartment of Neuroscience, Baylor College of Medicine, Houston, TX, United StatesCenter for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, United StatesDepartment of Electrical and Computer Engineering, Rice University, Houston, TX, United StatesDepartment of Neuroscience, Baylor College of Medicine, Houston, TX, United StatesCenter for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, United StatesDepartment of Electrical and Computer Engineering, Rice University, Houston, TX, United StatesDepartment of Neuroscience, Baylor College of Medicine, Houston, TX, United StatesCenter for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, United StatesDespite the enormous success of artificial neural networks (ANNs) in many disciplines, the characterization of their computations and the origin of key properties such as generalization and robustness remain open questions. Recent literature suggests that robust networks with good generalization properties tend to be biased toward processing low frequencies in images. To explore the frequency bias hypothesis further, we develop an algorithm that allows us to learn modulatory masks highlighting the essential input frequencies needed for preserving a trained network's performance. We achieve this by imposing invariance in the loss with respect to such modulations in the input frequencies. We first use our method to test the low-frequency preference hypothesis of adversarially trained or data-augmented networks. Our results suggest that adversarially robust networks indeed exhibit a low-frequency bias but we find this bias is also dependent on directions in frequency space. However, this is not necessarily true for other types of data augmentation. Our results also indicate that the essential frequencies in question are effectively the ones used to achieve generalization in the first place. Surprisingly, images seen through these modulatory masks are not recognizable and resemble texture-like patterns.https://www.frontiersin.org/articles/10.3389/frai.2022.890016/fullFourier analysissymmetryrobustnessgeneralizationneural networksdata augmentation |
| spellingShingle | Nikos Karantzas Nikos Karantzas Emma Besier Emma Besier Josue Ortega Caro Josue Ortega Caro Xaq Pitkow Xaq Pitkow Xaq Pitkow Andreas S. Tolias Andreas S. Tolias Andreas S. Tolias Ankit B. Patel Ankit B. Patel Ankit B. Patel Fabio Anselmi Fabio Anselmi Understanding Robustness and Generalization of Artificial Neural Networks Through Fourier Masks Frontiers in Artificial Intelligence Fourier analysis symmetry robustness generalization neural networks data augmentation |
| title | Understanding Robustness and Generalization of Artificial Neural Networks Through Fourier Masks |
| title_full | Understanding Robustness and Generalization of Artificial Neural Networks Through Fourier Masks |
| title_fullStr | Understanding Robustness and Generalization of Artificial Neural Networks Through Fourier Masks |
| title_full_unstemmed | Understanding Robustness and Generalization of Artificial Neural Networks Through Fourier Masks |
| title_short | Understanding Robustness and Generalization of Artificial Neural Networks Through Fourier Masks |
| title_sort | understanding robustness and generalization of artificial neural networks through fourier masks |
| topic | Fourier analysis symmetry robustness generalization neural networks data augmentation |
| url | https://www.frontiersin.org/articles/10.3389/frai.2022.890016/full |
| work_keys_str_mv | AT nikoskarantzas understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT nikoskarantzas understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT emmabesier understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT emmabesier understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT josueortegacaro understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT josueortegacaro understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT xaqpitkow understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT xaqpitkow understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT xaqpitkow understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT andreasstolias understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT andreasstolias understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT andreasstolias understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT ankitbpatel understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT ankitbpatel understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT ankitbpatel understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT fabioanselmi understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks AT fabioanselmi understandingrobustnessandgeneralizationofartificialneuralnetworksthroughfouriermasks |