Configural relations in humans and deep convolutional neural networks
Deep convolutional neural networks (DCNNs) have attracted considerable interest as useful devices and as possible windows into understanding perception and cognition in biological systems. In earlier work, we showed that DCNNs differ dramatically from human perceivers in that they have no sensitivit...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-03-01
|
| Series: | Frontiers in Artificial Intelligence |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frai.2022.961595/full |
| _version_ | 1811162616593121280 |
|---|---|
| author | Nicholas Baker Patrick Garrigan Austin Phillips Philip J. Kellman |
| author_facet | Nicholas Baker Patrick Garrigan Austin Phillips Philip J. Kellman |
| author_sort | Nicholas Baker |
| collection | DOAJ |
| description | Deep convolutional neural networks (DCNNs) have attracted considerable interest as useful devices and as possible windows into understanding perception and cognition in biological systems. In earlier work, we showed that DCNNs differ dramatically from human perceivers in that they have no sensitivity to global object shape. Here, we investigated whether those findings are symptomatic of broader limitations of DCNNs regarding the use of relations. We tested learning and generalization of DCNNs (AlexNet and ResNet-50) for several relations involving objects. One involved classifying two shapes in an otherwise empty field as same or different. Another involved enclosure. Every display contained a closed figure among contour noise fragments and one dot; correct responding depended on whether the dot was inside or outside the figure. The third relation we tested involved a classification that depended on which of two polygons had more sides. One polygon always contained a dot, and correct classification of each display depended on whether the polygon with the dot had a greater number of sides. We used DCNNs that had been trained on the ImageNet database, and we used both restricted and unrestricted transfer learning (connection weights at all layers could change with training). For the same-different experiment, there was little restricted transfer learning (82.2%). Generalization tests showed near chance performance for new shapes. Results for enclosure were at chance for restricted transfer learning and somewhat better for unrestricted (74%). Generalization with two new kinds of shapes showed reduced but above-chance performance (≈66%). Follow-up studies indicated that the networks did not access the enclosure relation in their responses. For the relation of more or fewer sides of polygons, DCNNs showed successful learning with polygons having 3–5 sides under unrestricted transfer learning, but showed chance performance in generalization tests with polygons having 6–10 sides. Experiments with human observers showed learning from relatively few examples of all of the relations tested and complete generalization of relational learning to new stimuli. These results using several different relations suggest that DCNNs have crucial limitations that derive from their lack of computations involving abstraction and relational processing of the sort that are fundamental in human perception. |
| first_indexed | 2024-04-10T06:32:15Z |
| format | Article |
| id | doaj.art-c35b2fde8f9e47f7bddd84a495e48e2f |
| institution | Directory Open Access Journal |
| issn | 2624-8212 |
| language | English |
| last_indexed | 2024-04-10T06:32:15Z |
| publishDate | 2023-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Artificial Intelligence |
| spelling | doaj.art-c35b2fde8f9e47f7bddd84a495e48e2f2023-03-01T06:00:51ZengFrontiers Media S.A.Frontiers in Artificial Intelligence2624-82122023-03-01510.3389/frai.2022.961595961595Configural relations in humans and deep convolutional neural networksNicholas Baker0Patrick Garrigan1Austin Phillips2Philip J. Kellman3Department of Psychology, Loyola University Chicago, Chicago, IL, United StatesDepartment of Psychology, Saint Joseph's University, Philadelphia, PA, United StatesUCLA Human Perception Laboratory, Department of Psychology, University of California, Los Angeles, Los Angeles, CA, United StatesUCLA Human Perception Laboratory, Department of Psychology, University of California, Los Angeles, Los Angeles, CA, United StatesDeep convolutional neural networks (DCNNs) have attracted considerable interest as useful devices and as possible windows into understanding perception and cognition in biological systems. In earlier work, we showed that DCNNs differ dramatically from human perceivers in that they have no sensitivity to global object shape. Here, we investigated whether those findings are symptomatic of broader limitations of DCNNs regarding the use of relations. We tested learning and generalization of DCNNs (AlexNet and ResNet-50) for several relations involving objects. One involved classifying two shapes in an otherwise empty field as same or different. Another involved enclosure. Every display contained a closed figure among contour noise fragments and one dot; correct responding depended on whether the dot was inside or outside the figure. The third relation we tested involved a classification that depended on which of two polygons had more sides. One polygon always contained a dot, and correct classification of each display depended on whether the polygon with the dot had a greater number of sides. We used DCNNs that had been trained on the ImageNet database, and we used both restricted and unrestricted transfer learning (connection weights at all layers could change with training). For the same-different experiment, there was little restricted transfer learning (82.2%). Generalization tests showed near chance performance for new shapes. Results for enclosure were at chance for restricted transfer learning and somewhat better for unrestricted (74%). Generalization with two new kinds of shapes showed reduced but above-chance performance (≈66%). Follow-up studies indicated that the networks did not access the enclosure relation in their responses. For the relation of more or fewer sides of polygons, DCNNs showed successful learning with polygons having 3–5 sides under unrestricted transfer learning, but showed chance performance in generalization tests with polygons having 6–10 sides. Experiments with human observers showed learning from relatively few examples of all of the relations tested and complete generalization of relational learning to new stimuli. These results using several different relations suggest that DCNNs have crucial limitations that derive from their lack of computations involving abstraction and relational processing of the sort that are fundamental in human perception.https://www.frontiersin.org/articles/10.3389/frai.2022.961595/fullperception of relationsdeep convolutional neural networksDCNNsdeep learningabstract relationsvisual relations |
| spellingShingle | Nicholas Baker Patrick Garrigan Austin Phillips Philip J. Kellman Configural relations in humans and deep convolutional neural networks Frontiers in Artificial Intelligence perception of relations deep convolutional neural networks DCNNs deep learning abstract relations visual relations |
| title | Configural relations in humans and deep convolutional neural networks |
| title_full | Configural relations in humans and deep convolutional neural networks |
| title_fullStr | Configural relations in humans and deep convolutional neural networks |
| title_full_unstemmed | Configural relations in humans and deep convolutional neural networks |
| title_short | Configural relations in humans and deep convolutional neural networks |
| title_sort | configural relations in humans and deep convolutional neural networks |
| topic | perception of relations deep convolutional neural networks DCNNs deep learning abstract relations visual relations |
| url | https://www.frontiersin.org/articles/10.3389/frai.2022.961595/full |
| work_keys_str_mv | AT nicholasbaker configuralrelationsinhumansanddeepconvolutionalneuralnetworks AT patrickgarrigan configuralrelationsinhumansanddeepconvolutionalneuralnetworks AT austinphillips configuralrelationsinhumansanddeepconvolutionalneuralnetworks AT philipjkellman configuralrelationsinhumansanddeepconvolutionalneuralnetworks |