Using human brain activity to guide machine learning
Abstract Machine learning is a field of computer science that builds algorithms that learn. In many cases, machine learning algorithms are used to recreate a human ability like adding a caption to a photo, driving a car, or playing a game. While the human brain has long served as a source of inspira...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2018-03-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-018-23618-6 |
| _version_ | 1818991405512523776 |
|---|---|
| author | Ruth C. Fong Walter J. Scheirer David D. Cox |
| author_facet | Ruth C. Fong Walter J. Scheirer David D. Cox |
| author_sort | Ruth C. Fong |
| collection | DOAJ |
| description | Abstract Machine learning is a field of computer science that builds algorithms that learn. In many cases, machine learning algorithms are used to recreate a human ability like adding a caption to a photo, driving a car, or playing a game. While the human brain has long served as a source of inspiration for machine learning, little effort has been made to directly use data collected from working brains as a guide for machine learning algorithms. Here we demonstrate a new paradigm of “neurally-weighted” machine learning, which takes fMRI measurements of human brain activity from subjects viewing images, and infuses these data into the training process of an object recognition learning algorithm to make it more consistent with the human brain. After training, these neurally-weighted classifiers are able to classify images without requiring any additional neural data. We show that our neural-weighting approach can lead to large performance gains when used with traditional machine vision features, as well as to significant improvements with already high-performing convolutional neural network features. The effectiveness of this approach points to a path forward for a new class of hybrid machine learning algorithms which take both inspiration and direct constraints from neuronal data. |
| first_indexed | 2024-12-20T20:09:45Z |
| format | Article |
| id | doaj.art-2e55a068e7f449f5ad9e29c3ff4f18bd |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-12-20T20:09:45Z |
| publishDate | 2018-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-2e55a068e7f449f5ad9e29c3ff4f18bd2022-12-21T19:27:50ZengNature PortfolioScientific Reports2045-23222018-03-018111010.1038/s41598-018-23618-6Using human brain activity to guide machine learningRuth C. Fong0Walter J. Scheirer1David D. Cox2Department of Engineering Science, University of Oxford, Information Engineering BuildingDepartment of Computer Science and Engineering, University of Notre Dame, Fitzpatrick Hall of EngineeringDepartment of Molecular and Cellular Biology, School of Engineering and Applied Sciences and Center for Brain Science, Harvard UniversityAbstract Machine learning is a field of computer science that builds algorithms that learn. In many cases, machine learning algorithms are used to recreate a human ability like adding a caption to a photo, driving a car, or playing a game. While the human brain has long served as a source of inspiration for machine learning, little effort has been made to directly use data collected from working brains as a guide for machine learning algorithms. Here we demonstrate a new paradigm of “neurally-weighted” machine learning, which takes fMRI measurements of human brain activity from subjects viewing images, and infuses these data into the training process of an object recognition learning algorithm to make it more consistent with the human brain. After training, these neurally-weighted classifiers are able to classify images without requiring any additional neural data. We show that our neural-weighting approach can lead to large performance gains when used with traditional machine vision features, as well as to significant improvements with already high-performing convolutional neural network features. The effectiveness of this approach points to a path forward for a new class of hybrid machine learning algorithms which take both inspiration and direct constraints from neuronal data.https://doi.org/10.1038/s41598-018-23618-6 |
| spellingShingle | Ruth C. Fong Walter J. Scheirer David D. Cox Using human brain activity to guide machine learning Scientific Reports |
| title | Using human brain activity to guide machine learning |
| title_full | Using human brain activity to guide machine learning |
| title_fullStr | Using human brain activity to guide machine learning |
| title_full_unstemmed | Using human brain activity to guide machine learning |
| title_short | Using human brain activity to guide machine learning |
| title_sort | using human brain activity to guide machine learning |
| url | https://doi.org/10.1038/s41598-018-23618-6 |
| work_keys_str_mv | AT ruthcfong usinghumanbrainactivitytoguidemachinelearning AT walterjscheirer usinghumanbrainactivitytoguidemachinelearning AT daviddcox usinghumanbrainactivitytoguidemachinelearning |