DeepAction: a MATLAB toolbox for automated classification of animal behavior in video
Abstract The identification of animal behavior in video is a critical but time-consuming task in many areas of research. Here, we introduce DeepAction, a deep learning-based toolbox for automatically annotating animal behavior in video. Our approach uses features extracted from raw video frames by a...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-02-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-29574-0 |
| _version_ | 1827984532847984640 |
|---|---|
| author | Carl Harris Kelly R. Finn Marie-Luise Kieseler Marvin R. Maechler Peter U. Tse |
| author_facet | Carl Harris Kelly R. Finn Marie-Luise Kieseler Marvin R. Maechler Peter U. Tse |
| author_sort | Carl Harris |
| collection | DOAJ |
| description | Abstract The identification of animal behavior in video is a critical but time-consuming task in many areas of research. Here, we introduce DeepAction, a deep learning-based toolbox for automatically annotating animal behavior in video. Our approach uses features extracted from raw video frames by a pretrained convolutional neural network to train a recurrent neural network classifier. We evaluate the classifier on two benchmark rodent datasets and one octopus dataset. We show that it achieves high accuracy, requires little training data, and surpasses both human agreement and most comparable existing methods. We also create a confidence score for classifier output, and show that our method provides an accurate estimate of classifier performance and reduces the time required by human annotators to review and correct automatically-produced annotations. We release our system and accompanying annotation interface as an open-source MATLAB toolbox. |
| first_indexed | 2024-04-09T23:01:17Z |
| format | Article |
| id | doaj.art-fc5b9e01b8114f88ac12adfd92476864 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-09T23:01:17Z |
| publishDate | 2023-02-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-fc5b9e01b8114f88ac12adfd924768642023-03-22T10:59:28ZengNature PortfolioScientific Reports2045-23222023-02-0113111910.1038/s41598-023-29574-0DeepAction: a MATLAB toolbox for automated classification of animal behavior in videoCarl Harris0Kelly R. Finn1Marie-Luise Kieseler2Marvin R. Maechler3Peter U. Tse4Department of Psychological and Brain Science, Dartmouth CollegeDepartment of Psychological and Brain Science, Dartmouth CollegeDepartment of Psychological and Brain Science, Dartmouth CollegeDepartment of Psychological and Brain Science, Dartmouth CollegeDepartment of Psychological and Brain Science, Dartmouth CollegeAbstract The identification of animal behavior in video is a critical but time-consuming task in many areas of research. Here, we introduce DeepAction, a deep learning-based toolbox for automatically annotating animal behavior in video. Our approach uses features extracted from raw video frames by a pretrained convolutional neural network to train a recurrent neural network classifier. We evaluate the classifier on two benchmark rodent datasets and one octopus dataset. We show that it achieves high accuracy, requires little training data, and surpasses both human agreement and most comparable existing methods. We also create a confidence score for classifier output, and show that our method provides an accurate estimate of classifier performance and reduces the time required by human annotators to review and correct automatically-produced annotations. We release our system and accompanying annotation interface as an open-source MATLAB toolbox.https://doi.org/10.1038/s41598-023-29574-0 |
| spellingShingle | Carl Harris Kelly R. Finn Marie-Luise Kieseler Marvin R. Maechler Peter U. Tse DeepAction: a MATLAB toolbox for automated classification of animal behavior in video Scientific Reports |
| title | DeepAction: a MATLAB toolbox for automated classification of animal behavior in video |
| title_full | DeepAction: a MATLAB toolbox for automated classification of animal behavior in video |
| title_fullStr | DeepAction: a MATLAB toolbox for automated classification of animal behavior in video |
| title_full_unstemmed | DeepAction: a MATLAB toolbox for automated classification of animal behavior in video |
| title_short | DeepAction: a MATLAB toolbox for automated classification of animal behavior in video |
| title_sort | deepaction a matlab toolbox for automated classification of animal behavior in video |
| url | https://doi.org/10.1038/s41598-023-29574-0 |
| work_keys_str_mv | AT carlharris deepactionamatlabtoolboxforautomatedclassificationofanimalbehaviorinvideo AT kellyrfinn deepactionamatlabtoolboxforautomatedclassificationofanimalbehaviorinvideo AT marieluisekieseler deepactionamatlabtoolboxforautomatedclassificationofanimalbehaviorinvideo AT marvinrmaechler deepactionamatlabtoolboxforautomatedclassificationofanimalbehaviorinvideo AT peterutse deepactionamatlabtoolboxforautomatedclassificationofanimalbehaviorinvideo |