OpenApePose, a database of annotated ape photographs for pose estimation
Because of their close relationship with humans, non-human apes (chimpanzees, bonobos, gorillas, orangutans, and gibbons, including siamangs) are of great scientific interest. The goal of understanding their complex behavior would be greatly advanced by the ability to perform video-based pose tracki...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
eLife Sciences Publications Ltd
2023-12-01
|
| Series: | eLife |
| Subjects: | |
| Online Access: | https://elifesciences.org/articles/86873 |
| _version_ | 1797396965867126784 |
|---|---|
| author | Nisarg Desai Praneet Bala Rebecca Richardson Jessica Raper Jan Zimmermann Benjamin Hayden |
| author_facet | Nisarg Desai Praneet Bala Rebecca Richardson Jessica Raper Jan Zimmermann Benjamin Hayden |
| author_sort | Nisarg Desai |
| collection | DOAJ |
| description | Because of their close relationship with humans, non-human apes (chimpanzees, bonobos, gorillas, orangutans, and gibbons, including siamangs) are of great scientific interest. The goal of understanding their complex behavior would be greatly advanced by the ability to perform video-based pose tracking. Tracking, however, requires high-quality annotated datasets of ape photographs. Here we present OpenApePose, a new public dataset of 71,868 photographs, annotated with 16 body landmarks of six ape species in naturalistic contexts. We show that a standard deep net (HRNet-W48) trained on ape photos can reliably track out-of-sample ape photos better than networks trained on monkeys (specifically, the OpenMonkeyPose dataset) and on humans (COCO) can. This trained network can track apes almost as well as the other networks can track their respective taxa, and models trained without one of the six ape species can track the held-out species better than the monkey and human models can. Ultimately, the results of our analyses highlight the importance of large, specialized databases for animal tracking systems and confirm the utility of our new ape database. |
| first_indexed | 2024-03-09T01:03:16Z |
| format | Article |
| id | doaj.art-ec95565267dc4abf859dbdc0ca33b4a3 |
| institution | Directory Open Access Journal |
| issn | 2050-084X |
| language | English |
| last_indexed | 2024-03-09T01:03:16Z |
| publishDate | 2023-12-01 |
| publisher | eLife Sciences Publications Ltd |
| record_format | Article |
| series | eLife |
| spelling | doaj.art-ec95565267dc4abf859dbdc0ca33b4a32023-12-11T14:58:37ZengeLife Sciences Publications LtdeLife2050-084X2023-12-011210.7554/eLife.86873OpenApePose, a database of annotated ape photographs for pose estimationNisarg Desai0https://orcid.org/0000-0003-3210-9409Praneet Bala1https://orcid.org/0000-0002-2144-1986Rebecca Richardson2Jessica Raper3https://orcid.org/0000-0002-0964-9944Jan Zimmermann4Benjamin Hayden5Department of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, United StatesDepartment of Computer Science, University of Minnesota, Minneapolis, United StatesEmory National Primate Research Center, Emory University, Atlanta, United StatesEmory National Primate Research Center, Emory University, Atlanta, United StatesDepartment of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, United StatesDepartment of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, United StatesBecause of their close relationship with humans, non-human apes (chimpanzees, bonobos, gorillas, orangutans, and gibbons, including siamangs) are of great scientific interest. The goal of understanding their complex behavior would be greatly advanced by the ability to perform video-based pose tracking. Tracking, however, requires high-quality annotated datasets of ape photographs. Here we present OpenApePose, a new public dataset of 71,868 photographs, annotated with 16 body landmarks of six ape species in naturalistic contexts. We show that a standard deep net (HRNet-W48) trained on ape photos can reliably track out-of-sample ape photos better than networks trained on monkeys (specifically, the OpenMonkeyPose dataset) and on humans (COCO) can. This trained network can track apes almost as well as the other networks can track their respective taxa, and models trained without one of the six ape species can track the held-out species better than the monkey and human models can. Ultimately, the results of our analyses highlight the importance of large, specialized databases for animal tracking systems and confirm the utility of our new ape database.https://elifesciences.org/articles/86873apespose estimationdeep learningbehavior trackingdataset |
| spellingShingle | Nisarg Desai Praneet Bala Rebecca Richardson Jessica Raper Jan Zimmermann Benjamin Hayden OpenApePose, a database of annotated ape photographs for pose estimation eLife apes pose estimation deep learning behavior tracking dataset |
| title | OpenApePose, a database of annotated ape photographs for pose estimation |
| title_full | OpenApePose, a database of annotated ape photographs for pose estimation |
| title_fullStr | OpenApePose, a database of annotated ape photographs for pose estimation |
| title_full_unstemmed | OpenApePose, a database of annotated ape photographs for pose estimation |
| title_short | OpenApePose, a database of annotated ape photographs for pose estimation |
| title_sort | openapepose a database of annotated ape photographs for pose estimation |
| topic | apes pose estimation deep learning behavior tracking dataset |
| url | https://elifesciences.org/articles/86873 |
| work_keys_str_mv | AT nisargdesai openapeposeadatabaseofannotatedapephotographsforposeestimation AT praneetbala openapeposeadatabaseofannotatedapephotographsforposeestimation AT rebeccarichardson openapeposeadatabaseofannotatedapephotographsforposeestimation AT jessicaraper openapeposeadatabaseofannotatedapephotographsforposeestimation AT janzimmermann openapeposeadatabaseofannotatedapephotographsforposeestimation AT benjaminhayden openapeposeadatabaseofannotatedapephotographsforposeestimation |