Exploring deep learning techniques for wild animal behaviour classification using animal‐borne accelerometers
Abstract Machine learning‐based behaviour classification using acceleration data is a powerful tool in bio‐logging research. Deep learning architectures such as convolutional neural networks (CNN), long short‐term memory (LSTM) and self‐attention mechanism as well as related training techniques have...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-04-01
|
| Series: | Methods in Ecology and Evolution |
| Subjects: | |
| Online Access: | https://doi.org/10.1111/2041-210X.14294 |
| _version_ | 1797226587011153920 |
|---|---|
| author | Ryoma Otsuka Naoya Yoshimura Kei Tanigaki Shiho Koyama Yuichi Mizutani Ken Yoda Takuya Maekawa |
| author_facet | Ryoma Otsuka Naoya Yoshimura Kei Tanigaki Shiho Koyama Yuichi Mizutani Ken Yoda Takuya Maekawa |
| author_sort | Ryoma Otsuka |
| collection | DOAJ |
| description | Abstract Machine learning‐based behaviour classification using acceleration data is a powerful tool in bio‐logging research. Deep learning architectures such as convolutional neural networks (CNN), long short‐term memory (LSTM) and self‐attention mechanism as well as related training techniques have been extensively studied in human activity recognition. However, they have rarely been used in wild animal studies. The main challenges of acceleration‐based wild animal behaviour classification include data shortages, class imbalance problems, various types of noise in data due to differences in individual behaviour and where the loggers were attached and complexity in data due to complex animal‐specific behaviours, which may have limited the application of deep learning techniques in this area. To overcome these challenges, we explored the effectiveness of techniques for efficient model training: data augmentation, manifold mixup and pre‐training of deep learning models with unlabelled data, using datasets from two species of wild seabirds and state‐of‐the‐art deep learning model architectures. Data augmentation improved the overall model performance when one of the various techniques (none, scaling, jittering, permutation, time‐warping and rotation) was randomly applied to each data during mini‐batch training. Manifold mixup also improved model performance, but not as much as random data augmentation. Pre‐training with unlabelled data did not improve model performance. The state‐of‐the‐art deep learning models, including a model consisting of four CNN layers, an LSTM layer and a multi‐head attention layer, as well as its modified version with shortcut connection, showed better performance among other comparative models. Using only raw acceleration data as inputs, these models outperformed classic machine learning approaches that used 119 handcrafted features. Our experiments showed that deep learning techniques are promising for acceleration‐based behaviour classification of wild animals and highlighted some challenges (e.g. effective use of unlabelled data). There is scope for greater exploration of deep learning techniques in wild animal studies (e.g. advanced data augmentation, multimodal sensor data use, transfer learning and self‐supervised learning). We hope that this study will stimulate the development of deep learning techniques for wild animal behaviour classification using time‐series sensor data. |
| first_indexed | 2024-04-24T14:27:16Z |
| format | Article |
| id | doaj.art-cb35496c9f3240b79c3757becf300e08 |
| institution | Directory Open Access Journal |
| issn | 2041-210X |
| language | English |
| last_indexed | 2024-04-24T14:27:16Z |
| publishDate | 2024-04-01 |
| publisher | Wiley |
| record_format | Article |
| series | Methods in Ecology and Evolution |
| spelling | doaj.art-cb35496c9f3240b79c3757becf300e082024-04-03T04:38:58ZengWileyMethods in Ecology and Evolution2041-210X2024-04-0115471673110.1111/2041-210X.14294Exploring deep learning techniques for wild animal behaviour classification using animal‐borne accelerometersRyoma Otsuka0Naoya Yoshimura1Kei Tanigaki2Shiho Koyama3Yuichi Mizutani4Ken Yoda5Takuya Maekawa6Graduate School of Information Science and Technology Osaka University Suita Osaka JapanGraduate School of Information Science and Technology Osaka University Suita Osaka JapanGraduate School of Information Science and Technology Osaka University Suita Osaka JapanGraduate School of Environmental Studies Nagoya University Nagoya Aichi JapanGraduate School of Environmental Studies Nagoya University Nagoya Aichi JapanGraduate School of Environmental Studies Nagoya University Nagoya Aichi JapanGraduate School of Information Science and Technology Osaka University Suita Osaka JapanAbstract Machine learning‐based behaviour classification using acceleration data is a powerful tool in bio‐logging research. Deep learning architectures such as convolutional neural networks (CNN), long short‐term memory (LSTM) and self‐attention mechanism as well as related training techniques have been extensively studied in human activity recognition. However, they have rarely been used in wild animal studies. The main challenges of acceleration‐based wild animal behaviour classification include data shortages, class imbalance problems, various types of noise in data due to differences in individual behaviour and where the loggers were attached and complexity in data due to complex animal‐specific behaviours, which may have limited the application of deep learning techniques in this area. To overcome these challenges, we explored the effectiveness of techniques for efficient model training: data augmentation, manifold mixup and pre‐training of deep learning models with unlabelled data, using datasets from two species of wild seabirds and state‐of‐the‐art deep learning model architectures. Data augmentation improved the overall model performance when one of the various techniques (none, scaling, jittering, permutation, time‐warping and rotation) was randomly applied to each data during mini‐batch training. Manifold mixup also improved model performance, but not as much as random data augmentation. Pre‐training with unlabelled data did not improve model performance. The state‐of‐the‐art deep learning models, including a model consisting of four CNN layers, an LSTM layer and a multi‐head attention layer, as well as its modified version with shortcut connection, showed better performance among other comparative models. Using only raw acceleration data as inputs, these models outperformed classic machine learning approaches that used 119 handcrafted features. Our experiments showed that deep learning techniques are promising for acceleration‐based behaviour classification of wild animals and highlighted some challenges (e.g. effective use of unlabelled data). There is scope for greater exploration of deep learning techniques in wild animal studies (e.g. advanced data augmentation, multimodal sensor data use, transfer learning and self‐supervised learning). We hope that this study will stimulate the development of deep learning techniques for wild animal behaviour classification using time‐series sensor data.https://doi.org/10.1111/2041-210X.14294acceleration sensoranimal behaviour classificationbio‐loggingdata augmentationdeep learningmachine learning |
| spellingShingle | Ryoma Otsuka Naoya Yoshimura Kei Tanigaki Shiho Koyama Yuichi Mizutani Ken Yoda Takuya Maekawa Exploring deep learning techniques for wild animal behaviour classification using animal‐borne accelerometers Methods in Ecology and Evolution acceleration sensor animal behaviour classification bio‐logging data augmentation deep learning machine learning |
| title | Exploring deep learning techniques for wild animal behaviour classification using animal‐borne accelerometers |
| title_full | Exploring deep learning techniques for wild animal behaviour classification using animal‐borne accelerometers |
| title_fullStr | Exploring deep learning techniques for wild animal behaviour classification using animal‐borne accelerometers |
| title_full_unstemmed | Exploring deep learning techniques for wild animal behaviour classification using animal‐borne accelerometers |
| title_short | Exploring deep learning techniques for wild animal behaviour classification using animal‐borne accelerometers |
| title_sort | exploring deep learning techniques for wild animal behaviour classification using animal borne accelerometers |
| topic | acceleration sensor animal behaviour classification bio‐logging data augmentation deep learning machine learning |
| url | https://doi.org/10.1111/2041-210X.14294 |
| work_keys_str_mv | AT ryomaotsuka exploringdeeplearningtechniquesforwildanimalbehaviourclassificationusinganimalborneaccelerometers AT naoyayoshimura exploringdeeplearningtechniquesforwildanimalbehaviourclassificationusinganimalborneaccelerometers AT keitanigaki exploringdeeplearningtechniquesforwildanimalbehaviourclassificationusinganimalborneaccelerometers AT shihokoyama exploringdeeplearningtechniquesforwildanimalbehaviourclassificationusinganimalborneaccelerometers AT yuichimizutani exploringdeeplearningtechniquesforwildanimalbehaviourclassificationusinganimalborneaccelerometers AT kenyoda exploringdeeplearningtechniquesforwildanimalbehaviourclassificationusinganimalborneaccelerometers AT takuyamaekawa exploringdeeplearningtechniquesforwildanimalbehaviourclassificationusinganimalborneaccelerometers |