Supervised versus unsupervised approaches to classification of accelerometry data
Abstract Sophisticated animal‐borne sensor systems are increasingly providing novel insight into how animals behave and move. Despite their widespread use in ecology, the diversity and expanding quality and quantity of data they produce have created a need for robust analytical methods for biologica...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-05-01
|
| Series: | Ecology and Evolution |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/ece3.10035 |
| _version_ | 1797776581263884288 |
|---|---|
| author | Maitreyi Sur Jonathan C. Hall Joseph Brandt Molly Astell Sharon A. Poessel Todd E. Katzner |
| author_facet | Maitreyi Sur Jonathan C. Hall Joseph Brandt Molly Astell Sharon A. Poessel Todd E. Katzner |
| author_sort | Maitreyi Sur |
| collection | DOAJ |
| description | Abstract Sophisticated animal‐borne sensor systems are increasingly providing novel insight into how animals behave and move. Despite their widespread use in ecology, the diversity and expanding quality and quantity of data they produce have created a need for robust analytical methods for biological interpretation. Machine learning tools are often used to meet this need. However, their relative effectiveness is not well known and, in the case of unsupervised tools, given that they do not use validation data, their accuracy can be difficult to assess. We evaluated the effectiveness of supervised (n = 6), semi‐supervised (n = 1), and unsupervised (n = 2) approaches to analyzing accelerometry data collected from critically endangered California condors (Gymnogyps californianus). Unsupervised K‐means and EM (expectation–maximization) clustering approaches performed poorly, with adequate classification accuracies of <0.8 but very low values for kappa statistics (range: −0.02 to 0.06). The semi‐supervised nearest mean classifier was moderately effective at classification, with an overall classification accuracy of 0.61 but effective classification only of two of the four behavioral classes. Supervised random forest (RF) and k‐nearest neighbor (kNN) machine learning models were most effective at classification across all behavior types, with overall accuracies >0.81. Kappa statistics were also highest for RF and kNN, in most cases substantially greater than for other modeling approaches. Unsupervised modeling, which is commonly used for the classification of a priori‐defined behaviors in telemetry data, can provide useful information but likely is instead better suited to post hoc definition of generalized behavioral states. This work also shows the potential for substantial variation in classification accuracy among different machine learning approaches and among different metrics of accuracy. As such, when analyzing biotelemetry data, best practices appear to call for the evaluation of several machine learning techniques and several measures of accuracy for each dataset under consideration. |
| first_indexed | 2024-03-12T22:51:54Z |
| format | Article |
| id | doaj.art-498b0077eff843f4bf53e502df60fb53 |
| institution | Directory Open Access Journal |
| issn | 2045-7758 |
| language | English |
| last_indexed | 2024-03-12T22:51:54Z |
| publishDate | 2023-05-01 |
| publisher | Wiley |
| record_format | Article |
| series | Ecology and Evolution |
| spelling | doaj.art-498b0077eff843f4bf53e502df60fb532023-07-20T08:50:55ZengWileyEcology and Evolution2045-77582023-05-01135n/an/a10.1002/ece3.10035Supervised versus unsupervised approaches to classification of accelerometry dataMaitreyi Sur0Jonathan C. Hall1Joseph Brandt2Molly Astell3Sharon A. Poessel4Todd E. Katzner5Conservation Science Global, Inc. West Cape May New Jersey USADepartment of Biology Eastern Michigan University Ypsilanti Michigan USAU.S. Fish and Wildlife Service, Hopper Mountain National Wildlife Refuge Complex Ventura California USAU.S. Fish and Wildlife Service, Hopper Mountain National Wildlife Refuge Complex Ventura California USAU.S. Geological Survey, Forest and Rangeland Ecosystem Science Center Boise Idaho USAU.S. Geological Survey, Forest and Rangeland Ecosystem Science Center Boise Idaho USAAbstract Sophisticated animal‐borne sensor systems are increasingly providing novel insight into how animals behave and move. Despite their widespread use in ecology, the diversity and expanding quality and quantity of data they produce have created a need for robust analytical methods for biological interpretation. Machine learning tools are often used to meet this need. However, their relative effectiveness is not well known and, in the case of unsupervised tools, given that they do not use validation data, their accuracy can be difficult to assess. We evaluated the effectiveness of supervised (n = 6), semi‐supervised (n = 1), and unsupervised (n = 2) approaches to analyzing accelerometry data collected from critically endangered California condors (Gymnogyps californianus). Unsupervised K‐means and EM (expectation–maximization) clustering approaches performed poorly, with adequate classification accuracies of <0.8 but very low values for kappa statistics (range: −0.02 to 0.06). The semi‐supervised nearest mean classifier was moderately effective at classification, with an overall classification accuracy of 0.61 but effective classification only of two of the four behavioral classes. Supervised random forest (RF) and k‐nearest neighbor (kNN) machine learning models were most effective at classification across all behavior types, with overall accuracies >0.81. Kappa statistics were also highest for RF and kNN, in most cases substantially greater than for other modeling approaches. Unsupervised modeling, which is commonly used for the classification of a priori‐defined behaviors in telemetry data, can provide useful information but likely is instead better suited to post hoc definition of generalized behavioral states. This work also shows the potential for substantial variation in classification accuracy among different machine learning approaches and among different metrics of accuracy. As such, when analyzing biotelemetry data, best practices appear to call for the evaluation of several machine learning techniques and several measures of accuracy for each dataset under consideration.https://doi.org/10.1002/ece3.10035accelerometeranimal behaviorCalifornia condorclassificationmachine learning |
| spellingShingle | Maitreyi Sur Jonathan C. Hall Joseph Brandt Molly Astell Sharon A. Poessel Todd E. Katzner Supervised versus unsupervised approaches to classification of accelerometry data Ecology and Evolution accelerometer animal behavior California condor classification machine learning |
| title | Supervised versus unsupervised approaches to classification of accelerometry data |
| title_full | Supervised versus unsupervised approaches to classification of accelerometry data |
| title_fullStr | Supervised versus unsupervised approaches to classification of accelerometry data |
| title_full_unstemmed | Supervised versus unsupervised approaches to classification of accelerometry data |
| title_short | Supervised versus unsupervised approaches to classification of accelerometry data |
| title_sort | supervised versus unsupervised approaches to classification of accelerometry data |
| topic | accelerometer animal behavior California condor classification machine learning |
| url | https://doi.org/10.1002/ece3.10035 |
| work_keys_str_mv | AT maitreyisur supervisedversusunsupervisedapproachestoclassificationofaccelerometrydata AT jonathanchall supervisedversusunsupervisedapproachestoclassificationofaccelerometrydata AT josephbrandt supervisedversusunsupervisedapproachestoclassificationofaccelerometrydata AT mollyastell supervisedversusunsupervisedapproachestoclassificationofaccelerometrydata AT sharonapoessel supervisedversusunsupervisedapproachestoclassificationofaccelerometrydata AT toddekatzner supervisedversusunsupervisedapproachestoclassificationofaccelerometrydata |