Animal hunt: bioacoustics animal recognition application
This project aims to create a bioacoustics classification model that can be used for real- time identification of animals based on their sounds in mobile applications. The first part of the project will focus on developing a bioacoustics classification model for the backend of the application. Th...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/175290 |
| _version_ | 1811679420449030144 |
|---|---|
| author | Low, Ren Hwa |
| author2 | Owen Noel Newton Fernando |
| author_facet | Owen Noel Newton Fernando Low, Ren Hwa |
| author_sort | Low, Ren Hwa |
| collection | NTU |
| description | This project aims to create a bioacoustics classification model that can be used for real-
time identification of animals based on their sounds in mobile applications. The first part
of the project will focus on developing a bioacoustics classification model for the backend
of the application. The second part of the project will emphasize on the deployment of the
model and optimizing its inference performance for edge devices.
To build effective bioacoustics classification models, a substantial amount of labelled data
is often required. The primary challenge for many bioacoustics tasks lies in the scarcity of
training data, especially for rare and endangered species. Furthermore, challenges arise
not only from a scarcity of data, but also from concerns regarding data quality. Many
datasets exhibit weak labelling and are often plagued by background noise and
overlapping vocalizations from different species.
To address the data limitations, this study reframes the bioacoustics classification task as a
few-shot learning problem, primarily relying on transfer learning through pre-trained
global bird embedding models such as BirdNET and Perch, known for their exceptional
generalization capabilities to other non-bird taxa. The performance of their embeddings
was evaluated on three diverse datasets specific to Singapore.
We also propose a pipeline to derive an annotated dataset for supervised learning through
the use of MixIT, a sound separation model designed to isolate background noise and
overlapping vocalizations, and RIBBIT, a bioacoustics tool. RIBBIT can not only identify
the output channel containing the isolated target vocalizations but also generate strongly
labeled data by providing temporal information about the audio events within each
recording.
Our findings demonstrate the superior performance of these large-scale acoustic bird
classifier models in comparison to general audio event detection models for bioacoustics
classification tasks, which can be further improved by applying separation to classifier
training data, addressing the issue of limited high quality training data. |
| first_indexed | 2024-10-01T03:08:52Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/175290 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:08:52Z |
| publishDate | 2024 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1752902024-04-26T15:44:40Z Animal hunt: bioacoustics animal recognition application Low, Ren Hwa Owen Noel Newton Fernando School of Computer Science and Engineering OFernando@ntu.edu.sg Computer and Information Science Bioacoustics Few-shot learning Source seperation Transfer learning Data augmentation Edge inference This project aims to create a bioacoustics classification model that can be used for real- time identification of animals based on their sounds in mobile applications. The first part of the project will focus on developing a bioacoustics classification model for the backend of the application. The second part of the project will emphasize on the deployment of the model and optimizing its inference performance for edge devices. To build effective bioacoustics classification models, a substantial amount of labelled data is often required. The primary challenge for many bioacoustics tasks lies in the scarcity of training data, especially for rare and endangered species. Furthermore, challenges arise not only from a scarcity of data, but also from concerns regarding data quality. Many datasets exhibit weak labelling and are often plagued by background noise and overlapping vocalizations from different species. To address the data limitations, this study reframes the bioacoustics classification task as a few-shot learning problem, primarily relying on transfer learning through pre-trained global bird embedding models such as BirdNET and Perch, known for their exceptional generalization capabilities to other non-bird taxa. The performance of their embeddings was evaluated on three diverse datasets specific to Singapore. We also propose a pipeline to derive an annotated dataset for supervised learning through the use of MixIT, a sound separation model designed to isolate background noise and overlapping vocalizations, and RIBBIT, a bioacoustics tool. RIBBIT can not only identify the output channel containing the isolated target vocalizations but also generate strongly labeled data by providing temporal information about the audio events within each recording. Our findings demonstrate the superior performance of these large-scale acoustic bird classifier models in comparison to general audio event detection models for bioacoustics classification tasks, which can be further improved by applying separation to classifier training data, addressing the issue of limited high quality training data. Bachelor's degree 2024-04-23T06:31:52Z 2024-04-23T06:31:52Z 2024 Final Year Project (FYP) Low, R. H. (2024). Animal hunt: bioacoustics animal recognition application. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/175290 https://hdl.handle.net/10356/175290 en application/pdf image/jpeg Nanyang Technological University |
| spellingShingle | Computer and Information Science Bioacoustics Few-shot learning Source seperation Transfer learning Data augmentation Edge inference Low, Ren Hwa Animal hunt: bioacoustics animal recognition application |
| title | Animal hunt: bioacoustics animal recognition application |
| title_full | Animal hunt: bioacoustics animal recognition application |
| title_fullStr | Animal hunt: bioacoustics animal recognition application |
| title_full_unstemmed | Animal hunt: bioacoustics animal recognition application |
| title_short | Animal hunt: bioacoustics animal recognition application |
| title_sort | animal hunt bioacoustics animal recognition application |
| topic | Computer and Information Science Bioacoustics Few-shot learning Source seperation Transfer learning Data augmentation Edge inference |
| url | https://hdl.handle.net/10356/175290 |
| work_keys_str_mv | AT lowrenhwa animalhuntbioacousticsanimalrecognitionapplication |