Robust source counting and DOA estimation using spatial pseudo-spectrum and convolutional neural network
Many signal processing-based methods for sound source direction-of-arrival estimation produce a spatial pseudo-spectrum of which the local maxima strongly indicate the source directions. Due to different levels of noise, reverberation and different number of overlapping sources, the spatial pseudo-s...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/144539 |
| _version_ | 1811688328490123264 |
|---|---|
| author | Nguyen, Thi Ngoc Tho Gan, Woon-Seng Ranjan, Rishabh Jones, Douglas L. |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Nguyen, Thi Ngoc Tho Gan, Woon-Seng Ranjan, Rishabh Jones, Douglas L. |
| author_sort | Nguyen, Thi Ngoc Tho |
| collection | NTU |
| description | Many signal processing-based methods for sound source direction-of-arrival estimation produce a spatial pseudo-spectrum of which the local maxima strongly indicate the source directions. Due to different levels of noise, reverberation and different number of overlapping sources, the spatial pseudo-spectra are noisy even after smoothing. In addition, the number of sources is often unknown. As a result, selecting the peaks from these spectra is susceptible to error. Convolutional neural network has been successfully applied to many image processing problems in general and direction-of-arrival estimation in particular. In addition, deep learning-based methods for direction-of-arrival estimation show good generalization to different environments. We propose to use a 2D convolutional neural network with multi-task learning to robustly estimate the number of sources and the directions-of-arrival from short-time spatial pseudo-spectra, which have useful directional information from audio input signals. This approach reduces the tendency of the neural network to learn unwanted association between sound classes and directional information, and helps the network generalize to unseen sound classes. The simulation and experimental results show that the proposed methods outperform other directional-of-arrival estimation methods in different levels of noise and reverberation, and different number of sources. |
| first_indexed | 2024-10-01T05:30:28Z |
| format | Journal Article |
| id | ntu-10356/144539 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:30:28Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | ntu-10356/1445392020-11-11T07:36:02Z Robust source counting and DOA estimation using spatial pseudo-spectrum and convolutional neural network Nguyen, Thi Ngoc Tho Gan, Woon-Seng Ranjan, Rishabh Jones, Douglas L. School of Electrical and Electronic Engineering Engineering Direction-of-arrival Estimation Convolutional Neural Network Many signal processing-based methods for sound source direction-of-arrival estimation produce a spatial pseudo-spectrum of which the local maxima strongly indicate the source directions. Due to different levels of noise, reverberation and different number of overlapping sources, the spatial pseudo-spectra are noisy even after smoothing. In addition, the number of sources is often unknown. As a result, selecting the peaks from these spectra is susceptible to error. Convolutional neural network has been successfully applied to many image processing problems in general and direction-of-arrival estimation in particular. In addition, deep learning-based methods for direction-of-arrival estimation show good generalization to different environments. We propose to use a 2D convolutional neural network with multi-task learning to robustly estimate the number of sources and the directions-of-arrival from short-time spatial pseudo-spectra, which have useful directional information from audio input signals. This approach reduces the tendency of the neural network to learn unwanted association between sound classes and directional information, and helps the network generalize to unseen sound classes. The simulation and experimental results show that the proposed methods outperform other directional-of-arrival estimation methods in different levels of noise and reverberation, and different number of sources. Accepted version This research was conducted at Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU), which is a collaboration between Singapore Telecommunications Limited (Singtel) and Nanyang Technological University (NTU) that is funded by the Singapore Government through the Industry Alignment Fund - Industry Collaboration Projects Grant. The authors would like to thank Mr. Kenneth Ooi and Mr. Sathish Jayabalan for helping us with the data collection for the experiment. 2020-11-11T07:33:06Z 2020-11-11T07:33:06Z 2020 Journal Article Nguyen, T. N. T., Gan, W.-S., Ranjan R., & Jones, D. L. (2020). Robust source counting and DOA estimation using spatial pseudo-spectrum and convolutional neural network. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28, 2626-2637. doi: 10.1109/TASLP.2020.3019646 2329-9304 https://hdl.handle.net/10356/144539 10.1109/TASLP.2020.3019646 28 2626 2637 en IEEE/ACM Transactions on Audio, Speech, and Language Processing © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: https://doi.org/10.1109/TASLP.2020.3019646 application/pdf |
| spellingShingle | Engineering Direction-of-arrival Estimation Convolutional Neural Network Nguyen, Thi Ngoc Tho Gan, Woon-Seng Ranjan, Rishabh Jones, Douglas L. Robust source counting and DOA estimation using spatial pseudo-spectrum and convolutional neural network |
| title | Robust source counting and DOA estimation using spatial pseudo-spectrum and convolutional neural network |
| title_full | Robust source counting and DOA estimation using spatial pseudo-spectrum and convolutional neural network |
| title_fullStr | Robust source counting and DOA estimation using spatial pseudo-spectrum and convolutional neural network |
| title_full_unstemmed | Robust source counting and DOA estimation using spatial pseudo-spectrum and convolutional neural network |
| title_short | Robust source counting and DOA estimation using spatial pseudo-spectrum and convolutional neural network |
| title_sort | robust source counting and doa estimation using spatial pseudo spectrum and convolutional neural network |
| topic | Engineering Direction-of-arrival Estimation Convolutional Neural Network |
| url | https://hdl.handle.net/10356/144539 |
| work_keys_str_mv | AT nguyenthingoctho robustsourcecountinganddoaestimationusingspatialpseudospectrumandconvolutionalneuralnetwork AT ganwoonseng robustsourcecountinganddoaestimationusingspatialpseudospectrumandconvolutionalneuralnetwork AT ranjanrishabh robustsourcecountinganddoaestimationusingspatialpseudospectrumandconvolutionalneuralnetwork AT jonesdouglasl robustsourcecountinganddoaestimationusingspatialpseudospectrumandconvolutionalneuralnetwork |