Sound-Based Construction Activity Monitoring with Deep Learning
Automated construction monitoring assists site managers in managing safety, schedule, and productivity effectively. Existing research focuses on identifying construction sounds to determine the type of construction activity. However, there are two major limitations: the inability to handle a mixed s...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Buildings |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-5309/12/11/1947 |
| _version_ | 1827647012891262976 |
|---|---|
| author | Wuyue Xiong Xuenan Xu Long Chen Jian Yang |
| author_facet | Wuyue Xiong Xuenan Xu Long Chen Jian Yang |
| author_sort | Wuyue Xiong |
| collection | DOAJ |
| description | Automated construction monitoring assists site managers in managing safety, schedule, and productivity effectively. Existing research focuses on identifying construction sounds to determine the type of construction activity. However, there are two major limitations: the inability to handle a mixed sound environment in which multiple construction activity sounds occur simultaneously, and the inability to precisely locate the start and end times of each individual construction activity. This research aims to fill this gap through developing an innovative deep learning-based method. The proposed model combines the benefits of Convolutional Neural Network (CNN) for extracting features and Recurrent Neural Network (RNN) for leveraging contextual information to handle construction environments with polyphony and noise. In addition, the dual threshold output permits exact identification of the start and finish timings of individual construction activities. Before training and testing with construction sounds collected from a modular construction factory, the model has been pre-trained with publicly available general sound event data. All of the innovative designs have been confirmed by an ablation study, and two extended experiments were also performed to verify the versatility of the present model in additional construction environments or activities. This model has great potential to be used for autonomous monitoring of construction activities. |
| first_indexed | 2024-03-09T19:13:21Z |
| format | Article |
| id | doaj.art-7aca0c8f82de4672a7eb4027e5e5d773 |
| institution | Directory Open Access Journal |
| issn | 2075-5309 |
| language | English |
| last_indexed | 2024-03-09T19:13:21Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Buildings |
| spelling | doaj.art-7aca0c8f82de4672a7eb4027e5e5d7732023-11-24T04:00:13ZengMDPI AGBuildings2075-53092022-11-011211194710.3390/buildings12111947Sound-Based Construction Activity Monitoring with Deep LearningWuyue Xiong0Xuenan Xu1Long Chen2Jian Yang3Shanghai Key Laboratory for Digital Maintenance of Buildings and Infrastructure, School of Naval Architecture, Ocean & Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, ChinaMoE Key Lab of Artificial Intelligence, X-LANCE Lab, Department of Computer Science and Engineering, AI Institute, Shanghai Jiao Tong University, Shanghai 200240, ChinaSchool of Architecture Building and Civil Engineering, Loughborough University, Loughborough LE11 3TU, UKShanghai Key Laboratory for Digital Maintenance of Buildings and Infrastructure, School of Naval Architecture, Ocean & Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, ChinaAutomated construction monitoring assists site managers in managing safety, schedule, and productivity effectively. Existing research focuses on identifying construction sounds to determine the type of construction activity. However, there are two major limitations: the inability to handle a mixed sound environment in which multiple construction activity sounds occur simultaneously, and the inability to precisely locate the start and end times of each individual construction activity. This research aims to fill this gap through developing an innovative deep learning-based method. The proposed model combines the benefits of Convolutional Neural Network (CNN) for extracting features and Recurrent Neural Network (RNN) for leveraging contextual information to handle construction environments with polyphony and noise. In addition, the dual threshold output permits exact identification of the start and finish timings of individual construction activities. Before training and testing with construction sounds collected from a modular construction factory, the model has been pre-trained with publicly available general sound event data. All of the innovative designs have been confirmed by an ablation study, and two extended experiments were also performed to verify the versatility of the present model in additional construction environments or activities. This model has great potential to be used for autonomous monitoring of construction activities.https://www.mdpi.com/2075-5309/12/11/1947construction monitoringsound event detectionconvolutional recurrent neural networkdeep learning |
| spellingShingle | Wuyue Xiong Xuenan Xu Long Chen Jian Yang Sound-Based Construction Activity Monitoring with Deep Learning Buildings construction monitoring sound event detection convolutional recurrent neural network deep learning |
| title | Sound-Based Construction Activity Monitoring with Deep Learning |
| title_full | Sound-Based Construction Activity Monitoring with Deep Learning |
| title_fullStr | Sound-Based Construction Activity Monitoring with Deep Learning |
| title_full_unstemmed | Sound-Based Construction Activity Monitoring with Deep Learning |
| title_short | Sound-Based Construction Activity Monitoring with Deep Learning |
| title_sort | sound based construction activity monitoring with deep learning |
| topic | construction monitoring sound event detection convolutional recurrent neural network deep learning |
| url | https://www.mdpi.com/2075-5309/12/11/1947 |
| work_keys_str_mv | AT wuyuexiong soundbasedconstructionactivitymonitoringwithdeeplearning AT xuenanxu soundbasedconstructionactivitymonitoringwithdeeplearning AT longchen soundbasedconstructionactivitymonitoringwithdeeplearning AT jianyang soundbasedconstructionactivitymonitoringwithdeeplearning |