Semantic Image Segmentation Based Cable Vibration Frequency Visual Monitoring Using Modified Convolutional Neural Network with Pixel-wise Weighting Strategy
Attributed to the explosive adoption of large-span spatial structures and infrastructures as a critical damage-sensitive element, there is a pressing need to monitor cable vibration frequency to inspect the structural health. Neither existing acceleration sensor-utilized contact methods nor conventi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-04-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/8/1466 |
| _version_ | 1797538189368360960 |
|---|---|
| author | Han Yang Hong-Cheng Xu Shuang-Jian Jiao Feng-De Yin |
| author_facet | Han Yang Hong-Cheng Xu Shuang-Jian Jiao Feng-De Yin |
| author_sort | Han Yang |
| collection | DOAJ |
| description | Attributed to the explosive adoption of large-span spatial structures and infrastructures as a critical damage-sensitive element, there is a pressing need to monitor cable vibration frequency to inspect the structural health. Neither existing acceleration sensor-utilized contact methods nor conventional computer vision-based photogrammetry methods have, to date, addressed the defects of lack in cost-effectiveness and compatibility with real-world situations. In this study, a state-of-the-art method based on modified convolutional neural network semantic image segmentation, which is compatible with extensively varying real-world backgrounds, is presented for cable vibration frequency remote and visual monitoring. Modifications of the underlying network framework lie in adopting simpler feature extractors and introducing class weights to loss function by pixel-wise weighting strategies. Nine convolutional neural networks were established and modified. Discrete images with varying real-world backgrounds were captured to train and validate network models. Continuous videos with different cable pixel-to-total pixel (C-T) ratios were captured to test the networks and derive vibration frequencies. Various metrics were leveraged to evaluate the effectiveness of network models. The optimal C-T ratio was also studied to provide guidelines for the parameter setting of monitoring systems in further research and practical application. Training and validation accuracies of nine networks were all reported higher than 90%. A network model with ResNet-50 as feature extractor and uniform prior weighting showed the most superior learning and generalization ability, of which the <i>Precision</i> reached 0.9973, <i>F</i><sup>1</sup> reached 0.9685, and intersection over union (<i>IoU</i>) reached 0.8226 when utilizing images with the optimal C-T ratio of 0.04 as testing set. Contrasted with that sampled by acceleration sensor, the first two order vibration frequencies derived by the most superior network from video with the optimal C-T ratio had merely ignorable absolute percentage errors of 0.41% and 0.36%, substantiating the effectiveness of the proposed method. |
| first_indexed | 2024-03-10T12:27:01Z |
| format | Article |
| id | doaj.art-6b06575e0a6a49f6b1a4f09b4226ace0 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T12:27:01Z |
| publishDate | 2021-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-6b06575e0a6a49f6b1a4f09b4226ace02023-11-21T14:57:36ZengMDPI AGRemote Sensing2072-42922021-04-01138146610.3390/rs13081466Semantic Image Segmentation Based Cable Vibration Frequency Visual Monitoring Using Modified Convolutional Neural Network with Pixel-wise Weighting StrategyHan Yang0Hong-Cheng Xu1Shuang-Jian Jiao2Feng-De Yin3Department of Civil Engineering, College of Engineering, Ocean University of China, Qingdao 266100, ChinaDepartment of Civil Engineering, College of Engineering, Ocean University of China, Qingdao 266100, ChinaDepartment of Civil Engineering, College of Engineering, Ocean University of China, Qingdao 266100, ChinaDepartment of Civil Engineering, College of Engineering, Ocean University of China, Qingdao 266100, ChinaAttributed to the explosive adoption of large-span spatial structures and infrastructures as a critical damage-sensitive element, there is a pressing need to monitor cable vibration frequency to inspect the structural health. Neither existing acceleration sensor-utilized contact methods nor conventional computer vision-based photogrammetry methods have, to date, addressed the defects of lack in cost-effectiveness and compatibility with real-world situations. In this study, a state-of-the-art method based on modified convolutional neural network semantic image segmentation, which is compatible with extensively varying real-world backgrounds, is presented for cable vibration frequency remote and visual monitoring. Modifications of the underlying network framework lie in adopting simpler feature extractors and introducing class weights to loss function by pixel-wise weighting strategies. Nine convolutional neural networks were established and modified. Discrete images with varying real-world backgrounds were captured to train and validate network models. Continuous videos with different cable pixel-to-total pixel (C-T) ratios were captured to test the networks and derive vibration frequencies. Various metrics were leveraged to evaluate the effectiveness of network models. The optimal C-T ratio was also studied to provide guidelines for the parameter setting of monitoring systems in further research and practical application. Training and validation accuracies of nine networks were all reported higher than 90%. A network model with ResNet-50 as feature extractor and uniform prior weighting showed the most superior learning and generalization ability, of which the <i>Precision</i> reached 0.9973, <i>F</i><sup>1</sup> reached 0.9685, and intersection over union (<i>IoU</i>) reached 0.8226 when utilizing images with the optimal C-T ratio of 0.04 as testing set. Contrasted with that sampled by acceleration sensor, the first two order vibration frequencies derived by the most superior network from video with the optimal C-T ratio had merely ignorable absolute percentage errors of 0.41% and 0.36%, substantiating the effectiveness of the proposed method.https://www.mdpi.com/2072-4292/13/8/1466cable vibration frequencyremote and visual monitoringconvolutional neural networksemantic image segmentationdeep learningstructural health monitoring |
| spellingShingle | Han Yang Hong-Cheng Xu Shuang-Jian Jiao Feng-De Yin Semantic Image Segmentation Based Cable Vibration Frequency Visual Monitoring Using Modified Convolutional Neural Network with Pixel-wise Weighting Strategy Remote Sensing cable vibration frequency remote and visual monitoring convolutional neural network semantic image segmentation deep learning structural health monitoring |
| title | Semantic Image Segmentation Based Cable Vibration Frequency Visual Monitoring Using Modified Convolutional Neural Network with Pixel-wise Weighting Strategy |
| title_full | Semantic Image Segmentation Based Cable Vibration Frequency Visual Monitoring Using Modified Convolutional Neural Network with Pixel-wise Weighting Strategy |
| title_fullStr | Semantic Image Segmentation Based Cable Vibration Frequency Visual Monitoring Using Modified Convolutional Neural Network with Pixel-wise Weighting Strategy |
| title_full_unstemmed | Semantic Image Segmentation Based Cable Vibration Frequency Visual Monitoring Using Modified Convolutional Neural Network with Pixel-wise Weighting Strategy |
| title_short | Semantic Image Segmentation Based Cable Vibration Frequency Visual Monitoring Using Modified Convolutional Neural Network with Pixel-wise Weighting Strategy |
| title_sort | semantic image segmentation based cable vibration frequency visual monitoring using modified convolutional neural network with pixel wise weighting strategy |
| topic | cable vibration frequency remote and visual monitoring convolutional neural network semantic image segmentation deep learning structural health monitoring |
| url | https://www.mdpi.com/2072-4292/13/8/1466 |
| work_keys_str_mv | AT hanyang semanticimagesegmentationbasedcablevibrationfrequencyvisualmonitoringusingmodifiedconvolutionalneuralnetworkwithpixelwiseweightingstrategy AT hongchengxu semanticimagesegmentationbasedcablevibrationfrequencyvisualmonitoringusingmodifiedconvolutionalneuralnetworkwithpixelwiseweightingstrategy AT shuangjianjiao semanticimagesegmentationbasedcablevibrationfrequencyvisualmonitoringusingmodifiedconvolutionalneuralnetworkwithpixelwiseweightingstrategy AT fengdeyin semanticimagesegmentationbasedcablevibrationfrequencyvisualmonitoringusingmodifiedconvolutionalneuralnetworkwithpixelwiseweightingstrategy |