Symmetrically Stacked Long Short-Term Memory Networks for Fall Event Recognition Using Compact Convolutional Neural Networks-Based Tracker
In recent years, the advancement of pattern recognition algorithms, specifically the deep learning-related techniques, have propelled a tremendous amount of researches in fall event recognition systems. It is important to detect a fall incident as early as possible, whereby a slight delay in providi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-02-01
|
| Series: | Symmetry |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-8994/14/2/293 |
| _version_ | 1797476404734984192 |
|---|---|
| author | Nur Ayuni Mohamed Mohd Asyraf Zulkifley Nor Azwan Mohamed Kamari Zulaikha Kadim |
| author_facet | Nur Ayuni Mohamed Mohd Asyraf Zulkifley Nor Azwan Mohamed Kamari Zulaikha Kadim |
| author_sort | Nur Ayuni Mohamed |
| collection | DOAJ |
| description | In recent years, the advancement of pattern recognition algorithms, specifically the deep learning-related techniques, have propelled a tremendous amount of researches in fall event recognition systems. It is important to detect a fall incident as early as possible, whereby a slight delay in providing immediate assistance can cause severe unrecoverable injuries. One of the main challenges in fall event recognition is the imbalanced training data between fall and no-fall events, where a real-life fall incident is a sporadic event that occurs infrequently. Most of the recent techniques produce a lot of false alarms, as it is hard to train them to cover a wide range of fall situations. Hence, this paper aims to detect the exact fall frame in a video sequence, as such it will not be dependent on the whole clip of the video sequence. Our proposed approach consists of a two-stage module where the first stage employs a compact convolutional neural network tracker to generate the object trajectory information. Features of interest will be sampled from the generated trajectory paths, which will be fed as the input to the second stage. The next stage network then models the temporal dependencies of the trajectory information using symmetrical Long Short-Term Memory (LSTM) architecture. This two-stage module is a novel approach as most of the techniques rely on the detection module rather than the tracking module. The simulation experiments were tested using Fall Detection Dataset (FDD). The proposed approach obtains an expected average overlap of 0.167, which is the best performance compared to Multi-Domain Network (MDNET) and Tree-structured Convolutional Neural Network (TCNN) trackers. Furthermore, the proposed 3-layers of stacked LSTM architecture also performs the best compared to the vanilla recurrent neural network and single-layer LSTM. This approach can be further improved if the tracker model is firstly pre-tuned in offline mode with respect to a specific type of object of interest, rather than a general object. |
| first_indexed | 2024-03-09T20:58:17Z |
| format | Article |
| id | doaj.art-49d7e7017cb64a15afaa82c3018cab90 |
| institution | Directory Open Access Journal |
| issn | 2073-8994 |
| language | English |
| last_indexed | 2024-03-09T20:58:17Z |
| publishDate | 2022-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Symmetry |
| spelling | doaj.art-49d7e7017cb64a15afaa82c3018cab902023-11-23T22:16:19ZengMDPI AGSymmetry2073-89942022-02-0114229310.3390/sym14020293Symmetrically Stacked Long Short-Term Memory Networks for Fall Event Recognition Using Compact Convolutional Neural Networks-Based TrackerNur Ayuni Mohamed0Mohd Asyraf Zulkifley1Nor Azwan Mohamed Kamari2Zulaikha Kadim3Department of Electrical, Electronic and Systems Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, Bangi 43600, Selangor, MalaysiaDepartment of Electrical, Electronic and Systems Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, Bangi 43600, Selangor, MalaysiaDepartment of Electrical, Electronic and Systems Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, Bangi 43600, Selangor, MalaysiaAdvanced Informatics Lab, MIMOS Berhad, Technology Park Malaysia, Kuala Lumpur 57000, Wilayah Persekutuan Kuala Lumpur, MalaysiaIn recent years, the advancement of pattern recognition algorithms, specifically the deep learning-related techniques, have propelled a tremendous amount of researches in fall event recognition systems. It is important to detect a fall incident as early as possible, whereby a slight delay in providing immediate assistance can cause severe unrecoverable injuries. One of the main challenges in fall event recognition is the imbalanced training data between fall and no-fall events, where a real-life fall incident is a sporadic event that occurs infrequently. Most of the recent techniques produce a lot of false alarms, as it is hard to train them to cover a wide range of fall situations. Hence, this paper aims to detect the exact fall frame in a video sequence, as such it will not be dependent on the whole clip of the video sequence. Our proposed approach consists of a two-stage module where the first stage employs a compact convolutional neural network tracker to generate the object trajectory information. Features of interest will be sampled from the generated trajectory paths, which will be fed as the input to the second stage. The next stage network then models the temporal dependencies of the trajectory information using symmetrical Long Short-Term Memory (LSTM) architecture. This two-stage module is a novel approach as most of the techniques rely on the detection module rather than the tracking module. The simulation experiments were tested using Fall Detection Dataset (FDD). The proposed approach obtains an expected average overlap of 0.167, which is the best performance compared to Multi-Domain Network (MDNET) and Tree-structured Convolutional Neural Network (TCNN) trackers. Furthermore, the proposed 3-layers of stacked LSTM architecture also performs the best compared to the vanilla recurrent neural network and single-layer LSTM. This approach can be further improved if the tracker model is firstly pre-tuned in offline mode with respect to a specific type of object of interest, rather than a general object.https://www.mdpi.com/2073-8994/14/2/293fall event recognitionCompact Convolutional Neural NetworksSymmetrical Recurrent Neural Networks |
| spellingShingle | Nur Ayuni Mohamed Mohd Asyraf Zulkifley Nor Azwan Mohamed Kamari Zulaikha Kadim Symmetrically Stacked Long Short-Term Memory Networks for Fall Event Recognition Using Compact Convolutional Neural Networks-Based Tracker Symmetry fall event recognition Compact Convolutional Neural Networks Symmetrical Recurrent Neural Networks |
| title | Symmetrically Stacked Long Short-Term Memory Networks for Fall Event Recognition Using Compact Convolutional Neural Networks-Based Tracker |
| title_full | Symmetrically Stacked Long Short-Term Memory Networks for Fall Event Recognition Using Compact Convolutional Neural Networks-Based Tracker |
| title_fullStr | Symmetrically Stacked Long Short-Term Memory Networks for Fall Event Recognition Using Compact Convolutional Neural Networks-Based Tracker |
| title_full_unstemmed | Symmetrically Stacked Long Short-Term Memory Networks for Fall Event Recognition Using Compact Convolutional Neural Networks-Based Tracker |
| title_short | Symmetrically Stacked Long Short-Term Memory Networks for Fall Event Recognition Using Compact Convolutional Neural Networks-Based Tracker |
| title_sort | symmetrically stacked long short term memory networks for fall event recognition using compact convolutional neural networks based tracker |
| topic | fall event recognition Compact Convolutional Neural Networks Symmetrical Recurrent Neural Networks |
| url | https://www.mdpi.com/2073-8994/14/2/293 |
| work_keys_str_mv | AT nurayunimohamed symmetricallystackedlongshorttermmemorynetworksforfalleventrecognitionusingcompactconvolutionalneuralnetworksbasedtracker AT mohdasyrafzulkifley symmetricallystackedlongshorttermmemorynetworksforfalleventrecognitionusingcompactconvolutionalneuralnetworksbasedtracker AT norazwanmohamedkamari symmetricallystackedlongshorttermmemorynetworksforfalleventrecognitionusingcompactconvolutionalneuralnetworksbasedtracker AT zulaikhakadim symmetricallystackedlongshorttermmemorynetworksforfalleventrecognitionusingcompactconvolutionalneuralnetworksbasedtracker |