Three-Dimensional Foot Position Estimation Based on Footprint Shadow Image Processing and Deep Learning for Smart Trampoline Fitness System
In the wake of COVID-19, the digital fitness market combining health equipment and ICT technologies is experiencing unexpected high growth. A smart trampoline fitness system is a new representative home exercise equipment for muscle strengthening and rehabilitation exercises. Recognizing the motions...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/18/6922 |
| _version_ | 1797482567286390784 |
|---|---|
| author | Se-Kyung Park Jun-Kyu Park Hong-In Won Seung-Hwan Choi Chang-Hyun Kim Suwoong Lee Min Young Kim |
| author_facet | Se-Kyung Park Jun-Kyu Park Hong-In Won Seung-Hwan Choi Chang-Hyun Kim Suwoong Lee Min Young Kim |
| author_sort | Se-Kyung Park |
| collection | DOAJ |
| description | In the wake of COVID-19, the digital fitness market combining health equipment and ICT technologies is experiencing unexpected high growth. A smart trampoline fitness system is a new representative home exercise equipment for muscle strengthening and rehabilitation exercises. Recognizing the motions of the user and evaluating user activity is critical for implementing its self-guided exercising system. This study aimed to estimate the three-dimensional positions of the user’s foot using deep learning-based image processing algorithms for footprint shadow images acquired from the system. The proposed system comprises a jumping fitness trampoline; an upward-looking camera with a wide-angle and fish-eye lens; and an embedded board to process deep learning algorithms. Compared with our previous approach, which suffered from a geometric calibration process, a camera calibration method for highly distorted images, and algorithmic sensitivity to environmental changes such as illumination conditions, the proposed deep learning algorithm utilizes end-to-end learning without calibration. The network is configured with a modified Fast-RCNN based on ResNet-50, where the region proposal network is modified to process location regression different from box regression. To verify the effectiveness and accuracy of the proposed algorithm, a series of experiments are performed using a prototype system with a robotic manipulator to handle a foot mockup. The three root mean square errors corresponding to X, Y, and Z directions were revealed to be 8.32, 15.14, and 4.05 mm, respectively. Thus, the system can be utilized for motion recognition and performance evaluation of jumping exercises. |
| first_indexed | 2024-03-09T22:34:15Z |
| format | Article |
| id | doaj.art-de9bf3a6ba234958aca4c011c4ee0e3a |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T22:34:15Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-de9bf3a6ba234958aca4c011c4ee0e3a2023-11-23T18:51:19ZengMDPI AGSensors1424-82202022-09-012218692210.3390/s22186922Three-Dimensional Foot Position Estimation Based on Footprint Shadow Image Processing and Deep Learning for Smart Trampoline Fitness SystemSe-Kyung Park0Jun-Kyu Park1Hong-In Won2Seung-Hwan Choi3Chang-Hyun Kim4Suwoong Lee5Min Young Kim6Ansan R&D Campus, LG Innotek, Ansan 15588, KoreaRenewable Energy Solution Group, Korea Electric Power Research Institute (KEPRI), Naju 58277, KoreaAdvanced Mechatronics Research Group, Daegyeong Division, Korea Institute of Industrial Technology, Daegu 42994, KoreaAdvanced Mechatronics Research Group, Daegyeong Division, Korea Institute of Industrial Technology, Daegu 42994, KoreaAdvanced Mechatronics Research Group, Daegyeong Division, Korea Institute of Industrial Technology, Daegu 42994, KoreaAdvanced Mechatronics Research Group, Daegyeong Division, Korea Institute of Industrial Technology, Daegu 42994, KoreaSchool of Electronics Engineering, Kyungpook National University, Daegu 41566, KoreaIn the wake of COVID-19, the digital fitness market combining health equipment and ICT technologies is experiencing unexpected high growth. A smart trampoline fitness system is a new representative home exercise equipment for muscle strengthening and rehabilitation exercises. Recognizing the motions of the user and evaluating user activity is critical for implementing its self-guided exercising system. This study aimed to estimate the three-dimensional positions of the user’s foot using deep learning-based image processing algorithms for footprint shadow images acquired from the system. The proposed system comprises a jumping fitness trampoline; an upward-looking camera with a wide-angle and fish-eye lens; and an embedded board to process deep learning algorithms. Compared with our previous approach, which suffered from a geometric calibration process, a camera calibration method for highly distorted images, and algorithmic sensitivity to environmental changes such as illumination conditions, the proposed deep learning algorithm utilizes end-to-end learning without calibration. The network is configured with a modified Fast-RCNN based on ResNet-50, where the region proposal network is modified to process location regression different from box regression. To verify the effectiveness and accuracy of the proposed algorithm, a series of experiments are performed using a prototype system with a robotic manipulator to handle a foot mockup. The three root mean square errors corresponding to X, Y, and Z directions were revealed to be 8.32, 15.14, and 4.05 mm, respectively. Thus, the system can be utilized for motion recognition and performance evaluation of jumping exercises.https://www.mdpi.com/1424-8220/22/18/6922smart fitnesstrampoline3D foot contact position estimationwide-angle camerafootprint shadowimage processing |
| spellingShingle | Se-Kyung Park Jun-Kyu Park Hong-In Won Seung-Hwan Choi Chang-Hyun Kim Suwoong Lee Min Young Kim Three-Dimensional Foot Position Estimation Based on Footprint Shadow Image Processing and Deep Learning for Smart Trampoline Fitness System Sensors smart fitness trampoline 3D foot contact position estimation wide-angle camera footprint shadow image processing |
| title | Three-Dimensional Foot Position Estimation Based on Footprint Shadow Image Processing and Deep Learning for Smart Trampoline Fitness System |
| title_full | Three-Dimensional Foot Position Estimation Based on Footprint Shadow Image Processing and Deep Learning for Smart Trampoline Fitness System |
| title_fullStr | Three-Dimensional Foot Position Estimation Based on Footprint Shadow Image Processing and Deep Learning for Smart Trampoline Fitness System |
| title_full_unstemmed | Three-Dimensional Foot Position Estimation Based on Footprint Shadow Image Processing and Deep Learning for Smart Trampoline Fitness System |
| title_short | Three-Dimensional Foot Position Estimation Based on Footprint Shadow Image Processing and Deep Learning for Smart Trampoline Fitness System |
| title_sort | three dimensional foot position estimation based on footprint shadow image processing and deep learning for smart trampoline fitness system |
| topic | smart fitness trampoline 3D foot contact position estimation wide-angle camera footprint shadow image processing |
| url | https://www.mdpi.com/1424-8220/22/18/6922 |
| work_keys_str_mv | AT sekyungpark threedimensionalfootpositionestimationbasedonfootprintshadowimageprocessinganddeeplearningforsmarttrampolinefitnesssystem AT junkyupark threedimensionalfootpositionestimationbasedonfootprintshadowimageprocessinganddeeplearningforsmarttrampolinefitnesssystem AT honginwon threedimensionalfootpositionestimationbasedonfootprintshadowimageprocessinganddeeplearningforsmarttrampolinefitnesssystem AT seunghwanchoi threedimensionalfootpositionestimationbasedonfootprintshadowimageprocessinganddeeplearningforsmarttrampolinefitnesssystem AT changhyunkim threedimensionalfootpositionestimationbasedonfootprintshadowimageprocessinganddeeplearningforsmarttrampolinefitnesssystem AT suwoonglee threedimensionalfootpositionestimationbasedonfootprintshadowimageprocessinganddeeplearningforsmarttrampolinefitnesssystem AT minyoungkim threedimensionalfootpositionestimationbasedonfootprintshadowimageprocessinganddeeplearningforsmarttrampolinefitnesssystem |