On-Device Deep Personalization for Robust Activity Data Collection
One of the biggest challenges of activity data collection is the need to rely on users and keep them engaged to continually provide labels. Recent breakthroughs in mobile platforms have proven effective in bringing deep neural networks powered intelligence into mobile devices. This study proposes a...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/1/41 |
| _version_ | 1797543709628170240 |
|---|---|
| author | Nattaya Mairittha Tittaya Mairittha Sozo Inoue |
| author_facet | Nattaya Mairittha Tittaya Mairittha Sozo Inoue |
| author_sort | Nattaya Mairittha |
| collection | DOAJ |
| description | One of the biggest challenges of activity data collection is the need to rely on users and keep them engaged to continually provide labels. Recent breakthroughs in mobile platforms have proven effective in bringing deep neural networks powered intelligence into mobile devices. This study proposes a novel on-device personalization for data labeling for an activity recognition system using mobile sensing. The key idea behind this system is that estimated activities personalized for a specific individual user can be used as feedback to motivate user contribution and improve data labeling quality. First, we exploited fine-tuning using a Deep Recurrent Neural Network to address the lack of sufficient training data and minimize the need for training deep learning on mobile devices from scratch. Second, we utilized a model pruning technique to reduce the computation cost of on-device personalization without affecting the accuracy. Finally, we built a robust activity data labeling system by integrating the two techniques outlined above, allowing the mobile application to create a personalized experience for the user. To demonstrate the proposed model’s capability and feasibility, we developed and deployed the proposed system to realistic settings. For our experimental setup, we gathered more than 16,800 activity windows from 12 activity classes using smartphone sensors. We empirically evaluated the proposed quality by comparing it with a baseline using machine learning. Our results indicate that the proposed system effectively improved activity accuracy recognition for individual users and reduced cost and latency for inference for mobile devices. Based on our findings, we highlight critical and promising future research directions regarding the design of efficient activity data collection with on-device personalization. |
| first_indexed | 2024-03-10T13:49:32Z |
| format | Article |
| id | doaj.art-86c682fa4f934255a55ef7bb30fee56d |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T13:49:32Z |
| publishDate | 2020-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-86c682fa4f934255a55ef7bb30fee56d2023-11-21T02:17:29ZengMDPI AGSensors1424-82202020-12-012114110.3390/s21010041On-Device Deep Personalization for Robust Activity Data CollectionNattaya Mairittha0Tittaya Mairittha1Sozo Inoue2Graduate School of Engineering, Kyushu Institute of Technology, 1-1 Sensui-cho, Tobata-ku, Kitakyushu-shi, Fukuoka 804-8550, JapanGraduate School of Engineering, Kyushu Institute of Technology, 1-1 Sensui-cho, Tobata-ku, Kitakyushu-shi, Fukuoka 804-8550, JapanGraduate School of Engineering, Kyushu Institute of Technology, 1-1 Sensui-cho, Tobata-ku, Kitakyushu-shi, Fukuoka 804-8550, JapanOne of the biggest challenges of activity data collection is the need to rely on users and keep them engaged to continually provide labels. Recent breakthroughs in mobile platforms have proven effective in bringing deep neural networks powered intelligence into mobile devices. This study proposes a novel on-device personalization for data labeling for an activity recognition system using mobile sensing. The key idea behind this system is that estimated activities personalized for a specific individual user can be used as feedback to motivate user contribution and improve data labeling quality. First, we exploited fine-tuning using a Deep Recurrent Neural Network to address the lack of sufficient training data and minimize the need for training deep learning on mobile devices from scratch. Second, we utilized a model pruning technique to reduce the computation cost of on-device personalization without affecting the accuracy. Finally, we built a robust activity data labeling system by integrating the two techniques outlined above, allowing the mobile application to create a personalized experience for the user. To demonstrate the proposed model’s capability and feasibility, we developed and deployed the proposed system to realistic settings. For our experimental setup, we gathered more than 16,800 activity windows from 12 activity classes using smartphone sensors. We empirically evaluated the proposed quality by comparing it with a baseline using machine learning. Our results indicate that the proposed system effectively improved activity accuracy recognition for individual users and reduced cost and latency for inference for mobile devices. Based on our findings, we highlight critical and promising future research directions regarding the design of efficient activity data collection with on-device personalization.https://www.mdpi.com/1424-8220/21/1/41activity recognitiondata collectionon-device personalizationdeep learningfine-tuningsmartphone sensors |
| spellingShingle | Nattaya Mairittha Tittaya Mairittha Sozo Inoue On-Device Deep Personalization for Robust Activity Data Collection Sensors activity recognition data collection on-device personalization deep learning fine-tuning smartphone sensors |
| title | On-Device Deep Personalization for Robust Activity Data Collection |
| title_full | On-Device Deep Personalization for Robust Activity Data Collection |
| title_fullStr | On-Device Deep Personalization for Robust Activity Data Collection |
| title_full_unstemmed | On-Device Deep Personalization for Robust Activity Data Collection |
| title_short | On-Device Deep Personalization for Robust Activity Data Collection |
| title_sort | on device deep personalization for robust activity data collection |
| topic | activity recognition data collection on-device personalization deep learning fine-tuning smartphone sensors |
| url | https://www.mdpi.com/1424-8220/21/1/41 |
| work_keys_str_mv | AT nattayamairittha ondevicedeeppersonalizationforrobustactivitydatacollection AT tittayamairittha ondevicedeeppersonalizationforrobustactivitydatacollection AT sozoinoue ondevicedeeppersonalizationforrobustactivitydatacollection |