Federated Meta-Learning with Attention for Diversity-Aware Human Activity Recognition
The ubiquity of smartphones equipped with multiple sensors has provided the possibility of automatically recognizing of human activity, which can benefit intelligent applications such as smart homes, health monitoring, and aging care. However, there are two major barriers to deploying an activity re...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/3/1083 |
| _version_ | 1797623262920835072 |
|---|---|
| author | Qiang Shen Haotian Feng Rui Song Donglei Song Hao Xu |
| author_facet | Qiang Shen Haotian Feng Rui Song Donglei Song Hao Xu |
| author_sort | Qiang Shen |
| collection | DOAJ |
| description | The ubiquity of smartphones equipped with multiple sensors has provided the possibility of automatically recognizing of human activity, which can benefit intelligent applications such as smart homes, health monitoring, and aging care. However, there are two major barriers to deploying an activity recognition model in real-world scenarios. Firstly, deep learning models for activity recognition use a large amount of sensor data, which are privacy-sensitive and hence cannot be shared or uploaded to a centralized server. Secondly, divergence in the distribution of sensory data exists among multiple individuals due to their diverse behavioral patterns and lifestyles, which contributes to difficulty in recognizing activity for large-scale users or ’cold-starts’ for new users. To address these problems, we propose DivAR, a diversity-aware activity recognition framework based on a federated Meta-Learning architecture, which can extract general sensory features shared among individuals by a centralized embedding network and individual-specific features by attention module in each decentralized network. Specifically, we first classify individuals into multiple clusters according to their behavioral patterns and social factors. We then apply meta-learning in the architecture of federated learning, where a centralized meta-model learns common feature representations that can be transferred across all clusters of individuals, and multiple decentralized cluster-specific models are utilized to learn cluster-specific features. For each cluster-specific model, a CNN-based attention module learns cluster-specific features from the global model. In this way, by training with sensory data locally, privacy-sensitive information existing in sensory data can be preserved. To evaluate the model, we conduct two data collection experiments by collecting sensor readings from naturally used smartphones annotated with activity information in the real-life environment and constructing two multi-individual heterogeneous datasets. In addition, social characteristics including personality, mental health state, and behavior patterns are surveyed using questionnaires. Finally, extensive empirical results demonstrate that the proposed diversity-aware activity recognition model has a relatively better generalization ability and achieves competitive performance on multi-individual activity recognition tasks. |
| first_indexed | 2024-03-11T09:26:11Z |
| format | Article |
| id | doaj.art-83b9ea84b29e475e9ee0cc7a5f8f0da6 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T09:26:11Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-83b9ea84b29e475e9ee0cc7a5f8f0da62023-11-16T17:55:54ZengMDPI AGSensors1424-82202023-01-01233108310.3390/s23031083Federated Meta-Learning with Attention for Diversity-Aware Human Activity RecognitionQiang Shen0Haotian Feng1Rui Song2Donglei Song3Hao Xu4College of Computer Science and Technology, Jilin University, Changchun 130012, ChinaCollege of Computer Science and Technology, Jilin University, Changchun 130012, ChinaSchool of Artificial Intelligence, Jilin University, Changchun 130012, ChinaCollege of Computer Science and Technology, Jilin University, Changchun 130012, ChinaCollege of Computer Science and Technology, Jilin University, Changchun 130012, ChinaThe ubiquity of smartphones equipped with multiple sensors has provided the possibility of automatically recognizing of human activity, which can benefit intelligent applications such as smart homes, health monitoring, and aging care. However, there are two major barriers to deploying an activity recognition model in real-world scenarios. Firstly, deep learning models for activity recognition use a large amount of sensor data, which are privacy-sensitive and hence cannot be shared or uploaded to a centralized server. Secondly, divergence in the distribution of sensory data exists among multiple individuals due to their diverse behavioral patterns and lifestyles, which contributes to difficulty in recognizing activity for large-scale users or ’cold-starts’ for new users. To address these problems, we propose DivAR, a diversity-aware activity recognition framework based on a federated Meta-Learning architecture, which can extract general sensory features shared among individuals by a centralized embedding network and individual-specific features by attention module in each decentralized network. Specifically, we first classify individuals into multiple clusters according to their behavioral patterns and social factors. We then apply meta-learning in the architecture of federated learning, where a centralized meta-model learns common feature representations that can be transferred across all clusters of individuals, and multiple decentralized cluster-specific models are utilized to learn cluster-specific features. For each cluster-specific model, a CNN-based attention module learns cluster-specific features from the global model. In this way, by training with sensory data locally, privacy-sensitive information existing in sensory data can be preserved. To evaluate the model, we conduct two data collection experiments by collecting sensor readings from naturally used smartphones annotated with activity information in the real-life environment and constructing two multi-individual heterogeneous datasets. In addition, social characteristics including personality, mental health state, and behavior patterns are surveyed using questionnaires. Finally, extensive empirical results demonstrate that the proposed diversity-aware activity recognition model has a relatively better generalization ability and achieves competitive performance on multi-individual activity recognition tasks.https://www.mdpi.com/1424-8220/23/3/1083human activity recognitionfederated learningmeta learning |
| spellingShingle | Qiang Shen Haotian Feng Rui Song Donglei Song Hao Xu Federated Meta-Learning with Attention for Diversity-Aware Human Activity Recognition Sensors human activity recognition federated learning meta learning |
| title | Federated Meta-Learning with Attention for Diversity-Aware Human Activity Recognition |
| title_full | Federated Meta-Learning with Attention for Diversity-Aware Human Activity Recognition |
| title_fullStr | Federated Meta-Learning with Attention for Diversity-Aware Human Activity Recognition |
| title_full_unstemmed | Federated Meta-Learning with Attention for Diversity-Aware Human Activity Recognition |
| title_short | Federated Meta-Learning with Attention for Diversity-Aware Human Activity Recognition |
| title_sort | federated meta learning with attention for diversity aware human activity recognition |
| topic | human activity recognition federated learning meta learning |
| url | https://www.mdpi.com/1424-8220/23/3/1083 |
| work_keys_str_mv | AT qiangshen federatedmetalearningwithattentionfordiversityawarehumanactivityrecognition AT haotianfeng federatedmetalearningwithattentionfordiversityawarehumanactivityrecognition AT ruisong federatedmetalearningwithattentionfordiversityawarehumanactivityrecognition AT dongleisong federatedmetalearningwithattentionfordiversityawarehumanactivityrecognition AT haoxu federatedmetalearningwithattentionfordiversityawarehumanactivityrecognition |