An Empirical Study on Multimodal Activity Clustering of Android Applications
Recently, researchers have started looking at Android activities to improve user interface (UI) design. Since similar activities in Android have similar functional behaviors, activity clustering is a fundamental step toward efficient Android app development. Well-grouped activities are useful not on...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10138412/ |
| _version_ | 1797808485234114560 |
|---|---|
| author | Sungmin Choi Hyeon-Tae Seo Yo-Sub Han |
| author_facet | Sungmin Choi Hyeon-Tae Seo Yo-Sub Han |
| author_sort | Sungmin Choi |
| collection | DOAJ |
| description | Recently, researchers have started looking at Android activities to improve user interface (UI) design. Since similar activities in Android have similar functional behaviors, activity clustering is a fundamental step toward efficient Android app development. Well-grouped activities are useful not only for UI design, but also for app design, development, and testing. However, there are no studies on activity clustering yet, and no activity dataset with labels and categories. The purpose of this study is to use the Rico dataset to know (i) whether the Rico dataset can be used for activity clustering, (ii) how useful activity attributes expressed in XML are for activity clustering, and (iii) how useful fusion with activity image and attributes is for activity clustering. We generate various activity latent vectors using a CNN autoencoder for the Rico dataset. Then, we produce a sequence-to-sequence latent vector from the semantic properties of the Rico dataset. Finally, by fusing the two models, we propose an activity clustering approach using multimodal learning. Since there are no labels in the dataset, we make 2000 labeled data for evaluation. The experimental results show that the activity clustering works well by fusing the semantic activity latent vector and the seq2seq latent vector. Especially, activity attributes such as component and position information are effective for activity clustering and help to boost the performance better than real activity images or Rico. Research findings on clustering and newly created labeled data can be a starting point for various studies on Android activity. |
| first_indexed | 2024-03-13T06:38:13Z |
| format | Article |
| id | doaj.art-87076564d77f412786568953aa57512c |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T06:38:13Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-87076564d77f412786568953aa57512c2023-06-08T23:01:15ZengIEEEIEEE Access2169-35362023-01-0111535985361410.1109/ACCESS.2023.328098510138412An Empirical Study on Multimodal Activity Clustering of Android ApplicationsSungmin Choi0https://orcid.org/0009-0007-3374-3926Hyeon-Tae Seo1Yo-Sub Han2https://orcid.org/0000-0002-7211-6657Department of Computer Science, Yonsei University, Seoul, Republic of KoreaKT Research and Development Center, Seoul, Republic of KoreaDepartment of Computer Science, Yonsei University, Seoul, Republic of KoreaRecently, researchers have started looking at Android activities to improve user interface (UI) design. Since similar activities in Android have similar functional behaviors, activity clustering is a fundamental step toward efficient Android app development. Well-grouped activities are useful not only for UI design, but also for app design, development, and testing. However, there are no studies on activity clustering yet, and no activity dataset with labels and categories. The purpose of this study is to use the Rico dataset to know (i) whether the Rico dataset can be used for activity clustering, (ii) how useful activity attributes expressed in XML are for activity clustering, and (iii) how useful fusion with activity image and attributes is for activity clustering. We generate various activity latent vectors using a CNN autoencoder for the Rico dataset. Then, we produce a sequence-to-sequence latent vector from the semantic properties of the Rico dataset. Finally, by fusing the two models, we propose an activity clustering approach using multimodal learning. Since there are no labels in the dataset, we make 2000 labeled data for evaluation. The experimental results show that the activity clustering works well by fusing the semantic activity latent vector and the seq2seq latent vector. Especially, activity attributes such as component and position information are effective for activity clustering and help to boost the performance better than real activity images or Rico. Research findings on clustering and newly created labeled data can be a starting point for various studies on Android activity.https://ieeexplore.ieee.org/document/10138412/Activity clusteringautoencoderCNNdeep learningsequence-to-sequenceRico |
| spellingShingle | Sungmin Choi Hyeon-Tae Seo Yo-Sub Han An Empirical Study on Multimodal Activity Clustering of Android Applications IEEE Access Activity clustering autoencoder CNN deep learning sequence-to-sequence Rico |
| title | An Empirical Study on Multimodal Activity Clustering of Android Applications |
| title_full | An Empirical Study on Multimodal Activity Clustering of Android Applications |
| title_fullStr | An Empirical Study on Multimodal Activity Clustering of Android Applications |
| title_full_unstemmed | An Empirical Study on Multimodal Activity Clustering of Android Applications |
| title_short | An Empirical Study on Multimodal Activity Clustering of Android Applications |
| title_sort | empirical study on multimodal activity clustering of android applications |
| topic | Activity clustering autoencoder CNN deep learning sequence-to-sequence Rico |
| url | https://ieeexplore.ieee.org/document/10138412/ |
| work_keys_str_mv | AT sungminchoi anempiricalstudyonmultimodalactivityclusteringofandroidapplications AT hyeontaeseo anempiricalstudyonmultimodalactivityclusteringofandroidapplications AT yosubhan anempiricalstudyonmultimodalactivityclusteringofandroidapplications AT sungminchoi empiricalstudyonmultimodalactivityclusteringofandroidapplications AT hyeontaeseo empiricalstudyonmultimodalactivityclusteringofandroidapplications AT yosubhan empiricalstudyonmultimodalactivityclusteringofandroidapplications |