Automatic Classification of Photos by Tourist Attractions Using Deep Learning Model and Image Feature Vector Clustering
With the rise of social media platforms, tourists tend to share their experiences in the form of texts, photos, and videos on social media. These user-generated contents (UGC) play an important role in shaping tourism destination images (TDI) and directly affect the decision-making process of touris...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-04-01
|
| Series: | ISPRS International Journal of Geo-Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2220-9964/11/4/245 |
| _version_ | 1827628524012306432 |
|---|---|
| author | Jiyeon Kim Youngok Kang |
| author_facet | Jiyeon Kim Youngok Kang |
| author_sort | Jiyeon Kim |
| collection | DOAJ |
| description | With the rise of social media platforms, tourists tend to share their experiences in the form of texts, photos, and videos on social media. These user-generated contents (UGC) play an important role in shaping tourism destination images (TDI) and directly affect the decision-making process of tourists. Among UGCs, photos represent tourists’ visual preferences for a specific area. Paying attention to the value of photos, several studies have attempted to analyze them using deep learning technology. However, the research methods that analyze tourism photos using recent deep learning technology have a limitation in that they cannot properly classify unique photos appearing in specific tourist attractions with predetermined photo categories such as Places365 or ImageNet dataset or it takes a lot of time and effort to build a separate training dataset to train the model and to generate a tourism photo classification category according to a specific tourist destination. The purpose of this study is to propose a method of automatically classifying tourist photos by tourist attractions by applying the methods of the image feature vector clustering and the deep learning model. To this end, first, we collected photos attached to reviews posted by foreign tourists on TripAdvisor. Second, we embedded individual images as 512-dimensional feature vectors using the VGG16 network pre-trained with Places365 and reduced them to two dimensions with t-SNE(t-Distributed Stochastic Neighbor Embedding). Then, clusters were extracted through HDBSCAN(Hierarchical Clustering and Density-Based Spatial Clustering of Applications with Noise) analysis and set as a regional image category. Finally, the Siamese Network was applied to remove noise photos within the cluster and classify photos according to the category. In addition, this study attempts to confirm the validity of the proposed method by applying it to two representative tourist attractions such as ‘Gyeongbokgung Palace’ and ‘Insadong’ in Seoul. As a result, it was possible to identify which visual elements of tourist attractions are attractive to tourists. This method has the advantages in that it is not necessary to create a classification category in advance, it is possible to flexibly extract categories for each tourist destination, and it is able to improve classification performance even with a rather small volume of a dataset. |
| first_indexed | 2024-03-09T13:34:42Z |
| format | Article |
| id | doaj.art-0a785d4d80074158af4ea1f9e57b65e5 |
| institution | Directory Open Access Journal |
| issn | 2220-9964 |
| language | English |
| last_indexed | 2024-03-09T13:34:42Z |
| publishDate | 2022-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | ISPRS International Journal of Geo-Information |
| spelling | doaj.art-0a785d4d80074158af4ea1f9e57b65e52023-11-30T21:13:41ZengMDPI AGISPRS International Journal of Geo-Information2220-99642022-04-0111424510.3390/ijgi11040245Automatic Classification of Photos by Tourist Attractions Using Deep Learning Model and Image Feature Vector ClusteringJiyeon Kim0Youngok Kang1Department of Social Studies, Ewha Womans University, Seoul 03760, KoreaDepartment of Social Studies, Ewha Womans University, Seoul 03760, KoreaWith the rise of social media platforms, tourists tend to share their experiences in the form of texts, photos, and videos on social media. These user-generated contents (UGC) play an important role in shaping tourism destination images (TDI) and directly affect the decision-making process of tourists. Among UGCs, photos represent tourists’ visual preferences for a specific area. Paying attention to the value of photos, several studies have attempted to analyze them using deep learning technology. However, the research methods that analyze tourism photos using recent deep learning technology have a limitation in that they cannot properly classify unique photos appearing in specific tourist attractions with predetermined photo categories such as Places365 or ImageNet dataset or it takes a lot of time and effort to build a separate training dataset to train the model and to generate a tourism photo classification category according to a specific tourist destination. The purpose of this study is to propose a method of automatically classifying tourist photos by tourist attractions by applying the methods of the image feature vector clustering and the deep learning model. To this end, first, we collected photos attached to reviews posted by foreign tourists on TripAdvisor. Second, we embedded individual images as 512-dimensional feature vectors using the VGG16 network pre-trained with Places365 and reduced them to two dimensions with t-SNE(t-Distributed Stochastic Neighbor Embedding). Then, clusters were extracted through HDBSCAN(Hierarchical Clustering and Density-Based Spatial Clustering of Applications with Noise) analysis and set as a regional image category. Finally, the Siamese Network was applied to remove noise photos within the cluster and classify photos according to the category. In addition, this study attempts to confirm the validity of the proposed method by applying it to two representative tourist attractions such as ‘Gyeongbokgung Palace’ and ‘Insadong’ in Seoul. As a result, it was possible to identify which visual elements of tourist attractions are attractive to tourists. This method has the advantages in that it is not necessary to create a classification category in advance, it is possible to flexibly extract categories for each tourist destination, and it is able to improve classification performance even with a rather small volume of a dataset.https://www.mdpi.com/2220-9964/11/4/245image feature vectorclusteringSiamese Networkautomatic classification of tourist photosdeep learning model |
| spellingShingle | Jiyeon Kim Youngok Kang Automatic Classification of Photos by Tourist Attractions Using Deep Learning Model and Image Feature Vector Clustering ISPRS International Journal of Geo-Information image feature vector clustering Siamese Network automatic classification of tourist photos deep learning model |
| title | Automatic Classification of Photos by Tourist Attractions Using Deep Learning Model and Image Feature Vector Clustering |
| title_full | Automatic Classification of Photos by Tourist Attractions Using Deep Learning Model and Image Feature Vector Clustering |
| title_fullStr | Automatic Classification of Photos by Tourist Attractions Using Deep Learning Model and Image Feature Vector Clustering |
| title_full_unstemmed | Automatic Classification of Photos by Tourist Attractions Using Deep Learning Model and Image Feature Vector Clustering |
| title_short | Automatic Classification of Photos by Tourist Attractions Using Deep Learning Model and Image Feature Vector Clustering |
| title_sort | automatic classification of photos by tourist attractions using deep learning model and image feature vector clustering |
| topic | image feature vector clustering Siamese Network automatic classification of tourist photos deep learning model |
| url | https://www.mdpi.com/2220-9964/11/4/245 |
| work_keys_str_mv | AT jiyeonkim automaticclassificationofphotosbytouristattractionsusingdeeplearningmodelandimagefeaturevectorclustering AT youngokkang automaticclassificationofphotosbytouristattractionsusingdeeplearningmodelandimagefeaturevectorclustering |