| Summary: | Recommender systems play an essential role in enhancing user experiences by providing personalized content and suggestions, thereby improving user engagement and satisfaction. However, a major challenge faced by recommender systems is data sparsity, where real-world datasets often lack comprehensive user-item interaction data, resulting in suboptimal performance. Additionally, their dependence on limited user-item interaction data makes them susceptible to over-fitting and poor generalization, further compromising their effectiveness.
A promising solution to mitigate data sparsity and improve the generalization capabilities of recommender systems involves the integration of data augmentation techniques. Data augmentation artificially expands datasets by creating new or modified copies of existing data. In model training, augmented data can be utilized in two primary ways: it can either be directly fed into the neural network or used in conjunction with contrastive learning. Both approaches are aimed at refining the model parameters in response to new variations introduced by the augmented data. This process is crucial for enabling the model to learn rich and discriminative representations, particularly under the constraint of sparse data.
However, generating effective data augmentations in recommender systems is challenging. Firstly, the inherent issue of data sparsity in recommender systems makes generating meaningful augmented data without introducing noise or bias a complex task. Second, the complexity of user-item interactions, influenced by user/item features, temporal dynamics of preferences, and specific contextual scenarios, further complicates this task.
This dissertation aims to develop effective data augmentation methods with contrastive learning to enhance model generalization and mitigate the data sparsity issue in recommender systems. Firstly, we analyze the original user-item interaction graph and propose a data augmentation method that minimizes the introduction of noise or bias, ensuring the integrity of the underlying data structure. Secondly, we explore the incorporation of side information by designing a systematic data augmentation method based on item features, thus leveraging additional contextual information to improve recommendations. Lastly, we integrate multimodal features and the sequential order of user interactions into our data augmentation strategy, providing a robust solution that captures the complexity and richness of real-world user behavior. By addressing these aspects collectively, this dissertation demonstrates how a multifaceted data augmentation strategy can significantly enhance the performance of recommender systems. Specifically, the research is divided into three chapters, each addressing a specific research problem:
Research problem 1 Existing data augmentation methods predominantly focus on randomly removing edges from the user-item interaction graph, which overlooks the importance of differentiating between informative and irrelevant or noisy edges in the augmented graph. We present an advanced method utilizing graph diffusion, which smooths neighborhood interactions across the graph and transforms the original unweighted graph into a weighted one. The weights, based on the structural importance of each edge, facilitate the maintenance of an efficient neighborhood for each node in the diffusion graph. Particularly, we propose a Graph Diffusion Contrastive Learning (GDCL) framework for recommendation, where the diffusion graph is encoded to preserve heterogeneity, and a symmetric contrastive learning objective contrasts local node representations of the diffusion graph with the user-item interaction graph.
Research problem 2 Current feature-based data augmentation methods in recommender systems generally involve random alterations of features, such as dropout, shuffling, or perturbing embeddings with random noise. However, these methods mostly rely on arbitrary data augmentations, chosen through a process of trial-and-error. This reliance on non-systematic methods may constrain their generalizability and adversely affect their overall performance. This study first examines a sequential recommendation framework based on item text features, finding that anisotropy in pre-trained text embeddings can impair performance. To address this, a whitening transformation is applied to reconfigure the pre-trained text embedding distribution into an isotropic form, significantly enhancing model performance. However, an empirical analysis indicates that the whitening may adversely affect the manifold of items with similar textual semantics. To mitigate this, we first introduce an ensemble framework WhitenRec+, which combines fully and partially whitened representations via a simple summation. Then, we refine its architecture by proposing a Dual-view Whitening method for Sequential Recommendation (DWSRec). DWSRec utilizes diverse views of whitened embeddings to alternately update the attention heads within the transformer model, effectively acting as data augmentation and improving overall performance.
Research problem 3 Most existing methods focus on augmenting a single type of feature and are often unable to explore augmentations with user behavior sequences across different types of features. In this work, we propose a novel Multimodal Pre-training for Sequential Recommendation (MP4SR) framework, which utilizes contrastive losses to capture the correlation among different modality sequences of users and different modality sequences of users and items. MP4SR employs a sequence mixup strategy for fusing different modality sequences and leverages contrastive learning at the sequence-to-sequence and sequence-to-item levels. This multimodal pre-training approach serves as an effective regularizer, optimizing the parameter space for recommendation tasks.
In conclusion, this dissertation proposes methods that develop different data augmentations across various data structures to enhance the performance of recommender systems. Extensive experiments on real-world datasets validate the effectiveness of these methods.
|
|---|