Development of a Multimodal Movie Recommendation System Using Graph Neural Networks and Fusion of Textual and Visual Features

Subject Areas : information technology

Ali Mosaddegh ¹ , Danial Barati ² , kiarash fazilat ³

1 - MSc student, department of engineering, West Tehran Bracnh, Islamic Azad University, Tehran, Iran
2 - MSc student, department of engineering, West Tehran Bracnh, Islamic Azad University, Tehran, Iran
3 - MSc student, department of engineering, West Tehran Bracnh, Islamic Azad University, Tehran, Iran

Received: 2024-12-27 Accepted : 2024-12-31 Published : 2025-03-04

Keywords: Recommendation System, Graph Neural Networks, Multimodal Data, Movie Recommender, Feature Fusion,

Abstract :

This study focuses on the design and development of a multimodal movie recommendation system utilizing Graph Neural Networks (GCN). The primary goal of the system is to improve the accuracy and quality of recommendations by integrating multimodal information, including textual and visual features of movies. In this model, the user-movie interaction graph was used as the main structure to model relationships between users and movies through nodes and edges. Textual features of movies were extracted using embedding models, while visual features were extracted using convolutional neural networks, and these features were then fused into graph nodes. The GCN was employed to learn interactive features and predict user preferences. Experimental results demonstrated that the proposed model, despite fluctuations in loss and mean squared error (MSE), achieved relative improvements in accuracy and convergence compared to baseline methods. The interaction graph also highlighted the diversity of user preferences and the importance of high-interaction movies. Additionally, this study provides suggestions for enhancing the model, such as employing real-world datasets, advanced fusion algorithms, and improving interpretability. The proposed model serves as a foundation for designing more advanced and personalized recommendation systems.

References:

[1]. Xia, L., Yang, Y., Chen, Z., Yang, Z., & Zhu, S. (2024). Movie Recommendation with Poster Attention via Multi-modal Transformer Feature Fusion. arXiv preprint arXiv:2407.09157.
[2]. Wu, X., Huang, A., Yang, H., He, H., Tai, Y., & Zhang, W. (2024). Towards Bridging the Cross-modal Semantic Gap for Multi-modal Recommendation. arXiv preprint arXiv:2407.05420.
[3]. Tokala, S., Nagaram, J., Enduri, M. K., & Lakshmi, T. J. (2024, June). Enhanced Movie Recommender system using Deep Learning Techniques. In 2024 3rd International Conference on Computational Modelling, Simulation and Optimization (ICCMSO) (pp. 71-75). IEEE.
[4]. Malitesta, D. (2024). Graph neural networks for recommendation leveraging multimodal information.
[5]. Burabak, M., & Aytekin, T. (2024). SynerGraph: An Integrated Graph Convolution Network for Multimodal Recommendation. arXiv preprint arXiv:2405.19031.
[6]. Mouhiha, M., Oualhaj, O. A., & Mabrouk, A. (2024, May). Enhancing Movie Recommendations: A Deep Neural Network Approach with MovieLens Case Study. In 2024 International Wireless Communications and Mobile Computing (IWCMC) (pp. 1303-1308). IEEE.
[7]. He, J., Zhang, L., Cao, W., Yang, M., Li, M., Zhao, Z., & Leung, M. F. (2024, May). Multi-modal Bayesian Recommendation System. In 2024 IEEE 6th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC) (Vol. 6, pp. 141-145). IEEE.
[8]. Wei, X., Dou, J., Wang, S., Zhang, Y., Hou, B., & Wang, F. (2024, May). Multi-view Sequence Recommendation Model. In 2024 IEEE 6th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC) (Vol. 6, pp. 645-648). IEEE.
[9]. Malitesta, D., Cornacchia, G., Pomo, C., Merra, F. A., Di Noia, T., & Di Sciascio, E. (2018). Formalizing multimedia recommendation through multimodal deep learning. ACM Transactions on Recommender Systems.
[10]. Siet, S., Peng, S., Ilkhomjon, S., Kang, M., & Park, D. S. (2024). Enhancing sequence movie recommendation system using deep learning and kmeans. Applied Sciences, 14(6), 2505.
[11]. Kim, W. H., Kim, G. W., & Kim, J. C. (2024). Multi-Modal Deep Learning based Metadata Extensions for Video Clipping. International Journal on Advanced Science, Engineering & Information Technology, 14(1).
[12]. Zhan, Y., Xie, H., Huan, H., & Che, S. (2024, February). Research on Movie Recommendation Algorithm based on Deep Learning. In 2024 International Conference on Integrated Circuits and Communication Systems (ICICACS) (pp. 1-6). IEEE.
[13]. Peng, S., Siet, S., Ilkhomjon, S., Kim, D. Y., & Park, D. S. (2024). Integration of deep reinforcement learning with collaborative filtering for movie recommendation systems. Applied Sciences, 14(3), 1155.
[14]. Li, J., Yang, C., Ye, G., & Nguyen, Q. V. H. (2024). Graph neural networks with deep mutual learning for designing multi-modal recommendation systems. Information Sciences, 654, 119815.