Tag-based search on Flickr by combining folksiness information and image visual features

Subject Areas : Information Technology in Engineering Design (ITED) Journal

Received: 2015-01-26 Accepted : 2015-01-26 Published : 2015-01-21

Keywords:

Abstract :

Abstract In recent years, there are explosive photo sharing websites such as Flickr that users can upload and annotate images with arbitrary keywords called tags. Consequently users have been overwhelmed by the huge numbers of images that effective retrieval technology for this volume of images is needed. In tag-based image retrieval most existing methods use tags or visual characteristics and do not consider users information. In this paper we propose a system for tag-based image retrieval on Flickr. Base on extracted information from ternary relationships between users, images and tags in folksonomy and simultaneously visual features that extracted from images, the similarity between images and tags derive and use for tag-based image retrieval. A folksonomy can be viewed as a three-dimensional space of users, tags, and images; consequently this three-dimensional space can be projected onto three two-dimensional matrices user-tag, user-image and tag-image. We then extract Tag-Image similarity from tag-item matrix. To calculate similarity between images based on visual features we use cosine similarity between local features. To validate the effectiveness of our proposed approach on real-world web image datasets, we conduct extensive experiments on the image dataset NUS-WIDE. Since the NUS-WIDE dataset do not contain user information and only contain visual features and tag information, we use Flickr API to access users tag assignments. Experimental evaluations demonstrate that using simultaneously folksonomy and image visual features can improve results of tag-based image retrieval than using separated folksonomy information or image visual features.

References:

1. R. Datta, D. Joshi, J. Li, and J. Wang, “Image retrieval: ideas, influences, and trends of the new age,” ACM Computing Surveys (CSUR), vol. 40, no. 2, pp. 5,2008. 2. Heung-Nam Kim, Majdi Rawashdeh , Abdullah Alghamdi , Abdulmotaleb El Saddik, “Folksonomy- based personalized search and ranking in social media services,” Information Systems vol. 37, no. 1, pp. 61-76, 2012. 3. Yue Gao, Meng Wang, Zheng-Jun Zha, Jialie Shen, XuelongLi, Xindong Wu, “Visual-Textual Joint Relevance Learning forTag-Based Social Image Search,” IEEE TRANSACTIONS ON IMAGE PROCESSING, 2011 4. S.A. Golder, B.A. Huberman, “Usage patterns of collaborative tagging systems,” Journal of Information Science vol. 32, no. 2, pp. 198–208 , 2006. 5. Xiaoming Zhang, XiaojianZhao, ZhoujunLi, JialiXia, RameshJain, WenhanChao, “Social image tagging using graph-based reinforcement on multi-type interrelated objects,” Signal Processing vol. 93 , no. 8 ,pp. 2178-2189, 2013. 6. M. Kato, H. Ohshima, S. Oyama, and K. Tanaka, “Can social tagging improve web image search?,” Lecture Notes in Computer Science, vol.5175, pp. 235–249, 2008. 7. L. Chen, D. Xu, W. Tsang, and J. Luo, “Tag-based web photo retrieval improved by batch mode re-tagging,” in Proceedings of IEEE InternationalConference on Computer Vision and Pattern Recognition, 2010. 8. K. Yang, M. Wang, X. S. Hua, and H. J. Zhang, “Social image search with diverse relevance ranking,” in Proceedings of ACM Conference on Multimedia Modeling, pp. 174–184, 2009. 9. X. Li, C. G. M. Snoek, and M. Worring, “Learning tag relevance by neighbor voting for social image retrieval,” in Proceedings of ACM conference on multimedia information retrieval, pp. 180–187, 2008. 10. Jitao Sang, Changsheng Xu, and Jing Liu. “User-Aware Image Tag Refinement via Ternary Semantic Analysis,” IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 14, NO. 3, 2012. 11. Sihyoung Lee, Wesley De Neve, Yong Man Ro. “Tag refinement in an image folksonomy using visual similarity and tag co-occurrence statistics,” Signal Processing: Image Communication vol.25 , no.10,pp. 761-773 , 2010. 12. G. Zhu, S. Yan, and Y. Ma, “Image tag refinement towards low rank, content-tag prior and error sparsity,” in Proceedings of ACM International Conference on Multimedia, pp. pp.461-470, 2010. 13. X. Li, C. G. Snoek, and M. Worring, “Learning social tag relevance by neighbor voting,” IEEE Transaction on Multimedia, vol. 11, no. 7, pp.1310–1322, 2009. خطی فضای حالت بندی ایستای وایف با استفاده از سوئیچینگ ارائه یک مدل زمان 123 14. J. Bu, S. Tan, C. Chen, C. Wang, H. Wu, L. Zhang, and X. He, “Music recommendation by unified hypergraph: combining social media information and music content,” in Proceedings of ACM International Conference on Multimedia, pp. 391-400 , 2010. 15. Sare Gul Sevil , Onur Kucuktunc,Pinar Duygulu, Fazli Can. “Automatic tag expansion using visual similarity for photo sharing websites,” Multimedia Tools and Applications, vol.49 , no. 1, pp. 81-99,2010. 16. D. Liu, M. Wang, L. Yang, X. S. Hua, and H. Zhang, “Tag quality improvement for social images,” in Proceedings of IEEE InternationalConference on Multimedia, pp. 350–353, 2009. 17. Ilaria Bartolini, Marco Patella , Corrado Romani, “SHIATSU: tagging and retrieving videos without worries,” Multimedia Tools and Applications, pp. 1-29, 2013. 18. A.Hotho,R.J aschke, C.Schmitz,G.Stumme, “Information retrieval in folksonomies: search and ranking,” in:Proceeding sof the Third European Semantic Web Conference ,pp.411–426, 2006. 19. Sihyoung Lee, Wesley De Neve, Konstantinos N. Plataniotis, Yong Man Ro. “MAP-based image tag recommendation using a visual folksonomy,” Pattern Recognition Letters, vol. 31, no. 9, pp. 976-982, 2010. 20. K. Bischoff, C.S. Firan, W. Nejdl, R. Paiu, “Can all tags be used for search?, ” in: Proceeding of the 17th ACM Conference on Information and Knowledge Management, pp. 203–212, 2008. 21. X. Li, L. Guo, Y.E. Zhao, “Tag-based social interest discovery,” in: Proceeding of the 17th International Conference on World Wide Web, pp. 675–684, 2008. 22. M. Deshpande, G. Karypis, “Item-based top-n recommendation algorithms,” ACM Transactions on Information Systems, vol.22, no. 1, pp. 143–177, 2004. 23. R.J aschke, L.Marinho,A.Hotho,L.Schmidt-Thieme,G.Stumme, “Tag recommendations in social bookmarking systems ,” AI Communications, vol.21 ,no. 4, pp.231–247, 2008. 24. H.N.Kim,A.T.Ji,I.Ha,G.S.Jo, “Collaborative filtering based on collaborative tagging for enhancing the quality of recommendation,” Electronic Commerce Research and Applications, vol. 9, no. 1, pp.73–83, 2010. 25. S. Xu, S. Bao, B. Fei, Z. Su, Y. Yu, “Exploring folksonomy for personalized search, ” in: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 155–162, 2008. 26. V. Zanardi, L. Capra, “Social ranking: uncovering relevant content using tag-based recommender systems,” in: Proceedings of the 2008 ACM Conference on Recommender Systems, pp. 51–58, 2008. 27. Ba Quan Truong, Aixin Sun, Sourav S. Bhowmick, “Content is Still King: The Effect of Neighbor Voting Schemes on Tag Relevance for Social Image Retrieval,” Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. ACM, 2012. ی ـ مهندس ر طراحـی د ات ـ اوری اطلاع ـ ه فن ـ مجل 124 28. Aixin Sun, Sourav S. Bhowmick, Khanh Tran Nam Nguyen. “Tag-Based Social Image Retrieval: An Empirical Evaluation,” Journal of the American Society for Information Science and Technology (JASIST), vol.62, no. 12, pp.2364–2381, 2011. 29. Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng. “NUSWIDE: A Real-World Web Image Database from National University of Singapore,” Proceedings of the ACM International Conference on Image and Video Retrieval. ACM, 2009.

_||_