International Journal of Information Technology and Computer Science (IJITCS)
IJITCS Vol. 11, No. 1, 8 Jan. 2019
Cover page and Table of Contents: PDF (size: 421KB)
Selecting Appropriate Metrics for Evaluation of Recommender Systems
Full Text (PDF, 421KB), PP.14-23
Views: 0 Downloads: 0
Author(s)
Index Terms
Precision, Recall, Root Mean Square Error, Mean Absolute Error, Normalized Mean Average Error
Abstract
The abundance of information on the web makes it difficult for users to find items that meet their information need effectively. To deal with this issue, a large number of recommender systems based on different recommender approaches were developed which have been used successfully in a wide variety of domains such as e-commerce, e-learning, e-resources, and e-government among others. Moreover, in order for a recommender system to generate good quality of recommendations, it is essential for a researcher to find the most suitable evaluation metric which best matches a given recommender algorithm and a recommender's task. However, with the availability of several recommender tasks, recommender algorithms, and evaluation metrics, it is often difficult for a researcher to find their best combination. This paper aims to discuss various evaluation metrics in order to help researchers to select the most appropriate metric which matches a given task and an algorithm so as to provide good quality of recommendations.
Cite This Paper
Bhupesh Rawat, Sanjay K.Dwivedi, "Selecting Appropriate Metrics for Evaluation of Recommender Systems", International Journal of Information Technology and Computer Science(IJITCS), Vol.11, No.1, pp.14-23, 2019. DOI:10.5815/ijitcs.2019.01.02
Reference
[1]G.Adomavicius, A. Tuzhilin, “Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions”,IEEE Transactions on Knowledge and Data Engineering, Vol.17, no.6,pp.734–749,2005.
[2]J. Lu, D. Wu, M. Mao, W. Wang, G. Zhang, “Recommender system application developments: A survey, Decision support system, vol.74, pp.12-32,2015.
[3]S.S. Sohail, J. Siddiqui, R. Ali, “Classifications of Recommender Systems: A review”, Journal of Engineering Science and Technology Review,vol.10,no.4,pp.132-153, 2017.
[4]P. Resnick, N. Iakavou, M. Sushak, P. Bergsstrom and J. Riedl, “Grouplens: An open architecture for collaborative filtering of Netnews”, In Proceedings of the computer supported cooperative work conference, 1994.
[5]D. Herath and L. Jayaratne, “A Personalized Web Content Recommendation System for E-Learners in E-Learning Environment”, In Proceedings of National Information Technology Conference, pp.89-95, 2017.
[6]R. Rafter, “Evaluation and Conversation in Collaborative Filtering”, PhD Thesis, University College Dublin, College of Engineering Mathematical and Physical Sciences, 2010.
[7]Y. Want, M. Wang, W. Xu, “A Sentiment-Enhanced Hybrid Recommender System for Movie Recommendation: A Big Data Analytics Framework”, Wireless Communications and Mobile Computing, 2018.
[8]G. Badaro, H. Hajj, W. EI-Hajj,Nachman, “A Hybrid Approach with Collaborative Filtering for Recommender Systems”, In 9th International Conference on Wireless Communications and Mobile Computing,pp.349-354,2013.
[9]E. Cano and M. Morisio, “Hybrid recommender systems: A systematic literature review”, Intelligent Data Analysis, vol.21,no.6,pp.1487-1524, 2017.
[10]G. Shani, D. Heckerman, and R.I. Brafman “An mdpbased recommender system”, Journal of Machine Learning Research, vol.6,pp.1265-1295, 2005.
[11]D. Braziunas and C. Boutilier, “Local Utility Elicitation in GAI Models”, In Proceedings of the Twenty first Conference on Uncertainty in Artificial Intelligence, pp.42-49, 2005,
[12]J. Herlocker, J. Konstan, J.L.Terveen, J. Reidl, “Evaluating Collaborative Filtering Recommender Systems”,ACM Transaction on Information Systems, ,vol.22,no.1,pp.5-53, 2004.
[13]F .M. Harper and J. Konstan, “The Movie Lens Dataset: History and Context”, ACM Transactions on Interactive Intelligent Systems, vol.5, 2015
[14]M. Hahsler, “Recommender lab: A Framework for Developing and Testing Recommendation Algorithms”, Southern Methodist University, 2011.
[15]A. Gilotte, C. Calauzenes, T. Nedelec, A. Abraham, S. Dolle, “ Offline A/B testing for Recommender Systems”,In Proceedings of the 11th ACM International Conference on Web Search and Data Mining,pp.198-206,2018.
[16]B. Marlin, R. S. Zemel, S. Roweis, M. Slaney , “Collaborative filtering and the missing at random assumption”, In Proceedings of the 23rd conference on uncertainty in artificial intelligence,pp.5054,2007.
[17]R. Mu, and X. Zeng, “Collaborative Filtering Recommendation Algorithm Based on Knowledge Graph”, Mathematical Problems in Engineering, 2018.
[18]R. Kohavi, R. Longbotham, D. Sommerfield, and R .M. Henne, “Controlled experiments on the web: survey and practical guide”, Data Mining and Knowledge Discovery, vol.18, no.1, pp.140-181, 2009.
[19]J. Lu, D. Wu, M. Mao, W. Wang, and G. Zhang, “Reco-Mmender system application developments: a survey”, Decision support system, vol.74, pp.12-32, 2015.
[20]K. Goldberg, T. Roeder, D. Gupta and C. Perkins, “Eigentaste: A constant time collaborative filtering algorithm”, Information Retrieval, vol.4, no.2, pp.133- 151, 2001.
[21]J.W. Perry, K. Kent, M.M. Berry, “ Machine literature searching X. Machine language; factors underlying its design and development”, American Documentation, vol.6,no.4, 1955.
[22]S.M. Beitzel, “On Understanding and Classifying Web Queries (Ph.D. thesis)”, Department of Computer Science,Illinois Institute Technology,Chicago,2006.
[23]D.C. Blair, “Information retrieval”, 2nd ed. c.j. van rijsbergen. london. Journal of the American Society for Information Science, vol.30, no.6, pp.374-375, 1979.
[24]F.H.D .Olmo and E. Gaudioso, “Evaluation of recommender systems: A new approach”, Expert Systems with Applications, vol.35, no.3, pp.790-804, 2018.
[25]T.Arsan, E. Koksal, Z. Bozkus, “Comparison of collaborative filtering algorithms with various similarity measures for movie recommendation”, International Journal of computer science engineering and application, vol.6, no.3, 2016.[26]G. Salton, “The smart document retrieval project”, In Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, pp.356-358 ,1991.
[27]Z. Zhong, T. Xu, F. Wang, T. Tang, “Text case based reasoning framework for fault diagnosis and predication by cloud computing. Mathematical problems in engineering, 2018.
[28]Y. Xin, L. Kong, Z. Liu, Y. Chen,Y.Li, H.Zhu,M.Gao, M,H.Hou,C.Wang, “ Machine Learning and Deep Learning Methods for Cybersecurity”,IEEE Access,2018.