International Journal of Information Technology and Computer Science (IJITCS)
IJITCS Vol. 11, No. 5, 8 May 2019
Cover page and Table of Contents: PDF (size: 358KB)
Factors Affecting Users‟ Measure of Interest: A Study of the Effect of Task, Document Difficulty and Document Familiarity
Full Text (PDF, 358KB), PP.47-57
Views: 0 Downloads: 0
Author(s)
Stephen Akuma
1,*
Chrisina Jayne
2
Index Terms
Document difficulty, Document familiarity, implicit indicators, Information Retrieval, Task Type
Abstract
Data on the web is constantly growing which may affect users’ ability to find relevant information within a reasonable time limit. Some of the factors previously studied that affect users searching behaviour are task difficulty and topic familiarity. In this paper, we consider a set of implicit feedback parameters to investigate how document difficulty and document familiarity affects users searching behaviour in a task-specific context. An experiment was conducted and data was collected from 77 undergraduate students of Computer science. Users’ implicit features and explicit ratings of document difficulty and familiarity were captured and logged through a plugin in Firefox browser. Implicit feedback parameters were correlated with user ratings for document difficulty and familiarity. The result showed no correlation between implicit feedback parameters and the rating for document familiarity. There was, however, a negative correlation between user mouse activities and document difficulty ratings.
Also, the dataset of all the participants in the experiment was grouped according to task type and analysed. The result showed that their behaviour varies according to task type. Our findings provide more insight into studying the moderating factors that affect user searching behaviour.
Cite This Paper
Stephen Akuma, Chrisina Jayne, "Factors Affecting Users’ Measure of Interest: A Study of the Effect of Task, Document Difficulty and Document Familiarity", International Journal of Information Technology and Computer Science(IJITCS), Vol.11, No.5, pp.47-57, 2019. DOI:10.5815/ijitcs.2019.05.06
Reference
[1]G. Jawaheer, P. Weller and P. Kostkova, '"Modeling User Preferences in Recommender Systems: A Classification Framework for Explicit and Implicit User Feedback," ACM Transactions on Interactive Intelligent Systems, vol. 4, pp. 1-26.
[2]X. Zhu, J. Huang, B. Zhou, A. Li and Y. Jia, '"Real-time personalized twitter search based on semantic expansion and quality model," Neurocomputing, vol. 254, pp. 1339-1351.
[3]S. Akuma, R. Iqbal, C. Jayne and F. Doctor, '"Comparative analysis of relevance feedback methods based on two user studies," Comput.Hum.Behav., vol. 60, 7, pp. 138-146.
[4]B. Mobasher, R. Cooley and J. Srivastava, '"Automatic Personalization Based on Web Usage Mining," Commun ACM, vol. 43, no. 8, pp. 142-151.
[5]A. Alhindi, U. Kruschwitz, C. Fox and M. Albakour, '"Profile-Based Summarisation for Web Site Navigation,"ACM Transactions on Information Systems, vol. 33, no. 1, pp. 1-40.
[6]J. Liu, C. Liu and N. Belkin, '"Examining the effects of task topic familiarity on searchers' behaviors in different task types," Proceedings of the ASIST Annual Meeting, vol. 50, no. 1.
[7]N.J. Belkin, D. Hienert, P. Mayr and C. Shah, '"Data requirements for evaluation of personalization of information retrieval - A position paper," CEUR Workshop Proc., vol. 1866.
[8]S. Akuma, C. Jayne, R. Iqbal and F. Doctor, '"Implicit predictive indicators: Mouse activity and dwell time," IFIP Advances in Information and Communication Technology, vol. 436, September 19-21, pp. 162-171.
[9]D. Kelly and C. Cool, '"The effects of topic familiarity on information search behavior," Proceedings of the ACM International Conference on Digital Libraries, pp. 74-75.
[10]J. Liu, C. Liu, J. Gwizdka and N.J. Belkin, '"Can search systems detect users' task difficulty? Some behavioral signals," SIGIR 2010 Proceedings - 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 845-846.
[11]H. Lee and N. Pang, '"Understanding the effects of task and topical knowledge in the evaluation of websites as information patch," J.Doc., vol. 74, no. 1, pp. 162-186.
[12]S.K. Bhavnani, '"Domain-specific search strategies for the effective retrieval of healthcare and shopping information," Conference on Human Factors in Computing Systems - Proceedings, pp. 610-611.
[13]S.K. Bhavnani, '"Important cognitive components of domain-specific search knowledge,", pp. 571-578.
[14]B. Allen, '"Topic knowledge and online catalog search formulation," Library Quarterly, vol. 61, no. 2, pp. 188-213.
[15]X. Zhang, H.G.B. Anghelescu and X. Yuan, '"Domain knowledge, search behaviour, and search effectiveness of engineering and science students: An exploratory study," Information Research, vol. 10, no. 2.
[16]Y. Li and N.J. Belkin, '"A faceted approach to conceptualizing tasks in information seeking," Information Processing and Management, vol. 44, no. 6, pp. 1822-1837.
[17]P. Bennett, K. Collins-Thompson, D. Kelly, R. White and Y. Zhang, '"Overview of the Special Issue on Contextual Search and Recommendation," ACM Transactions on Information Systems, vol. 33, no. 1, pp. 1-7.
[18]D.K. Limbu, A.M. Connor, R. Pears and S.G. MacDonell, '"Improving web search using contextual retrieval," ITNG 2009 - 6th International Conference on Information Technology: New Generations, pp. 1329-1334.
[19]M. Busby, '"Learn Google,", 2003.
[20]R. Iqbal, A. Grzywaczewski, J. Halloran, F. Doctor and K. Iqbal, '"Design implications for task-specific search utilities for retrieval and reengineering of code," Enterprise Information Systems, pp. 1751-7575.
[21]A. Crescenzi, R. Capra and J. Arguello, '"Time pressure, user satisfaction and task difficulty," Proceedings of the ASIST Annual Meeting, vol. 50, no. 1.
[22]J. Mao, Y. Liu, K. Zhou, J. Nie, M. Zhang, S. Ma, J. Sun and H. Luo, '"When does Relevance Mean Usefulness and User Satisfaction in Web Search?" SIGIR '16 Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, July 17 -21, 2016, pp. 463-472.
[23]J. Jiang, D. He, D. Kelly and J. Allan, '"Understandingephemeral state of relevance,", March 07-11, 2017, pp. 137-146.
[24]K. Järvelin and P. Ingwersen, '"Information seeking research needs extension toward tasks and technology," Information Research, vol. 10, no. 1.
[25]R.A. Hamid, J.A. Thom and D.N.F.A. Iskandar, '"Effects of relevance criteria and subjective factors on web image searching behaviour," J.Inf.Sci., vol. 43, no. 6, pp. 786-800.
[26]C.C. Kuhlthau, '"A principle of uncertainty for information seeking," Journal of Documentation, vol. 49, no. 4, pp. 339-355.
[27]T.D. Wilson, '"Models in information behaviour research," Journal of Documentation, vol. 55, no. 3, pp. 249-270.
[28]R. Mehrotra and E. Yilmaz, '"Terms, topics & tasks: Enhanced user modelling for better personalization," ICTIR - Proc. ACM SIGIR Int. Conf. Theory Inf. Retr., pp. 131-140.
[29]M. Kellar, C. Watters and M. Shepherd, '"A field study characterizing Web-based information-seeking tasks," J.Am.Soc.Inf.Sci.Technol., vol. 58, no. 7, pp. 999-1018.
[30]R.W. White and D. Kelly, '"A study on the effects of personalization and task information on implicit feedback performance," International Conference on Information and Knowledge Management, Proceedings, pp. 297-306.
[31]D.-. Liu and I.-. Wu, '"Collaborative relevance assessment for task-based knowledge support," Decis.Support Syst., vol. 44, no. 2, pp. 524-543.
[32]D. Kelly and N.J. Belkin, '"Display time as implicit feedback: Understanding task effects," Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 377-384.
[33]C. Liu, X. Zhang and W. Huang, '"The exploration of objective task difficulty and domain knowledge effects on users' query formulation," Proceedings of the Association for Information Science and Technology, vol. 53, no. 1, pp. 1-9.
[34]M.J. Cole, X. Zhang, C. Liu, N.J. Belkin and J. Gwizdka, '"Knowledge effects on document selection in search results pages," SIGIR'11 - Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1219-1220.
[35]D. Kelly, J. Arguello, A. Edwards and W. Wu, '"Development and Evaluation of Search Tasks for IIR Experiments using a Cognitive Complexity Framework,", September 27-30, 2015.
[36]R. Iqbal, A. Grzywaczewski, A. James, F. Doctor and J. Halloran, '"Investigating the value of retention actions as a source of relevance information in the software development environment," Proceedings of the 2012 IEEE 16th International Conference on Computer Supported Cooperative Work in Design, CSCWD 2012, pp. 121-127.
[37]R. Capra, J. Arguello and Y. Zhang, '"The Effects of Search Task Determinability on Search Behavior,", April, 2017, pp. 108-121.
[38]H.A. Hembrooke, L.A. Granka, G.K. Gay and E.D. Liddy, '"The effects of expertise and feedback on search term selection and subsequent learning," J.Am.Soc.Inf.Sci.Technol., vol. 56, no. 8, pp. 861-871.
[39]X. Zhang, H.G.B. Anghelescu and X. Yuan, '"Domain knowledge, search behaviour, and search effectiveness of engineering and science students: An exploratory study," Information Research, vol. 10, no. 2.
[40]R.W. White, S.T. Dumais and J. Teevan, '"Characterizing the influence of domain expertise on web search behavior," Proceedings of the 2nd ACM International Conference on Web Search and Data Mining, WSDM'09, pp. 132-141.
[41]C.B. Gómez Ferragud, J.J. Solaz-Portolés and V. Sanjosé, '"Effects of topic familiarity on analogical transfer in problem-solving: A think-aloud study of two singular cases," Eurasia J.Math.Sci.Technol.Educ., vol. 11, no. 4, pp. 875-887.
[42]G. Pasi, '"Implicit feedback through user-system interactions for defining user models in personalized search," Procedia Comput. Sci., vol. 39, no. C, pp. 8-11.
[43]P. Borlund, '"The IIR evaluation model: A framework for evaluation of interactive information retrieval systems," Information Research, vol. 8, no. 3.
[44]G.V. Glass, P.D. Peckham and J.R. Sanders, '"Consequences of failure to meet assumptions underlying fixed effects analyses of variance and covariance," Rev. Educ. Res., vol. 42, pp. 237-288.
[45]M.R. Harwell, E.N. Rubinstein, W.S. Hayes and C.C. Olds, '"Summarizing Monte Carlo results in methodological research: the one- and two-factor fixed effects ANOVA cases," J. Educ. Stat, vol. 17, pp. 315-339.
[46]L.M. Lix, J.C. Keselman and H.J. Keselman, '"Consequences of assumption violations revisited: A quantitative review of alternatives to the one-way analysis of variance F test," Rev. Educ. Res., vol. 66, pp. 579-619.