Test design of IIR evaluation

Abstract:

This talk addresses methodological issues of interactive information retrieval (IIR) evaluation in terms of what it entails to study users' use and interaction with IR systems.  In contrast to classic IR systems evaluation, which focuses only on IR systems performance and relies on the Cranfield model, IIR evaluation, and hence the test design, is far more complex due to the user involvement and the broader scope of testing.  Thus, the talk focuses on test design by taking a look into the toolbox of IIR test design with reference to data collection methods and test procedure.  This includes the use of assigned search tasks, known as simulated work task situations.

Biography:

Pia Borlund is professor of interactive information retrieval at Oslo and Akershus University College of Applied Sciences, the Department of Archivistics, Library and Information Science, Norway. She is also docent at the School of Information Sciences, University of Tampere, Finland.

The research expertise of Pia Borlund is on methods for evaluation of systems that support interactive information retrieval.  She has conducted research on frameworks and guidelines for performance evaluation of interactive information retrieval systems centred on the context instrument 'simulated work task situation' by involvement of users.  Her current research focuses on methodological issues, test design and recommendations for evaluation of user-based performance and search interaction.

Pia Borlund is actively involved in the research community and she acts as referee for a large number of conferences and journals.  She is member of editorial boards for Information Processing & Management (IPM), Journal of Information Science (JIS), Journal of Documentation (JDoc), Journal of Educational Media & Library Science, and Journal of Information Science Theory and Practice.

Previously, Pia Borlund was professor at the Royal School of Library and Information Science, Denmark, where she also served  as Pro Rector, Head of Research, and Head of Department of Information Studies.

Professor Borlund earned her PhD from Aabo Akademi University in Finland and her Master and Bachelor degrees from the Royal School of Library and Information Science, Denmark.