Thank you to all those who attended the ECIR 2013 tutorial on Practical Online Retrieval Evaluation that Filip Radlinski and I gave today. We updated some of the slides from the version that was provided on the conference USB stick. Below are the updated slides and links to additional resources.
- PPT slides (based on the 2011 SIGIR tutorial by Yisong Yue and Filip Radlinski, the slides of this earlier tutorial are available on Yisong’s website) [PDF version without animations]
- Demo scripts: these are based on Yisong’s material, available from his website
- Additional reference implementations of Balanced Interleave, Team Draft, and Probabilistic Interleaving in python are part of our Learning to Rank Framework, released with our WSDM 2013 paper Reusing Historical Interaction Data For Faster Online Learning To Rank For IR by Katja Hofmann, Anne Schuth, Shimon Whiteson, and Maarten de Rijke
For additional details and further reading, please see the references cited in the slides. A detailed list of references is included at the end of the presentation.
Katja Hofmann 