Accurately interpreting clickthrough data as implicit feedback

Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Geri Gay

Research output: Chapter in Book/Report/Conference proceedingConference contribution

736 Citations (Scopus)

Abstract

This paper examines the reliability of implicit feedback generated from clickthrough data in WWW search. Analyzing the users' decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes the interpretation of clicks as absolute relevance judgments difficult, we show that relative preferences derived from clicks are reasonably accurate on average.

Original languageEnglish (US)
Title of host publicationSIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
Pages154-161
Number of pages8
DOIs
StatePublished - Dec 1 2005
Event28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005 - Salvador, Brazil
Duration: Aug 15 2005Aug 19 2005

Publication series

NameSIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Other

Other28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005
CountryBrazil
CitySalvador
Period8/15/058/19/05

Fingerprint

Feedback
World Wide Web

All Science Journal Classification (ASJC) codes

  • Information Systems

Cite this

Joachims, T., Granka, L., Pan, B., Hembrooke, H., & Gay, G. (2005). Accurately interpreting clickthrough data as implicit feedback. In SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 154-161). (SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval). https://doi.org/10.1145/1076034.1076063
Joachims, Thorsten ; Granka, Laura ; Pan, Bing ; Hembrooke, Helene ; Gay, Geri. / Accurately interpreting clickthrough data as implicit feedback. SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2005. pp. 154-161 (SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval).
@inproceedings{44a322ed179447da92bce2e07f4aa81f,
title = "Accurately interpreting clickthrough data as implicit feedback",
abstract = "This paper examines the reliability of implicit feedback generated from clickthrough data in WWW search. Analyzing the users' decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes the interpretation of clicks as absolute relevance judgments difficult, we show that relative preferences derived from clicks are reasonably accurate on average.",
author = "Thorsten Joachims and Laura Granka and Bing Pan and Helene Hembrooke and Geri Gay",
year = "2005",
month = "12",
day = "1",
doi = "10.1145/1076034.1076063",
language = "English (US)",
isbn = "1595930345",
series = "SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval",
pages = "154--161",
booktitle = "SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval",

}

Joachims, T, Granka, L, Pan, B, Hembrooke, H & Gay, G 2005, Accurately interpreting clickthrough data as implicit feedback. in SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 154-161, 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005, Salvador, Brazil, 8/15/05. https://doi.org/10.1145/1076034.1076063

Accurately interpreting clickthrough data as implicit feedback. / Joachims, Thorsten; Granka, Laura; Pan, Bing; Hembrooke, Helene; Gay, Geri.

SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2005. p. 154-161 (SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Accurately interpreting clickthrough data as implicit feedback

AU - Joachims, Thorsten

AU - Granka, Laura

AU - Pan, Bing

AU - Hembrooke, Helene

AU - Gay, Geri

PY - 2005/12/1

Y1 - 2005/12/1

N2 - This paper examines the reliability of implicit feedback generated from clickthrough data in WWW search. Analyzing the users' decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes the interpretation of clicks as absolute relevance judgments difficult, we show that relative preferences derived from clicks are reasonably accurate on average.

AB - This paper examines the reliability of implicit feedback generated from clickthrough data in WWW search. Analyzing the users' decision process using eyetracking and comparing implicit feedback against manual relevance judgments, we conclude that clicks are informative but biased. While this makes the interpretation of clicks as absolute relevance judgments difficult, we show that relative preferences derived from clicks are reasonably accurate on average.

UR - http://www.scopus.com/inward/record.url?scp=84885665252&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84885665252&partnerID=8YFLogxK

U2 - 10.1145/1076034.1076063

DO - 10.1145/1076034.1076063

M3 - Conference contribution

AN - SCOPUS:84885665252

SN - 1595930345

SN - 9781595930347

T3 - SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

SP - 154

EP - 161

BT - SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

ER -

Joachims T, Granka L, Pan B, Hembrooke H, Gay G. Accurately interpreting clickthrough data as implicit feedback. In SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2005. p. 154-161. (SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval). https://doi.org/10.1145/1076034.1076063