Coverage, relevance, and ranking: The impact of query operators on Web search engine results

Caroline M. Eastman, Bernard J. Jansen

Research output: Contribution to journalReview article

78 Citations (Scopus)

Abstract

Research has reported that about 10% of Web searchers utilize advanced query operators, with the other 90% using extremely simple queries. It is often assumed that the use of query operators, such as Boolean operators and phrase searching, improves the effectiveness of Web searching. We test this assumption by examining the effects of query operators on the performance of three major Web search engines. We selected one hundred queries from the transaction log of a Web search service. Each of these original queries contained query operators such as AND, OR, MUST APPEAR (+), or PHRASE (" "). We then removed the operators from these one hundred advanced queries. We submitted both the original and modified queries to three major Web search engines; a total of 600 queries were submitted and 5,748 documents evaluated. We compared the results from the original queries with the operators to the results from the modified queries without the operators. We examined the results for changes in coverage, relative precision, and ranking of relevant documents. The use of most query operators had no significant effect on coverage, relative precision, or ranking, although the effect varied depending on the search engine. We discuss implications for the effectiveness of searching techniques as currently taught, for future information retrieval system design, and for future research.

Original languageEnglish (US)
Pages (from-to)383-411
Number of pages29
JournalACM Transactions on Information Systems
Volume21
Issue number4
DOIs
StatePublished - Oct 1 2003

Fingerprint

Search engines
Information retrieval systems
Systems analysis
Web search
Query
Operator
Ranking
Search engine

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Business, Management and Accounting(all)
  • Computer Science Applications

Cite this

@article{cdb19fe6cfa3402cbe52822528efc586,
title = "Coverage, relevance, and ranking: The impact of query operators on Web search engine results",
abstract = "Research has reported that about 10{\%} of Web searchers utilize advanced query operators, with the other 90{\%} using extremely simple queries. It is often assumed that the use of query operators, such as Boolean operators and phrase searching, improves the effectiveness of Web searching. We test this assumption by examining the effects of query operators on the performance of three major Web search engines. We selected one hundred queries from the transaction log of a Web search service. Each of these original queries contained query operators such as AND, OR, MUST APPEAR (+), or PHRASE ({"} {"}). We then removed the operators from these one hundred advanced queries. We submitted both the original and modified queries to three major Web search engines; a total of 600 queries were submitted and 5,748 documents evaluated. We compared the results from the original queries with the operators to the results from the modified queries without the operators. We examined the results for changes in coverage, relative precision, and ranking of relevant documents. The use of most query operators had no significant effect on coverage, relative precision, or ranking, although the effect varied depending on the search engine. We discuss implications for the effectiveness of searching techniques as currently taught, for future information retrieval system design, and for future research.",
author = "Eastman, {Caroline M.} and Jansen, {Bernard J.}",
year = "2003",
month = "10",
day = "1",
doi = "10.1145/944012.944015",
language = "English (US)",
volume = "21",
pages = "383--411",
journal = "ACM Transactions on Information Systems",
issn = "1046-8188",
publisher = "Association for Computing Machinery (ACM)",
number = "4",

}

Coverage, relevance, and ranking : The impact of query operators on Web search engine results. / Eastman, Caroline M.; Jansen, Bernard J.

In: ACM Transactions on Information Systems, Vol. 21, No. 4, 01.10.2003, p. 383-411.

Research output: Contribution to journalReview article

TY - JOUR

T1 - Coverage, relevance, and ranking

T2 - The impact of query operators on Web search engine results

AU - Eastman, Caroline M.

AU - Jansen, Bernard J.

PY - 2003/10/1

Y1 - 2003/10/1

N2 - Research has reported that about 10% of Web searchers utilize advanced query operators, with the other 90% using extremely simple queries. It is often assumed that the use of query operators, such as Boolean operators and phrase searching, improves the effectiveness of Web searching. We test this assumption by examining the effects of query operators on the performance of three major Web search engines. We selected one hundred queries from the transaction log of a Web search service. Each of these original queries contained query operators such as AND, OR, MUST APPEAR (+), or PHRASE (" "). We then removed the operators from these one hundred advanced queries. We submitted both the original and modified queries to three major Web search engines; a total of 600 queries were submitted and 5,748 documents evaluated. We compared the results from the original queries with the operators to the results from the modified queries without the operators. We examined the results for changes in coverage, relative precision, and ranking of relevant documents. The use of most query operators had no significant effect on coverage, relative precision, or ranking, although the effect varied depending on the search engine. We discuss implications for the effectiveness of searching techniques as currently taught, for future information retrieval system design, and for future research.

AB - Research has reported that about 10% of Web searchers utilize advanced query operators, with the other 90% using extremely simple queries. It is often assumed that the use of query operators, such as Boolean operators and phrase searching, improves the effectiveness of Web searching. We test this assumption by examining the effects of query operators on the performance of three major Web search engines. We selected one hundred queries from the transaction log of a Web search service. Each of these original queries contained query operators such as AND, OR, MUST APPEAR (+), or PHRASE (" "). We then removed the operators from these one hundred advanced queries. We submitted both the original and modified queries to three major Web search engines; a total of 600 queries were submitted and 5,748 documents evaluated. We compared the results from the original queries with the operators to the results from the modified queries without the operators. We examined the results for changes in coverage, relative precision, and ranking of relevant documents. The use of most query operators had no significant effect on coverage, relative precision, or ranking, although the effect varied depending on the search engine. We discuss implications for the effectiveness of searching techniques as currently taught, for future information retrieval system design, and for future research.

UR - http://www.scopus.com/inward/record.url?scp=2442457056&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=2442457056&partnerID=8YFLogxK

U2 - 10.1145/944012.944015

DO - 10.1145/944012.944015

M3 - Review article

AN - SCOPUS:2442457056

VL - 21

SP - 383

EP - 411

JO - ACM Transactions on Information Systems

JF - ACM Transactions on Information Systems

SN - 1046-8188

IS - 4

ER -