Embedding Training Within Warnings Improves Skills of Identifying Phishing Webpages

Aiping Xiong, Robert W. Proctor, Weining Yang, Ninghui Li

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Objective: Evaluate the effectiveness of training embedded within security warnings to identify phishing webpages. Background: More than 20 million malware and phishing warnings are shown to users of Google Safe Browsing every week. Substantial click-through rate is still evident, and a common issue reported is that users lack understanding of the warnings. Nevertheless, each warning provides an opportunity to train users about phishing and how to avoid phishing attacks. Method: To test use of phishing-warning instances as opportunities to train users’ phishing webpage detection skills, we conducted an online experiment contrasting the effectiveness of the current Chrome phishing warning with two training-embedded warning interfaces. The experiment consisted of three phases. In Phase 1, participants made login decisions on 10 webpages with the aid of warning. After a distracting task, participants made legitimacy judgments for 10 different login webpages without warnings in Phase 2. To test the long-term effect of the training, participants were invited back a week later to participate in Phase 3, which was conducted similarly as Phase 2. Results: Participants differentiated legitimate and fraudulent webpages better than chance. Performance was similar for all interfaces in Phase 1 for which the warning aid was present. However, training-embedded interfaces provided better protection than the Chrome phishing warning on both subsequent phases. Conclusion: Embedded training is a complementary strategy to compensate for lack of phishing webpage detection skill when phishing warning is absent. Application: Potential applications include development of training-embedded warnings to enable security training at scale.

Original languageEnglish (US)
Pages (from-to)577-595
Number of pages19
JournalHuman Factors
Volume61
Issue number4
DOIs
StatePublished - Jun 1 2019

Fingerprint

Illegitimacy
Experiments
lack
experiment
search engine
legitimacy
Malware

All Science Journal Classification (ASJC) codes

  • Human Factors and Ergonomics
  • Applied Psychology
  • Behavioral Neuroscience

Cite this

Xiong, Aiping ; Proctor, Robert W. ; Yang, Weining ; Li, Ninghui. / Embedding Training Within Warnings Improves Skills of Identifying Phishing Webpages. In: Human Factors. 2019 ; Vol. 61, No. 4. pp. 577-595.
@article{3dbfed53446e4dd6a15a96548d60e5f9,
title = "Embedding Training Within Warnings Improves Skills of Identifying Phishing Webpages",
abstract = "Objective: Evaluate the effectiveness of training embedded within security warnings to identify phishing webpages. Background: More than 20 million malware and phishing warnings are shown to users of Google Safe Browsing every week. Substantial click-through rate is still evident, and a common issue reported is that users lack understanding of the warnings. Nevertheless, each warning provides an opportunity to train users about phishing and how to avoid phishing attacks. Method: To test use of phishing-warning instances as opportunities to train users’ phishing webpage detection skills, we conducted an online experiment contrasting the effectiveness of the current Chrome phishing warning with two training-embedded warning interfaces. The experiment consisted of three phases. In Phase 1, participants made login decisions on 10 webpages with the aid of warning. After a distracting task, participants made legitimacy judgments for 10 different login webpages without warnings in Phase 2. To test the long-term effect of the training, participants were invited back a week later to participate in Phase 3, which was conducted similarly as Phase 2. Results: Participants differentiated legitimate and fraudulent webpages better than chance. Performance was similar for all interfaces in Phase 1 for which the warning aid was present. However, training-embedded interfaces provided better protection than the Chrome phishing warning on both subsequent phases. Conclusion: Embedded training is a complementary strategy to compensate for lack of phishing webpage detection skill when phishing warning is absent. Application: Potential applications include development of training-embedded warnings to enable security training at scale.",
author = "Aiping Xiong and Proctor, {Robert W.} and Weining Yang and Ninghui Li",
year = "2019",
month = "6",
day = "1",
doi = "10.1177/0018720818810942",
language = "English (US)",
volume = "61",
pages = "577--595",
journal = "Human Factors",
issn = "0018-7208",
publisher = "SAGE Publications Inc.",
number = "4",

}

Embedding Training Within Warnings Improves Skills of Identifying Phishing Webpages. / Xiong, Aiping; Proctor, Robert W.; Yang, Weining; Li, Ninghui.

In: Human Factors, Vol. 61, No. 4, 01.06.2019, p. 577-595.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Embedding Training Within Warnings Improves Skills of Identifying Phishing Webpages

AU - Xiong, Aiping

AU - Proctor, Robert W.

AU - Yang, Weining

AU - Li, Ninghui

PY - 2019/6/1

Y1 - 2019/6/1

N2 - Objective: Evaluate the effectiveness of training embedded within security warnings to identify phishing webpages. Background: More than 20 million malware and phishing warnings are shown to users of Google Safe Browsing every week. Substantial click-through rate is still evident, and a common issue reported is that users lack understanding of the warnings. Nevertheless, each warning provides an opportunity to train users about phishing and how to avoid phishing attacks. Method: To test use of phishing-warning instances as opportunities to train users’ phishing webpage detection skills, we conducted an online experiment contrasting the effectiveness of the current Chrome phishing warning with two training-embedded warning interfaces. The experiment consisted of three phases. In Phase 1, participants made login decisions on 10 webpages with the aid of warning. After a distracting task, participants made legitimacy judgments for 10 different login webpages without warnings in Phase 2. To test the long-term effect of the training, participants were invited back a week later to participate in Phase 3, which was conducted similarly as Phase 2. Results: Participants differentiated legitimate and fraudulent webpages better than chance. Performance was similar for all interfaces in Phase 1 for which the warning aid was present. However, training-embedded interfaces provided better protection than the Chrome phishing warning on both subsequent phases. Conclusion: Embedded training is a complementary strategy to compensate for lack of phishing webpage detection skill when phishing warning is absent. Application: Potential applications include development of training-embedded warnings to enable security training at scale.

AB - Objective: Evaluate the effectiveness of training embedded within security warnings to identify phishing webpages. Background: More than 20 million malware and phishing warnings are shown to users of Google Safe Browsing every week. Substantial click-through rate is still evident, and a common issue reported is that users lack understanding of the warnings. Nevertheless, each warning provides an opportunity to train users about phishing and how to avoid phishing attacks. Method: To test use of phishing-warning instances as opportunities to train users’ phishing webpage detection skills, we conducted an online experiment contrasting the effectiveness of the current Chrome phishing warning with two training-embedded warning interfaces. The experiment consisted of three phases. In Phase 1, participants made login decisions on 10 webpages with the aid of warning. After a distracting task, participants made legitimacy judgments for 10 different login webpages without warnings in Phase 2. To test the long-term effect of the training, participants were invited back a week later to participate in Phase 3, which was conducted similarly as Phase 2. Results: Participants differentiated legitimate and fraudulent webpages better than chance. Performance was similar for all interfaces in Phase 1 for which the warning aid was present. However, training-embedded interfaces provided better protection than the Chrome phishing warning on both subsequent phases. Conclusion: Embedded training is a complementary strategy to compensate for lack of phishing webpage detection skill when phishing warning is absent. Application: Potential applications include development of training-embedded warnings to enable security training at scale.

UR - http://www.scopus.com/inward/record.url?scp=85059962070&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85059962070&partnerID=8YFLogxK

U2 - 10.1177/0018720818810942

DO - 10.1177/0018720818810942

M3 - Article

C2 - 30526089

AN - SCOPUS:85059962070

VL - 61

SP - 577

EP - 595

JO - Human Factors

JF - Human Factors

SN - 0018-7208

IS - 4

ER -