Detecting insider threats in a real corporate database of computer usage activity

Ted E. Senator, Henry G. Goldberg, Alex Memory, William T. Young, Brad Rees, Robert Pierce, Daniel Huang, Matthew Reardon, David A. Bader, Edmond Chow, Irfan Essa, Joshua Jones, Vinay Bettadapura, Duen Horng Chau, Oded Green, Oguz Kaya, Anita Zakrzewska, Erica Briscoe, Rudolph L. Mappus, Robert MccollLora G. Weiss, Thomas G. Dietterich, Alan Fern, Weng Keen Wong, Shubhomoy Das, Andrew Emmott, Jed Irvine, Jay Yoon Lee, Danai Koutra, Christos Faloutsos, Daniel Corkill, Lisa Friedland, Amanda Gentzel, David Jensen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

54 Citations (Scopus)

Abstract

This paper reports on methods and results of an applied research project by a team consisting of SAIC and four universities to develop, integrate, and evaluate new approaches to detect the weak signals characteristic of insider threats on organizations' information systems. Our system combines structural and semantic information from a real corporate database of monitored activity on their users' computers to detect independently developed red team inserts of malicious insider activities. We have developed and applied multiple algorithms for anomaly detection based on suspected scenarios of malicious insider behavior, indicators of unusual activities, high-dimensional statistical patterns, temporal sequences, and normal graph evolution. Algorithms and representations for dynamic graph processing provide the ability to scale as needed for enterprise-level deployments on real-Time data streams. We have also developed a visual language for specifying combinations of features, baselines, peer groups, time periods, and algorithms to detect anomalies suggestive of instances of insider threat behavior. We defined over 100 data features in seven categories based on approximately 5.5 million actions per day from approximately 5,500 users. We have achieved area under the ROC curve values of up to 0.979 and lift values of 65 on the top 50 user-days identified on two months of real data.

Original languageEnglish (US)
Title of host publicationKDD 2013 - 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
EditorsRajesh Parekh, Jingrui He, Dhillon S. Inderjit, Paul Bradley, Yehuda Koren, Rayid Ghani, Ted E. Senator, Robert L. Grossman, Ramasamy Uthurusamy
PublisherAssociation for Computing Machinery
Pages1393-1401
Number of pages9
ISBN (Electronic)9781450321747
DOIs
StatePublished - Aug 11 2013
Event19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2013 - Chicago, United States
Duration: Aug 11 2013Aug 14 2013

Publication series

NameProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
VolumePart F128815

Other

Other19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2013
CountryUnited States
CityChicago
Period8/11/138/14/13

Fingerprint

Visual languages
Information systems
Semantics
Processing
Industry

All Science Journal Classification (ASJC) codes

  • Software
  • Information Systems

Cite this

Senator, T. E., Goldberg, H. G., Memory, A., Young, W. T., Rees, B., Pierce, R., ... Jensen, D. (2013). Detecting insider threats in a real corporate database of computer usage activity. In R. Parekh, J. He, D. S. Inderjit, P. Bradley, Y. Koren, R. Ghani, T. E. Senator, R. L. Grossman, ... R. Uthurusamy (Eds.), KDD 2013 - 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1393-1401). [2488213] (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; Vol. Part F128815). Association for Computing Machinery. https://doi.org/10.1145/2487575.2488213
Senator, Ted E. ; Goldberg, Henry G. ; Memory, Alex ; Young, William T. ; Rees, Brad ; Pierce, Robert ; Huang, Daniel ; Reardon, Matthew ; Bader, David A. ; Chow, Edmond ; Essa, Irfan ; Jones, Joshua ; Bettadapura, Vinay ; Chau, Duen Horng ; Green, Oded ; Kaya, Oguz ; Zakrzewska, Anita ; Briscoe, Erica ; Mappus, Rudolph L. ; Mccoll, Robert ; Weiss, Lora G. ; Dietterich, Thomas G. ; Fern, Alan ; Wong, Weng Keen ; Das, Shubhomoy ; Emmott, Andrew ; Irvine, Jed ; Lee, Jay Yoon ; Koutra, Danai ; Faloutsos, Christos ; Corkill, Daniel ; Friedland, Lisa ; Gentzel, Amanda ; Jensen, David. / Detecting insider threats in a real corporate database of computer usage activity. KDD 2013 - 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. editor / Rajesh Parekh ; Jingrui He ; Dhillon S. Inderjit ; Paul Bradley ; Yehuda Koren ; Rayid Ghani ; Ted E. Senator ; Robert L. Grossman ; Ramasamy Uthurusamy. Association for Computing Machinery, 2013. pp. 1393-1401 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining).
@inproceedings{caf8b538e55c4b009c412b3b25f14b26,
title = "Detecting insider threats in a real corporate database of computer usage activity",
abstract = "This paper reports on methods and results of an applied research project by a team consisting of SAIC and four universities to develop, integrate, and evaluate new approaches to detect the weak signals characteristic of insider threats on organizations' information systems. Our system combines structural and semantic information from a real corporate database of monitored activity on their users' computers to detect independently developed red team inserts of malicious insider activities. We have developed and applied multiple algorithms for anomaly detection based on suspected scenarios of malicious insider behavior, indicators of unusual activities, high-dimensional statistical patterns, temporal sequences, and normal graph evolution. Algorithms and representations for dynamic graph processing provide the ability to scale as needed for enterprise-level deployments on real-Time data streams. We have also developed a visual language for specifying combinations of features, baselines, peer groups, time periods, and algorithms to detect anomalies suggestive of instances of insider threat behavior. We defined over 100 data features in seven categories based on approximately 5.5 million actions per day from approximately 5,500 users. We have achieved area under the ROC curve values of up to 0.979 and lift values of 65 on the top 50 user-days identified on two months of real data.",
author = "Senator, {Ted E.} and Goldberg, {Henry G.} and Alex Memory and Young, {William T.} and Brad Rees and Robert Pierce and Daniel Huang and Matthew Reardon and Bader, {David A.} and Edmond Chow and Irfan Essa and Joshua Jones and Vinay Bettadapura and Chau, {Duen Horng} and Oded Green and Oguz Kaya and Anita Zakrzewska and Erica Briscoe and Mappus, {Rudolph L.} and Robert Mccoll and Weiss, {Lora G.} and Dietterich, {Thomas G.} and Alan Fern and Wong, {Weng Keen} and Shubhomoy Das and Andrew Emmott and Jed Irvine and Lee, {Jay Yoon} and Danai Koutra and Christos Faloutsos and Daniel Corkill and Lisa Friedland and Amanda Gentzel and David Jensen",
year = "2013",
month = "8",
day = "11",
doi = "10.1145/2487575.2488213",
language = "English (US)",
series = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",
publisher = "Association for Computing Machinery",
pages = "1393--1401",
editor = "Rajesh Parekh and Jingrui He and Inderjit, {Dhillon S.} and Paul Bradley and Yehuda Koren and Rayid Ghani and Senator, {Ted E.} and Grossman, {Robert L.} and Ramasamy Uthurusamy",
booktitle = "KDD 2013 - 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

}

Senator, TE, Goldberg, HG, Memory, A, Young, WT, Rees, B, Pierce, R, Huang, D, Reardon, M, Bader, DA, Chow, E, Essa, I, Jones, J, Bettadapura, V, Chau, DH, Green, O, Kaya, O, Zakrzewska, A, Briscoe, E, Mappus, RL, Mccoll, R, Weiss, LG, Dietterich, TG, Fern, A, Wong, WK, Das, S, Emmott, A, Irvine, J, Lee, JY, Koutra, D, Faloutsos, C, Corkill, D, Friedland, L, Gentzel, A & Jensen, D 2013, Detecting insider threats in a real corporate database of computer usage activity. in R Parekh, J He, DS Inderjit, P Bradley, Y Koren, R Ghani, TE Senator, RL Grossman & R Uthurusamy (eds), KDD 2013 - 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining., 2488213, Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, vol. Part F128815, Association for Computing Machinery, pp. 1393-1401, 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2013, Chicago, United States, 8/11/13. https://doi.org/10.1145/2487575.2488213

Detecting insider threats in a real corporate database of computer usage activity. / Senator, Ted E.; Goldberg, Henry G.; Memory, Alex; Young, William T.; Rees, Brad; Pierce, Robert; Huang, Daniel; Reardon, Matthew; Bader, David A.; Chow, Edmond; Essa, Irfan; Jones, Joshua; Bettadapura, Vinay; Chau, Duen Horng; Green, Oded; Kaya, Oguz; Zakrzewska, Anita; Briscoe, Erica; Mappus, Rudolph L.; Mccoll, Robert; Weiss, Lora G.; Dietterich, Thomas G.; Fern, Alan; Wong, Weng Keen; Das, Shubhomoy; Emmott, Andrew; Irvine, Jed; Lee, Jay Yoon; Koutra, Danai; Faloutsos, Christos; Corkill, Daniel; Friedland, Lisa; Gentzel, Amanda; Jensen, David.

KDD 2013 - 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ed. / Rajesh Parekh; Jingrui He; Dhillon S. Inderjit; Paul Bradley; Yehuda Koren; Rayid Ghani; Ted E. Senator; Robert L. Grossman; Ramasamy Uthurusamy. Association for Computing Machinery, 2013. p. 1393-1401 2488213 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; Vol. Part F128815).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Detecting insider threats in a real corporate database of computer usage activity

AU - Senator, Ted E.

AU - Goldberg, Henry G.

AU - Memory, Alex

AU - Young, William T.

AU - Rees, Brad

AU - Pierce, Robert

AU - Huang, Daniel

AU - Reardon, Matthew

AU - Bader, David A.

AU - Chow, Edmond

AU - Essa, Irfan

AU - Jones, Joshua

AU - Bettadapura, Vinay

AU - Chau, Duen Horng

AU - Green, Oded

AU - Kaya, Oguz

AU - Zakrzewska, Anita

AU - Briscoe, Erica

AU - Mappus, Rudolph L.

AU - Mccoll, Robert

AU - Weiss, Lora G.

AU - Dietterich, Thomas G.

AU - Fern, Alan

AU - Wong, Weng Keen

AU - Das, Shubhomoy

AU - Emmott, Andrew

AU - Irvine, Jed

AU - Lee, Jay Yoon

AU - Koutra, Danai

AU - Faloutsos, Christos

AU - Corkill, Daniel

AU - Friedland, Lisa

AU - Gentzel, Amanda

AU - Jensen, David

PY - 2013/8/11

Y1 - 2013/8/11

N2 - This paper reports on methods and results of an applied research project by a team consisting of SAIC and four universities to develop, integrate, and evaluate new approaches to detect the weak signals characteristic of insider threats on organizations' information systems. Our system combines structural and semantic information from a real corporate database of monitored activity on their users' computers to detect independently developed red team inserts of malicious insider activities. We have developed and applied multiple algorithms for anomaly detection based on suspected scenarios of malicious insider behavior, indicators of unusual activities, high-dimensional statistical patterns, temporal sequences, and normal graph evolution. Algorithms and representations for dynamic graph processing provide the ability to scale as needed for enterprise-level deployments on real-Time data streams. We have also developed a visual language for specifying combinations of features, baselines, peer groups, time periods, and algorithms to detect anomalies suggestive of instances of insider threat behavior. We defined over 100 data features in seven categories based on approximately 5.5 million actions per day from approximately 5,500 users. We have achieved area under the ROC curve values of up to 0.979 and lift values of 65 on the top 50 user-days identified on two months of real data.

AB - This paper reports on methods and results of an applied research project by a team consisting of SAIC and four universities to develop, integrate, and evaluate new approaches to detect the weak signals characteristic of insider threats on organizations' information systems. Our system combines structural and semantic information from a real corporate database of monitored activity on their users' computers to detect independently developed red team inserts of malicious insider activities. We have developed and applied multiple algorithms for anomaly detection based on suspected scenarios of malicious insider behavior, indicators of unusual activities, high-dimensional statistical patterns, temporal sequences, and normal graph evolution. Algorithms and representations for dynamic graph processing provide the ability to scale as needed for enterprise-level deployments on real-Time data streams. We have also developed a visual language for specifying combinations of features, baselines, peer groups, time periods, and algorithms to detect anomalies suggestive of instances of insider threat behavior. We defined over 100 data features in seven categories based on approximately 5.5 million actions per day from approximately 5,500 users. We have achieved area under the ROC curve values of up to 0.979 and lift values of 65 on the top 50 user-days identified on two months of real data.

UR - http://www.scopus.com/inward/record.url?scp=84959879166&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84959879166&partnerID=8YFLogxK

U2 - 10.1145/2487575.2488213

DO - 10.1145/2487575.2488213

M3 - Conference contribution

AN - SCOPUS:84959879166

T3 - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

SP - 1393

EP - 1401

BT - KDD 2013 - 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

A2 - Parekh, Rajesh

A2 - He, Jingrui

A2 - Inderjit, Dhillon S.

A2 - Bradley, Paul

A2 - Koren, Yehuda

A2 - Ghani, Rayid

A2 - Senator, Ted E.

A2 - Grossman, Robert L.

A2 - Uthurusamy, Ramasamy

PB - Association for Computing Machinery

ER -

Senator TE, Goldberg HG, Memory A, Young WT, Rees B, Pierce R et al. Detecting insider threats in a real corporate database of computer usage activity. In Parekh R, He J, Inderjit DS, Bradley P, Koren Y, Ghani R, Senator TE, Grossman RL, Uthurusamy R, editors, KDD 2013 - 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery. 2013. p. 1393-1401. 2488213. (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). https://doi.org/10.1145/2487575.2488213