Screenomics: A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them

Byron Reeves, Nilam Ram, Thomas N. Robinson, James J. Cummings, C. Lee Giles, Jennifer Pan, Agnese Chiatti, Mj Cho, Katie Roehrick, Xiao Yang, Anupriya Gagneja, Miriam Brinberg, Daniel Muise, Yingdan Lu, Mufan Luo, Andrew Fitzgerald, Leo Yeykelis

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Digital experiences capture an increasingly large part of life, making them a preferred, if not required, method to describe and theorize about human behavior. Digital media also shape behavior by enabling people to switch between different content easily, and create unique threads of experiences that pass quickly through numerous information categories. Current methods of recording digital experiences provide only partial reconstructions of digital lives that weave–often within seconds–among multiple applications, locations, functions, and media. We describe an end-to-end system for capturing and analyzing the “screenome” of life in media, i.e., the record of individual experiences represented as a sequence of screens that people view and interact with over time. The system includes software that collects screenshots, extracts text and images, and allows searching of a screenshot database. We discuss how the system can be used to elaborate current theories about psychological processing of technology, and suggest new theoretical questions that are enabled by multiple timescale analyses. Capabilities of the system are highlighted with eight research examples that analyze screens from adults who have generated data within the system. We end with a discussion of future uses, limitations, theory, and privacy.

Original languageEnglish (US)
JournalHuman-Computer Interaction
DOIs
StatePublished - Jan 1 2019

Fingerprint

Digital storage
Life Change Events
Switches
Technology
Psychological Theory
Privacy
Processing
Information Systems
Software
Databases
Research

All Science Journal Classification (ASJC) codes

  • Applied Psychology
  • Human-Computer Interaction

Cite this

Reeves, Byron ; Ram, Nilam ; Robinson, Thomas N. ; Cummings, James J. ; Giles, C. Lee ; Pan, Jennifer ; Chiatti, Agnese ; Cho, Mj ; Roehrick, Katie ; Yang, Xiao ; Gagneja, Anupriya ; Brinberg, Miriam ; Muise, Daniel ; Lu, Yingdan ; Luo, Mufan ; Fitzgerald, Andrew ; Yeykelis, Leo. / Screenomics : A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them. In: Human-Computer Interaction. 2019.
@article{abc69ccf1c3048e0926491266c79c995,
title = "Screenomics: A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them",
abstract = "Digital experiences capture an increasingly large part of life, making them a preferred, if not required, method to describe and theorize about human behavior. Digital media also shape behavior by enabling people to switch between different content easily, and create unique threads of experiences that pass quickly through numerous information categories. Current methods of recording digital experiences provide only partial reconstructions of digital lives that weave–often within seconds–among multiple applications, locations, functions, and media. We describe an end-to-end system for capturing and analyzing the “screenome” of life in media, i.e., the record of individual experiences represented as a sequence of screens that people view and interact with over time. The system includes software that collects screenshots, extracts text and images, and allows searching of a screenshot database. We discuss how the system can be used to elaborate current theories about psychological processing of technology, and suggest new theoretical questions that are enabled by multiple timescale analyses. Capabilities of the system are highlighted with eight research examples that analyze screens from adults who have generated data within the system. We end with a discussion of future uses, limitations, theory, and privacy.",
author = "Byron Reeves and Nilam Ram and Robinson, {Thomas N.} and Cummings, {James J.} and Giles, {C. Lee} and Jennifer Pan and Agnese Chiatti and Mj Cho and Katie Roehrick and Xiao Yang and Anupriya Gagneja and Miriam Brinberg and Daniel Muise and Yingdan Lu and Mufan Luo and Andrew Fitzgerald and Leo Yeykelis",
year = "2019",
month = "1",
day = "1",
doi = "10.1080/07370024.2019.1578652",
language = "English (US)",
journal = "Human-Computer Interaction",
issn = "0737-0024",
publisher = "Taylor and Francis Ltd.",

}

Reeves, B, Ram, N, Robinson, TN, Cummings, JJ, Giles, CL, Pan, J, Chiatti, A, Cho, M, Roehrick, K, Yang, X, Gagneja, A, Brinberg, M, Muise, D, Lu, Y, Luo, M, Fitzgerald, A & Yeykelis, L 2019, 'Screenomics: A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them', Human-Computer Interaction. https://doi.org/10.1080/07370024.2019.1578652

Screenomics : A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them. / Reeves, Byron; Ram, Nilam; Robinson, Thomas N.; Cummings, James J.; Giles, C. Lee; Pan, Jennifer; Chiatti, Agnese; Cho, Mj; Roehrick, Katie; Yang, Xiao; Gagneja, Anupriya; Brinberg, Miriam; Muise, Daniel; Lu, Yingdan; Luo, Mufan; Fitzgerald, Andrew; Yeykelis, Leo.

In: Human-Computer Interaction, 01.01.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Screenomics

T2 - A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them

AU - Reeves, Byron

AU - Ram, Nilam

AU - Robinson, Thomas N.

AU - Cummings, James J.

AU - Giles, C. Lee

AU - Pan, Jennifer

AU - Chiatti, Agnese

AU - Cho, Mj

AU - Roehrick, Katie

AU - Yang, Xiao

AU - Gagneja, Anupriya

AU - Brinberg, Miriam

AU - Muise, Daniel

AU - Lu, Yingdan

AU - Luo, Mufan

AU - Fitzgerald, Andrew

AU - Yeykelis, Leo

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Digital experiences capture an increasingly large part of life, making them a preferred, if not required, method to describe and theorize about human behavior. Digital media also shape behavior by enabling people to switch between different content easily, and create unique threads of experiences that pass quickly through numerous information categories. Current methods of recording digital experiences provide only partial reconstructions of digital lives that weave–often within seconds–among multiple applications, locations, functions, and media. We describe an end-to-end system for capturing and analyzing the “screenome” of life in media, i.e., the record of individual experiences represented as a sequence of screens that people view and interact with over time. The system includes software that collects screenshots, extracts text and images, and allows searching of a screenshot database. We discuss how the system can be used to elaborate current theories about psychological processing of technology, and suggest new theoretical questions that are enabled by multiple timescale analyses. Capabilities of the system are highlighted with eight research examples that analyze screens from adults who have generated data within the system. We end with a discussion of future uses, limitations, theory, and privacy.

AB - Digital experiences capture an increasingly large part of life, making them a preferred, if not required, method to describe and theorize about human behavior. Digital media also shape behavior by enabling people to switch between different content easily, and create unique threads of experiences that pass quickly through numerous information categories. Current methods of recording digital experiences provide only partial reconstructions of digital lives that weave–often within seconds–among multiple applications, locations, functions, and media. We describe an end-to-end system for capturing and analyzing the “screenome” of life in media, i.e., the record of individual experiences represented as a sequence of screens that people view and interact with over time. The system includes software that collects screenshots, extracts text and images, and allows searching of a screenshot database. We discuss how the system can be used to elaborate current theories about psychological processing of technology, and suggest new theoretical questions that are enabled by multiple timescale analyses. Capabilities of the system are highlighted with eight research examples that analyze screens from adults who have generated data within the system. We end with a discussion of future uses, limitations, theory, and privacy.

UR - http://www.scopus.com/inward/record.url?scp=85062969904&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062969904&partnerID=8YFLogxK

U2 - 10.1080/07370024.2019.1578652

DO - 10.1080/07370024.2019.1578652

M3 - Article

AN - SCOPUS:85062969904

JO - Human-Computer Interaction

JF - Human-Computer Interaction

SN - 0737-0024

ER -