Advancing the theory and practice of system evaluation: a case study in geovisual analytics of social media

Alexander Savelyev, Alan M. MacEachren

Research output: Contribution to journalArticlepeer-review

Abstract

This paper advances the state-of-the-art in methodology design for empirical evaluation of (geo)visual analytics software. Specifically, we describe the process of design, development and application of a prototypical user study tailored to the evaluation of complex geovisual analytics tools that focus on social media analysis. We fist perform a synthesis of existing theory and best practices for software evaluation of comparable systems. We then demonstrate how the product of said synthesis–a methodological ‘check list’–can be used to inform a proof-of-concept user study of an actual geovisual analytics software system. The resulting user study design accommodates for the use of real geographic social media datasets, the complexity of the intended analytical process, and for the learning challenges faced by the participants working with a fully-functional and mature geovisual analytics application, and is likely representative of a wide range of evaluation scenarios in (geo)visual analytics. A complete summary of all the study instruments is included to encourage their scrutiny, reuse and modification by others. Finally, we have discovered that participants’ curiosity and desire for autonomy played a noticeable role in the evaluation process–something not previously reported.

Original languageEnglish (US)
Pages (from-to)202-221
Number of pages20
JournalInternational Journal of Cartography
Volume6
Issue number2
DOIs
StatePublished - May 3 2020

All Science Journal Classification (ASJC) codes

  • Earth and Planetary Sciences (miscellaneous)
  • Computers in Earth Sciences
  • Geography, Planning and Development

Fingerprint Dive into the research topics of 'Advancing the theory and practice of system evaluation: a case study in geovisual analytics of social media'. Together they form a unique fingerprint.

Cite this