Evaluating visual explanations for similarity-based recommendations: User perception and performance

Chun Hua Tsai, Peter Brusilovsky

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recommender system helps users to reduce information overload. In recent years, enhancing explainability in recommender systems has drawn more and more attention in the field of Human-Computer Interaction (HCI). However, it is not clear whether a user-preferred explanation interface can maintain the same level of performance while the users are exploring or comparing the recommendations. In this paper, we introduced a participatory process of designing explanation interfaces with multiple explanatory goals for three similarity-based recommendation models. We investigate the relations of user perception and performance with two user studies. In the first study (N=15), we conducted card-sorting and semi-interview to identify the user preferred interfaces. In the second study (N=18), we carry out a performance-focused evaluation of six explanation interfaces. The result suggests that the user-preferred interface may not guarantee the same level of performance.

Original languageEnglish (US)
Title of host publicationACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization
PublisherAssociation for Computing Machinery, Inc
Pages22-30
Number of pages9
ISBN (Electronic)9781450360210
DOIs
StatePublished - Jun 7 2019
Event27th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2019 - Larnaca, Cyprus
Duration: Jun 9 2019Jun 12 2019

Publication series

NameACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization

Conference

Conference27th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2019
CountryCyprus
CityLarnaca
Period6/9/196/12/19

Fingerprint

Recommender systems
Human computer interaction
Sorting

All Science Journal Classification (ASJC) codes

  • Software

Cite this

Tsai, C. H., & Brusilovsky, P. (2019). Evaluating visual explanations for similarity-based recommendations: User perception and performance. In ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization (pp. 22-30). (ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization). Association for Computing Machinery, Inc. https://doi.org/10.1145/3320435.3320465
Tsai, Chun Hua ; Brusilovsky, Peter. / Evaluating visual explanations for similarity-based recommendations : User perception and performance. ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. Association for Computing Machinery, Inc, 2019. pp. 22-30 (ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization).
@inproceedings{57d7530e074c46a4b18978904349142b,
title = "Evaluating visual explanations for similarity-based recommendations: User perception and performance",
abstract = "Recommender system helps users to reduce information overload. In recent years, enhancing explainability in recommender systems has drawn more and more attention in the field of Human-Computer Interaction (HCI). However, it is not clear whether a user-preferred explanation interface can maintain the same level of performance while the users are exploring or comparing the recommendations. In this paper, we introduced a participatory process of designing explanation interfaces with multiple explanatory goals for three similarity-based recommendation models. We investigate the relations of user perception and performance with two user studies. In the first study (N=15), we conducted card-sorting and semi-interview to identify the user preferred interfaces. In the second study (N=18), we carry out a performance-focused evaluation of six explanation interfaces. The result suggests that the user-preferred interface may not guarantee the same level of performance.",
author = "Tsai, {Chun Hua} and Peter Brusilovsky",
year = "2019",
month = "6",
day = "7",
doi = "10.1145/3320435.3320465",
language = "English (US)",
series = "ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization",
publisher = "Association for Computing Machinery, Inc",
pages = "22--30",
booktitle = "ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization",

}

Tsai, CH & Brusilovsky, P 2019, Evaluating visual explanations for similarity-based recommendations: User perception and performance. in ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization, Association for Computing Machinery, Inc, pp. 22-30, 27th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2019, Larnaca, Cyprus, 6/9/19. https://doi.org/10.1145/3320435.3320465

Evaluating visual explanations for similarity-based recommendations : User perception and performance. / Tsai, Chun Hua; Brusilovsky, Peter.

ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. Association for Computing Machinery, Inc, 2019. p. 22-30 (ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Evaluating visual explanations for similarity-based recommendations

T2 - User perception and performance

AU - Tsai, Chun Hua

AU - Brusilovsky, Peter

PY - 2019/6/7

Y1 - 2019/6/7

N2 - Recommender system helps users to reduce information overload. In recent years, enhancing explainability in recommender systems has drawn more and more attention in the field of Human-Computer Interaction (HCI). However, it is not clear whether a user-preferred explanation interface can maintain the same level of performance while the users are exploring or comparing the recommendations. In this paper, we introduced a participatory process of designing explanation interfaces with multiple explanatory goals for three similarity-based recommendation models. We investigate the relations of user perception and performance with two user studies. In the first study (N=15), we conducted card-sorting and semi-interview to identify the user preferred interfaces. In the second study (N=18), we carry out a performance-focused evaluation of six explanation interfaces. The result suggests that the user-preferred interface may not guarantee the same level of performance.

AB - Recommender system helps users to reduce information overload. In recent years, enhancing explainability in recommender systems has drawn more and more attention in the field of Human-Computer Interaction (HCI). However, it is not clear whether a user-preferred explanation interface can maintain the same level of performance while the users are exploring or comparing the recommendations. In this paper, we introduced a participatory process of designing explanation interfaces with multiple explanatory goals for three similarity-based recommendation models. We investigate the relations of user perception and performance with two user studies. In the first study (N=15), we conducted card-sorting and semi-interview to identify the user preferred interfaces. In the second study (N=18), we carry out a performance-focused evaluation of six explanation interfaces. The result suggests that the user-preferred interface may not guarantee the same level of performance.

UR - http://www.scopus.com/inward/record.url?scp=85068091026&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068091026&partnerID=8YFLogxK

U2 - 10.1145/3320435.3320465

DO - 10.1145/3320435.3320465

M3 - Conference contribution

AN - SCOPUS:85068091026

T3 - ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization

SP - 22

EP - 30

BT - ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization

PB - Association for Computing Machinery, Inc

ER -

Tsai CH, Brusilovsky P. Evaluating visual explanations for similarity-based recommendations: User perception and performance. In ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. Association for Computing Machinery, Inc. 2019. p. 22-30. (ACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization). https://doi.org/10.1145/3320435.3320465