Hierarchical clustering for discrimination discovery: A top-down approach

Neda Nasiriani, Anna Squicciarini, Zara Saldanha, Sanchit Goel, Nicola Zannone

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Today, data is an essential part of many decision-making processes in businesses and social life through the use of various machine learning techniques. These methods can easily perpetuate human bias in the data and result in discrimination. Despite a growing interest in data discrimination discovery and removal, to date there is a lack of a general and robust framework to distinguish discriminatory decision-making processes from non-discriminatory ones. In this work, we present a generic framework that helps detect possible discrimination by analyzing historical data and associated decisions using a top-down unsupervised approach, which we refer to as hierarchical clustering. Our approach is highly adaptive as it gradually 'learns' users' inherent groups, and clusters their records using cohesiveness and density of points in the dataset. Moreover, we propose a progressive attribute-selection method to choose statistically relevant attributes, thus reducing the effect of noise. Finally, we adopt a recursive notion of cluster profile that is homogeneous w.r.t. decision labels. This allows for deeper insights on the data and on the decision-making underlying the final user classification. Our framework is able to identify both positive and negative bias resulting in discrimination. We also highlight patterns of discrimination revealed by the homogeneous cluster centroids, which otherwise could not be captured.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE 2nd International Conference on Artificial Intelligence and Knowledge Engineering, AIKE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages187-194
Number of pages8
ISBN (Electronic)9781728114880
DOIs
StatePublished - Jun 1 2019
Event2nd IEEE International Conference on Artificial Intelligence and Knowledge Engineering, AIKE 2019 - Cagliari, Sardinia, Italy
Duration: Jun 3 2019Jun 5 2019

Publication series

NameProceedings - IEEE 2nd International Conference on Artificial Intelligence and Knowledge Engineering, AIKE 2019

Conference

Conference2nd IEEE International Conference on Artificial Intelligence and Knowledge Engineering, AIKE 2019
CountryItaly
CityCagliari, Sardinia
Period6/3/196/5/19

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Science Applications
  • Information Systems and Management

Fingerprint Dive into the research topics of 'Hierarchical clustering for discrimination discovery: A top-down approach'. Together they form a unique fingerprint.

  • Cite this

    Nasiriani, N., Squicciarini, A., Saldanha, Z., Goel, S., & Zannone, N. (2019). Hierarchical clustering for discrimination discovery: A top-down approach. In Proceedings - IEEE 2nd International Conference on Artificial Intelligence and Knowledge Engineering, AIKE 2019 (pp. 187-194). [8791737] (Proceedings - IEEE 2nd International Conference on Artificial Intelligence and Knowledge Engineering, AIKE 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/AIKE.2019.00041