Abstract
Multifactor dimensionality reduction (MDR) was developed as a method for detecting statistical patterns of epistasis. The overall goal of MDR is to change the representation space of the data to make interactions easier to detect. It is well known that machine learning methods may not provide robust models when the class variable (e.g. case-control status) is imbalanced and accuracy is used as the fitness measure. This is because most methods learn patterns that are relevant for the larger of the two classes. The goal of this study was to evaluate three different strategies for improving the power of MDR to detect epistasis in imbalanced datasets. The methods evaluated were: (1) over-sampling that resamples with replacement the smaller class until the data are balanced, (2) under-sampling that randomly removes subjects from the larger class until the data are balanced, and (3) balanced accuracy [(sensitivity+specificity)/2] as the fitness function with and without an adjusted threshold. These three methods were compared using simulated data with two-locus epistatic interactions of varying heritability (0.01, 0.025, 0.05, 0.1, 0.2, 0.3, 0.4) and minor allele frequency (0.2, 0.4) that were embedded in 100 replicate datasets of varying sample sizes (400, 800, 1600). Each dataset was generated with different ratios of cases to controls (1:1, 1:2, 1:4). We found that the balanced accuracy function with an adjusted threshold significantly outperformed both over-sampling and under-sampling and fully recovered the power. These results suggest that balanced accuracy should be used instead of accuracy for the MDR analysis of epistasis in imbalanced datasets.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 306-315 |
| Number of pages | 10 |
| Journal | Genetic Epidemiology |
| Volume | 31 |
| Issue number | 4 |
| DOIs | |
| State | Published - May 1 2007 |
Fingerprint
All Science Journal Classification (ASJC) codes
- Epidemiology
- Genetics(clinical)
Cite this
}
A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reduction. / Velez, Digna R.; White, Bill C.; Motsinger, Alison A.; Bush, William S.; Ritchie, Marylyn Deriggi; Williams, Scott M.; Moore, Jason H.
In: Genetic Epidemiology, Vol. 31, No. 4, 01.05.2007, p. 306-315.Research output: Contribution to journal › Article
TY - JOUR
T1 - A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reduction
AU - Velez, Digna R.
AU - White, Bill C.
AU - Motsinger, Alison A.
AU - Bush, William S.
AU - Ritchie, Marylyn Deriggi
AU - Williams, Scott M.
AU - Moore, Jason H.
PY - 2007/5/1
Y1 - 2007/5/1
N2 - Multifactor dimensionality reduction (MDR) was developed as a method for detecting statistical patterns of epistasis. The overall goal of MDR is to change the representation space of the data to make interactions easier to detect. It is well known that machine learning methods may not provide robust models when the class variable (e.g. case-control status) is imbalanced and accuracy is used as the fitness measure. This is because most methods learn patterns that are relevant for the larger of the two classes. The goal of this study was to evaluate three different strategies for improving the power of MDR to detect epistasis in imbalanced datasets. The methods evaluated were: (1) over-sampling that resamples with replacement the smaller class until the data are balanced, (2) under-sampling that randomly removes subjects from the larger class until the data are balanced, and (3) balanced accuracy [(sensitivity+specificity)/2] as the fitness function with and without an adjusted threshold. These three methods were compared using simulated data with two-locus epistatic interactions of varying heritability (0.01, 0.025, 0.05, 0.1, 0.2, 0.3, 0.4) and minor allele frequency (0.2, 0.4) that were embedded in 100 replicate datasets of varying sample sizes (400, 800, 1600). Each dataset was generated with different ratios of cases to controls (1:1, 1:2, 1:4). We found that the balanced accuracy function with an adjusted threshold significantly outperformed both over-sampling and under-sampling and fully recovered the power. These results suggest that balanced accuracy should be used instead of accuracy for the MDR analysis of epistasis in imbalanced datasets.
AB - Multifactor dimensionality reduction (MDR) was developed as a method for detecting statistical patterns of epistasis. The overall goal of MDR is to change the representation space of the data to make interactions easier to detect. It is well known that machine learning methods may not provide robust models when the class variable (e.g. case-control status) is imbalanced and accuracy is used as the fitness measure. This is because most methods learn patterns that are relevant for the larger of the two classes. The goal of this study was to evaluate three different strategies for improving the power of MDR to detect epistasis in imbalanced datasets. The methods evaluated were: (1) over-sampling that resamples with replacement the smaller class until the data are balanced, (2) under-sampling that randomly removes subjects from the larger class until the data are balanced, and (3) balanced accuracy [(sensitivity+specificity)/2] as the fitness function with and without an adjusted threshold. These three methods were compared using simulated data with two-locus epistatic interactions of varying heritability (0.01, 0.025, 0.05, 0.1, 0.2, 0.3, 0.4) and minor allele frequency (0.2, 0.4) that were embedded in 100 replicate datasets of varying sample sizes (400, 800, 1600). Each dataset was generated with different ratios of cases to controls (1:1, 1:2, 1:4). We found that the balanced accuracy function with an adjusted threshold significantly outperformed both over-sampling and under-sampling and fully recovered the power. These results suggest that balanced accuracy should be used instead of accuracy for the MDR analysis of epistasis in imbalanced datasets.
UR - http://www.scopus.com/inward/record.url?scp=34247338434&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=34247338434&partnerID=8YFLogxK
U2 - 10.1002/gepi.20211
DO - 10.1002/gepi.20211
M3 - Article
C2 - 17323372
AN - SCOPUS:34247338434
VL - 31
SP - 306
EP - 315
JO - Genetic Epidemiology
JF - Genetic Epidemiology
SN - 0741-0395
IS - 4
ER -