Developing subdomain allocation algorithms based on spatial and communicational constraints to accelerate dust storm simulation

Zhipeng Gui, Manzhu Yu, Chaowei Yang, Yunfeng Jiang, Songqing Chen, Jizhe Xia, Qunying Huang, Kai Liu, Zhenlong Li, Mohammed Anowarul Hassan, Baoxuan Jin

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Dust storm has serious disastrous impacts on environment, human health, and assets. The developments and applications of dust storm models have contributed significantly to better understand and predict the distribution, intensity and structure of dust storms. However, dust storm simulation is a data and computing intensive process. To improve the computing performance, high performance computing has been widely adopted by dividing the entire study area into multiple subdomains and allocating each subdomain on different computing nodes in a parallel fashion. Inappropriate allocation may introduce imbalanced task loads and unnecessary communications among computing nodes. Therefore, allocation is a key factor that may impact the efficiency of parallel process. An allocation algorithm is expected to consider the computing cost and communication cost for each computing node to minimize total execution time and reduce overall communication cost for the entire simulation. This research introduces three algorithms to optimize the allocation by considering the spatial and communicational constraints: 1) an Integer Linear Programming (ILP) based algorithm from combinational optimization perspective; 2) a K-Means and Kernighan-Lin combined heuristic algorithm (K&K) integrating geometric and coordinate-free methods by merging local and global partitioning; 3) an automatic seeded region growing based geometric and local partitioning algorithm (ASRG). The performance and effectiveness of the three algorithms are compared based on different factors. Further, we adopt the K&K algorithm as the demonstrated algorithm for the experiment of dust model simulation with the non-hydrostatic mesoscale model (NMMdust) and compared the performance with the MPI default sequential allocation. The results demonstrate that K&K method significantly improves the simulation performance with better subdomain allocation. This method can also be adopted for other relevant atmospheric and numerical modeling.

Original languageEnglish (US)
Article numbere0152250
JournalPloS one
Volume11
Issue number4
DOIs
StatePublished - Apr 2016

Fingerprint

dust storms
Dust
Communication
Costs and Cost Analysis
Computing Methodologies
Costs
Linear Programming
Heuristic algorithms
linear programming
Merging
assets
Linear programming
dust
human health
simulation models
Health
methodology
Efficiency

All Science Journal Classification (ASJC) codes

  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)
  • General

Cite this

Gui, Zhipeng ; Yu, Manzhu ; Yang, Chaowei ; Jiang, Yunfeng ; Chen, Songqing ; Xia, Jizhe ; Huang, Qunying ; Liu, Kai ; Li, Zhenlong ; Hassan, Mohammed Anowarul ; Jin, Baoxuan. / Developing subdomain allocation algorithms based on spatial and communicational constraints to accelerate dust storm simulation. In: PloS one. 2016 ; Vol. 11, No. 4.
@article{be39eb44da954848a8e1506aece5d735,
title = "Developing subdomain allocation algorithms based on spatial and communicational constraints to accelerate dust storm simulation",
abstract = "Dust storm has serious disastrous impacts on environment, human health, and assets. The developments and applications of dust storm models have contributed significantly to better understand and predict the distribution, intensity and structure of dust storms. However, dust storm simulation is a data and computing intensive process. To improve the computing performance, high performance computing has been widely adopted by dividing the entire study area into multiple subdomains and allocating each subdomain on different computing nodes in a parallel fashion. Inappropriate allocation may introduce imbalanced task loads and unnecessary communications among computing nodes. Therefore, allocation is a key factor that may impact the efficiency of parallel process. An allocation algorithm is expected to consider the computing cost and communication cost for each computing node to minimize total execution time and reduce overall communication cost for the entire simulation. This research introduces three algorithms to optimize the allocation by considering the spatial and communicational constraints: 1) an Integer Linear Programming (ILP) based algorithm from combinational optimization perspective; 2) a K-Means and Kernighan-Lin combined heuristic algorithm (K&K) integrating geometric and coordinate-free methods by merging local and global partitioning; 3) an automatic seeded region growing based geometric and local partitioning algorithm (ASRG). The performance and effectiveness of the three algorithms are compared based on different factors. Further, we adopt the K&K algorithm as the demonstrated algorithm for the experiment of dust model simulation with the non-hydrostatic mesoscale model (NMMdust) and compared the performance with the MPI default sequential allocation. The results demonstrate that K&K method significantly improves the simulation performance with better subdomain allocation. This method can also be adopted for other relevant atmospheric and numerical modeling.",
author = "Zhipeng Gui and Manzhu Yu and Chaowei Yang and Yunfeng Jiang and Songqing Chen and Jizhe Xia and Qunying Huang and Kai Liu and Zhenlong Li and Hassan, {Mohammed Anowarul} and Baoxuan Jin",
year = "2016",
month = "4",
doi = "10.1371/journal.pone.0152250",
language = "English (US)",
volume = "11",
journal = "PLoS One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "4",

}

Gui, Z, Yu, M, Yang, C, Jiang, Y, Chen, S, Xia, J, Huang, Q, Liu, K, Li, Z, Hassan, MA & Jin, B 2016, 'Developing subdomain allocation algorithms based on spatial and communicational constraints to accelerate dust storm simulation', PloS one, vol. 11, no. 4, e0152250. https://doi.org/10.1371/journal.pone.0152250

Developing subdomain allocation algorithms based on spatial and communicational constraints to accelerate dust storm simulation. / Gui, Zhipeng; Yu, Manzhu; Yang, Chaowei; Jiang, Yunfeng; Chen, Songqing; Xia, Jizhe; Huang, Qunying; Liu, Kai; Li, Zhenlong; Hassan, Mohammed Anowarul; Jin, Baoxuan.

In: PloS one, Vol. 11, No. 4, e0152250, 04.2016.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Developing subdomain allocation algorithms based on spatial and communicational constraints to accelerate dust storm simulation

AU - Gui, Zhipeng

AU - Yu, Manzhu

AU - Yang, Chaowei

AU - Jiang, Yunfeng

AU - Chen, Songqing

AU - Xia, Jizhe

AU - Huang, Qunying

AU - Liu, Kai

AU - Li, Zhenlong

AU - Hassan, Mohammed Anowarul

AU - Jin, Baoxuan

PY - 2016/4

Y1 - 2016/4

N2 - Dust storm has serious disastrous impacts on environment, human health, and assets. The developments and applications of dust storm models have contributed significantly to better understand and predict the distribution, intensity and structure of dust storms. However, dust storm simulation is a data and computing intensive process. To improve the computing performance, high performance computing has been widely adopted by dividing the entire study area into multiple subdomains and allocating each subdomain on different computing nodes in a parallel fashion. Inappropriate allocation may introduce imbalanced task loads and unnecessary communications among computing nodes. Therefore, allocation is a key factor that may impact the efficiency of parallel process. An allocation algorithm is expected to consider the computing cost and communication cost for each computing node to minimize total execution time and reduce overall communication cost for the entire simulation. This research introduces three algorithms to optimize the allocation by considering the spatial and communicational constraints: 1) an Integer Linear Programming (ILP) based algorithm from combinational optimization perspective; 2) a K-Means and Kernighan-Lin combined heuristic algorithm (K&K) integrating geometric and coordinate-free methods by merging local and global partitioning; 3) an automatic seeded region growing based geometric and local partitioning algorithm (ASRG). The performance and effectiveness of the three algorithms are compared based on different factors. Further, we adopt the K&K algorithm as the demonstrated algorithm for the experiment of dust model simulation with the non-hydrostatic mesoscale model (NMMdust) and compared the performance with the MPI default sequential allocation. The results demonstrate that K&K method significantly improves the simulation performance with better subdomain allocation. This method can also be adopted for other relevant atmospheric and numerical modeling.

AB - Dust storm has serious disastrous impacts on environment, human health, and assets. The developments and applications of dust storm models have contributed significantly to better understand and predict the distribution, intensity and structure of dust storms. However, dust storm simulation is a data and computing intensive process. To improve the computing performance, high performance computing has been widely adopted by dividing the entire study area into multiple subdomains and allocating each subdomain on different computing nodes in a parallel fashion. Inappropriate allocation may introduce imbalanced task loads and unnecessary communications among computing nodes. Therefore, allocation is a key factor that may impact the efficiency of parallel process. An allocation algorithm is expected to consider the computing cost and communication cost for each computing node to minimize total execution time and reduce overall communication cost for the entire simulation. This research introduces three algorithms to optimize the allocation by considering the spatial and communicational constraints: 1) an Integer Linear Programming (ILP) based algorithm from combinational optimization perspective; 2) a K-Means and Kernighan-Lin combined heuristic algorithm (K&K) integrating geometric and coordinate-free methods by merging local and global partitioning; 3) an automatic seeded region growing based geometric and local partitioning algorithm (ASRG). The performance and effectiveness of the three algorithms are compared based on different factors. Further, we adopt the K&K algorithm as the demonstrated algorithm for the experiment of dust model simulation with the non-hydrostatic mesoscale model (NMMdust) and compared the performance with the MPI default sequential allocation. The results demonstrate that K&K method significantly improves the simulation performance with better subdomain allocation. This method can also be adopted for other relevant atmospheric and numerical modeling.

UR - http://www.scopus.com/inward/record.url?scp=84962900891&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84962900891&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0152250

DO - 10.1371/journal.pone.0152250

M3 - Article

C2 - 27044039

AN - SCOPUS:84962900891

VL - 11

JO - PLoS One

JF - PLoS One

SN - 1932-6203

IS - 4

M1 - e0152250

ER -