Abstract

This paper presents an implementation of phantom-cell adaptive mesh refinement (AMR) on a graphics processing unit (GPU) using CLAMR, a cell-based mini-app that runs on a variety of next-generation platforms. Phantom-cell AMR is a hybrid method of cell-based AMR and patch-based AMR that provides a separation of physics and mesh codes. By designing a structure that allows each level of the mesh to be independent, there are minimal development requirements that are needed to convert regular grid applications to AMR. The decoupling of physics and mesh codes through these phantom cells improves composability and creates an easy pathway toward implementing AMR codes on Exascale systems, specifically targeting GPUs. Physics and mesh codes can be accelerated individually, allowing for fewer dependencies and more opportunities for optimization. A complete implementation of phantom-cell AMR on a GPU with opencl is presented for the purpose of showing the simplicity of porting the algorithms to accelerator-based architectures and the performance and optimization improvements that are made as a result.

References

1.
Dunning
,
D.
,
Marts
,
W.
,
Robey
,
R. W.
, and
Bridges
,
P.
,
2020
, “
Adaptive Mesh Refinement in the Fast Lane
,”
J. Comput. Phys.
,
406
, p.
109193
. 10.1016/j.jcp.2019.109193
2.
Berger
,
M. J.
, and
Oliger
,
J.
,
1984
, “
Adaptive Mesh Refinement for Hyperbolic Partial Differential Equations
,”
J. Comput. Phys.
,
53
(
3
), pp.
484
512
. 10.1016/0021-9991(84)90073-1
3.
Berger
,
M. J.
, and
LeVeque
,
R. J.
,
1998
, “
Adaptive Mesh Refinement Using Wave-propagation Algorithms for Hyperbolic Systems
,”
SIAM J. Numer. Anal.
,
35
(
6
), pp.
2298
2316
. 10.1137/S0036142997315974
4.
Beckingsale
,
D.
, and
Gaudin
,
W.
,
2015
, “
Andrew Herdman, and Stephen Jarvis. Resident Block-structured Adaptive Mesh Refinement on Thousands of Graphics Processing Units
,”
44th International Conference on Parallel Processing (ICPP)
, IEEE.
5.
Sætra
,
M. L.
,
Brodtkorb
,
A. R.
, and
Lie
,
K-A.
,
2015
, “
Efficient GPU-Implementation of Adaptive Mesh Refinement for the Shallow-Water Equations
,”
J. Sci. Comput.
,
63
(
1
), pp.
23
48
. 10.1007/s10915-014-9883-4
6.
Young
,
D. P.
,
Melvin
,
R. G.
,
Bieterman
,
M. B.
,
Johnson
,
F. T.
,
Samant
,
S. S.
, and
Bussoletti
,
J. E.
,
1991
, “
A Locally Refined Rectangular Grid Finite Element Method: Application to Computational Fluid Dynamics and Computational Physics
,”
J. Comput. Phys.
,
92
(
1
), pp.
1
66
. 10.1016/0021-9991(91)90291-R
7.
DeZeeuw
,
D.
, and
Powell
,
K. G.
,
1993
, “
An Adaptively Refined Cartesian Mesh Solver for the Euler Equations
,”
J. Comput. Phys.
,
104
(
1
), pp.
56
68
. 10.1006/jcph.1993.1007
8.
Gittings
,
M.
,
Weaver
,
R.
,
Clover
,
M.
,
Betlach
,
T.
,
Byrne
,
N.
,
Coker
,
R.
,
Dendy
,
E.
,
Hueckstaedt
,
R.
,
New
,
K.
,
Oakes
,
W. R.
, and
Ranta
,
D.
,
2008
, “
The RAGE Radiation-Hydrodynamic Code
,”
Comput. Sci. Discovery
,
1
(
1
), p.
015005
. 10.1088/1749-4699/1/1/015005
9.
Khokhlov
,
A. M.
,
1998
, “
Fully Threaded Tree Algorithms for Adaptive Refinement Fluid Dynamics Simulations
,”
J. Comput. Phys.
,
143
(
2
), pp.
519
543
. 10.1006/jcph.1998.9998
10.
Griebel
,
M.
, and
Zumbusch
,
G.
,
1999
, “
Parallel Multigrid in An Adaptive PDE Solver Based on Hashing and Space-Filling Curves
,”
Parallel Comput.
,
25
(
7
), pp.
827
843
. 10.1016/S0167-8191(99)00020-4
11.
F-S Lien
,
H. J.i.
, and
Yee
,
E.
,
2010
, “
A New Adaptive Mesh Refinement Data Structure With An Application to Detonation
,”
J. Comput. Phys.
,
229
(
23
), pp.
8981
8993
. 10.1016/j.jcp.2010.08.023
12.
Nicholaeff
,
D.
,
Davis
,
N.
,
Trujillo
,
D.
, and
Robey
,
R. W.
,
2012
, “
Cell-Based Adaptive Mesh Refinement Implemented With General Purpose Graphics Processing Units
,”
Technical Report LA-UR-11-07127
.
13.
Robey
,
R. N.
,
Nicholaeff
,
D.
, and
Robey
,
R. W.
,
2013
, “
Hash-Based Algorithms for Discretized Data
,”
SIAM J. Sci. Comput.
,
35
(
4
), pp.
C346
C368
. 10.1137/120873686
14.
Tumblin
,
R.
,
Ahrens
,
P.
,
Hartse
,
S.
, and
Robey
,
R. W.
,
2015
, “
Parallel Compact Hash Algorithms for Computational Meshes
,”
SIAM J. Sci. Comput.
,
37
(
1
), pp.
C31
C53
. 10.1137/13093371X
15.
Leveque
,
R. J.
,
2002
,
Finite Volume Methods for Hyperbolic Problems
,
Cambridge University Press
,
Cambridge
.
16.
Davis
,
S. F.
,
1984
, “
TVD Finite Difference Schemes, and Artificial Viscosity
,”
ICASE Report no. 84–20
.
National Aeronautics and Space Administration
.
USA
.
17.
Davis
,
S. F.
,
1987
, “
A Simplified TVD Finite Difference Scheme Via Artificial Viscosity
,”
SIAM J. Sci. Stat. Comput.
,
8
(
1
), pp.
1
18
. 10.1137/0908002
18.
Yee
,
H. C.
,
1987
, “
Construction of Explicit and Implicit Symmetric TVD Schemes and Their Applications
,”
J. Comput. Phys.
,
68
(
1
), pp.
151
179
. 10.1016/0021-9991(87)90049-0
19.
Sweby
,
P. K.
,
1984
, “
High Resolution Schemes Using Flux Limiters for Hyperbolic Conservation Laws
,”
SIAM J. Numer. Anal.
,
21
(
5
), pp.
995
1011
. 10.1137/0721062
20.
Kuehn
,
J. A.
,
2019
, “
An Approach to Quantifying Representativeness of Workload Proxies
,”
JOWOG-34, Albuquerque, NM, July 15–18
,
Sandia National Laboratories
,
Albuquerque, NM
.
21.
Harrell
,
S. L.
,
Kitson
,
J.
,
Bird
,
R.
,
Pennycook
,
S. J.
,
Sewall
,
J.
,
Jacobsen
,
D.
,
Asanza
,
D. N.
,
Hsu
,
A.
,
Carrillo
,
H. C.
,
Kim
,
H.
, and
Robey
,
R.
,
2018
, “
Effective Performance Portability
,”
2018 IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC (P3HPC)
,
IEEE
,
Dallas, TX
, pp.
24
36
.
You do not currently have access to this content.