Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions docs/src/parallelization.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ the same time, the latter takes precedence.
If you use time integration methods from
[OrdinaryDiffEq.jl](https://github.com/SciML/OrdinaryDiffEq.jl)
and want to use multiple threads therein, you need to set the keyword argument
`thread = Trixi.True()` (or `thread = OrdinaryDiffEq.True()`)
`thread = Trixi.Threaded()`
of the algorithms, as described in the
[section on time integration methods](@ref time-integration).

Expand Down Expand Up @@ -127,11 +127,11 @@ julia> HDF5.API.set_libraries!("/usr/lib/x86_64-linux-gnu/hdf5/openmpi/libhdf5.s
```
After the preferences are set, restart the Julia REPL again.

!!! note
If multiple MPI installations are present on a system (as is typically the case on a cluster), calling
`MPIPreferences.use_system_binary()` may lead to an undesired selection of the MPI implementation -
make sure to check the `LocalPreferences.toml` in any case.
You can also check at runtime of your Julia session the MPI configuration with
!!! note
If multiple MPI installations are present on a system (as is typically the case on a cluster), calling
`MPIPreferences.use_system_binary()` may lead to an undesired selection of the MPI implementation -
make sure to check the `LocalPreferences.toml` in any case.
You can also check at runtime of your Julia session the MPI configuration with
`using MPI; MPI.versioninfo()`.

### [Usage](@id parallel_usage)
Expand Down Expand Up @@ -251,14 +251,14 @@ julia> using HDF5
julia> HDF5.API.set_libraries!("/path/to/your/libhdf5.so", "/path/to/your/libhdf5_hl.so")
```
For more information see also the
[documentation of HDF5.jl](https://juliaio.github.io/HDF5.jl/stable/mpi/).
[documentation of HDF5.jl](https://juliaio.github.io/HDF5.jl/stable/mpi/).

To install the MPI-enabled `HDF5` library, i.e., the `libhdf5.so` and `libhdf5_hl.so` files,
To install the MPI-enabled `HDF5` library, i.e., the `libhdf5.so` and `libhdf5_hl.so` files,
there are two options.
First, there might be a package available from the package manager, e.g., `libhdf5-openmpi-dev` for
`OpenMPI` or `libhdf5-mpich-dev` for `MPICH`.
First, there might be a package available from the package manager, e.g., `libhdf5-openmpi-dev` for
`OpenMPI` or `libhdf5-mpich-dev` for `MPICH`.
On `Ubuntu`, you find a list of available `libhdf5-` packages [here](https://packages.ubuntu.com/search?keywords=libhdf5-&searchon=names&suite=all&section=all).
For other Linux distributions you can consult the package manager of your distribution or the third-party webpage https://pkgs.org/.
For other Linux distributions you can consult the package manager of your distribution or the third-party webpage https://pkgs.org/.

The second option is to manually build the library on your system.
To do so, download the latest release from the [HDF5 download page](https://www.hdfgroup.org/download-hdf5/source-code/).
Expand Down
6 changes: 3 additions & 3 deletions docs/src/time_integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ are the following. Further documentation can be found in the
from Trixi.jl.
- If you start Julia with multiple threads and want to use them also in the time
integration method from OrdinaryDiffEq.jl, you need to pass the keyword argument
`thread = Trixi.True()` (or `thread = OrdinaryDiffEq.True()`) to the algorithm, e.g.,
`RDPK3SpFSAL49(thread = Trixi.True())` or
`CarpenterKennedy2N54(thread = Trixi.True(), williamson_condition = false)`.
`thread = Trixi.Threaded()` to the algorithm, e.g.,
`RDPK3SpFSAL49(thread = Trixi.Threaded())` or
`CarpenterKennedy2N54(thread = Trixi.Threaded(), williamson_condition = false)`.
For more information on using thread-based parallelism in Trixi.jl, please refer to
[Shared-memory parallelization with threads](@ref).
- If you use error-based step size control (see also the section on
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,6 @@ callbacks = CallbackSet(summary_callback, analysis_callback, alive_callback,

###############################################################################
# run the simulation
sol = solve(ode, SSPRK54(thread = Trixi.True());
sol = solve(ode, SSPRK54(thread = Trixi.Threaded());
dt = 1.0, # solve needs some value here but it will be overwritten by the stepsize_callback
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ callbacks = CallbackSet(summary_callback,

# Run the simulation
###############################################################################
sol = solve(ode, SSPRK104(; thread = Trixi.True());
sol = solve(ode, SSPRK104(; thread = Trixi.Threaded());
dt = 1.0, # overwritten by the `stepsize_callback`
ode_default_options()...,
callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,6 @@ stage_limiter! = PositivityPreservingLimiterZhangShu(thresholds = (5.0e-6, 5.0e-
###############################################################################
# Run the simulation

sol = solve(ode, SSPRK33(; stage_limiter! = stage_limiter!, thread = Trixi.True());
sol = solve(ode, SSPRK33(; stage_limiter! = stage_limiter!, thread = Trixi.Threaded());
dt = 1.6e-5, # Fixed timestep works decent here
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,6 @@ stage_limiter! = PositivityPreservingLimiterZhangShu(thresholds = (5.0e-6, 5.0e-

###############################################################################
# run the simulation
sol = solve(ode, SSPRK43(; stage_limiter! = stage_limiter!, thread = Trixi.True());
sol = solve(ode, SSPRK43(; stage_limiter! = stage_limiter!, thread = Trixi.Threaded());
dt = 5e-7, # Reducing initial timestep allows AMR interval of 2 instead of 1
adaptive = true, ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ callbacks = CallbackSet(summary_callback,
# run the simulation

time_int_tol = 1e-6
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.True());
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.Threaded());
abstol = time_int_tol, reltol = time_int_tol,
adaptive = true, dt = 1e-3, # needed only for tests/CI
ode_default_options()..., callback = callbacks);
2 changes: 1 addition & 1 deletion examples/p4est_2d_dgsem/elixir_euler_subsonic_cylinder.jl
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,6 @@ callbacks = CallbackSet(summary_callback, analysis_callback, alive_callback, sav
# run the simulation
sol = solve(ode,
CarpenterKennedy2N54(williamson_condition = false;
thread = Trixi.True());
thread = Trixi.Threaded());
dt = 1.0, # solve needs some value here but it will be overwritten by the stepsize_callback
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,6 @@ stage_limiter! = PositivityPreservingLimiterZhangShu(thresholds = (5.0e-7, 1.0e-
# We supply a small initial timestep to be able to use a larger AMR interval (3 instead of 1) throughout the simulation.
# This pays off almost immediately as only the first couple timesteps use this timestep before it is ramped up.
dt0 = 1e-8
sol = solve(ode, SSPRK43(; stage_limiter! = stage_limiter!, thread = Trixi.True());
sol = solve(ode, SSPRK43(; stage_limiter! = stage_limiter!, thread = Trixi.Threaded());
adaptive = true, dt = dt0,
ode_default_options()..., callback = callbacks);
2 changes: 1 addition & 1 deletion examples/p4est_2d_dgsem/elixir_mhd_rotor_cfl_ramp.jl
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ callbacks = CallbackSet(summary_callback,
# run the simulation

sol = solve(ode,
CarpenterKennedy2N54(thread = Trixi.True(),
CarpenterKennedy2N54(thread = Trixi.Threaded(),
williamson_condition = false);
dt = 1.0, # solve needs some value here but it will be overwritten by the stepsize_callback
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,6 @@ callbacks = CallbackSet(summary_callback, analysis_callback, alive_callback, sav
###############################################################################
# run the simulation

sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.True());
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.Threaded());
abstol = 1e-8, reltol = 1e-8,
ode_default_options()..., callback = callbacks)
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,6 @@ callbacks = CallbackSet(summary_callback, analysis_callback, alive_callback,

###############################################################################

sol = solve(ode, SSPRK54(thread = Trixi.True());
sol = solve(ode, SSPRK54(thread = Trixi.Threaded());
dt = dt_restart,
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ callbacks = CallbackSet(summary_callback,
###############################################################################
# run the simulation

ode_algorithm = SSPRK43(thread = Trixi.True())
ode_algorithm = SSPRK43(thread = Trixi.Threaded())

time_int_tol = 1e-4
sol = solve(ode, ode_algorithm;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -146,5 +146,5 @@ callbacks = CallbackSet(summary_callback,

sol = solve(ode,
CarpenterKennedy2N54(williamson_condition = false,
thread = Trixi.True());
thread = Trixi.Threaded());
dt = 1.0, ode_default_options()..., callback = callbacks)
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ callbacks = CallbackSet(summary_callback,
# run the simulation

# Moderate number of threads (e.g. 4) advisable to speed things up
ode_alg = RDPK3SpFSAL49(thread = Trixi.True())
ode_alg = RDPK3SpFSAL49(thread = Trixi.Threaded())
time_int_tol = 1e-7
sol = solve(ode, ode_alg;
# not necessary, added for overwriting in tests
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,6 @@ callbacks = CallbackSet(summary_callback,

# Use a Runge-Kutta method with automatic (error based) time step size control
# Enable threading of the RK method for better performance on multiple threads
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.True());
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.Threaded());
abstol = 1.0e-6, reltol = 1.0e-6,
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,6 @@ callbacks = CallbackSet(summary_callback,

# Use a Runge-Kutta method with automatic (error based) time step size control
# Enable threading of the RK method for better performance on multiple threads
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.True());
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.Threaded());
abstol = 1.0e-6, reltol = 1.0e-6,
ode_default_options()..., callback = callbacks);
2 changes: 1 addition & 1 deletion examples/p4est_3d_dgsem/elixir_euler_tandem_spheres.jl
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,6 @@ callbacks = CallbackSet(summary_callback,
###############################################################################

tols = 1e-5
sol = solve(ode, RDPK3SpFSAL35(thread = Trixi.True());
sol = solve(ode, RDPK3SpFSAL35(thread = Trixi.Threaded());
abstol = tols, reltol = tols,
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,6 @@ callbacks = CallbackSet(summary_callback,
# run the simulation

time_int_tol = 1e-8
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.True());
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.Threaded());
abstol = time_int_tol, reltol = time_int_tol,
ode_default_options()..., callback = callbacks)
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,6 @@ callbacks = CallbackSet(summary_callback,

# Use a Runge-Kutta method with automatic (error based) time step size control
# Enable threading of the RK method for better performance on multiple threads
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.True());
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.Threaded());
abstol = 1.0e-6, reltol = 1.0e-6,
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ volume_integral_fluxdiff = VolumeIntegralFluxDifferencing(volume_flux)
# `volume_integral_stabilized` is used instead of the `volume_integral_default`.
indicator = IndicatorEntropyChange(maximum_entropy_increase = 0.0)

# Adaptive volume integral using the entropy production comparison indicator to perform the
# Adaptive volume integral using the entropy production comparison indicator to perform the
# stabilized/EC volume integral when needed and keeping the weak form if it is more diffusive.
volume_integral = VolumeIntegralAdaptive(volume_integral_default = volume_integral_weakform,
volume_integral_stabilized = volume_integral_fluxdiff,
Expand Down Expand Up @@ -67,6 +67,7 @@ callbacks = CallbackSet(summary_callback,
###############################################################################
# run the simulation

sol = solve(ode, CarpenterKennedy2N54(williamson_condition = false, thread = Trixi.True());
sol = solve(ode,
CarpenterKennedy2N54(williamson_condition = false, thread = Trixi.Threaded());
dt = stepsize_callback(ode),
ode_default_options()..., callback = callbacks);
2 changes: 1 addition & 1 deletion examples/tree_2d_dgsem/elixir_euler_astro_jet_amr_scO2.jl
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,6 @@ stage_limiter! = PositivityPreservingLimiterZhangShu(thresholds = (5.0e-6, 5.0e-
# run the simulation

# use adaptive time stepping based on error estimates
sol = solve(ode, SSPRK43(; stage_limiter! = stage_limiter!, thread = Trixi.True());
sol = solve(ode, SSPRK43(; stage_limiter! = stage_limiter!, thread = Trixi.Threaded());
controller = PIDController(0.55, -0.27, 0.05),
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,6 @@ callbacks = CallbackSet(summary_callback,
###############################################################################
# run the simulation

sol = solve(ode, SSPRK54(thread = Trixi.True());
sol = solve(ode, SSPRK54(thread = Trixi.Threaded());
dt = 1.0, # solve needs some value here but it will be overwritten by the stepsize_callback
ode_default_options()..., callback = callbacks);
2 changes: 1 addition & 1 deletion examples/tree_3d_dgsem/elixir_euler_sedov_scO2.jl
Original file line number Diff line number Diff line change
Expand Up @@ -100,5 +100,5 @@ callbacks = CallbackSet(summary_callback,
###############################################################################
# run the simulation

sol = solve(ode, SSPRK54(thread = Trixi.True()); dt = 1.0,
sol = solve(ode, SSPRK54(thread = Trixi.Threaded()); dt = 1.0,
ode_default_options()..., callback = callbacks);
Original file line number Diff line number Diff line change
Expand Up @@ -99,5 +99,5 @@ callbacks = CallbackSet(summary_callback,
###############################################################################
# run the simulation

sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.True()); adaptive = true,
sol = solve(ode, RDPK3SpFSAL49(thread = Trixi.Threaded()); adaptive = true,
ode_default_options()..., callback = callbacks)
2 changes: 2 additions & 0 deletions src/Trixi.jl
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ using T8code
using RecipesBase: RecipesBase
using RecursiveArrayTools: VectorOfArray
using Static: Static, One, True, False
# OrdinaryDiffEq v7+ uses FastBroadcast.Threaded() for the thread argument; older versions use Static.True()
const Threaded = isdefined(DiffEqBase, :Threaded) ? DiffEqBase.Threaded : True
@reexport using StaticArrays: SVector
using StaticArrays: StaticArrays, MVector, MArray, SMatrix, @SMatrix
using StrideArrays: PtrArray, StrideArray, StaticInt
Expand Down
6 changes: 3 additions & 3 deletions test/test_threaded.jl
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Trixi.MPI.Barrier(Trixi.mpi_comm())
@test_trixi_include(joinpath(EXAMPLES_DIR, "tree_2d_dgsem",
"elixir_advection_restart.jl"),
alg=CarpenterKennedy2N54(williamson_condition = false,
thread = Trixi.True()),
thread = Trixi.Threaded()),
# Expected errors are exactly the same as in the serial test!
l2=[8.005068880114254e-6],
linf=[6.39093577996519e-5])
Expand Down Expand Up @@ -148,7 +148,7 @@ Trixi.MPI.Barrier(Trixi.mpi_comm())
@test_trixi_include(joinpath(EXAMPLES_DIR, "tree_2d_dgsem",
"elixir_advection_diffusion.jl"),
initial_refinement_level=2, tspan=(0.0, 0.4), polydeg=5,
alg=RDPK3SpFSAL49(thread = Trixi.True()),
alg=RDPK3SpFSAL49(thread = Trixi.Threaded()),
l2=[4.0915532997994255e-6],
linf=[2.3040850347877395e-5])

Expand Down Expand Up @@ -363,7 +363,7 @@ end
@trixi_testset "elixir_euler_curved.jl with threaded time integration" begin
@test_trixi_include(joinpath(EXAMPLES_DIR, "dgmulti_2d",
"elixir_euler_curved.jl"),
alg=RDPK3SpFSAL49(thread = Trixi.True()),
alg=RDPK3SpFSAL49(thread = Trixi.Threaded()),
l2=[
1.720916434676505e-5,
1.5928649356300228e-5,
Expand Down
Loading