How to define what a numerical scheme (or plotter or grid setup) does

Table of Contents

• Construct the task graph
• Add action sets
• Ensure that action sets get all the data they are working on
• Minimise data movements
• Branching of the task graph

The introduce a new numerical scheme page shows how a Solver.NumericalScheme (or Solver.GridConstruction, or Solver.Plotter) adds data to a solver and ensures that these data are also tied to the mesh properly.

This is all about static setups. On this page, we discuss how to add dynamic behaviour.

ExaHyPE's current rewrite uses task graphs to model all behaviour. You find the prototype of such a task graph in exahype.api.fv.RusanovWithExplicitEuler.construct_output(). In general, any construct_output() from a numerical scheme, plot strategy or mesh setup is responsible for returning a task graph which describes what is to be done in this step.

Construct the task graph

reconstruction_task        = peano4.api.taskgraph.Task(name=f"{self._solver.name}_Reconstruct and apply BCs",    entity=peano4.api.taskgraph.Task.GridEntity.CELL)
update_cell_task           = peano4.api.taskgraph.Task(name=f"{self._solver.name}_Update cell",                  entity=peano4.api.taskgraph.Task.GridEntity.CELL)
project_on_faces_task      = peano4.api.taskgraph.Task(name=f"{self._solver.name}_Project on faces",             entity=peano4.api.taskgraph.Task.GridEntity.CELL)
compute_cfl_condition_task = peano4.api.taskgraph.Task(name=f"{self._solver.name}_Compute global CFL condition", entity=peano4.api.taskgraph.Task.GridEntity.CELL)
 
update_cell_task.add_dependency(reconstruction_task)
project_on_faces_task.add_dependency(update_cell_task)
compute_cfl_condition_task.add_dependency(update_cell_task)
 
time_step.add_task(reconstruction_task)
time_step.add_task(update_cell_task)
time_step.add_task(project_on_faces_task)
time_step.add_task(compute_cfl_condition_task)

The CFL condition has a task of its own: The update task's action set (exahype2.api.fv.actionsets.UpdateCell) stores the maximum eigenvalue of every patch it updates in the cell. The task "Compute global CFL condition" (exahype2.api.fv.actionsets.ComputeGlobalCFLCondition) then reduces these values and derives the next global time step size from them. It only depends on the update, i.e. it can run in parallel to the projection onto the faces.

Add action sets

Ensure that action sets get all the data they are working on

Minimise data movements

Branching of the task graph

If you need branching, you have to model tasks as parallel. See OPtimistic time stepping for Euler. And then you add guards that ensure that always only one branch works. SO we have to work with masking, as real branching is not supported by the task graph.

Why exclusivity doesn't fix the abort

Peano decides per sweep, not per cell, whether a sweep loads and stores a data object. The "Update cell" sweep declares QWithHalo as READ_AND_DISCARD. So it takes the patch off the stack for every cell and doesn't put it back, even for cells where UpdateCell doesn't fire. The run proves this. It aborts in time step 0, where every cell takes the traditional branch and UpdateCellShell fires nowhere. Still, the shell sweep tries to load QWithHalo and finds nothing.

Fix

At the task level your declarations are right: both siblings read the halo patch and are done with it. With the sequential compiler the two become consecutive sweeps, so only the second may discard. I'd let propagate_use_statements() handle this. Whenever a step declares READ_AND_DISCARD and a later step reads the same data before anything writes it, the compiler would upgrade the discard to READ_AND_WRITE. That would keep your declarations, work for any compiler that puts sibling tasks one after the other, and fit next to the pass I just added. Shall I implement that?

So we need a read-write. If these are smart pointers, it should not harm.