peano4.api.taskgraph.compiler.CellTaskProducerConsumer Namespace Reference

Functions
List["peano4.api.solversteps.Step"]	translate_with_sequential (Graph graph, str identifier, "peano4.api.Project" peano_project)
	Split tasks writing cell data into producer and consumer, and map each task onto its own mesh traversal.
List["peano4.api.solversteps.Step"]	translate_with_fusion (Graph graph, str identifier, "peano4.api.Project" peano_project)
	Split tasks writing cell data into producer and consumer, and fuse tasks into as few mesh traversals as possible.
List["peano4.api.solversteps.Step"]	_translate (Graph graph, str identifier, "peano4.api.Project" peano_project, bool use_fusion)
	Task graph compiler which splits tasks writing cell data into a producer spawning tasks and a consumer waiting for them (work in progress)
str	_task_number_attribute_name (Task task, int action_set_index)
	CamelCase version of the task's name followed by ActionSet, the action set index and TaskNumber.
Task	_copy_task (Task task)
	Shallow copy of a task with its own lists and dictionaries, so we can alter the copy's dependencies without touching the original graph.
bool	_exclusively_writes_cell_data (Task task)
	True if task declares at least one cell submodel as WRITE or READ_AND_WRITE, and no face or vertex submodel as WRITE or READ_AND_WRITE.
List[str]	_has_events_after_touch_cell_first_time (Task task)
	List the events which the task's action sets realise and which a producer would invoke before its spawned tasks have necessarily terminated: unprepareTraversal(), endTraversal() and the touch...LastTime() events.
	_split_cell_tasks (Graph graph)
	Build the producer-consumer graph.
List[str]	_non_static_attribute_names (str attributes)
	Names of the non-static attributes declared in the C++ snippet returned by an action set's get_attributes().
str	_generator_name (submodel)
	Name of the generator class which realises the submodel in C++.
bool	_is_held_via_smart_pointer (submodel)
	True if the submodel's generator holds the data via a smart pointer, i.e.
	_enumerators_used_by (Task task, str task_body)
	Names of the vertex and face enumerators which a task body might use, i.e.
	_warn_about_unsafe_task (Task task, int action_set_index, "peano4.api.solversteps.ActionSet" action_set, str task_body)
	Issue the warnings about task bodies which might not be safe.
	_spawn_tasks (split_tasks, "peano4.api.datamodel.DaStGen2" task_labels)
	Make the producers spawn tasks and the consumers wait for them, and add one task number per spawning action set to the task labels.

Variables
tuple	_SMART_POINTER_HINT

Function Documentation

_copy_task()

Task peano4.api.taskgraph.compiler.CellTaskProducerConsumer._copy_task ( Task  task )

protected

Shallow copy of a task with its own lists and dictionaries, so we can alter the copy's dependencies without touching the original graph.

The action sets and the submodels themselves are shared.

Definition at line 274 of file CellTaskProducerConsumer.py.

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._split_cell_tasks().

Here is the caller graph for this function:

_enumerators_used_by()

peano4.api.taskgraph.compiler.CellTaskProducerConsumer._enumerators_used_by ( Task  task, str  task_body  )

protected

Names of the vertex and face enumerators which a task body might use, i.e.

which occur in the generated C++ text. They are realised as peano4::datamanagement::FaceEnumerator or peano4::datamanagement::VertexEnumerator and hold pointers into Peano's stacks, so the spawned task has to capture copies (peano4.api.taskgraph.compiler.SpawnCellTaskActionSet.SpawnCellTaskActionSet).

Returns

([str], [str]) Names of the face enumerators and of the vertex enumerators.

Definition at line 441 of file CellTaskProducerConsumer.py.

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._spawn_tasks(), and peano4.api.taskgraph.compiler.CellTaskProducerConsumer._warn_about_unsafe_task().

Here is the caller graph for this function:

_exclusively_writes_cell_data()

bool peano4.api.taskgraph.compiler.CellTaskProducerConsumer._exclusively_writes_cell_data ( Task  task )

protected

True if task declares at least one cell submodel as WRITE or READ_AND_WRITE, and no face or vertex submodel as WRITE or READ_AND_WRITE.

peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate()

Definition at line 294 of file CellTaskProducerConsumer.py.

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._split_cell_tasks().

Here is the caller graph for this function:

_generator_name()

str peano4.api.taskgraph.compiler.CellTaskProducerConsumer._generator_name (   submodel )

protected

Name of the generator class which realises the submodel in C++.

Definition at line 414 of file CellTaskProducerConsumer.py.

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._is_held_via_smart_pointer().

Here is the caller graph for this function:

_has_events_after_touch_cell_first_time()

List[str] peano4.api.taskgraph.compiler.CellTaskProducerConsumer._has_events_after_touch_cell_first_time ( Task  task )

protected

List the events which the task's action sets realise and which a producer would invoke before its spawned tasks have necessarily terminated: unprepareTraversal(), endTraversal() and the touch...LastTime() events.

An event counts if the action set yields code for it.

Returns

[str] One entry "<action set name>::<event>" per such event. The list is empty if the task can be split safely.

Definition at line 311 of file CellTaskProducerConsumer.py.

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._split_cell_tasks().

Here is the caller graph for this function:

_is_held_via_smart_pointer()

bool peano4.api.taskgraph.compiler.CellTaskProducerConsumer._is_held_via_smart_pointer (   submodel )

protected

True if the submodel's generator holds the data via a smart pointer, i.e.

if copies of a C++ object share the data rather than cloning it.

Definition at line 424 of file CellTaskProducerConsumer.py.

References peano4.api.taskgraph.compiler.CellTaskProducerConsumer._generator_name().

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._warn_about_unsafe_task().

Here is the call graph for this function:

Here is the caller graph for this function:

_non_static_attribute_names()

List[str] peano4.api.taskgraph.compiler.CellTaskProducerConsumer._non_static_attribute_names ( str  attributes )

protected

Names of the non-static attributes declared in the C++ snippet returned by an action set's get_attributes().

This is a heuristic: We split the snippet into statements, skip all statements which are static or contain a bracket (functions), and take the last identifier before an optional array size or initialisation.

Definition at line 391 of file CellTaskProducerConsumer.py.

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._warn_about_unsafe_task().

Here is the caller graph for this function:

_spawn_tasks()

peano4.api.taskgraph.compiler.CellTaskProducerConsumer._spawn_tasks (   split_tasks, "peano4.api.datamodel.DaStGen2"  task_labels  )

protected

Make the producers spawn tasks and the consumers wait for them, and add one task number per spawning action set to the task labels.

peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate()

Parameters

split_tasks	[(Task, Task, Task)] Triples of original task, producer and consumer.
task_labels	peano4.api.datamodel.DaStGen2 The task labels. This routine adds the attributes.

Returns

int Number of attributes added to the task labels.

Definition at line 518 of file CellTaskProducerConsumer.py.

References peano4.api.taskgraph.compiler.CellTaskProducerConsumer._enumerators_used_by(), peano4.api.taskgraph.compiler.CellTaskProducerConsumer._task_number_attribute_name(), and peano4.api.taskgraph.compiler.CellTaskProducerConsumer._warn_about_unsafe_task().

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate().

Here is the call graph for this function:

Here is the caller graph for this function:

_split_cell_tasks()

peano4.api.taskgraph.compiler.CellTaskProducerConsumer._split_cell_tasks ( Graph  graph )

protected

Build the producer-consumer graph.

peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate()

Returns

(peano4.api.taskgraph.Graph, [(Task, Task, Task)]) The new graph and the list of (original task, producer, consumer) triples. Producer and consumer must not be fused.

Definition at line 341 of file CellTaskProducerConsumer.py.

References peano4.api.taskgraph.compiler.CellTaskProducerConsumer._copy_task(), peano4.api.taskgraph.compiler.CellTaskProducerConsumer._exclusively_writes_cell_data(), and peano4.api.taskgraph.compiler.CellTaskProducerConsumer._has_events_after_touch_cell_first_time().

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate().

Here is the call graph for this function:

Here is the caller graph for this function:

_task_number_attribute_name()

str peano4.api.taskgraph.compiler.CellTaskProducerConsumer._task_number_attribute_name ( Task  task, int  action_set_index  )

protected

CamelCase version of the task's name followed by ActionSet, the action set index and TaskNumber.

Words are separated by whitespace or underscores, and only the first letter of each word is capitalised, so existing camel case and acronyms are preserved. "EulerFVSmooth_Update cell" with action set index 0 for example becomes EulerFVSmoothUpdateCellActionSet0TaskNumber.

Definition at line 259 of file CellTaskProducerConsumer.py.

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._spawn_tasks().

Here is the caller graph for this function:

_translate()

List["peano4.api.solversteps.Step"] peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate ( Graph  graph, str  identifier, "peano4.api.Project"  peano_project, bool  use_fusion  )

protected

Task graph compiler which splits tasks writing cell data into a producer spawning tasks and a consumer waiting for them (work in progress)

Do not use this routine directly. Pick peano4.api.taskgraph.compiler.CellTaskProducerConsumer.translate_with_sequential() or peano4.api.taskgraph.compiler.CellTaskProducerConsumer.translate_with_fusion() instead.

This compiler adds a preamble which rewrites the task graph, and then hands the rewritten graph over to either peano4.api.taskgraph.compiler.Sequential.translate() or peano4.api.taskgraph.compiler.FuseSteps.translate().

Preamble

Every task which exclusively writes cell data is replaced by two tasks. A task exclusively writes cell data if it declares at least one cell submodel as WRITE or READ_AND_WRITE, but no face or vertex submodel as WRITE or READ_AND_WRITE (peano4.api.taskgraph.compiler.CellTaskProducerConsumer._exclusively_writes_cell_data()). It may read face or vertex data. All other tasks are kept as they are.

A task is only split if this is safe (peano4.api.taskgraph.compiler.CellTaskProducerConsumer._has_events_after_touch_cell_first_time()): The producer spawns tasks which might still run after the producer's mesh traversal. If any action set of the task yields code for unprepareTraversal(), endTraversal() or one of the touch...LastTime() events, these events might be invoked before the spawned tasks have terminated. Such a task is hence not split, and the compiler writes a warning. You can enforce the split by setting the task's flag peano4.api.taskgraph.Task.Task.may_spawn_background_tasks to True, which skips this check. If the flag is False, the task is never split, and the compiler writes no warning.

The two tasks are:

1. The producer, named after the original task plus "Producer". It inherits all action sets and data declarations of the original task.
2. The consumer, named after the original task plus "Consumer". It works on cells, too, and depends on the producer. It declares every cell submodel that the producer writes (WRITE or READ_AND_WRITE) as READ_AND_WRITE.

Every task that depended on the original task depends on the consumer instead. Producer and consumer are marked as non-fusable, i.e. if we use FuseSteps, it never puts them into the same mesh traversal. A consumer however might be fused with subsequent tasks. In an ExaHyPE 2 setup, for example, the initial condition yields a producer and a consumer, and the latter is fused with the subsequent projection onto the faces.

The preamble does not alter the graph passed in. It works on a copy of the graph's tasks.

Spawning tasks

If a peano_project is given, the producer spawns the touchCellFirstTime() events of its action sets as tasks, and the consumer waits for these tasks (peano4.api.taskgraph.compiler.CellTaskProducerConsumer._spawn_tasks()):

• Every action set of the producer whose touchCellFirstTime() has a non-empty body is wrapped into a peano4.api.taskgraph.compiler.SpawnCellTaskActionSet.SpawnCellTaskActionSet. The wrapper evaluates the guard. If it holds, the wrapper takes a new task number from the tarch, stores it in the cell's task label (see below), and spawns the body as a task with this number. The task depends on the task spawned by the previous wrapped action set of the same producer, so the bodies are executed in their original order.
• Action sets whose touchCellFirstTime() has an empty body, such as exahype2.api.fv.actionsets.ApplyDirichletBoundaryConditions.ApplyDirichletBoundaryConditions, are not part of the producer-consumer pattern. They remain plain action sets of the producer, i.e. all their events are invoked by the producer.
• The same holds for action sets which do not want to run in a task (peano4.api.solversteps.ActionSet.ActionSet.may_run_in_background_task()), such as exahype2.api.fv.actionsets.ReadjustTimeStepSizeForOptimisticGlobalTimestepping.ReadjustTimeStepSizeForOptimisticGlobalTimestepping. Their touchCellFirstTime() is invoked directly by the producer, i.e. possibly while tasks spawned by previous action sets of the same producer still run. They hence must not depend on these tasks.
• The consumer is given a cousin peano4.api.taskgraph.compiler.WaitForCellTaskActionSet.WaitForCellTaskActionSet per wrapped action set. If the cell's task label holds a valid task number, it waits for this task, returns the number to the task enumerator and resets the label to NoNumber=-1.
• All other events of the wrapped action sets remain within the producer.

The spawned tasks outlive touchCellFirstTime(). The compiler therefore warns about constructs which might not be safe within a task. It does not inspect the code but only searches the generated C++ text for the names of data and attributes, i.e. the warnings state that a task might access something. Every warning names the generator which realises the data in question and suggests a fix. It has to be accepted or the generation is aborted (peano4.api.Logger.Logger.warn_and_confirm()):

• The task body might access fine or coarse grid cell data which is not held via a smart pointer, e.g. via peano4.api.datamodel.PatchToDoubleArray or peano4.api.datamodel.PatchToDoubleArrayOnHeap. The task copies the data, and writes to the copy are lost. The fix is a smart-pointer generator such as peano4.api.datamodel.PatchToDoubleArrayWithSmartPointer. Heap-based generators do not suffice, as they deep-copy their data whenever Peano copies an object.
• The task body might access vertex or face data via an enumerator whose data is not held via a smart pointer. The enumerators hold pointers to the objects on Peano's stacks, which are invalid once touchCellFirstTime() has terminated. The task therefore always captures them as peano4::datamanagement::FaceHeapDataEnumerator or peano4::datamanagement::VertexHeapDataEnumerator, which copy the faces or vertices. Without smart pointers, these copies are deep, i.e. writes are lost. With smart pointers, no warning is issued.
• The task body might use a non-static attribute of the action set. The task accesses an object which is destroyed after the producer's mesh traversal.
• An action set of the producer has to be modified by the user (peano4.api.solversteps.ActionSet.ActionSet.user_should_modify_template()). Such a producer is not altered, i.e. it spawns no tasks.

Task labels

The compiler adds a new cell data type <identifier>_TaskLabels to the project's data model. It is modelled via DaStGen2 and holds one integer per producer action set with a non-empty touchCellFirstTime(). The integer's name is the CamelCase version of the original task's name followed by ActionSet, the action set's index within the task and TaskNumber, e.g. EulerFVSmoothInitialConditionActionSet0TaskNumber (peano4.api.taskgraph.compiler.CellTaskProducerConsumer._task_number_attribute_name()). The labels furthermore hold a static tarch::multicore::TaskEnumerator (getTaskEnumerator()), from which the producers take their task numbers and to which the consumers return them. The producer declares the task labels as WRITE, the consumer as READ_AND_DISCARD: The producer sets the label in every sweep, whether it spawns a task or not, so the consumer's reset does not have to be kept. If the consumer declared READ_AND_WRITE, a fused producer step that directly follows a consumer would read the labels, too (peano4.api.taskgraph.compiler.PropagateUseStatements), and the labels would be relayed through steps of other phases which run before the labels are ever stored.

Parameters

graph	peano4.api.taskgraph.Graph The task graph to compile. Assumed to be acyclic.
identifier	str peano4.api.taskgraph.compiler.FuseSteps.translate()
peano_project	peano4.api.Project Project the task labels are added to. If None, no task labels are added and the producers spawn no tasks.
use_fusion	bool If True, hand the rewritten graph over to FuseSteps, otherwise to Sequential.

Returns

[peano4.api.solversteps.Step] peano4.api.taskgraph.compiler.FuseSteps.translate()

Definition at line 70 of file CellTaskProducerConsumer.py.

References peano4.api.taskgraph.compiler.CellTaskProducerConsumer._spawn_tasks(), and peano4.api.taskgraph.compiler.CellTaskProducerConsumer._split_cell_tasks().

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer.translate_with_fusion(), and peano4.api.taskgraph.compiler.CellTaskProducerConsumer.translate_with_sequential().

Here is the call graph for this function:

Here is the caller graph for this function:

_warn_about_unsafe_task()

peano4.api.taskgraph.compiler.CellTaskProducerConsumer._warn_about_unsafe_task ( Task  task, int  action_set_index, "peano4.api.solversteps.ActionSet"  action_set, str  task_body  )

protected

Issue the warnings about task bodies which might not be safe.

peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate()

We do not inspect the code. We only search the generated C++ text for the names of data and attributes, i.e. the warnings say that a task might access something.

Definition at line 470 of file CellTaskProducerConsumer.py.

References peano4.api.taskgraph.compiler.CellTaskProducerConsumer._enumerators_used_by(), peano4.api.taskgraph.compiler.CellTaskProducerConsumer._is_held_via_smart_pointer(), and peano4.api.taskgraph.compiler.CellTaskProducerConsumer._non_static_attribute_names().

Referenced by peano4.api.taskgraph.compiler.CellTaskProducerConsumer._spawn_tasks().

Here is the call graph for this function:

Here is the caller graph for this function:

translate_with_fusion()

List["peano4.api.solversteps.Step"] peano4.api.taskgraph.compiler.CellTaskProducerConsumer.translate_with_fusion	(	Graph	graph,
		str	identifier,
		"peano4.api.Project"	peano_project
	)

Split tasks writing cell data into producer and consumer, and fuse tasks into as few mesh traversals as possible.

You can use this task graph compiler in your ExaHyPE 2 project through

project.compiler = peano4.api.taskgraph.compiler.CellTaskProducerConsumer.translate_with_fusion

The rewritten graph is handed over to peano4.api.taskgraph.compiler.FuseSteps.translate(), which never fuses a producer with its consumer. See peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate() for a description of the rewrite and of the parameters.

Definition at line 45 of file CellTaskProducerConsumer.py.

References peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate().

Here is the call graph for this function:

translate_with_sequential()

List["peano4.api.solversteps.Step"] peano4.api.taskgraph.compiler.CellTaskProducerConsumer.translate_with_sequential	(	Graph	graph,
		str	identifier,
		"peano4.api.Project"	peano_project
	)

Split tasks writing cell data into producer and consumer, and map each task onto its own mesh traversal.

You can use this task graph compiler in your ExaHyPE 2 project through

project.compiler = peano4.api.taskgraph.compiler.CellTaskProducerConsumer.translate_with_sequential

The rewritten graph is handed over to peano4.api.taskgraph.compiler.Sequential.translate(). See peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate() for a description of the rewrite and of the parameters.

Definition at line 21 of file CellTaskProducerConsumer.py.

References peano4.api.taskgraph.compiler.CellTaskProducerConsumer._translate().

Here is the call graph for this function:

Variable Documentation

_SMART_POINTER_HINT

tuple peano4.api.taskgraph.compiler.CellTaskProducerConsumer._SMART_POINTER_HINT

protected

Initial value:

=  (
    "Realise the data via a smart-pointer generator, e.g. peano4.api.datamodel.PatchToDoubleArrayWithSmartPointer. "
    "Heap-based generators such as peano4.api.datamodel.PatchToDoubleArrayOnHeap do not suffice, as they deep-copy "
    "their data whenever an object is copied, and Peano copies the objects when it moves them between its stacks."
)

Definition at line 434 of file CellTaskProducerConsumer.py.