 |
Peano
|
|
Peano
|
Task fusion is one of the many magic knobs that can make a huge difference in your application performance and also is one key tool to faciliate GPU offloading.
This page dives into some design principles behind the task fusion and provides a high-level overview as well as programming guidelines. Details on how the fusion is implemented are provided in the documentation of the namespace tarch::multicore::taskfusion. From there, you can also navigate to a high-level overview of what task fusion strategies we have on the table, which is all content available from tarch::multicore::orchestration::FusionImplementation. This page however is written for programmers who want to use/adopt the task fusion within their own code.
To be able to fuse tasks, we have to assign tasks are unique type number. This is not the number of a task as we use it for dependency management, but one that identifies its type. A very simple way to do this is to create a static marker either within the task class or somewhere in your code that is set once and then used for all instances of this task.
While it is possible to hardcode a task number, it is not particularly elegant and reasonable: If you later combine two different applications, you have to be absolutely sure that they do not use the same type number for different tasks. It is safer to rely on the peano4::parallel::getTaskType() factory method to produce unique identifiers.
This code snippet spawns a task passing in a functor and hence mirrors code as you would write it in TBB or plain C++. The interesting thing in the present snippet is the argument residualTaskType. By default, we would pass in tarch::multicore::Task::DontFuse. As we hand over a valid id, we implicitly signal to the runtime that many tasks with this functor will be spawned, and the runtime should try to fuse them into one big task if possible.
In a proper task class, you can make the task type a static field. ExaHyPE for example gives each enclave task an attribute
by default. It then initialises this similar to
For each task class, we now have a dedicated task type number. We can also use this one outside of the class later on for the orchestration (see below).
Once a task has a valid task type id, its tarch::multicore::Task::canFuse() will return true and hence permit task fusion. If you want, you can still overwrite this routine and disable task fusion on a per-task level, i.e. even though a task might have a valid type which labels it as fusable in principle, nothing stops a user from labelling specific tasks as not fusable, i.e. to veto any fusion.
As soon as your task type is not tarch::multicore::Task::DontFuse, the task's canFuse() routine will return true (unless you overwrite it again manually in the case where you have an explicit class type). Consequently, Peano's technical architecture will embed the spawned tasks into a queue and try to fuse the tasks with the same type id if there are many of the same type ready.
The task orchestration is controlled through tarch::multicore::setOrchestration(). It sets a global instane of tarch::multicore::orchestration::Strategy which determines from hereon how the task fusion (and overall orchestration) behaves.
From a task fusion point of view, there are only two functions relevant in the orchestration:
In short, the first function of the orchestration determines how the task fusion is realised, while the second steers which tasks are offloaded when.
The default orchestration is determined through tarch::multicore::orchestration::createDefaultStrategy(). It should also ready enable fusion.
You can reset the strategy manually. Most extension's Python interfaces provide a routine for this (e.g. ExaHyPE's exahype2.project.Project.set_multicore_orchestration()).
Before you continue, it is key to validate that the fusion in principle is working. It makes no sense to directly start performance assessments or even tuning. For this, switch to Peano's statistics mode. In Python, it is set through
but most people favour settting it through a string argument and to offer it to a script user to set it externally:
Once the statistics mode is switched on, we can track how many tasks there are, on average, in the queue: We follow the guidelines on logging and statistics:
| Column | Counter |
|---|---|
| 0 | time (always used to plot) |
| 1 | exahype2::EnclaveBookkeeping::memory-allocations |
| 2 | mpi wait times |
| 3 | tarch::multicore::BSPConcurrencyLevelStatisticsIdentifier |
| 4 | tarch::multicore::SpawnedFusableTasksStatisticsIdentifier |
| 5 | tarch::multicore::SpawnedTasksStatisticsIdentifier |
| 6 | tarch::multicore::taskfusion::DirectlyInsertFusedTaskIntoReadyQueueStatisticsIdentifier(0) |
| 7 | tarch::multicore::taskfusion::FusableTasksQueue::SizeOfFusableTaskQueueStatisticsIdentifier(0) |
| 8 | tarch::multicore::taskfusion::ProcessOneReadyTask::WaitedForOneReadyTaskFromFusableTasksStatisticsIdentifier(0) |
| 9 | tarch::multicore::taskfusion::ProcessSetOfReadyTasks::ProcessFusedTasksStatisticsIdentifier(0) |
| 10 | grid-control-events |
This table tells us which column numbers we have to pick per metric that we study. We next will refer to a series of plots of certain metrics, which we create through calls similar to
We pass in a 3 here, but would refer to entry from column 3 in the table above. The values in brackets in the table above allow you to distinguish different task types btw. In the present example, there is only one task type in the code that can be fused.
A first check crossvalidates the BSP levels. Plotting the quantity tarch::multicore::BSPConcurrencyLevelStatisticsIdentifier gives us an overview of the fork-join parallelism at any point in time. Please consult the documentation of this flag and check that there's the correct number of fork-join tasks, i.e. subpartitions per rank, such that we produce tasks sufficiently quickly. For the task fusion, it is important to cross-validate that we work with a reasonable number of BSP partitions, such that tasks are produced quickly enough.
We next check tarch::multicore::SpawnedTasksStatisticsIdentifier. If no tasks are spawned, we do not have to continue to study any fusion. It is convenient to plot tarch::multicore::SpawnedFusableTasksStatisticsIdentifier at the same time, as we are really interested in fusable tasks only.
If the plot resembles something alike
then all is fine: Here, we issue a lot of tasks, and all of them are actually also fusable. It is ok if there's fewer fusable tasks, but there should be a healthy number, so we can actually fuse.
Finally, we plot the following quantities (maybe all in one go by picking column indicies separated by columns, i.e. similar to 1,2,3,8):
The flags tarch::multicore::taskfusion::MapFusedTaskOntoNativeTaskStatisticsIdentifier, tarch::multicore::taskfusion::DirectlyInsertFusedTaskIntoReadyQueueStatisticsIdentifier and tarch::multicore::taskfusion::LogReadyTask::SubmitFusedTasksStatisticsIdentifier. clarify how the fused tasks have been handled. Their sum should equal what we get out from tarch::multicore::SpawnedFusableTasksStatisticsIdentifier.
In the example above, we see that a lot of tasks are spawned. All of these tasks are also fusable, as the red and green triangles overlap. Unfortunately, all fusable tasks are then directly mapped onto native tasks. This indicates that the chosen task orchestration does not work. Likely, an unfortunate orchestration has been chosen.
The quantity tarch::multicore::taskfusion::ProcessOneReadyTask::ProcessOneReadyTaskFromFusableTasksStatisticsIdentifier tells us how many fusable tasks have been manually removed from the fusable queue, as the runtime requested the task outcome asap. Along the same lines, tarch::multicore::taskfusion::ProcessOneReadyTask::WaitedForOneReadyTaskFromFusableTasksStatisticsIdentifier tracks how often the runtime tried to extract a task from the queue but it had, at that point, already been processed by someone else. Both counters should be low compared to the total number of tasks as explained in their documentation.
In the example above, all outcomes are rather promising: We see that all spawned tasks are also fusable, and that they are all inserted directly into the ready queue. These three symbols overlap. Every now and then, the runtime has to wait for some ready task, but that happens infrequently, which means the fusion is kicking in most of the time.
Of particular relevance in practice are tarch::multicore::taskfusion::FusableTasksQueue::SizeOfFusableTaskQueueStatisticsIdentifier and tarch::multicore::taskfusion::ProcessSetOfReadyTasks::ProcessFusedTasksStatisticsIdentifier. They track the size of the fused queue and how many tasks have been extracted per fusion.
In the example above, we inserted around 250 tasks into the queue, which then immediately got fused into one big meta task hosting 250 tasks. Nothing happened after that for 60s, before we enter a certain sequence. Per step within this sequence, we insert around 250 tasks, which then seemed to be grabbed always in one go.
One of the purposes of task fusion - besides a reduction of overheads - is the capability to introduce bespoke compute kernels that handle a whole set of tasks instead of a single one. They might even take them and ship them to a GPU. For this, you have to overwrite tarch::multicore::Task::fuse(). The default is just a loop over tasks.
In our latest write-up, we call these fusion kernels executor.
The default orchestration strategies do not distinguish different task types and they are not particularly bespoke. In most cases, you will have to tweak them to get good performance. Below are a few natural tweaks. To implement them, I recommend to create a new subclass of tarch::multicore::orchestration::Strategy based upon an existing default strategy and to amend it to your needs.
An example of such a bespoke, tailored orchestration is mghype::MultigridDefaultMulticoreOrchestration.
Whenever tarch::multicore::taskfusion::ProcessReadyTask::run() kicks off, it invokes updateFuseStatistics() on the orchestration. We receive
This allows to make statements on the load, as the stats are only updated if the underlying tasks is not processed yet.
Let the task return instruction.maxTasks=0 to disable the fusion completely. In this case, the fusable tasks are spawned as native tasks. This reduces overhead, but has to be used carefully: On the one hand, tasks will not enter their fuse() operation anymore. On the other hand, task queue stats (size of queues, average size of queues, process statistics) won't be updated anymore.
If the average number of ready tasks in the queue is higher than the max fused tasks every time a process ready task is invoked, the handling of the fused tasks is too slow. Again, we have two options.
We could start to spawn more tasks as native tasks on the host. The CPU will be "more busy" with work, and the fused processing will get fewer tasks overall to process. In return, they have more time to handle their bunch of tasks, e.g. on an accelerator.
If you have only one single type of tasks, decisions are easy. However, most codes feature different (fusable) task types and hence want to make different fusion decisions depending on the type. In this case, your fuse() routine resembles
This is obviously bad style. It is way more elegant to use static values of classes rather than hard-coded constants.
If you don't know the task numbers, you have to use a solution similar to the one in mghype::MultigridDefaultMulticoreOrchestration:
1.9.8