See page DSL Syntax for details on writing the timestep function for a new kernel.

This page details how to use this function in a solver. The implementation of the Rusanov kernel is used as an example throughout.

Generating C++ Functions

Each category of solver uses a python script to generate any required code, usually a file called kernels.py. For the Rusanov kernel this file is python/exahype2/solvers/fv/rusanov/kernels.py. Create a function in this file which is used to generate the C++ files from the DSL. Since the timestep function must be specified for each dimension, generating a base header file which checks the dimension is useful.

def create_rusanov_kernel_definitions():
  if not os.path.exists("kernels"):
      os.makedirs("kernels")
  
  file = open("kernels/rusanov.h", "w")
  file.write("""#if Dimensions == 2
#include "rusanov2d.h"
#elif Dimensions == 3
#include "rusanov3d.h"
#endif
""")
  
  create_rusanov_kernel_definitions_2d()
  create_rusanov_kernel_definitions_3d()

The create_kernel_definitions_2d() and create_kernel_definitions_3d() should usually be very similar, so only one with unspecified dimension is described here.

The create_kernel_definitions() function has three main parts: setting template and function arguments, generating strings containing the C++ code, and writing to files.

Setting Template and Function Parameters

The template parameters and function parameters are passed as individual lists to the parsing function. Each element of the list is of type SyntaxTree.Argument. The parameters used for the Rusanov kernels are

template_parameters = [DSL.SyntaxTree.Argument("NumberOfVolumesPerAxisInPatch", DSL.SyntaxTree.TInteger()),
                         DSL.SyntaxTree.Argument("HaloSize", DSL.SyntaxTree.TInteger()),
                         DSL.SyntaxTree.Argument("NumberOfUnknowns", DSL.SyntaxTree.TInteger()),
                         DSL.SyntaxTree.Argument("NumberOfAuxiliaryVariables", DSL.SyntaxTree.TInteger()),
                         DSL.SyntaxTree.Argument("EvaluateFlux", DSL.SyntaxTree.TBoolean()),
                         DSL.SyntaxTree.Argument("EvaluateNonconservativeProduct", DSL.SyntaxTree.TBoolean()),
                         DSL.SyntaxTree.Argument("EvaluateSource", DSL.SyntaxTree.TBoolean()),
                         DSL.SyntaxTree.Argument("EvaluateMaximumEigenvalueAfterTimeStep", DSL.SyntaxTree.TBoolean())]
 
  functor_arguments = [DSL.SyntaxTree.Argument("flux", DSL.SyntaxTree.TCustom("const FluxFunctor&")),
                       DSL.SyntaxTree.Argument("nonconservativeProduct", DSL.SyntaxTree.TCustom("const NonconservativeProductFunctor&")),
                       DSL.SyntaxTree.Argument("sourceTerm", DSL.SyntaxTree.TCustom("const SourceFunctor&")),
                       DSL.SyntaxTree.Argument("maxEigenvalue", DSL.SyntaxTree.TCustom("const MaxEigenvalueFunctor&"))]

Generating Function Strings

A Python string is generated using Parser().parse(timestepFunction, template_parameters, functor_arguments, namespaces, is_stateless, use_accelerator) Its parameters are:

timestepFunction: the function that defines the kernel
template_parameters: as previously defined
functor_arguments: as previously defined
namespaces: a list of strings, each specifying a layer of a namespace. The generated function is defined to be within this namespace.
is_stateless: a boolean which defaults to False. If the generated code is stateless, a template parameter is added at the start to specify a solver. Functor calls are removed from the function parameters, and calls to them are replaced with calls to static alternatives.
use_accelerator: a boolean which defaults to False. If True, an integer targetDevice parameter is added.

Calling this function returns the root node to an abstract syntax tree representing the kernel. The C++ code can then be generated by calling its print methods. The options currently are print_cpp(), print_omp() and print_tree(), which does not produce valid code and is for debugging only. The FunctionDefinition node, which is the root node, also has functions for generating declarations for code organisation (print_declaration()). It also can generate functions with a timer for benchmarking (print_declaration_with_timer(), print_definition_with_timer()).

Writing to Files

From here the process is standard Python. For consistency the generated files should be within a kernels folder. It is also recommended to use separate header files for declarations, and separate files for different dimensions. Certain include files are also required as seen in the following example.

if not os.path.exists("kernels"):
      os.makedirs("kernels")
  
  file = open("kernels/rusanov3d.h", "w")
  file.write(f"""#pragma once
#include "exahype2/CellData.h"
#include "exahype2/VolumeIndex.h"
#include "exahype2/fv/PatchUtils.h"
#include "exahype2/fv/rusanov/Functors.h"
#include "peano4/utils/Loop.h"
#include "tarch/timing/Measurement.h"
#include "tarch/timing/Watch.h"
             
{rusanov_kernel_declaration}
{rusanov_call_with_measurement_declaration}
{rusanov_stateless_kernel_declaration}
{rusanov_stateless_call_with_measurement_declaration}
{rusanov_accelerator_kernel_declaration}
{rusanov_accelerator_call_with_measurement_declaration}
 
#if defined(GPUOffloadingOMP)
{rusanov_omp_accelerator_kernel_declaration}
{rusanov_omp_accelerator_call_with_measurement_declaration}
#endif
 
#include "rusanov3d.cpph"
""")
  file.close()

Generating C++ Function Calls

The solver also needs to know how to call the new kernels, which should also be defined in kernels.py. A function is defined which returns a Python string containing the C++ code used to call the timestep function. It checks which version of the function is required, such as whether its stateless or uses an accelerator and returns the relevant call.

If the code isn't stateless there are usually functor parameters which have to be passed in. There are also template parameters which need to be determined which specify whether certain parts of the kernel are required. This is best seen by example in create_compute_Riemann_kernel_for_Rusanov_dsl() in kernels.py.