This stack hosts data of constant size which are stored on the heap and subject to smart pointers. More...

#include <STDVectorStackOverSmartPointers.h>

Collaboration diagram for peano4::stacks::STDVectorStackOverSmartPointers< T >:

[legend]

Data Structures
class	PopBlockStackView
	This class represents a whole block of the tree. More...

class	PushBlockStackView

Public Member Functions
	STDVectorStackOverSmartPointers ()

	STDVectorStackOverSmartPointers (const STDVectorStackOverSmartPointers< T > &)

STDVectorStackOverSmartPointers< T > &	operator= (const STDVectorStackOverSmartPointers< T > &stack)
	One is allowed to copy a stack but it has to be empty.

void	clone (const STDVectorStackOverSmartPointers< T > &data)
	Clone data into the current object on which clone() is called.

T	pop ()
	Pops element from a stack.

T &	top (int shift=0)
	Get top element or shiftth top element.

const T &	top (int shift=0) const
	Get top element or shiftth top element.

void	push (const T &element)
	Pushes element to a stack.

PopBlockStackView	popBlock (int numberOfElements)
	This operation grabs numberOfElements from the input stack en block and returns a view to it.

PushBlockStackView	pushBlock (int numberOfElements)
	Push a block on the output stack.

int	size () const

bool	empty () const

void	clear ()

void	startSend (peano4::grid::TraversalObserver::SendReceiveContext context, int rank, int tag, MPI_Comm comm)
	Start the actual send process.

void	startReceive (peano4::grid::TraversalObserver::SendReceiveContext context, int rank, int tag, MPI_Comm comm, int numberOfElements)
	Counterpart of startSend()

bool	tryToFinishSendOrReceive ()
	Semantics are alost an exact 1:1 copy of the superclass implementation.

int	sendingOrReceiving () const
	I need this one to find out whether I'm waiting for data.

void	reverse ()
	Reversing a stream is something I need extremely rarely.

std::string	toString () const

Private Member Functions
void	gather ()
	Allocate buffer and gather.

void	prepareGatheredDataBuffer ()
	Mere alloc.

void	scatter ()
	Scatter received gathered data.

void	gatherMetaData ()
	Allocate buffer and gather.

void	prepareMetaDataBuffers ()
	Mere alloc.

void	scatterDebugMetaData ()

Private Attributes
std::vector< T >	_data
	This is the attribute holding all the temporary stacks.

T::DoFType *	_gatheredData
	See MPI routines below.

int *	_gatheredMetaData
	Meta data with size of memory chunks.

double *	_gatheredDebugMetaData
	Debug data.

IOMode	_ioMode

int	_ioRank

int	_ioTag

MPI_Request *	_ioMPIRequest

MPI_Request *	_ioMetaDataMPIRequest

MPI_Request *	_ioDebugMetaDataMPIRequest

Static Private Attributes
static tarch::logging::Log	_log
	Logging device.

Detailed Description

template<class T>
class peano4::stacks::STDVectorStackOverSmartPointers< T >

This stack hosts data of constant size which are stored on the heap and subject to smart pointers.

We assume that that the object stored on the stack

all point to a memory region of the same size
offer a field T::Cardinality which tells us what this size is
defines a native pointer value which points to actual data

Once we work with smart pointers, we can work with shallow copies as data travel over the individual stacks. This implies that we "only" have to make the clones for the data exchange between threads and ranks actual deep copies.

Special care is however required if we work with solver localisation, i.e. solvers which do not store data in each and every grid entity. If a solver decides not to store a certain piece of data, the underlying value pointer will exist but will point to nullptr. So we may not attempt to deep copy it, but instead need to ensure that the cloned vector will hold nullptr, too.

MPI

One tricky aspect in the context of smart pointers is actually the MPI data exchange which has to take into account that data are scattered over the heap. See the documentation of the send/receive operations. They discuss how the business with the required deep copies actually works. Anyway, to make it work, we need the used datatype T to have two fields:

A type expression DoFType;
A constexpr int Cardinality.

The other tricky thing is that some mesh entities might hold nullptrs. We now have two options:

We actually only flatten the non-nullptr data.
We send out a huge array with "all" flattened data and take into account that some data within this array hold garbage, as the actual data actually dd point to nullptr.

We go down the latter route. It would be no problem to squeeze out areas in the flattened/gathered area which correspond to nullptr. Yet, if we do so we do not know a priori how big the outgoing data will be. This implies that we have to send out the data in two chunks: First the meta data and then we can trigger the corresponding receive call with a matching size. This is too complicated for me. I just send out everything - including rubbish.

Comparison to normal STD stack

The normal stack works with the STL stack yet never shrinks it. This is to avoid too many memory relocations. Instead, it holds an integer counter which always clarifies how many elements in the STL container hole valid data.

The present container works with the "native" STL container.

Parameters

T	Hosts usually pointer which is backed up by a smart pointer, i.e. deleted as soon as noone needs it anymore. T has to provide a clone() operation.

Definition at line 92 of file STDVectorStackOverSmartPointers.h.

Constructor & Destructor Documentation

◆ STDVectorStackOverSmartPointers() [1/2]

template<class T >

peano4::stacks::STDVectorStackOverSmartPointers< T >::STDVectorStackOverSmartPointers ( )

◆ STDVectorStackOverSmartPointers() [2/2]

template<class T >

peano4::stacks::STDVectorStackOverSmartPointers< T >::STDVectorStackOverSmartPointers ( const STDVectorStackOverSmartPointers< T > & )

Member Function Documentation

◆ clear()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::clear ( )

Referenced by peano4.api.output.Makefile.Makefile::__init__().

Here is the caller graph for this function:

◆ clone()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::clone ( const STDVectorStackOverSmartPointers< T > & data )

Clone data into the current object on which clone() is called.

This operation overwrites the baseclass version. If we called the plain base class version, we would end up with a shallow copy. However, we want a semi-shallow copy: We replicate the meta data, and then we call the clone() of the actual vector over pointers. It is up to this routine if we craete a shallow copy or not.

Read the class documentation why I think that the clone() on the subvectors should create a deep copy. It is the section on intra-node parallelisation.

This operation has to use the default constructor of T to resize, as we want the destination data structure to be able to host the target data. So it has to issue all the news if required.

◆ empty()

template<class T >

bool peano4::stacks::STDVectorStackOverSmartPointers< T >::empty ( ) const

◆ gather()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::gather ( )

private

Allocate buffer and gather.

◆ gatherMetaData()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::gatherMetaData ( )

private

Allocate buffer and gather.

◆ operator=()

template<class T >

STDVectorStackOverSmartPointers< T > & peano4::stacks::STDVectorStackOverSmartPointers< T >::operator= ( const STDVectorStackOverSmartPointers< T > & stack )

One is allowed to copy a stack but it has to be empty.

Definition at line 189 of file STDVectorStackOverSmartPointers.h.

References peano4::stacks::STDVectorStackOverSmartPointers< T >::_data, and assertion.

◆ pop()

template<class T >

T peano4::stacks::STDVectorStackOverSmartPointers< T >::pop ( )

Pops element from a stack.

I have to be careful how and why I use std::move here, as we employ smart pointers. I found that a move after the return confuses the reference counting, and eventually leads to memory leaks.

◆ popBlock()

template<class T >

PopBlockStackView peano4::stacks::STDVectorStackOverSmartPointers< T >::popBlock ( int numberOfElements )

This operation grabs numberOfElements from the input stack en block and returns a view to it.

Subsequent pops do not affect this block anymore, i.e. the stack is reduced immediately.

Parameters

numberOfElements Size of the view

◆ prepareGatheredDataBuffer()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::prepareGatheredDataBuffer ( )

private

Mere alloc.

We use the tarch's allocation. Therefore, we can have aligned data here. Without the tarch, the code call would be

    _gatheredData     = new typename T::DoFType[ size() * T::Cardinality ];

◆ prepareMetaDataBuffers()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::prepareMetaDataBuffers ( )

private

Mere alloc.

We use the tarch's allocation. Therefore, we can have aligned data here. Without tarch, the call would be

    _gatheredDebugMetaData = new double[ Dimensions*2*size() ];

◆ push()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::push ( const T & element )

Pushes element to a stack.

◆ pushBlock()

template<class T >

PushBlockStackView peano4::stacks::STDVectorStackOverSmartPointers< T >::pushBlock ( int numberOfElements )

Push a block on the output stack.

Parameters

numberOfElements Size of the view

◆ reverse()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::reverse ( )

Reversing a stream is something I need extremely rarely.

The biggest application is the realisation of joins through peano4::parallel::SpacetreeSet::streamLocalVertexInformationToMasterThroughVerticalStacks(). Here, I need a streamed version of the tree to get the up-to-date data of the mesh in. However, I don't have streams. I have only stacks. So I map the stream idea to a stack.

◆ scatter()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::scatter ( )

private

Scatter received gathered data.

We run over the size keeping in mind that the target has been created over objects with the constructor argument T::ObjectConstruction::NoData. That is, each entry hosts a pointer, but this pointer will currently point to nullptr, as the object is not yet prepared to host any data. We run over the meta data to find out if this is ok, or if we actually want this stack entry to host data. If so, we use the "real" constructor and then copy the raw content over from the gathered data set.

◆ scatterDebugMetaData()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::scatterDebugMetaData ( )

private

◆ sendingOrReceiving()

template<class T >

int peano4::stacks::STDVectorStackOverSmartPointers< T >::sendingOrReceiving ( ) const

I need this one to find out whether I'm waiting for data.

Returns: A negative value if we don't send or receive anymore. Otherwise, we return the rank of the communication partner.

◆ size()

template<class T >

int peano4::stacks::STDVectorStackOverSmartPointers< T >::size ( ) const

Referenced by tarch.plotter.api.input.Patch.Patch::__repr__(), and peano4::stacks::STDVectorStackOverSmartPointers< T >::PopBlockStackView::PopBlockStackView().

Here is the caller graph for this function:

◆ startReceive()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::startReceive	(	peano4::grid::TraversalObserver::SendReceiveContext	context,
		int	rank,
		int	tag,
		MPI_Comm	comm,
		int	numberOfElements
	)

Counterpart of startSend()

We have to prepare the en bloc, raw data buffers for both the data and the meta data (if required). After that, we can trigger the two non-blocking receives.

See clone() for a description why we have to use resize() with the standard constructor here.

See also: startSend()

◆ startSend()

template<class T >

void peano4::stacks::STDVectorStackOverSmartPointers< T >::startSend	(	peano4::grid::TraversalObserver::SendReceiveContext	context,
		int	rank,
		int	tag,
		MPI_Comm	comm
	)

Start the actual send process.

Always pairs up with a finish... call.

Gathering the data

As we have to assume that the data of interest are scattered over the main memory, we first allocate one big buffer for the data. This is a buffer over T::DoFType. We then copy the chunks over and send this block rather than the "original" stack data.

The meta data has to be treated (almost) in the same way: In this case, we create manually a buffer over 2 * Dimensions * no of elements doubles and then copy the meta data over one by one. That means: we do not expect T's datatype to encapsulate the meta data - which also doesn't make any sense as T's data itself is scattered over the memory since it works with pointers.

Cleaning up

All clean ups are realised within tryToFinishSendOrReceive() where I have to delete the gathered buffer. To save memory overall, I can delete the pointers to the data scattered over the memory straightaway once all is copied over. As these pointers are shared smart pointers, this immediate delete might free memory as the shared pointer counter reduces to zero.

◆ top() [1/2]

template<class T >

T & peano4::stacks::STDVectorStackOverSmartPointers< T >::top ( int shift = 0 )

Get top element or shiftth top element.

We start to count with 0, i.e. a shift of 0 (default) really returns the top element. A shift of 3 returns the fourth element from the stack

◆ top() [2/2]

template<class T >

const T & peano4::stacks::STDVectorStackOverSmartPointers< T >::top ( int shift = 0 ) const

Get top element or shiftth top element.

We start to count with 0, i.e. a shift of 0 (default) really returns the top element. A shift of 3 returns the fourth element from the stack

◆ toString()

template<class T >

std::string peano4::stacks::STDVectorStackOverSmartPointers< T >::toString ( ) const

◆ tryToFinishSendOrReceive()

template<class T >

bool peano4::stacks::STDVectorStackOverSmartPointers< T >::tryToFinishSendOrReceive ( )

Semantics are alost an exact 1:1 copy of the superclass implementation.

Clean up

If a send or receive had been pending and just completed, we start the actual clean up process.

As we know that MPI sends in order, we do not have to test for the meta data. Once the real data has dropped in, we know that the meta data are in place, too.

Rationale

I originally wanted to call the superclass and just free the buffer if its result is true. However, at the time when the result comes back, I do not know if it has just switched from false to true or had been true all along. To avoid complicated logic, I decided to replicate the parent data and to inject the free into the replicated code.

Field Documentation

◆ _data

template<class T >

std::vector<T> peano4::stacks::STDVectorStackOverSmartPointers< T >::_data

private

This is the attribute holding all the temporary stacks.

Definition at line 102 of file STDVectorStackOverSmartPointers.h.

Referenced by peano4.api.datamodel.DaStGen2.DaStGen2Generator::construct_output(), peano4.api.datamodel.DaStGen2.DaStGen2GeneratorForObjectsWithSmartPointers::construct_output(), peano4.api.toolbox.particles.ParticleSet.ParticleSetGenerator_ScatteredOnHeap_IndexByList::construct_output(), peano4.api.toolbox.particles.ParticleSet.ParticleSetGenerator_ScatteredOnHeap_IndexByVector::construct_output(), peano4.api.toolbox.particles.ParticleSet.ParticleSetGenerator_ContinuousPerVertex::construct_output(), peano4.api.toolbox.particles.ParticleSet.ParticleSetGenerator_GlobalContinuous::construct_output(), peano4.api.datamodel.DaStGen2.DaStGen2Generator::get_header_file_include(), peano4.api.datamodel.DaStGen2.DaStGen2GeneratorForObjectsWithSmartPointers::get_header_file_include(), peano4.api.toolbox.particles.ParticleSet.AbstractParticleSetGenerator::get_header_file_include(), peano4.api.datamodel.DaStGen2.DaStGen2Generator::get_stack_container(), peano4.api.datamodel.DaStGen2.DaStGen2GeneratorForObjectsWithSmartPointers::get_stack_container(), peano4.api.datamodel.DynamicArrayOverPrimitivesToStdVector.DynamicArrayOverPrimitivesToStdVector::get_stack_container(), peano4.api.toolbox.particles.ParticleSet.AbstractParticleSetGenerator::get_stack_container(), peano4::stacks::STDVectorStackOverSmartPointers< T >::operator=(), and peano4::stacks::STDVectorStackOverSmartPointers< T >::PopBlockStackView::PopBlockStackView().

◆ _gatheredData

template<class T >

T::DoFType* peano4::stacks::STDVectorStackOverSmartPointers< T >::_gatheredData

private

See MPI routines below.

This routine holds the actual data in a continuous way.

Definition at line 109 of file STDVectorStackOverSmartPointers.h.

◆ _gatheredDebugMetaData

template<class T >

double* peano4::stacks::STDVectorStackOverSmartPointers< T >::_gatheredDebugMetaData

private

Debug data.

Is only used in debug mode to store the x and h values tied to each entry on the stack.

Definition at line 126 of file STDVectorStackOverSmartPointers.h.

◆ _gatheredMetaData

template<class T >

int* peano4::stacks::STDVectorStackOverSmartPointers< T >::_gatheredMetaData

private

Meta data with size of memory chunks.

The entries here hold either T::Cardinality or zero. In the latter case, any copy should reconstruct an entry with a nullptr. The entries effectively cluster _gatheredData into chunks.

Definition at line 118 of file STDVectorStackOverSmartPointers.h.