ParallelStudy< WorkType, ParallelDataType > Class Template Reference

#include <ParallelStudy.h>

Inheritance diagram for ParallelStudy< WorkType, ParallelDataType >:

Public Types
typedef MooseUtils::Buffer< WorkType >::iterator	work_iterator
typedef MooseUtils::Buffer< std::shared_ptr< ParallelDataType > >::iterator	parallel_data_iterator

Public Member Functions
	ParallelStudy (const libMesh::Parallel::Communicator &comm, const InputParameters &params, const std::string &name)
void	preExecute ()
	Pre-execute method that MUST be called before execute() and before adding work. More...
void	execute ()
	Execute method. More...
template<typename... Args>
MooseUtils::SharedPool< ParallelDataType >::PtrType	acquireParallelData (const THREAD_ID tid, Args &&... args)
	Acquire a parallel data object from the pool. More...
void	moveParallelDataToBuffer (std::shared_ptr< ParallelDataType > &data, const processor_id_type dest_pid)
	Moves parallel data objects to the send buffer to be communicated to processor dest_pid. More...
const ReceiveBuffer< ParallelDataType, ParallelStudy< WorkType, ParallelDataType > > &	receiveBuffer () const
	Gets the receive buffer. More...
const MooseUtils::Buffer< WorkType > &	workBuffer () const
	Gets the work buffer. More...
unsigned long long int	sendBufferPoolCreated () const
	Gets the total number of send buffer pools created. More...
unsigned long long int	parallelDataSent () const
	Gets the total number of parallel data objects sent from this processor. More...
unsigned long long int	buffersSent () const
	Gets the total number of buffers sent from this processor. More...
unsigned long long int	poolParallelDataCreated () const
	Gets the total number of parallel data created in all of the threaded pools. More...
unsigned long long int	localWorkStarted () const
	Gets the total amount of work started from this processor. More...
unsigned long long int	localWorkExecuted () const
	Gets the total amount of work executed on this processor. More...
unsigned long long int	totalWorkCompleted () const
	Gets the total amount of work completeed across all processors. More...
unsigned long long int	localChunksExecuted () const
	Gets the total number of chunks of work executed on this processor. More...
bool	currentlyExecuting () const
	Whether or not this object is currently in execute(). More...
bool	currentlyPreExecuting () const
	Whether or not this object is between preExecute() and execute(). More...
unsigned int	maxBufferSize () const
	Gets the max buffer size. More...
unsigned int	chunkSize () const
	Gets the chunk size. More...
unsigned int	clicksPerCommunication () const
	Gets the number of iterations to wait before communicating. More...
unsigned int	clicksPerRootCommunication () const
	Gets the number of iterations to wait before communicating with root. More...
unsigned int	clicksPerReceive () const
	Gets the number of iterations to wait before checking for new parallel data. More...
ParallelStudyMethod	method () const
	Gets the method. More...
void	reserveBuffer (const std::size_t size)
	Reserve size entries in the work buffer. More...
const Parallel::Communicator &	comm () const
processor_id_type	n_processors () const
processor_id_type	processor_id () const
void	moveWorkToBuffer (WorkType &work, const THREAD_ID tid)
	Adds work to the buffer to be executed. More...
void	moveWorkToBuffer (const work_iterator begin, const work_iterator end, const THREAD_ID tid)
void	moveWorkToBuffer (std::vector< WorkType > &work, const THREAD_ID tid)

Static Public Member Functions
static InputParameters	validParams ()

Protected Types
enum	MoveWorkError {   DURING_EXECUTION_DISABLED, PRE_EXECUTION_AND_EXECUTION_ONLY, PRE_EXECUTION_ONLY, PRE_EXECUTION_THREAD_0_ONLY,   CONTINUING_DURING_EXECUTING_WORK }
	Enum for providing useful errors during work addition in moveWorkError(). More...

Protected Member Functions
virtual std::unique_ptr< MooseUtils::Buffer< WorkType > >	createWorkBuffer ()
	Creates the work buffer. More...
virtual void	executeWork (const WorkType &work, const THREAD_ID tid)=0
	Pure virtual to be overridden that executes a single object of work on a given thread. More...
virtual void	moveWorkError (const MoveWorkError error, const WorkType *work=nullptr) const
	Virtual that allows for the customization of error text for moving work into the buffer. More...
virtual bool	alternateSmartEndingCriteriaMet ()
	Insertion point for derived classes to provide an alternate ending criteria for SMART execution. More...
virtual void	postExecuteChunk (const work_iterator, const work_iterator)
	Insertion point for acting on work that was just executed. More...
virtual void	preReceiveAndExecute ()
	Insertion point called just after trying to receive work and just before beginning work on the work buffer. More...
virtual void	postReceiveParallelData (const parallel_data_iterator begin, const parallel_data_iterator end)=0
	Pure virtual for acting on parallel data that has JUST been received and filled into the buffer. More...
virtual bool	workIsComplete (const WorkType &)
	Can be overridden to denote if a piece of work is not complete yet. More...
bool	buffersAreEmpty () const
	Whether or not ALL of the buffers are empty: Working buffer, threaded buffers, receive buffer, and send buffers. More...
void	moveContinuingWorkToBuffer (WorkType &Work)
	Moves work that is considered continuing for the purposes of the execution algorithm into the buffer. More...
void	moveContinuingWorkToBuffer (const work_iterator begin, const work_iterator end)

Protected Attributes
const processor_id_type	_pid
	This rank. More...
const std::string	_name
	Name for this object for use in error handling. More...
const InputParameters &	_params
	The InputParameters. More...
const ParallelStudyMethod	_method
	The study method. More...
bool	_has_alternate_ending_criteria
	Whether or not this object has alternate ending criteria. More...
const Parallel::Communicator &	_communicator

Private Member Functions
void	flushSendBuffers ()
	Flushes all parallel data out of the send buffers. More...
void	smartExecute ()
	Execute work using SMART. More...
void	harmExecute ()
	Execute work using HARM. More...
void	bsExecute ()
	Execute work using BS. More...
bool	receiveAndExecute ()
	Receive packets of parallel data from other processors and executes work. More...
void	executeAndBuffer (const std::size_t chunk_size)
	Execute a chunk of work and buffer. More...
void	canMoveWorkCheck (const THREAD_ID tid)
	Internal check for if it is allowed to currently add work in moveWorkToBuffer(). More...
void	postReceiveParallelDataInternal ()
	Internal method for acting on the parallel data that has just been received into the parallel buffer. More...

Private Attributes
const unsigned int	_min_buffer_size
	Minimum size of a SendBuffer. More...
const unsigned int	_max_buffer_size
	Number of objects to buffer before communication. More...
const Real	_buffer_growth_multiplier
	Multiplier for the buffer size for growing the buffer. More...
const Real	_buffer_shrink_multiplier
	Multiplier for the buffer size for shrinking the buffer. More...
const unsigned int	_chunk_size
	Number of objects to execute at once during communication. More...
const bool	_allow_new_work_during_execution
	Whether or not to allow the addition of new work to the buffer during execution. More...
const unsigned int	_clicks_per_communication
	Iterations to wait before communicating. More...
const unsigned int	_clicks_per_root_communication
	Iterations to wait before communicating with root. More...
const unsigned int	_clicks_per_receive
	Iterations to wait before checking for new objects. More...
Parallel::MessageTag	_parallel_data_buffer_tag
	MessageTag for sending parallel data. More...
std::vector< MooseUtils::SharedPool< ParallelDataType > >	_parallel_data_pools
	Pools for re-using destructed parallel data objects (one for each thread) More...
std::vector< std::vector< WorkType > >	_temp_threaded_work
	Threaded temprorary storage for work added while we're using the _work_buffer (one for each thread) More...
const std::unique_ptr< MooseUtils::Buffer< WorkType > >	_work_buffer
	Buffer for executing work. More...
const std::unique_ptr< ReceiveBuffer< ParallelDataType, ParallelStudy< WorkType, ParallelDataType > > >	_receive_buffer
	The receive buffer. More...
std::unordered_map< processor_id_type, std::unique_ptr< SendBuffer< ParallelDataType, ParallelStudy< WorkType, ParallelDataType > > > >	_send_buffers
	Send buffers for each processor. More...
unsigned long long int	_local_chunks_executed
	Number of chunks of work executed on this processor. More...
unsigned long long int	_local_work_completed
	Amount of work completed on this processor. More...
unsigned long long int	_local_work_started
	Amount of work started on this processor. More...
unsigned long long int	_local_work_executed
	Amount of work executed on this processor. More...
unsigned long long int	_total_work_started
	Amount of work started on all processors. More...
unsigned long long int	_total_work_completed
	Amount of work completed on all processors. More...
bool	_currently_executing
	Whether we are within execute() More...
bool	_currently_pre_executing
	Whether we are between preExecute() and execute() More...
bool	_currently_executing_work
	Whether or not we are currently within executeAndBuffer() More...

Detailed Description

template<typename WorkType, typename ParallelDataType>
class ParallelStudy< WorkType, ParallelDataType >

Definition at line 29 of file ParallelStudy.h.

Member Typedef Documentation

parallel_data_iterator

template<typename WorkType, typename ParallelDataType>

typedef MooseUtils::Buffer<std::shared_ptr<ParallelDataType> >::iterator ParallelStudy< WorkType, ParallelDataType >::parallel_data_iterator

Definition at line 34 of file ParallelStudy.h.

work_iterator

template<typename WorkType, typename ParallelDataType>

typedef MooseUtils::Buffer<WorkType>::iterator ParallelStudy< WorkType, ParallelDataType >::work_iterator

Definition at line 32 of file ParallelStudy.h.

Member Enumeration Documentation

MoveWorkError

template<typename WorkType, typename ParallelDataType>

enum ParallelStudy::MoveWorkError

protected

Enum for providing useful errors during work addition in moveWorkError().

Enumerator
DURING_EXECUTION_DISABLED
PRE_EXECUTION_AND_EXECUTION_ONLY
PRE_EXECUTION_ONLY
PRE_EXECUTION_THREAD_0_ONLY
CONTINUING_DURING_EXECUTING_WORK

Definition at line 182 of file ParallelStudy.h.

  {
    DURING_EXECUTION_DISABLED,
    PRE_EXECUTION_AND_EXECUTION_ONLY,
    PRE_EXECUTION_ONLY,
    PRE_EXECUTION_THREAD_0_ONLY,
    CONTINUING_DURING_EXECUTING_WORK
  };

Constructor & Destructor Documentation

ParallelStudy()

template<typename WorkType , typename ParallelDataType >

ParallelStudy< WorkType, ParallelDataType >::ParallelStudy	(	const libMesh::Parallel::Communicator &	comm,
		const InputParameters &	params,
		const std::string &	name
	)

Definition at line 370 of file ParallelStudy.h.

  : ParallelObject(comm),
    _pid(comm.rank()),
    _name(name),
    _params(params),

    _method((ParallelStudyMethod)(int)(params.get<MooseEnum>("method"))),
    _has_alternate_ending_criteria(false),
    _min_buffer_size(params.isParamSetByUser("min_buffer_size")
                         ? params.get<unsigned int>("min_buffer_size")
                         : params.get<unsigned int>("send_buffer_size")),
    _max_buffer_size(params.get<unsigned int>("send_buffer_size")),
    _buffer_growth_multiplier(params.get<Real>("buffer_growth_multiplier")),
    _buffer_shrink_multiplier(params.get<Real>("buffer_shrink_multiplier")),
    _chunk_size(params.get<unsigned int>("chunk_size")),
    _allow_new_work_during_execution(params.get<bool>("allow_new_work_during_execution")),

    _clicks_per_communication(params.get<unsigned int>("clicks_per_communication")),
    _clicks_per_root_communication(params.get<unsigned int>("clicks_per_root_communication")),
    _clicks_per_receive(params.get<unsigned int>("clicks_per_receive")),

    _parallel_data_buffer_tag(comm.get_unique_tag()),
    _parallel_data_pools(libMesh::n_threads()),
    _temp_threaded_work(libMesh::n_threads()),
    _work_buffer(createWorkBuffer()),
    _receive_buffer(std::make_unique<
                    ReceiveBuffer<ParallelDataType, ParallelStudy<WorkType, ParallelDataType>>>(
        comm, this, _method, _clicks_per_receive, _parallel_data_buffer_tag)),

    _currently_executing(false),
    _currently_pre_executing(false),
    _currently_executing_work(false)
{
#ifndef LIBMESH_HAVE_OPENMP
  if (libMesh::n_threads() != 1)
    mooseWarning(_name, ": Threading will not be used without OpenMP");
#endif

  if (_method != ParallelStudyMethod::SMART && _allow_new_work_during_execution)
    mooseError(_name,
               ": When allowing new work addition during execution\n",
               "('allow_new_work_during_execution = true'), the method must be SMART");
}

Member Function Documentation

acquireParallelData()

template<typename WorkType, typename ParallelDataType>

template<typename... Args>

MooseUtils::SharedPool<ParallelDataType>::PtrType ParallelStudy< WorkType, ParallelDataType >::acquireParallelData ( const THREAD_ID  tid, Args &&...  args  )

inline

Acquire a parallel data object from the pool.

Definition at line 73 of file ParallelStudy.h.

  {
    return _parallel_data_pools[tid].acquire(std::forward<Args>(args)...);
  }

alternateSmartEndingCriteriaMet()

template<typename WorkType , typename ParallelDataType >

bool ParallelStudy< WorkType, ParallelDataType >::alternateSmartEndingCriteriaMet ( )

protectedvirtual

Insertion point for derived classes to provide an alternate ending criteria for SMART execution.

Only called when _has_alternate_ending_criteria == true.

Definition at line 1202 of file ParallelStudy.h.

{
  mooseError(_name, ": Unimplemented alternateSmartEndingCriteriaMet()");
}

bsExecute()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::bsExecute ( )

private

Execute work using BS.

Definition at line 900 of file ParallelStudy.h.

{
  if (_has_alternate_ending_criteria)
    mooseError("ParallelStudy: Alternate ending criteria not yet supported for BS");
  if (_allow_new_work_during_execution)
    mooseError(_name, ": The addition of new work during execution is not supported by BS");
  mooseAssert(_method == ParallelStudyMethod::BS, "Should be called with BS only");

  Parallel::Request work_completed_probe_status;
  Parallel::Request work_completed_request;

  // Temp for use in sending the current value in a nonblocking sum instead of an updated value
  unsigned long long int temp;

  // Get the amount of work that were started in the whole domain
  comm().sum(_local_work_started, _total_work_started, work_completed_probe_status);

  // Keep working until done
  while (true)
  {
    bool receiving = false;
    bool sending = false;

    Parallel::Request some_left_request;
    unsigned int some_left = 0;
    unsigned int all_some_left = 1;

    do
    {
      _receive_buffer->receive();
      postReceiveParallelDataInternal();
      flushSendBuffers();

      receiving = _receive_buffer->currentlyReceiving();

      sending = false;
      for (auto & send_buffer : _send_buffers)
        sending = sending || send_buffer.second->currentlySending() ||
                  send_buffer.second->currentlyBuffered();

      if (!receiving && !sending && some_left_request.test() && all_some_left)
      {
        some_left = receiving || sending;
        comm().sum(some_left, all_some_left, some_left_request);
      }
    } while (receiving || sending || !some_left_request.test() || all_some_left);

    executeAndBuffer(_work_buffer->size());

    comm().barrier();

    if (work_completed_probe_status.test() && work_completed_request.test())
    {
      if (_total_work_started == _total_work_completed)
        return;

      temp = _local_work_completed;
      comm().sum(temp, _total_work_completed, work_completed_request);
    }
  }
}

buffersAreEmpty()

template<typename WorkType , typename ParallelDataType >

bool ParallelStudy< WorkType, ParallelDataType >::buffersAreEmpty ( ) const

protected

Whether or not ALL of the buffers are empty: Working buffer, threaded buffers, receive buffer, and send buffers.

Definition at line 1209 of file ParallelStudy.h.

{
  if (!_work_buffer->empty())
    return false;
  for (const auto & threaded_buffer : _temp_threaded_work)
    if (!threaded_buffer.empty())
      return false;
  if (_receive_buffer->currentlyReceiving())
    return false;
  for (const auto & map_pair : _send_buffers)
    if (map_pair.second->currentlySending() || map_pair.second->currentlyBuffered())
      return false;

  return true;
}

buffersSent()

template<typename WorkType , typename ParallelDataType >

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::buffersSent

(

)

const

Gets the total number of buffers sent from this processor.

Definition at line 1178 of file ParallelStudy.h.

Referenced by PerProcessorRayTracingResultsVectorPostprocessor::execute().

{
  unsigned long long int total_sent = 0;

  for (const auto & buffer : _send_buffers)
    total_sent += buffer.second->buffersSent();

  return total_sent;
}

canMoveWorkCheck()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::canMoveWorkCheck ( const THREAD_ID  tid )

private

Internal check for if it is allowed to currently add work in moveWorkToBuffer().

Definition at line 1050 of file ParallelStudy.h.

{
  if (_currently_executing)
  {
    if (!_allow_new_work_during_execution)
      moveWorkError(MoveWorkError::DURING_EXECUTION_DISABLED);
  }
  else if (!_currently_pre_executing)
  {
    if (_allow_new_work_during_execution)
      moveWorkError(MoveWorkError::PRE_EXECUTION_AND_EXECUTION_ONLY);
    else
      moveWorkError(MoveWorkError::PRE_EXECUTION_ONLY);
  }
  else if (tid != 0)
    moveWorkError(MoveWorkError::PRE_EXECUTION_THREAD_0_ONLY);
}

chunkSize()

template<typename WorkType, typename ParallelDataType>

unsigned int ParallelStudy< WorkType, ParallelDataType >::chunkSize ( ) const

inline

Gets the chunk size.

Definition at line 148 of file ParallelStudy.h.

{ return _chunk_size; }

clicksPerCommunication()

template<typename WorkType, typename ParallelDataType>

unsigned int ParallelStudy< WorkType, ParallelDataType >::clicksPerCommunication ( ) const

inline

Gets the number of iterations to wait before communicating.

Definition at line 153 of file ParallelStudy.h.

{ return _clicks_per_communication; }

clicksPerReceive()

template<typename WorkType, typename ParallelDataType>

unsigned int ParallelStudy< WorkType, ParallelDataType >::clicksPerReceive ( ) const

inline

Gets the number of iterations to wait before checking for new parallel data.

Definition at line 161 of file ParallelStudy.h.

{ return _clicks_per_receive; }

clicksPerRootCommunication()

template<typename WorkType, typename ParallelDataType>

unsigned int ParallelStudy< WorkType, ParallelDataType >::clicksPerRootCommunication ( ) const

inline

Gets the number of iterations to wait before communicating with root.

Definition at line 157 of file ParallelStudy.h.

{ return _clicks_per_root_communication; }

createWorkBuffer()

template<typename WorkType , typename ParallelDataType >

std::unique_ptr< MooseUtils::Buffer< WorkType > > ParallelStudy< WorkType, ParallelDataType >::createWorkBuffer ( )

protectedvirtual

Creates the work buffer.

This is virtual so that derived classes can use their own specialized buffers

Definition at line 419 of file ParallelStudy.h.

{
  std::unique_ptr<MooseUtils::Buffer<WorkType>> buffer;

  const auto buffer_type = _params.get<MooseEnum>("work_buffer_type");
  if (buffer_type == "lifo")
    buffer = std::make_unique<MooseUtils::LIFOBuffer<WorkType>>();
  else if (buffer_type == "circular")
    buffer = std::make_unique<MooseUtils::CircularBuffer<WorkType>>();
  else
    mooseError("Unknown work buffer type");

  return buffer;
}

currentlyExecuting()

template<typename WorkType, typename ParallelDataType>

bool ParallelStudy< WorkType, ParallelDataType >::currentlyExecuting ( ) const

inline

Whether or not this object is currently in execute().

Definition at line 135 of file ParallelStudy.h.

{ return _currently_executing; }

currentlyPreExecuting()

template<typename WorkType, typename ParallelDataType>

bool ParallelStudy< WorkType, ParallelDataType >::currentlyPreExecuting ( ) const

inline

Whether or not this object is between preExecute() and execute().

Definition at line 139 of file ParallelStudy.h.

{ return _currently_pre_executing; }

execute()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::execute

(

)

Execute method.

Definition at line 988 of file ParallelStudy.h.

{
  if (!_currently_pre_executing)
    mooseError(_name, ": preExecute() was not called before execute()");

  _currently_pre_executing = false;
  _currently_executing = true;

  switch (_method)
  {
    case ParallelStudyMethod::SMART:
      smartExecute();
      break;
    case ParallelStudyMethod::HARM:
      harmExecute();
      break;
    case ParallelStudyMethod::BS:
      bsExecute();
      break;
    default:
      mooseError("Unknown ParallelStudyMethod");
  }

  _currently_executing = false;

  // Sanity checks on if we're really done
  comm().barrier();

  if (!buffersAreEmpty())
    mooseError(_name, ": Buffers are not empty after execution");
}

executeAndBuffer()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::executeAndBuffer ( const std::size_t  chunk_size )

private

Execute a chunk of work and buffer.

Definition at line 497 of file ParallelStudy.h.

{
  _currently_executing_work = true;

  // If chunk_size > the number of objects left, this will properly grab all of them
  const auto begin = _work_buffer->beginChunk(chunk_size);
  const auto end = _work_buffer->endChunk(chunk_size);

  _local_chunks_executed++;

#ifdef LIBMESH_HAVE_OPENMP
#pragma omp parallel
#endif
  {
    const THREAD_ID tid =
#ifdef LIBMESH_HAVE_OPENMP
        omp_get_thread_num();
#else
        0;
#endif

#ifdef LIBMESH_HAVE_OPENMP
#pragma omp for schedule(dynamic, 20) nowait
#endif
    for (auto it = begin; it < end; ++it)
      executeWork(*it, tid);
  }

  // Increment the executed and completed counters
  _local_work_executed += std::distance(begin, end);
  for (auto it = begin; it != end; ++it)
    if (workIsComplete(*it))
      ++_local_work_completed;

  // Insertion point for derived classes to do something to the completed work
  // Example: Create ParallelData to spawn additional work on another processor
  postExecuteChunk(begin, end);

  // Remove the objects we just worked on from the buffer
  _work_buffer->eraseChunk(chunk_size);

  // If new work is allowed to be geneated during execution, it goes into _temp_threaded_work
  // during the threaded execution phase and then must be moved into the working buffer
  if (_allow_new_work_during_execution)
  {
    // Amount of work that needs to be moved into the main working buffer from
    // the temporary working buffer
    std::size_t threaded_work_size = 0;
    for (const auto & work_objects : _temp_threaded_work)
      threaded_work_size += work_objects.size();

    if (threaded_work_size)
    {
      // We don't ever want to decrease the capacity, so only set it if we need more entries
      if (_work_buffer->capacity() < _work_buffer->size() + threaded_work_size)
        _work_buffer->setCapacity(_work_buffer->size() + threaded_work_size);

      // Move the work into the buffer
      for (auto & threaded_work_vector : _temp_threaded_work)
      {
        for (auto & work : threaded_work_vector)
          _work_buffer->move(work);
        threaded_work_vector.clear();
      }

      // Variable that must be set when adding work so that the algorithm can keep count
      // of how much work still needs to be executed
      _local_work_started += threaded_work_size;
    }
  }

  if (_method == ParallelStudyMethod::HARM)
    flushSendBuffers();

  _currently_executing_work = false;
}

executeWork()

template<typename WorkType, typename ParallelDataType>

virtual void ParallelStudy< WorkType, ParallelDataType >::executeWork ( const WorkType &  work, const THREAD_ID  tid  )

protectedpure virtual

Pure virtual to be overridden that executes a single object of work on a given thread.

Implemented in ParallelRayStudy.

flushSendBuffers()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::flushSendBuffers ( )

private

Flushes all parallel data out of the send buffers.

Definition at line 610 of file ParallelStudy.h.

{
  for (auto & send_buffer_iter : _send_buffers)
    send_buffer_iter.second->forceSend();
}

harmExecute()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::harmExecute ( )

private

Execute work using HARM.

Definition at line 803 of file ParallelStudy.h.

{
  if (_has_alternate_ending_criteria)
    mooseError("ParallelStudy: Alternate ending criteria not yet supported for HARM");
  if (_allow_new_work_during_execution)
    mooseError(_name, ": The addition of new work during execution is not supported by HARM");
  mooseAssert(_method == ParallelStudyMethod::HARM, "Should be called with HARM only");

  // Request for the total amount of work started
  Parallel::Request work_started_request;
  // Requests for sending the amount of finished worked to every other processor
  std::vector<Parallel::Request> work_completed_requests(comm().size());
  // Whether or not the finished requests have been sent to each processor
  std::vector<bool> work_completed_requests_sent(comm().size(), false);
  // Values of work completed on this processor that are being sent to other processors
  std::vector<unsigned long long int> work_completed_requests_temps(comm().size(), 0);
  // Work completed by each processor
  std::vector<unsigned long long int> work_completed_per_proc(comm().size(), 0);
  // Tag for sending work finished
  const auto work_completed_requests_tag = comm().get_unique_tag();

  // Get the amount of work that was started in the whole domain
  comm().sum(_local_work_started, _total_work_started, work_started_request);

  // All work has been executed, so time to communicate
  flushSendBuffers();

  // HARM only does some communication based on times through the loop.
  // This counter will be used for that
  unsigned int communication_clicks = 0;

  Parallel::Status work_completed_probe_status;
  int work_completed_probe_flag;

  // Keep working until done
  while (true)
  {
    receiveAndExecute();

    flushSendBuffers();

    if (communication_clicks > comm().size())
    {
      // Receive messages about work being finished
      do
      {
        MPI_Iprobe(MPI_ANY_SOURCE,
                   work_completed_requests_tag.value(),
                   comm().get(),
                   &work_completed_probe_flag,
                   work_completed_probe_status.get());

        if (work_completed_probe_flag)
        {
          auto proc = work_completed_probe_status.source();
          comm().receive(proc, work_completed_per_proc[proc], work_completed_requests_tag);
        }
      } while (work_completed_probe_flag);

      _total_work_completed = std::accumulate(
          work_completed_per_proc.begin(), work_completed_per_proc.end(), _local_work_completed);

      // Reset
      communication_clicks = 0;
    }

    // Send messages about objects being finished
    for (processor_id_type pid = 0; pid < comm().size(); ++pid)
      if (pid != _pid &&
          (!work_completed_requests_sent[pid] || work_completed_requests[pid].test()) &&
          _local_work_completed > work_completed_requests_temps[pid])
      {
        work_completed_requests_temps[pid] = _local_work_completed;
        comm().send(pid,
                    work_completed_requests_temps[pid],
                    work_completed_requests[pid],
                    work_completed_requests_tag);
        work_completed_requests_sent[pid] = true;
      }

    // All procs agree on the amount of work started and we've finished all the work started
    if (work_started_request.test() && _total_work_started == _total_work_completed)
    {
      // Need to call the post wait work for all of the requests
      for (processor_id_type pid = 0; pid < comm().size(); ++pid)
        if (pid != _pid)
          work_completed_requests[pid].wait();

      return;
    }

    communication_clicks++;
  }
}

localChunksExecuted()

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::localChunksExecuted ( ) const

inline

Gets the total number of chunks of work executed on this processor.

Definition at line 130 of file ParallelStudy.h.

Referenced by PerProcessorRayTracingResultsVectorPostprocessor::execute().

{ return _local_chunks_executed; }

localWorkExecuted()

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::localWorkExecuted ( ) const

inline

Gets the total amount of work executed on this processor.

Definition at line 122 of file ParallelStudy.h.

Referenced by PerProcessorRayTracingResultsVectorPostprocessor::execute().

{ return _local_work_executed; }

localWorkStarted()

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::localWorkStarted ( ) const

inline

Gets the total amount of work started from this processor.

Definition at line 118 of file ParallelStudy.h.

Referenced by PerProcessorRayTracingResultsVectorPostprocessor::execute().

{ return _local_work_started; }

maxBufferSize()

template<typename WorkType, typename ParallelDataType>

unsigned int ParallelStudy< WorkType, ParallelDataType >::maxBufferSize ( ) const

inline

Gets the max buffer size.

Definition at line 144 of file ParallelStudy.h.

{ return _max_buffer_size; }

method()

template<typename WorkType, typename ParallelDataType>

ParallelStudyMethod ParallelStudy< WorkType, ParallelDataType >::method ( ) const

inline

Gets the method.

Definition at line 166 of file ParallelStudy.h.

{ return _method; }

moveContinuingWorkToBuffer() [1/2]

template<typename WorkType, typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::moveContinuingWorkToBuffer ( WorkType &  Work )

protected

Moves work that is considered continuing for the purposes of the execution algorithm into the buffer.

Definition at line 1128 of file ParallelStudy.h.

{
  if (_currently_executing_work)
    moveWorkError(MoveWorkError::CONTINUING_DURING_EXECUTING_WORK);

  _work_buffer->move(work);
}

moveContinuingWorkToBuffer() [2/2]

template<typename WorkType, typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::moveContinuingWorkToBuffer ( const work_iterator  begin, const work_iterator  end  )

protected

Definition at line 1138 of file ParallelStudy.h.

{
  if (_currently_executing_work)
    moveWorkError(MoveWorkError::CONTINUING_DURING_EXECUTING_WORK);

  const auto size = std::distance(begin, end);
  if (_work_buffer->capacity() < _work_buffer->size() + size)
    _work_buffer->setCapacity(_work_buffer->size() + size);

  for (auto it = begin; it != end; ++it)
    _work_buffer->move(*it);
}

moveParallelDataToBuffer()

template<typename WorkType , typename ParallelDataType>

void ParallelStudy< WorkType, ParallelDataType >::moveParallelDataToBuffer	(	std::shared_ptr< ParallelDataType > &	data,
		const processor_id_type	dest_pid
	)

Moves parallel data objects to the send buffer to be communicated to processor dest_pid.

Definition at line 576 of file ParallelStudy.h.

{
  mooseAssert(comm().size() > dest_pid, "Invalid processor ID");
  mooseAssert(_pid != dest_pid, "Processor ID is self");

  if (!_currently_executing && !_currently_pre_executing)
    mooseError(_name, ": Cannot sendParallelData() when not executing");

  // Get the send buffer for the proc this object is going to
  auto find_pair = _send_buffers.find(dest_pid);
  // Need to create a send buffer for said processor
  if (find_pair == _send_buffers.end())
    _send_buffers
        .emplace(dest_pid,
                 std::make_unique<
                     SendBuffer<ParallelDataType, ParallelStudy<WorkType, ParallelDataType>>>(
                     comm(),
                     this,
                     dest_pid,
                     _method,
                     _min_buffer_size,
                     _max_buffer_size,
                     _buffer_growth_multiplier,
                     _buffer_shrink_multiplier,
                     _parallel_data_buffer_tag))
        .first->second->moveObject(data);
  // Send buffer exists for this processor
  else
    find_pair->second->moveObject(data);
}

moveWorkError()

template<typename WorkType, typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::moveWorkError ( const MoveWorkError  error, const WorkType *  work = nullptr  ) const

protectedvirtual

Virtual that allows for the customization of error text for moving work into the buffer.

Definition at line 1022 of file ParallelStudy.h.

{
  if (error == MoveWorkError::DURING_EXECUTION_DISABLED)
    mooseError(_name,
               ": The moving of new work into the buffer during work execution requires\n",
               "that the parameter 'allow_new_work_during_execution = true'");
  if (error == MoveWorkError::PRE_EXECUTION_AND_EXECUTION_ONLY)
    mooseError(
        _name,
        ": Can only move work into the buffer in the pre-execution and execution phase\n(between "
        "preExecute() and the end of execute()");
  if (error == MoveWorkError::PRE_EXECUTION_ONLY)
    mooseError(_name,
               ": Can only move work into the buffer in the pre-execution phase\n(between "
               "preExecute() and execute()");
  if (error == MoveWorkError::PRE_EXECUTION_THREAD_0_ONLY)
    mooseError(_name,
               ": Can only move work into the buffer in the pre-execution phase\n(between "
               "preExecute() and execute()) on thread 0");
  if (error == CONTINUING_DURING_EXECUTING_WORK)
    mooseError(_name, ": Cannot move continuing work into the buffer during executeAndBuffer()");

  mooseError("Unknown MoveWorkError");
}

moveWorkToBuffer() [1/3]

template<typename WorkType, typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::moveWorkToBuffer	(	WorkType &	work,
		const THREAD_ID	tid
	)

Adds work to the buffer to be executed.

This will move the work into the buffer (with std::move), therefore the passed in work will be invalid after this call. For the purposes of the completion algorithm, this added work is considered NEW work.

During pre-execution (between preExecute() and execute()), this method can ONLY be called on thread 0.

During execute(), this method is thread safe and can be used to add work during execution.

Definition at line 1070 of file ParallelStudy.h.

{
  // Error checks for moving work into the buffer at unallowed times
  canMoveWorkCheck(tid);

  // Can move directly into the work buffer on thread 0 when we're not executing work
  if (!_currently_executing_work && tid == 0)
  {
    ++_local_work_started; // must ALWAYS increment when adding new work to the working buffer
    _work_buffer->move(work);
  }
  // Objects added during execution go into a temporary threaded vector (is thread safe) to be
  // moved into the working buffer when possible
  else
    _temp_threaded_work[tid].emplace_back(std::move(work));
}

moveWorkToBuffer() [2/3]

template<typename WorkType, typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::moveWorkToBuffer	(	const work_iterator	begin,
		const work_iterator	end,
		const THREAD_ID	tid
	)

Definition at line 1089 of file ParallelStudy.h.

{
  // Error checks for moving work into the buffer at unallowed times
  canMoveWorkCheck(tid);

  // Get work size beforehand so we can resize
  const auto size = std::distance(begin, end);

  // Can move directly into the work buffer on thread 0 when we're not executing work
  if (!_currently_executing_work && tid == 0)
  {
    if (_work_buffer->capacity() < _work_buffer->size() + size)
      _work_buffer->setCapacity(_work_buffer->size() + size);
    _local_work_started += size;
  }
  else
    _temp_threaded_work[tid].reserve(_temp_threaded_work[tid].size() + size);

  // Move the objects
  if (!_currently_executing_work && tid == 0)
    for (auto it = begin; it != end; ++it)
      _work_buffer->move(*it);
  else
    for (auto it = begin; it != end; ++it)
      _temp_threaded_work[tid].emplace_back(std::move(*it));
}

moveWorkToBuffer() [3/3]

template<typename WorkType, typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::moveWorkToBuffer	(	std::vector< WorkType > &	work,
		const THREAD_ID	tid
	)

Definition at line 1120 of file ParallelStudy.h.

{
  moveWorkToBuffer(work_vector.begin(), work_vector.end(), tid);
}

parallelDataSent()

template<typename WorkType , typename ParallelDataType >

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::parallelDataSent

(

)

const

Gets the total number of parallel data objects sent from this processor.

Definition at line 1166 of file ParallelStudy.h.

Referenced by PerProcessorRayTracingResultsVectorPostprocessor::execute().

{
  unsigned long long int total_sent = 0;

  for (const auto & buffer : _send_buffers)
    total_sent += buffer.second->objectsSent();

  return total_sent;
}

poolParallelDataCreated()

template<typename WorkType , typename ParallelDataType >

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::poolParallelDataCreated

(

)

const

Gets the total number of parallel data created in all of the threaded pools.

Definition at line 1190 of file ParallelStudy.h.

Referenced by PerProcessorRayTracingResultsVectorPostprocessor::execute().

{
  unsigned long long int num_created = 0;

  for (const auto & pool : _parallel_data_pools)
    num_created += pool.num_created();

  return num_created;
}

postExecuteChunk()

template<typename WorkType, typename ParallelDataType>

virtual void ParallelStudy< WorkType, ParallelDataType >::postExecuteChunk ( const work_iterator  , const work_iterator    )

inlineprotectedvirtual

Insertion point for acting on work that was just executed.

This is not called in threads.

Definition at line 219 of file ParallelStudy.h.

{}

postReceiveParallelData()

template<typename WorkType, typename ParallelDataType>

virtual void ParallelStudy< WorkType, ParallelDataType >::postReceiveParallelData ( const parallel_data_iterator  begin, const parallel_data_iterator  end  )

protectedpure virtual

Pure virtual for acting on parallel data that has JUST been received and filled into the buffer.

The parallel data in the range passed here will have its use count reduced by one if it still exists after this call.

postReceiveParallelDataInternal()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::postReceiveParallelDataInternal ( )

private

Internal method for acting on the parallel data that has just been received into the parallel buffer.

Definition at line 630 of file ParallelStudy.h.

{
  if (_receive_buffer->buffer().empty())
    return;

  // Let derived classes work on the data and then clear it after
  postReceiveParallelData(_receive_buffer->buffer().begin(), _receive_buffer->buffer().end());
  for (auto & data : _receive_buffer->buffer())
    if (data)
      data.reset();

  _receive_buffer->buffer().clear();
}

preExecute()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::preExecute

(

)

Pre-execute method that MUST be called before execute() and before adding work.

Definition at line 964 of file ParallelStudy.h.

{
  if (!buffersAreEmpty())
    mooseError(_name, ": Buffers are not empty in preExecute()");

  // Clear communication buffers
  for (auto & send_buffer_pair : _send_buffers)
    send_buffer_pair.second->clear();
  _send_buffers.clear();
  _receive_buffer->clear();

  // Clear counters
  _local_chunks_executed = 0;
  _local_work_completed = 0;
  _local_work_started = 0;
  _local_work_executed = 0;
  _total_work_started = 0;
  _total_work_completed = 0;

  _currently_pre_executing = true;
}

preReceiveAndExecute()

template<typename WorkType, typename ParallelDataType>

virtual void ParallelStudy< WorkType, ParallelDataType >::preReceiveAndExecute ( )

inlineprotectedvirtual

Insertion point called just after trying to receive work and just before beginning work on the work buffer.

Definition at line 225 of file ParallelStudy.h.

{}

receiveAndExecute()

template<typename WorkType , typename ParallelDataType >

bool ParallelStudy< WorkType, ParallelDataType >::receiveAndExecute ( )

private

Receive packets of parallel data from other processors and executes work.

Definition at line 646 of file ParallelStudy.h.

{
  bool executed_some = false;

  if (_receive_buffer->currentlyReceiving() && _method == ParallelStudyMethod::SMART)
    _receive_buffer->cleanupRequests();
  else
    _receive_buffer->receive();

  postReceiveParallelDataInternal();

  preReceiveAndExecute();

  while (!_work_buffer->empty())
  {
    executed_some = true;

    // Switch between tracing a chunk and buffering with SMART
    if (_method == ParallelStudyMethod::SMART)
    {
      // Look for extra work first so that these transfers can be finishing while we're executing
      // Start receives only if our work buffer is decently sized
      const bool start_receives_only = _work_buffer->size() > (2 * _chunk_size);
      _receive_buffer->receive(_work_buffer->size() > (2 * _chunk_size));
      if (!start_receives_only)
        postReceiveParallelDataInternal();

      // Execute some objects
      executeAndBuffer(_chunk_size);
    }
    // Execute all of them and then buffer with the other methods
    else
      executeAndBuffer(_work_buffer->size());
  }

  return executed_some;
}

receiveBuffer()

template<typename WorkType, typename ParallelDataType>

const ReceiveBuffer<ParallelDataType, ParallelStudy<WorkType, ParallelDataType> >& ParallelStudy< WorkType, ParallelDataType >::receiveBuffer ( ) const

inline

Gets the receive buffer.

Definition at line 88 of file ParallelStudy.h.

Referenced by PerProcessorRayTracingResultsVectorPostprocessor::execute().

  {
    return *_receive_buffer;
  }

reserveBuffer()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::reserveBuffer

(

const std::size_t

size

)

Reserve size entries in the work buffer.

This can only be used during the pre-execution phase (between preExecute() and execute()).

This is particularly useful when one wants to move many work objects into the buffer using moveWorkToBuffer() and wants to allocate the space ahead of time.

Definition at line 618 of file ParallelStudy.h.

{
  if (!_currently_pre_executing)
    mooseError(_name, ": Can only reserve in object buffer during pre-execution");

  // We don't ever want to decrease the capacity, so only set if we need more entries
  if (_work_buffer->capacity() < size)
    _work_buffer->setCapacity(size);
}

sendBufferPoolCreated()

template<typename WorkType , typename ParallelDataType >

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::sendBufferPoolCreated

(

)

const

Gets the total number of send buffer pools created.

Definition at line 1154 of file ParallelStudy.h.

Referenced by PerProcessorRayTracingResultsVectorPostprocessor::execute().

{
  unsigned long long int total = 0;

  for (const auto & buffer : _send_buffers)
    total += buffer.second->bufferPoolCreated();

  return total;
}

smartExecute()

template<typename WorkType , typename ParallelDataType >

void ParallelStudy< WorkType, ParallelDataType >::smartExecute ( )

private

Execute work using SMART.

Definition at line 686 of file ParallelStudy.h.

{
  mooseAssert(_method == ParallelStudyMethod::SMART, "Should be called with SMART only");

  // Request for the sum of the started work
  Parallel::Request started_request;
  // Request for the sum of the completed work
  Parallel::Request completed_request;

  // Temp for use in sending the current value in a nonblocking sum instead of an updated value
  unsigned long long int temp;

  // Whether or not to make the started request first, or after every finished request.
  // When allowing adding new work during the execution phase, the starting object counts could
  // change after right now, so we must update them after each finished request is complete.
  // When not allowing generation during propagation, we know the counts up front.
  const bool started_request_first = !_allow_new_work_during_execution;

  // Get the amount of work that was started in the whole domain, if applicable
  if (started_request_first)
    comm().sum(_local_work_started, _total_work_started, started_request);

  // Whether or not the started request has been made
  bool made_started_request = started_request_first;
  // Whether or not the completed request has been made
  bool made_completed_request = false;

  // Good time to get rid of whatever's currently in our SendBuffers
  flushSendBuffers();

  // Use these to try to delay some forced communication
  unsigned int non_executing_clicks = 0;
  unsigned int non_executing_root_clicks = 0;
  bool executed_some = true;

  // Keep executing work until it has all completed
  while (true)
  {
    executed_some = receiveAndExecute();

    if (executed_some)
    {
      non_executing_clicks = 0;
      non_executing_root_clicks = 0;
    }
    else
    {
      non_executing_clicks++;
      non_executing_root_clicks++;
    }

    if (non_executing_clicks >= _clicks_per_communication)
    {
      non_executing_clicks = 0;

      flushSendBuffers();
    }

    if (_has_alternate_ending_criteria)
    {
      if (buffersAreEmpty() && alternateSmartEndingCriteriaMet())
      {
        comm().barrier();
        return;
      }
    }
    else if (non_executing_root_clicks >= _clicks_per_root_communication)
    {
      non_executing_root_clicks = 0;

      // We need the starting work sum first but said request isn't complete yet
      if (started_request_first && !started_request.test())
        continue;

      // At this point, we need to make a request for the completed work sum
      if (!made_completed_request)
      {
        made_completed_request = true;
        temp = _local_work_completed;
        comm().sum(temp, _total_work_completed, completed_request);
        continue;
      }

      // We have the completed work sum
      if (completed_request.test())
      {
        // The starting work sum must be requested /after/ we have finishing counts and we
        // need to make the request for said sum
        if (!made_started_request)
        {
          made_started_request = true;
          temp = _local_work_started;
          comm().sum(temp, _total_work_started, started_request);
          continue;
        }

        // The starting work sum must be requested /after/ we have finishing sum and we
        // don't have the starting sum yet
        if (!started_request_first && !started_request.test())
          continue;

        // Started count is the same as the finished count - we're done!
        if (_total_work_started == _total_work_completed)
          return;

        // Next time around we should make a completed sum request
        made_completed_request = false;
        // If we need the starting work sum after the completed work sum, we need those now as well
        if (!started_request_first)
          made_started_request = false;
      }
    }
  }
}

totalWorkCompleted()

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::totalWorkCompleted ( ) const

inline

Gets the total amount of work completeed across all processors.

Definition at line 126 of file ParallelStudy.h.

Referenced by RayTracingStudyResult::getValue().

{ return _total_work_completed; }

validParams()

template<typename WorkType , typename ParallelDataType >

InputParameters ParallelStudy< WorkType, ParallelDataType >::validParams ( )

static

Definition at line 436 of file ParallelStudy.h.

{
  auto params = emptyInputParameters();

  params.addRangeCheckedParam<unsigned int>(
      "send_buffer_size", 100, "send_buffer_size > 0", "The size of the send buffer");
  params.addRangeCheckedParam<unsigned int>(
      "chunk_size",
      100,
      "chunk_size > 0",
      "The number of objects to process at one time during execution");
  params.addRangeCheckedParam<unsigned int>("clicks_per_communication",
                                            10,
                                            "clicks_per_communication >= 0",
                                            "Iterations to wait before communicating");
  params.addRangeCheckedParam<unsigned int>("clicks_per_root_communication",
                                            10,
                                            "clicks_per_root_communication > 0",
                                            "Iterations to wait before communicating with root");
  params.addRangeCheckedParam<unsigned int>("clicks_per_receive",
                                            1,
                                            "clicks_per_receive > 0",
                                            "Iterations to wait before checking for new objects");

  params.addParam<unsigned int>("min_buffer_size",
                                "The initial size of the SendBuffer and the floor for shrinking "
                                "it.  This defaults to send_buffer_size if not set (i.e. the "
                                "buffer won't change size)");
  params.addParam<Real>("buffer_growth_multiplier",
                        2.,
                        "How much to grow a SendBuffer by if the buffer completely fills and "
                        "dumps.  Will max at send_buffer_size");
  params.addRangeCheckedParam<Real>("buffer_shrink_multiplier",
                                    0.5,
                                    "0 < buffer_shrink_multiplier <= 1.0",
                                    "Multiplier (between 0 and 1) to apply to the current buffer "
                                    "size if it is force dumped.  Will stop at "
                                    "min_buffer_size.");

  params.addParam<bool>(
      "allow_new_work_during_execution",
      true,
      "Whether or not to allow the addition of new work to the work buffer during execution");

  MooseEnum methods("smart harm bs", "smart");
  params.addParam<MooseEnum>("method", methods, "The algorithm to use");

  MooseEnum work_buffers("lifo circular", "circular");
  params.addParam<MooseEnum>("work_buffer_type", work_buffers, "The work buffer type to use");

  params.addParamNamesToGroup(
      "send_buffer_size chunk_size clicks_per_communication clicks_per_root_communication "
      "clicks_per_receive min_buffer_size buffer_growth_multiplier buffer_shrink_multiplier method "
      "work_buffer_type allow_new_work_during_execution",
      "Advanced");

  return params;
}

workBuffer()

template<typename WorkType, typename ParallelDataType>

const MooseUtils::Buffer<WorkType>& ParallelStudy< WorkType, ParallelDataType >::workBuffer ( ) const

inline

Gets the work buffer.

Definition at line 96 of file ParallelStudy.h.

{ return *_work_buffer; }

workIsComplete()

template<typename WorkType, typename ParallelDataType>

virtual bool ParallelStudy< WorkType, ParallelDataType >::workIsComplete ( const WorkType &  )

inlineprotectedvirtual

Can be overridden to denote if a piece of work is not complete yet.

The complete terminology is used within the execution algorithms to determine if the study is complete.

Reimplemented in ParallelRayStudy.

Definition at line 243 of file ParallelStudy.h.

{ return true; }

Member Data Documentation

_allow_new_work_during_execution

template<typename WorkType, typename ParallelDataType>

const bool ParallelStudy< WorkType, ParallelDataType >::_allow_new_work_during_execution

private

Whether or not to allow the addition of new work to the buffer during execution.

Definition at line 322 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::ParallelStudy().

_buffer_growth_multiplier

template<typename WorkType, typename ParallelDataType>

const Real ParallelStudy< WorkType, ParallelDataType >::_buffer_growth_multiplier

private

Multiplier for the buffer size for growing the buffer.

Definition at line 316 of file ParallelStudy.h.

_buffer_shrink_multiplier

template<typename WorkType, typename ParallelDataType>

const Real ParallelStudy< WorkType, ParallelDataType >::_buffer_shrink_multiplier

private

Multiplier for the buffer size for shrinking the buffer.

Definition at line 318 of file ParallelStudy.h.

_chunk_size

template<typename WorkType, typename ParallelDataType>

const unsigned int ParallelStudy< WorkType, ParallelDataType >::_chunk_size

private

Number of objects to execute at once during communication.

Definition at line 320 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::chunkSize().

_clicks_per_communication

template<typename WorkType, typename ParallelDataType>

const unsigned int ParallelStudy< WorkType, ParallelDataType >::_clicks_per_communication

private

Iterations to wait before communicating.

Definition at line 325 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::clicksPerCommunication().

_clicks_per_receive

template<typename WorkType, typename ParallelDataType>

const unsigned int ParallelStudy< WorkType, ParallelDataType >::_clicks_per_receive

private

Iterations to wait before checking for new objects.

Definition at line 329 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::clicksPerReceive().

_clicks_per_root_communication

template<typename WorkType, typename ParallelDataType>

const unsigned int ParallelStudy< WorkType, ParallelDataType >::_clicks_per_root_communication

private

Iterations to wait before communicating with root.

Definition at line 327 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::clicksPerRootCommunication().

_currently_executing

template<typename WorkType, typename ParallelDataType>

bool ParallelStudy< WorkType, ParallelDataType >::_currently_executing

private

Whether we are within execute()

Definition at line 362 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::currentlyExecuting().

_currently_executing_work

template<typename WorkType, typename ParallelDataType>

bool ParallelStudy< WorkType, ParallelDataType >::_currently_executing_work

private

Whether or not we are currently within executeAndBuffer()

Definition at line 366 of file ParallelStudy.h.

_currently_pre_executing

template<typename WorkType, typename ParallelDataType>

bool ParallelStudy< WorkType, ParallelDataType >::_currently_pre_executing

private

Whether we are between preExecute() and execute()

Definition at line 364 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::currentlyPreExecuting().

_has_alternate_ending_criteria

template<typename WorkType, typename ParallelDataType>

bool ParallelStudy< WorkType, ParallelDataType >::_has_alternate_ending_criteria

protected

Whether or not this object has alternate ending criteria.

Definition at line 269 of file ParallelStudy.h.

_local_chunks_executed

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::_local_chunks_executed

private

Number of chunks of work executed on this processor.

Definition at line 349 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::localChunksExecuted().

_local_work_completed

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::_local_work_completed

private

Amount of work completed on this processor.

Definition at line 351 of file ParallelStudy.h.

_local_work_executed

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::_local_work_executed

private

Amount of work executed on this processor.

Definition at line 355 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::localWorkExecuted().

_local_work_started

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::_local_work_started

private

Amount of work started on this processor.

Definition at line 353 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::localWorkStarted().

_max_buffer_size

template<typename WorkType, typename ParallelDataType>

const unsigned int ParallelStudy< WorkType, ParallelDataType >::_max_buffer_size

private

Number of objects to buffer before communication.

Definition at line 314 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::maxBufferSize().

_method

template<typename WorkType, typename ParallelDataType>

const ParallelStudyMethod ParallelStudy< WorkType, ParallelDataType >::_method

protected

The study method.

Definition at line 267 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::method(), and ParallelStudy< std::shared_ptr< Ray >, Ray >::ParallelStudy().

_min_buffer_size

template<typename WorkType, typename ParallelDataType>

const unsigned int ParallelStudy< WorkType, ParallelDataType >::_min_buffer_size

private

Minimum size of a SendBuffer.

Definition at line 312 of file ParallelStudy.h.

_name

template<typename WorkType, typename ParallelDataType>

const std::string ParallelStudy< WorkType, ParallelDataType >::_name

protected

Name for this object for use in error handling.

Definition at line 263 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::ParallelStudy().

_parallel_data_buffer_tag

template<typename WorkType, typename ParallelDataType>

Parallel::MessageTag ParallelStudy< WorkType, ParallelDataType >::_parallel_data_buffer_tag

private

MessageTag for sending parallel data.

Definition at line 332 of file ParallelStudy.h.

_parallel_data_pools

template<typename WorkType, typename ParallelDataType>

std::vector<MooseUtils::SharedPool<ParallelDataType> > ParallelStudy< WorkType, ParallelDataType >::_parallel_data_pools

private

Pools for re-using destructed parallel data objects (one for each thread)

Definition at line 334 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::acquireParallelData().

_params

template<typename WorkType, typename ParallelDataType>

const InputParameters& ParallelStudy< WorkType, ParallelDataType >::_params

protected

The InputParameters.

Definition at line 265 of file ParallelStudy.h.

_pid

template<typename WorkType, typename ParallelDataType>

const processor_id_type ParallelStudy< WorkType, ParallelDataType >::_pid

protected

This rank.

Definition at line 261 of file ParallelStudy.h.

_receive_buffer

template<typename WorkType, typename ParallelDataType>

const std::unique_ptr<ReceiveBuffer<ParallelDataType, ParallelStudy<WorkType, ParallelDataType> > > ParallelStudy< WorkType, ParallelDataType >::_receive_buffer

private

The receive buffer.

Definition at line 341 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::receiveBuffer().

_send_buffers

template<typename WorkType, typename ParallelDataType>

std::unordered_map< processor_id_type, std::unique_ptr<SendBuffer<ParallelDataType, ParallelStudy<WorkType, ParallelDataType> > > > ParallelStudy< WorkType, ParallelDataType >::_send_buffers

private

Send buffers for each processor.

Definition at line 346 of file ParallelStudy.h.

_temp_threaded_work

template<typename WorkType, typename ParallelDataType>

std::vector<std::vector<WorkType> > ParallelStudy< WorkType, ParallelDataType >::_temp_threaded_work

private

Threaded temprorary storage for work added while we're using the _work_buffer (one for each thread)

Definition at line 336 of file ParallelStudy.h.

_total_work_completed

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::_total_work_completed

private

Amount of work completed on all processors.

Definition at line 359 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::totalWorkCompleted().

_total_work_started

template<typename WorkType, typename ParallelDataType>

unsigned long long int ParallelStudy< WorkType, ParallelDataType >::_total_work_started

private

Amount of work started on all processors.

Definition at line 357 of file ParallelStudy.h.

_work_buffer

template<typename WorkType, typename ParallelDataType>

const std::unique_ptr<MooseUtils::Buffer<WorkType> > ParallelStudy< WorkType, ParallelDataType >::_work_buffer

private

Buffer for executing work.

Definition at line 338 of file ParallelStudy.h.

Referenced by ParallelStudy< std::shared_ptr< Ray >, Ray >::workBuffer().

---

The documentation for this class was generated from the following file:

• ray_tracing/include/utils/ParallelStudy.h