RankMap Class Reference

Builds lists and maps that help in knowing which physical hardware nodes each rank is on. More...

#include <RankMap.h>

Inheritance diagram for RankMap:

Public Member Functions
	RankMap (const libMesh::Parallel::Communicator &comm, PerfGraph &perf_graph)
	Constructs and fills the map.
unsigned int	hardwareID (processor_id_type pid) const
	Returns the "hardware ID" (a unique ID given to each physical compute node in the job) for a given processor ID (rank)
const std::vector< processor_id_type > &	ranks (unsigned int hardware_id) const
	Returns the ranks that are on the given hardwareID (phsical node in the job)
const std::vector< unsigned int > &	rankHardwareIds () const
	Vector containing the hardware ID for each PID.

Protected Attributes
std::unordered_map< unsigned int, std::vector< processor_id_type > >	_hardware_id_to_ranks
	Map of hardware_id -> ranks on that node.
std::vector< unsigned int >	_rank_to_hardware_id
	Each entry corresponds to the hardware_id for that PID.

Private Member Functions
const Parallel::Communicator &	comm () const
processor_id_type	n_processors () const
processor_id_type	processor_id () const
PerfGraph &	perfGraph ()
	Get the PerfGraph.
PerfID	registerTimedSection (const std::string &section_name, const unsigned int level) const
	Call to register a named section for timing.
PerfID	registerTimedSection (const std::string &section_name, const unsigned int level, const std::string &live_message, const bool print_dots=true) const
	Call to register a named section for timing.
std::string	timedSectionName (const std::string &section_name) const

Static Private Member Functions
static InputParameters	validParams ()

Private Attributes
const Parallel::Communicator &	_communicator
MooseApp &	_pg_moose_app
	The MooseApp that owns the PerfGraph.
const std::string	_prefix
	A prefix to use for all sections.

Detailed Description

Builds lists and maps that help in knowing which physical hardware nodes each rank is on.

Note: large chunks of this code were originally committed by @dschwen in PR #12351

https://github.com/idaholab/moose/pull/12351

Definition at line 23 of file RankMap.h.

Constructor & Destructor Documentation

RankMap()

RankMap::RankMap	(	const libMesh::Parallel::Communicator &	comm,
		PerfGraph &	perf_graph
	)

Constructs and fills the map.

Definition at line 17 of file RankMap.C.

  : ParallelObject(comm), PerfGraphInterface(perf_graph, "RankMap")
{
  TIME_SECTION("construct", 2, "Constructing RankMap");
 
  auto num_procs = n_processors();
  _rank_to_hardware_id.resize(num_procs);
 
  Parallel::Communicator shmem_comm;
  _communicator.split_by_type(MPI_COMM_TYPE_SHARED, 0, MPI_INFO_NULL, shmem_comm);
 
  // This will be the world rank of the root process
  // from the shared memory communicator we're getting ready to create
  // Each process on the same node will end up with the same world_rank
  processor_id_type world_rank = processor_id();
 
  // Broadcast the world rank of the sub group root to all processes within this communicator
  shmem_comm.broadcast(world_rank, 0);
 
  // Send the info to everyone
  std::vector<processor_id_type> world_ranks(num_procs);
  _communicator.allgather(world_rank, world_ranks);
 
  // Map of world_rank to hardware_id
  std::map<unsigned int, unsigned int> world_rank_to_hardware_id;
 
  // Assign a contiguous unique numerical id to each shared memory group
  unsigned int next_id = 0;
 
  for (MooseIndex(world_ranks) pid = 0; pid < world_ranks.size(); pid++)
  {
    auto world_rank = world_ranks[pid];
 
    auto it = world_rank_to_hardware_id.lower_bound(world_rank);
 
    unsigned int current_id = 0;
 
    // If we've seen this world_rank before then use its already given ID
    if (it != world_rank_to_hardware_id.end() && it->first == world_rank)
      current_id = it->second;
    else // Create the new ID
    {
      current_id = next_id++;
 
      world_rank_to_hardware_id.emplace_hint(it, world_rank, current_id);
    }
 
    _rank_to_hardware_id[pid] = current_id;
 
    // Side-effect insertion utilized
    _hardware_id_to_ranks[current_id].emplace_back(pid);
  }
}

Member Function Documentation

hardwareID()

unsigned int RankMap::hardwareID ( processor_id_type  pid ) const

inline

Returns the "hardware ID" (a unique ID given to each physical compute node in the job) for a given processor ID (rank)

Definition at line 35 of file RankMap.h.

  {
    mooseAssert(pid < _communicator.size(), "PID out of range");
    return _rank_to_hardware_id[pid];
  }

Referenced by HardwareIDAux::computeValue().

perfGraph()

PerfGraph & PerfGraphInterface::perfGraph ( )

inherited

Get the PerfGraph.

Definition at line 86 of file PerfGraphInterface.C.

{
  return _pg_moose_app.perfGraph();
}

Referenced by CommonOutputAction::act(), PerfGraphData::finalize(), PerfGraphReporter::finalize(), and PerfGraphOutput::output().

rankHardwareIds()

const std::vector< unsigned int > & RankMap::rankHardwareIds ( ) const

inline

Vector containing the hardware ID for each PID.

Definition at line 56 of file RankMap.h.

{ return _rank_to_hardware_id; }

ranks()

const std::vector< processor_id_type > & RankMap::ranks ( unsigned int  hardware_id ) const

inline

Returns the ranks that are on the given hardwareID (phsical node in the job)

Definition at line 44 of file RankMap.h.

  {
    auto item = _hardware_id_to_ranks.find(hardware_id);
    if (item == _hardware_id_to_ranks.end())
      mooseError("hardware_id not found");
 
    return item->second;
  }

registerTimedSection() [1/2]

PerfID PerfGraphInterface::registerTimedSection ( const std::string &  section_name, const unsigned int  level  ) const

protectedinherited

Call to register a named section for timing.

Parameters

section_name	The name of the code section to be timed
level	The importance of the timer - lower is more important (0 will always come out)

Returns

The ID of the section - use when starting timing

Definition at line 61 of file PerfGraphInterface.C.

{
  const auto timed_section_name = timedSectionName(section_name);
  if (!moose::internal::getPerfGraphRegistry().sectionExists(timed_section_name))
    return moose::internal::getPerfGraphRegistry().registerSection(timed_section_name, level);
  else
    return moose::internal::getPerfGraphRegistry().sectionID(timed_section_name);
}

registerTimedSection() [2/2]

PerfID PerfGraphInterface::registerTimedSection ( const std::string &  section_name, const unsigned int  level, const std::string &  live_message, const bool  print_dots = true  ) const

protectedinherited

Call to register a named section for timing.

Parameters

section_name	The name of the code section to be timed
level	The importance of the timer - lower is more important (0 will always come out)
live_message	The message to be printed to the screen during execution
print_dots	Whether or not progress dots should be printed for this section

Returns

The ID of the section - use when starting timing

Definition at line 72 of file PerfGraphInterface.C.

{
  const auto timed_section_name = timedSectionName(section_name);
  if (!moose::internal::getPerfGraphRegistry().sectionExists(timed_section_name))
    return moose::internal::getPerfGraphRegistry().registerSection(
        timedSectionName(section_name), level, live_message, print_dots);
  else
    return moose::internal::getPerfGraphRegistry().sectionID(timed_section_name);
}

timedSectionName()

std::string PerfGraphInterface::timedSectionName ( const std::string &  section_name ) const

protectedinherited

Returns

The name of the timed section with the name section_name.

Optionally adds a prefix if one is defined.

Definition at line 55 of file PerfGraphInterface.C.

{
  return _prefix.empty() ? "" : (_prefix + "::") + section_name;
}

Referenced by PerfGraphInterface::registerTimedSection(), and PerfGraphInterface::registerTimedSection().

validParams()

InputParameters PerfGraphInterface::validParams ( )

staticinherited

Definition at line 16 of file PerfGraphInterface.C.

{
  InputParameters params = emptyInputParameters();
  return params;
}

Referenced by Convergence::validParams().

Member Data Documentation

_hardware_id_to_ranks

std::unordered_map<unsigned int, std::vector<processor_id_type> > RankMap::_hardware_id_to_ranks

protected

Map of hardware_id -> ranks on that node.

Definition at line 60 of file RankMap.h.

Referenced by RankMap(), and ranks().

_pg_moose_app

MooseApp& PerfGraphInterface::_pg_moose_app

protectedinherited

The MooseApp that owns the PerfGraph.

Definition at line 135 of file PerfGraphInterface.h.

Referenced by PerfGraphInterface::perfGraph().

_prefix

const std::string PerfGraphInterface::_prefix

protectedinherited

A prefix to use for all sections.

Definition at line 138 of file PerfGraphInterface.h.

Referenced by PerfGraphInterface::timedSectionName().

_rank_to_hardware_id

std::vector<unsigned int> RankMap::_rank_to_hardware_id

protected

Each entry corresponds to the hardware_id for that PID.

Definition at line 63 of file RankMap.h.

Referenced by hardwareID(), rankHardwareIds(), and RankMap().

---

The documentation for this class was generated from the following files:

• include/utils/RankMap.h
• src/utils/RankMap.C