Moose::Kokkos::System Class Reference

The Kokkos system class. More...

#include <KokkosSystem.h>

Inheritance diagram for Moose::Kokkos::System:

Classes
struct	Sparsity
	CSR format sparsity data. More...

Public Member Functions
	System (SystemBase &system)
	Constructor. More...
void	sync (const MemcpyType dir)
	Synchronize the active tagged vectors and matrices between host and device. More...
void	reinit ()
	Allocate the quadrature point vectors for active variable and tags and cache quadrature point values. More...
void	setActiveVariables (const std::set< MooseVariableFieldBase *> &vars)
	Set the active variables. More...
void	setActiveSolutionTags (const std::set< TagID > &tags)
	Set the active solution tags. More...
void	setActiveResidualTags (const std::set< TagID > &tags)
	Set the active residual tags. More...
void	setActiveMatrixTags (const std::set< TagID > &tags)
	Set the active matrix tags. More...
void	clearActiveVariables ()
	Clear the cached active variables. More...
void	clearActiveSolutionTags ()
	Clear the cached active solution tags. More...
void	clearActiveResidualTags ()
	Clear the cached active residual tags. More...
void	clearActiveMatrixTags ()
	Clear the cached active matrix tags. More...
const auto &	getDofMap () const
	Get the libMesh DOF map. More...
const auto &	getComm () const
	Get the libMesh communicator. More...
const auto &	getLocalCommList () const
	Get the list of local DOF indices to communicate. More...
const auto &	getGhostCommList () const
	Get the list of ghost DOF indices to communicate. More...
const auto &	getSparsity () const
	Get the sparisty pattern data. More...
KOKKOS_FUNCTION const auto &	getCoupling (unsigned int var) const
	Get the list of off-diagonal coupled variable numbers of a variable. More...
KOKKOS_FUNCTION bool	isVariableActive (unsigned int var, ContiguousSubdomainID subdomain) const
	Check whether a variable is active on a subdomain. More...
KOKKOS_FUNCTION bool	isResidualTagActive (TagID tag) const
	Check whether a residual tag is active. More...
KOKKOS_FUNCTION bool	isMatrixTagActive (TagID tag) const
	Check whether a matrix tag is active. More...
KOKKOS_FUNCTION bool	hasNodalBCMatrixTag (dof_id_type dof, TagID tag) const
	Check whether a local DOF index is associated with a nodal BC for an extra matrix tag. More...
KOKKOS_FUNCTION unsigned int	getFETypeID (unsigned int var) const
	Get the FE type ID of a variable. More...
KOKKOS_FUNCTION dof_id_type	getNumLocalDofs () const
	Get the number of local DOFs. More...
KOKKOS_FUNCTION dof_id_type	getNumGhostDofs () const
	Get the number of ghost DOFs. More...
KOKKOS_FUNCTION unsigned int	getMaxDofsPerElem (unsigned int var) const
	Get the maximum number of DOFs per element of a variable This number is local to each process. More...
KOKKOS_FUNCTION dof_id_type	getElemLocalDofIndex (ContiguousElementID elem, unsigned int i, unsigned int var) const
	Get the local DOF index of a variable for an element. More...
KOKKOS_FUNCTION dof_id_type	getNodeLocalDofIndex (ContiguousNodeID node, unsigned int i, unsigned int var) const
	Get the local DOF index of a variable for a node. More...
KOKKOS_FUNCTION dof_id_type	getElemGlobalDofIndex (ContiguousElementID elem, unsigned int i, unsigned int var) const
	Get the global DOF index of a variable for an element. More...
KOKKOS_FUNCTION dof_id_type	getNodeGlobalDofIndex (ContiguousNodeID node, unsigned int var) const
	Get the global DOF index of a variable for a node. More...
KOKKOS_FUNCTION dof_id_type	localToGlobalDofIndex (dof_id_type dof) const
	Get the global DOF index of a local DOF index. More...
KOKKOS_FUNCTION bool	isNodalDefined (ContiguousNodeID node, unsigned int var) const
	Get whether a variable is defined on a node. More...
KOKKOS_FUNCTION auto &	getVector (TagID tag) const
	Get a tagged Kokkos vector. More...
KOKKOS_FUNCTION auto &	getMatrix (TagID tag) const
	Get a tagged Kokkos matrix. More...
KOKKOS_FUNCTION Real &	getVectorDofValue (const dof_id_type dof, const TagID tag) const
	Get the DOF value of a tagged vector. More...
KOKKOS_FUNCTION ADReal	getVectorDofADValue (const dof_id_type dof, const TagID tag, const Real seed) const
	Get the DOF value of a tagged vector for automatic differentiation (AD) More...
KOKKOS_FUNCTION Real &	getVectorQpValue (const ElementInfo info, const dof_id_type qp, const unsigned int var, const TagID tag) const
	Get the quadrature value of a variable from a tagged vector. More...
KOKKOS_FUNCTION ADReal	getVectorQpADValue (const ElementInfo info, const dof_id_type offset, const dof_id_type qp, const unsigned int var, const TagID tag, const Real seed) const
	Get the quadrature value of a variable from a tagged vector for automatic differentiation (AD) More...
KOKKOS_FUNCTION Real3 &	getVectorQpGrad (const ElementInfo info, const dof_id_type qp, const unsigned int var, const TagID tag) const
	Get the quadrature gradient of a variable from a tagged vector. More...
KOKKOS_FUNCTION ADReal3	getVectorQpADGrad (const ElementInfo info, const Real33 jacobian, const dof_id_type offset, const dof_id_type qp, const unsigned int var, const TagID tag, const Real seed) const
	Get the quadrature gradient of a variable from a tagged vector for automatic differentiation (AD) More...
KOKKOS_FUNCTION Real	getVectorQpValueFace (const ElementInfo info, const unsigned int side, const unsigned int qp, const unsigned int var, const TagID tag) const
	Get the face quadrature value of a variable from a tagged vector. More...
KOKKOS_FUNCTION ADReal	getVectorQpADValueFace (const ElementInfo info, const unsigned int side, const unsigned int qp, const unsigned int var, const TagID tag, const Real seed) const
	Get the face quadrature value of a variable from a tagged vector for automatic differentiation (AD) More...
KOKKOS_FUNCTION Real3	getVectorQpGradFace (const ElementInfo info, const unsigned int side, const Real33 jacobian, const unsigned int qp, const unsigned int var, const TagID tag) const
	Get the face quadrature gradient of a variable from a tagged vector. More...
KOKKOS_FUNCTION ADReal3	getVectorQpADGradFace (const ElementInfo info, const unsigned int side, const Real33 jacobian, const unsigned int qp, const unsigned int var, const TagID tag, const Real seed) const
	Get the face quadrature gradient of a variable from a tagged vector for automatic differentiation (AD) More...
KOKKOS_FUNCTION Real &	getMatrixValue (dof_id_type row, dof_id_type col, TagID tag) const
	Get an entry from a tagged matrix. More...
KOKKOS_FUNCTION void	operator() (const ThreadID tid) const
	Kokkos function for caching variable values on element quadrature points. More...
PerfGraph &	perfGraph ()
	Get the PerfGraph. More...
KOKKOS_FUNCTION const Mesh &	kokkosMesh () const
	Get the const reference of the Kokkos mesh. More...
KOKKOS_FUNCTION const Assembly &	kokkosAssembly () const
	Get the const reference of the Kokkos assembly. More...
void	sync (const std::set< TagID > &tags, const MemcpyType dir)
	Synchronize the specified tagged vectors between host and device. More...
void	sync (const std::vector< TagID > &tags, const MemcpyType dir)
void	sync (const TagID tag, const MemcpyType dir)
auto &	getSystem ()
	Get the MOOSE system. More...
const auto &	getSystem () const

Static Public Member Functions
static InputParameters	validParams ()

Protected Member Functions
PerfID	registerTimedSection (const std::string &section_name, const unsigned int level) const
	Call to register a named section for timing. More...
PerfID	registerTimedSection (const std::string &section_name, const unsigned int level, const std::string &live_message, const bool print_dots=true) const
	Call to register a named section for timing. More...
std::string	timedSectionName (const std::string &section_name) const

Protected Attributes
MooseApp &	_pg_moose_app
	The MooseApp that owns the PerfGraph. More...
const std::string	_prefix
	A prefix to use for all sections. More...

Private Member Functions
void	setupVariables ()
	Setup variable data. More...
void	setupDofs ()
	Setup DOF data. More...
void	setupSparsity ()
	Setup sparsity data. More...
void	setupNodalBCDofs ()
	Mark the DOFs covered by nodal BCs. More...
void	getNodalBCDofs (const NodalBCBase *nbc, Array< bool > &dofs)
	Get the list of DOFs covered by a nodal BC. More...

Private Attributes
Thread	_thread
	Kokkos thread object. More...
SystemBase &	_system
	Reference of the MOOSE system. More...
const MooseMesh &	_mesh
	Reference of the MOOSE mesh. More...
const libMesh::DofMap &	_dof_map
	Reference of the libMesh DOF map. More...
const Parallel::Communicator &	_comm
	Reference of the libMesh communicator. More...
const unsigned int	_num_vars
	Number of variables. More...
const dof_id_type	_num_local_dofs
	Number of local DOFs. More...
const dof_id_type	_num_ghost_dofs
	Number of ghost DOFs. More...
Array< dof_id_type >	_local_to_global_dof_index
	Map from local DOF index to global DOF index. More...
Array< unsigned int >	_max_dofs_per_elem
	Maximum number of DOFs per element for each variable. More...
Array< unsigned int >	_var_fe_types
	FE type ID of each variable. More...
Array2D< bool >	_var_subdomain_active
	Whether each variable is active on subdomains. More...
Array< Array< unsigned int > >	_coupling
	Off-diagonal coupled variable numbers of each variable. More...
Array< unsigned int >	_active_variables
	List of active variable numbers. More...
Array< Array< bool > >	_nbc_matrix_tag_dof
	Flag whether each DOF is covered by a nodal BC for each matrix tag. More...
Sparsity	_sparsity
	Matrix sparsity pattern data. More...
Array< Vector >	_vectors
	Kokkos vectors and matrices on device. More...
Array< Matrix >	_matrices
Array< Array2D< Array< Real > > >	_qp_solutions
	Cached elemental quadrature values and gradients. More...
Array< Array2D< Array< Real3 > > >	_qp_solutions_grad
Array< Array2D< dof_id_type > >	_local_elem_dof_index
	Local DOF index of each variable. More...
Array< Array< dof_id_type > >	_local_node_dof_index
Array< TagID >	_active_solution_tags
	List of active tags. More...
Array< TagID >	_active_residual_tags
Array< TagID >	_active_matrix_tags
Array< bool >	_residual_tag_active
	Flag whether each tag is active. More...
Array< bool >	_matrix_tag_active
Array< Array< dof_id_type > >	_local_comm_list
	List of DOFs to send and receive. More...
Array< Array< dof_id_type > >	_ghost_comm_list

Detailed Description

The Kokkos system class.

Each nonlinear and auxiliary system in MOOSE has a corresponding Kokkos system.

Definition at line 34 of file KokkosSystem.h.

Constructor & Destructor Documentation

System()

Moose::Kokkos::System::System

(

SystemBase &

system

)

Constructor.

Parameters

system

The associated MOOSE system

Member Function Documentation

clearActiveMatrixTags()

void Moose::Kokkos::System::clearActiveMatrixTags ( )

inline

Clear the cached active matrix tags.

Definition at line 103 of file KokkosSystem.h.

  {
    _active_matrix_tags.destroy();
    _matrix_tag_active = false;
  }

clearActiveResidualTags()

void Moose::Kokkos::System::clearActiveResidualTags ( )

inline

Clear the cached active residual tags.

Definition at line 95 of file KokkosSystem.h.

  {
    _active_residual_tags.destroy();
    _residual_tag_active = false;
  }

clearActiveSolutionTags()

void Moose::Kokkos::System::clearActiveSolutionTags ( )

inline

Clear the cached active solution tags.

Definition at line 91 of file KokkosSystem.h.

{ _active_solution_tags.destroy(); }

clearActiveVariables()

void Moose::Kokkos::System::clearActiveVariables ( )

inline

Clear the cached active variables.

Definition at line 87 of file KokkosSystem.h.

{ _active_variables.destroy(); }

getComm()

const auto& Moose::Kokkos::System::getComm ( ) const

inline

Get the libMesh communicator.

Returns

The libMesh communicator

Definition at line 125 of file KokkosSystem.h.

{ return _comm; }

getCoupling()

KOKKOS_FUNCTION const auto& Moose::Kokkos::System::getCoupling ( unsigned int  var ) const

inline

Get the list of off-diagonal coupled variable numbers of a variable.

Parameters

var	The variable number

Returns

The list of off-diagonal coupled variable numbers

Definition at line 146 of file KokkosSystem.h.

Referenced by Moose::Kokkos::NodalBC::operator()(), Moose::Kokkos::NodalKernel::operator()(), Moose::Kokkos::Kernel::operator()(), and Moose::Kokkos::IntegratedBC::operator()().

{ return _coupling[var]; }

getDofMap()

const auto& Moose::Kokkos::System::getDofMap ( ) const

inline

Get the libMesh DOF map.

Returns

The libMesh DOF map

Definition at line 120 of file KokkosSystem.h.

{ return _system.dofMap(); }

getElemGlobalDofIndex()

KOKKOS_FUNCTION dof_id_type Moose::Kokkos::System::getElemGlobalDofIndex ( ContiguousElementID  elem, unsigned int  i, unsigned int  var  ) const

inline

Get the global DOF index of a variable for an element.

Parameters

elem	The contiguous element ID
i	The element-local DOF index
var	The variable number

Returns

The global DOF index

Definition at line 238 of file KokkosSystem.h.

  {
    return _local_to_global_dof_index[_local_elem_dof_index[var](i, elem)];
  }

getElemLocalDofIndex()

KOKKOS_FUNCTION dof_id_type Moose::Kokkos::System::getElemLocalDofIndex ( ContiguousElementID  elem, unsigned int  i, unsigned int  var  ) const

inline

Get the local DOF index of a variable for an element.

Parameters

elem	The contiguous element ID
i	The element-local DOF index
var	The variable number

Returns

The local DOF index

Definition at line 212 of file KokkosSystem.h.

Referenced by Moose::Kokkos::ResidualObject::accumulateTaggedElementalMatrix(), Moose::Kokkos::ResidualObject::accumulateTaggedElementalResidual(), getVectorQpADGrad(), getVectorQpADGradFace(), getVectorQpADValue(), getVectorQpADValueFace(), getVectorQpGradFace(), and getVectorQpValueFace().

  {
    return _local_elem_dof_index[var](i, elem);
  }

getFETypeID()

KOKKOS_FUNCTION unsigned int Moose::Kokkos::System::getFETypeID ( unsigned int  var ) const

inline

Get the FE type ID of a variable.

Parameters

var	The variable number

Returns

The FE type ID

Definition at line 184 of file KokkosSystem.h.

{ return _var_fe_types[var]; }

getGhostCommList()

const auto& Moose::Kokkos::System::getGhostCommList ( ) const

inline

Get the list of ghost DOF indices to communicate.

Returns

The list of ghost DOF indices to communicate

Definition at line 135 of file KokkosSystem.h.

{ return _ghost_comm_list; }

getLocalCommList()

const auto& Moose::Kokkos::System::getLocalCommList ( ) const

inline

Get the list of local DOF indices to communicate.

Returns

The list of local DOF indices to communicate

Definition at line 130 of file KokkosSystem.h.

{ return _local_comm_list; }

getMatrix()

KOKKOS_FUNCTION auto& Moose::Kokkos::System::getMatrix ( TagID  tag ) const

inline

Get a tagged Kokkos matrix.

Parameters

tag	The matrix tag

Returns

The Kokkos matrix

Definition at line 284 of file KokkosSystem.h.

{ return _matrices[tag]; }

getMatrixValue()

KOKKOS_FUNCTION Real& Moose::Kokkos::System::getMatrixValue ( dof_id_type  row, dof_id_type  col, TagID  tag  ) const

inline

Get an entry from a tagged matrix.

Parameters

row	The local row index
col	The global column index
tag	The matrix tag

Returns

The entry from the tagged matrix

Definition at line 443 of file KokkosSystem.h.

  {
    return _matrices[tag](row, col);
  }

getMaxDofsPerElem()

KOKKOS_FUNCTION unsigned int Moose::Kokkos::System::getMaxDofsPerElem ( unsigned int  var ) const

inline

Get the maximum number of DOFs per element of a variable This number is local to each process.

Parameters

var	The variable number

Returns

The maximum number of DOFs per element

Definition at line 201 of file KokkosSystem.h.

  {
    return _max_dofs_per_elem[var];
  }

getNodalBCDofs()

void Moose::Kokkos::System::getNodalBCDofs ( const NodalBCBase *  nbc, Array< bool > &  dofs  )

private

Get the list of DOFs covered by a nodal BC.

Parameters

nbc	The Kokkos nodal BC object
dofs	The flag whether each DOF is covered by the nodal BC

getNodeGlobalDofIndex()

KOKKOS_FUNCTION dof_id_type Moose::Kokkos::System::getNodeGlobalDofIndex ( ContiguousNodeID  node, unsigned int  var  ) const

inline

Get the global DOF index of a variable for a node.

Parameters

node	The contiguous node ID
var	The variable number

Returns

The global DOF index

Definition at line 250 of file KokkosSystem.h.

  {
    return _local_to_global_dof_index[_local_node_dof_index[var][node]];
  }

getNodeLocalDofIndex()

KOKKOS_FUNCTION dof_id_type Moose::Kokkos::System::getNodeLocalDofIndex ( ContiguousNodeID  node, unsigned int  i, unsigned int  var  ) const

inline

Get the local DOF index of a variable for a node.

Parameters

node	The contiguous node ID
idx	The node-local DOF index
var	The variable number

Returns

The local DOF index

Definition at line 225 of file KokkosSystem.h.

Referenced by Moose::Kokkos::ResidualObject::accumulateTaggedNodalMatrix(), and Moose::Kokkos::ResidualObject::accumulateTaggedNodalResidual().

  {
    return _local_node_dof_index[var][node] + i;
  }

getNumGhostDofs()

KOKKOS_FUNCTION dof_id_type Moose::Kokkos::System::getNumGhostDofs ( ) const

inline

Get the number of ghost DOFs.

Returns

The number of ghost DOFs

Definition at line 194 of file KokkosSystem.h.

{ return _num_ghost_dofs; }

getNumLocalDofs()

KOKKOS_FUNCTION dof_id_type Moose::Kokkos::System::getNumLocalDofs ( ) const

inline

Get the number of local DOFs.

Returns

The number of local DOFs

Definition at line 189 of file KokkosSystem.h.

{ return _num_local_dofs; }

getSparsity()

const auto& Moose::Kokkos::System::getSparsity ( ) const

inline

Get the sparisty pattern data.

Returns

The sparisty pattern data

Definition at line 140 of file KokkosSystem.h.

{ return _sparsity; }

getSystem() [1/2]

auto& Moose::Kokkos::System::getSystem ( )

inline

Get the MOOSE system.

Returns

The MOOSE system

Definition at line 113 of file KokkosSystem.h.

{ return _system; }

getSystem() [2/2]

const auto& Moose::Kokkos::System::getSystem ( ) const

inline

Definition at line 114 of file KokkosSystem.h.

{ return _system; }

getVector()

KOKKOS_FUNCTION auto& Moose::Kokkos::System::getVector ( TagID  tag ) const

inline

Get a tagged Kokkos vector.

Parameters

tag	The vector tag

Returns

The Kokkos vector

Definition at line 278 of file KokkosSystem.h.

{ return _vectors[tag]; }

getVectorDofADValue()

KOKKOS_FUNCTION ADReal Moose::Kokkos::System::getVectorDofADValue ( const dof_id_type  dof, const TagID  tag, const Real  seed  ) const

inline

Get the DOF value of a tagged vector for automatic differentiation (AD)

Parameters

dof	The local DOF index
tag	The vector tag
seed	The derivative seed

Returns

The DOF AD value with optional seed derivative

Definition at line 602 of file KokkosSystem.h.

Referenced by getVectorQpADGrad(), getVectorQpADGradFace(), getVectorQpADValue(), and getVectorQpADValueFace().

{
  ADReal value = _vectors[tag][dof];

  if (seed != 0)
    value.derivatives().insert(_local_to_global_dof_index[dof]) = seed;

  return value;
}

getVectorDofValue()

KOKKOS_FUNCTION Real& Moose::Kokkos::System::getVectorDofValue ( const dof_id_type  dof, const TagID  tag  ) const

inline

Get the DOF value of a tagged vector.

Parameters

dof	The local DOF index
tag	The vector tag

Returns

The DOF value

Definition at line 291 of file KokkosSystem.h.

Referenced by getVectorQpGradFace(), and getVectorQpValueFace().

  {
    return _vectors[tag][dof];
  }

getVectorQpADGrad()

KOKKOS_FUNCTION ADReal3 Moose::Kokkos::System::getVectorQpADGrad ( const ElementInfo  info, const Real33  jacobian, const dof_id_type  offset, const dof_id_type  qp, const unsigned int  var, const TagID  tag, const Real  seed  ) const

inline

Get the quadrature gradient of a variable from a tagged vector for automatic differentiation (AD)

Parameters

info	The element information object
jacobian	The inverse Jacobian matrix
offset	The offset into the global quadrature point index
qp	The local quadrature point index
var	The variable number
tag	The vector tag
seed	The derivative seed

Returns

The quadrature AD gradient

Definition at line 638 of file KokkosSystem.h.

{
  ADReal3 grad;

  if (seed == 0)
    grad = getVectorQpGrad(info, offset + qp, var, tag);
  else
  {
    auto fe = _var_fe_types[var];
    auto n_dofs = kokkosAssembly().getNumDofs(info.type, fe);
    auto & grad_phi = kokkosAssembly().getGradPhi(info.subdomain, info.type, fe);

    for (unsigned int i = 0; i < n_dofs; ++i)
      grad += getVectorDofADValue(getElemLocalDofIndex(info.id, i, var), tag, seed) *
              (jacobian * grad_phi(i, qp));
  }

  return grad;
}

getVectorQpADGradFace()

KOKKOS_FUNCTION ADReal3 Moose::Kokkos::System::getVectorQpADGradFace ( const ElementInfo  info, const unsigned int  side, const Real33  jacobian, const unsigned int  qp, const unsigned int  var, const TagID  tag, const Real  seed  ) const

inline

Get the face quadrature gradient of a variable from a tagged vector for automatic differentiation (AD)

Parameters

info	The element information object
side	The side index
jacobian	The inverse Jacobian matrix
qp	The local quadrature point index
var	The variable number
tag	The vector tag
seed	The derivative seed

Returns

The face quadrature AD gradient

Definition at line 725 of file KokkosSystem.h.

{
  auto fe = _var_fe_types[var];
  auto n_dofs = kokkosAssembly().getNumDofs(info.type, fe);
  auto & grad_phi = kokkosAssembly().getGradPhiFace(info.subdomain, info.type, fe)(side);

  ADReal3 grad = ADReal(0);

  for (unsigned int i = 0; i < n_dofs; ++i)
    grad += getVectorDofADValue(getElemLocalDofIndex(info.id, i, var), tag, seed) *
            (jacobian * grad_phi(i, qp));

  return grad;
}

getVectorQpADValue()

KOKKOS_FUNCTION ADReal Moose::Kokkos::System::getVectorQpADValue ( const ElementInfo  info, const dof_id_type  offset, const dof_id_type  qp, const unsigned int  var, const TagID  tag, const Real  seed  ) const

inline

Get the quadrature value of a variable from a tagged vector for automatic differentiation (AD)

Parameters

info	The element information object
offset	The offset into the global quadrature point index
qp	The local quadrature point index
var	The variable number
tag	The vector tag
seed	The derivative seed

Returns

The quadrature AD value

Definition at line 613 of file KokkosSystem.h.

{
  ADReal value = 0;

  if (seed == 0)
    value = getVectorQpValue(info, offset + qp, var, tag);
  else
  {
    auto fe = _var_fe_types[var];
    auto n_dofs = kokkosAssembly().getNumDofs(info.type, fe);
    auto & phi = kokkosAssembly().getPhi(info.subdomain, info.type, fe);

    for (unsigned int i = 0; i < n_dofs; ++i)
      value += getVectorDofADValue(getElemLocalDofIndex(info.id, i, var), tag, seed) * phi(i, qp);
  }

  return value;
}

getVectorQpADValueFace()

KOKKOS_FUNCTION ADReal Moose::Kokkos::System::getVectorQpADValueFace ( const ElementInfo  info, const unsigned int  side, const unsigned int  qp, const unsigned int  var, const TagID  tag, const Real  seed  ) const

inline

Get the face quadrature value of a variable from a tagged vector for automatic differentiation (AD)

Parameters

info	The element information object
side	The side index
qp	The local quadrature point index
var	The vriable number
tag	The vector tag
seed	The derivative seed

Returns

The face quadrature AD value

Definition at line 684 of file KokkosSystem.h.

{
  auto fe = _var_fe_types[var];
  auto n_dofs = kokkosAssembly().getNumDofs(info.type, fe);
  auto & phi = kokkosAssembly().getPhiFace(info.subdomain, info.type, fe)(side);

  ADReal value = 0;

  for (unsigned int i = 0; i < n_dofs; ++i)
    value += getVectorDofADValue(getElemLocalDofIndex(info.id, i, var), tag, seed) * phi(i, qp);

  return value;
}

getVectorQpGrad()

KOKKOS_FUNCTION Real3& Moose::Kokkos::System::getVectorQpGrad ( const ElementInfo  info, const dof_id_type  qp, const unsigned int  var, const TagID  tag  ) const

inline

Get the quadrature gradient of a variable from a tagged vector.

Parameters

info	The element information object
qp	The global quadrature point index
var	The variable number
tag	The vector tag

Returns

The quadrature gradient

Definition at line 344 of file KokkosSystem.h.

Referenced by getVectorQpADGrad().

  {
    return _qp_solutions_grad[tag](info.subdomain, var)[qp];
  }

getVectorQpGradFace()

KOKKOS_FUNCTION Real3 Moose::Kokkos::System::getVectorQpGradFace ( const ElementInfo  info, const unsigned int  side, const Real33  jacobian, const unsigned int  qp, const unsigned int  var, const TagID  tag  ) const

inline

Get the face quadrature gradient of a variable from a tagged vector.

Parameters

info	The element information object
side	The side index
jacobian	The inverse Jacobian matrix
qp	The local quadrature point index
var	The variable number
tag	The vector tag

Returns

The face quadrature gradient

Definition at line 704 of file KokkosSystem.h.

{
  auto fe = _var_fe_types[var];
  auto n_dofs = kokkosAssembly().getNumDofs(info.type, fe);
  auto & grad_phi = kokkosAssembly().getGradPhiFace(info.subdomain, info.type, fe)(side);

  Real3 grad = 0;

  for (unsigned int i = 0; i < n_dofs; ++i)
    grad += getVectorDofValue(getElemLocalDofIndex(info.id, i, var), tag) *
            (jacobian * grad_phi(i, qp));

  return grad;
}

getVectorQpValue()

KOKKOS_FUNCTION Real& Moose::Kokkos::System::getVectorQpValue ( const ElementInfo  info, const dof_id_type  qp, const unsigned int  var, const TagID  tag  ) const

inline

Get the quadrature value of a variable from a tagged vector.

Parameters

info	The element information object
qp	The global quadrature point index
var	The variable number
tag	The vector tag

Returns

The quadrature value

Definition at line 313 of file KokkosSystem.h.

Referenced by getVectorQpADValue().

  {
    return _qp_solutions[tag](info.subdomain, var)[qp];
  }

getVectorQpValueFace()

KOKKOS_FUNCTION Real Moose::Kokkos::System::getVectorQpValueFace ( const ElementInfo  info, const unsigned int  side, const unsigned int  qp, const unsigned int  var, const TagID  tag  ) const

inline

Get the face quadrature value of a variable from a tagged vector.

Parameters

info	The element information object
side	The side index
qp	The local quadrature point index
var	The vriable number
tag	The vector tag

Returns

The face quadrature value

Definition at line 665 of file KokkosSystem.h.

{
  auto fe = _var_fe_types[var];
  auto n_dofs = kokkosAssembly().getNumDofs(info.type, fe);
  auto & phi = kokkosAssembly().getPhiFace(info.subdomain, info.type, fe)(side);

  Real value = 0;

  for (unsigned int i = 0; i < n_dofs; ++i)
    value += getVectorDofValue(getElemLocalDofIndex(info.id, i, var), tag) * phi(i, qp);

  return value;
}

hasNodalBCMatrixTag()

KOKKOS_FUNCTION bool Moose::Kokkos::System::hasNodalBCMatrixTag ( dof_id_type  dof, TagID  tag  ) const

inline

Check whether a local DOF index is associated with a nodal BC for an extra matrix tag.

Parameters

dof	The local DOF index
tag	The extra matrix tag

Returns

Whether the local DOF index is covered by a nodal BC

Definition at line 175 of file KokkosSystem.h.

  {
    return _nbc_matrix_tag_dof[tag].isAlloc() && _nbc_matrix_tag_dof[tag][dof];
  }

isMatrixTagActive()

KOKKOS_FUNCTION bool Moose::Kokkos::System::isMatrixTagActive ( TagID  tag ) const

inline

Check whether a matrix tag is active.

Parameters

tag	The matrix tag

Returns

Whether the matrix tag is active

Definition at line 168 of file KokkosSystem.h.

{ return _matrix_tag_active[tag]; }

isNodalDefined()

KOKKOS_FUNCTION bool Moose::Kokkos::System::isNodalDefined ( ContiguousNodeID  node, unsigned int  var  ) const

inline

Get whether a variable is defined on a node.

Parameters

node	The contiguous node ID
var	The variable number

Returns

Whether the variable is defined on the node

Definition at line 269 of file KokkosSystem.h.

  {
    return _local_node_dof_index[var][node] != libMesh::DofObject::invalid_id;
  }

isResidualTagActive()

KOKKOS_FUNCTION bool Moose::Kokkos::System::isResidualTagActive ( TagID  tag ) const

inline

Check whether a residual tag is active.

Parameters

tag	The residual tag

Returns

Whether the residual tag is active

Definition at line 162 of file KokkosSystem.h.

{ return _residual_tag_active[tag]; }

isVariableActive()

KOKKOS_FUNCTION bool Moose::Kokkos::System::isVariableActive ( unsigned int  var, ContiguousSubdomainID  subdomain  ) const

inline

Check whether a variable is active on a subdomain.

Parameters

var	The variable number
subdomain	The contiguous subdomain ID

Returns

Whether the variable is active

Definition at line 153 of file KokkosSystem.h.

  {
    return _var_subdomain_active(var, subdomain);
  }

kokkosAssembly()

KOKKOS_FUNCTION const Assembly& Moose::Kokkos::AssemblyHolder::kokkosAssembly ( ) const

inlineinherited

Get the const reference of the Kokkos assembly.

Returns

The const reference of the Kokkos assembly depending on the architecture this function is being called on

Definition at line 569 of file KokkosAssembly.h.

Referenced by getVectorQpADGrad(), getVectorQpADGradFace(), getVectorQpADValue(), getVectorQpADValueFace(), getVectorQpGradFace(), getVectorQpValueFace(), Moose::Kokkos::SideReducer::operator()(), Moose::Kokkos::ElementReducer::operator()(), Moose::Kokkos::NodalReducer::operator()(), Moose::Kokkos::NodalUserObject::operator()(), Moose::Kokkos::SideUserObject::operator()(), Moose::Kokkos::ElementUserObject::operator()(), Moose::Kokkos::ADNodalBC::operator()(), Moose::Kokkos::ADKernel::operator()(), Moose::Kokkos::ADIntegratedBC::operator()(), Moose::Kokkos::AuxKernel::operator()(), Moose::Kokkos::Material::operator()(), Moose::Kokkos::NodalBC::operator()(), Moose::Kokkos::NodalKernel::operator()(), Moose::Kokkos::IntegratedBC::operator()(), and Moose::Kokkos::Kernel::operator()().

  {
    KOKKOS_IF_ON_HOST(return _assembly_host;)

    return _assembly_device;
  }

kokkosMesh()

KOKKOS_FUNCTION const Mesh& Moose::Kokkos::MeshHolder::kokkosMesh ( ) const

inlineinherited

Get the const reference of the Kokkos mesh.

Returns

The const reference of the Kokkos mesh depending on the architecture this function is being called on

Definition at line 452 of file KokkosMesh.h.

Referenced by Moose::Kokkos::Assembly::computePhysicalMap(), Moose::Kokkos::Kernel::operator()(), Moose::Kokkos::IntegratedBC::operator()(), Moose::Kokkos::Datum::q_point(), and KokkosBoundNodalKernel::skipOnBoundary().

  {
    KOKKOS_IF_ON_HOST(
        if (!_mesh_host.initialized()) mooseError(
            "kokkosMesh() was called too early. Kokkos mesh is available after problem "
            "initialization. Override initialSetup() if you need to setup your object data "
            "using the Kokkos mesh.");

        return _mesh_host;)

    return _mesh_device;
  }

localToGlobalDofIndex()

KOKKOS_FUNCTION dof_id_type Moose::Kokkos::System::localToGlobalDofIndex ( dof_id_type  dof ) const

inline

Get the global DOF index of a local DOF index.

Parameters

dof	The local DOF index

Returns

The global DOF index

Definition at line 259 of file KokkosSystem.h.

  {
    return _local_to_global_dof_index[dof];
  }

operator()()

KOKKOS_FUNCTION void Moose::Kokkos::System::operator()

(

const ThreadID

tid

)

const

Kokkos function for caching variable values on element quadrature points.

perfGraph()

PerfGraph & PerfGraphInterface::perfGraph ( )

inherited

Get the PerfGraph.

Definition at line 86 of file PerfGraphInterface.C.

Referenced by CommonOutputAction::act(), PerfGraphData::finalize(), PerfGraphReporter::finalize(), and PerfGraphOutput::output().

{
  return _pg_moose_app.perfGraph();
}

registerTimedSection() [1/2]

PerfID PerfGraphInterface::registerTimedSection ( const std::string &  section_name, const unsigned int  level  ) const

protectedinherited

Call to register a named section for timing.

Parameters

section_name	The name of the code section to be timed
level	The importance of the timer - lower is more important (0 will always come out)

Returns

The ID of the section - use when starting timing

Definition at line 61 of file PerfGraphInterface.C.

{
  const auto timed_section_name = timedSectionName(section_name);
  if (!moose::internal::getPerfGraphRegistry().sectionExists(timed_section_name))
    return moose::internal::getPerfGraphRegistry().registerSection(timed_section_name, level);
  else
    return moose::internal::getPerfGraphRegistry().sectionID(timed_section_name);
}

registerTimedSection() [2/2]

PerfID PerfGraphInterface::registerTimedSection ( const std::string &  section_name, const unsigned int  level, const std::string &  live_message, const bool  print_dots = true  ) const

protectedinherited

Call to register a named section for timing.

Parameters

section_name	The name of the code section to be timed
level	The importance of the timer - lower is more important (0 will always come out)
live_message	The message to be printed to the screen during execution
print_dots	Whether or not progress dots should be printed for this section

Returns

The ID of the section - use when starting timing

Definition at line 72 of file PerfGraphInterface.C.

{
  const auto timed_section_name = timedSectionName(section_name);
  if (!moose::internal::getPerfGraphRegistry().sectionExists(timed_section_name))
    return moose::internal::getPerfGraphRegistry().registerSection(
        timedSectionName(section_name), level, live_message, print_dots);
  else
    return moose::internal::getPerfGraphRegistry().sectionID(timed_section_name);
}

reinit()

void Moose::Kokkos::System::reinit

(

)

Allocate the quadrature point vectors for active variable and tags and cache quadrature point values.

setActiveMatrixTags()

void Moose::Kokkos::System::setActiveMatrixTags

(

const std::set< TagID > &

tags

)

Set the active matrix tags.

Parameters

vars	The active matrix tags

setActiveResidualTags()

void Moose::Kokkos::System::setActiveResidualTags

(

const std::set< TagID > &

tags

)

Set the active residual tags.

Parameters

tags	The active residual tags

setActiveSolutionTags()

void Moose::Kokkos::System::setActiveSolutionTags

(

const std::set< TagID > &

tags

)

Set the active solution tags.

Parameters

tags	The active solution tags

setActiveVariables()

void Moose::Kokkos::System::setActiveVariables

(

const std::set< MooseVariableFieldBase *> &

vars

)

Set the active variables.

Parameters

vars	The active MOOSE variables

setupDofs()

void Moose::Kokkos::System::setupDofs ( )

private

Setup DOF data.

setupNodalBCDofs()

void Moose::Kokkos::System::setupNodalBCDofs ( )

private

Mark the DOFs covered by nodal BCs.

setupSparsity()

void Moose::Kokkos::System::setupSparsity ( )

private

Setup sparsity data.

setupVariables()

void Moose::Kokkos::System::setupVariables ( )

private

Setup variable data.

sync() [1/4]

void Moose::Kokkos::System::sync

(

const MemcpyType

dir

)

Synchronize the active tagged vectors and matrices between host and device.

Parameters

dir	Copy direction

sync() [2/4]

void Moose::Kokkos::System::sync	(	const std::set< TagID > &	tags,
		const MemcpyType	dir
	)

Synchronize the specified tagged vectors between host and device.

Parameters

tags	The vector tags
dir	Copy direction

sync() [3/4]

void Moose::Kokkos::System::sync	(	const std::vector< TagID > &	tags,
		const MemcpyType	dir
	)

sync() [4/4]

void Moose::Kokkos::System::sync	(	const TagID	tag,
		const MemcpyType	dir
	)

timedSectionName()

std::string PerfGraphInterface::timedSectionName ( const std::string &  section_name ) const

protectedinherited

Returns

The name of the timed section with the name section_name.

Optionally adds a prefix if one is defined.

Definition at line 55 of file PerfGraphInterface.C.

Referenced by PerfGraphInterface::registerTimedSection().

{
  return _prefix.empty() ? "" : (_prefix + "::") + section_name;
}

validParams()

InputParameters PerfGraphInterface::validParams ( )

staticinherited

Definition at line 16 of file PerfGraphInterface.C.

Referenced by Convergence::validParams().

{
  InputParameters params = emptyInputParameters();
  return params;
}

Member Data Documentation

_active_matrix_tags

Array<TagID> Moose::Kokkos::System::_active_matrix_tags

private

Definition at line 573 of file KokkosSystem.h.

Referenced by clearActiveMatrixTags().

_active_residual_tags

Array<TagID> Moose::Kokkos::System::_active_residual_tags

private

Definition at line 572 of file KokkosSystem.h.

Referenced by clearActiveResidualTags().

_active_solution_tags

Array<TagID> Moose::Kokkos::System::_active_solution_tags

private

List of active tags.

Definition at line 571 of file KokkosSystem.h.

Referenced by clearActiveSolutionTags().

_active_variables

Array<unsigned int> Moose::Kokkos::System::_active_variables

private

List of active variable numbers.

Definition at line 566 of file KokkosSystem.h.

Referenced by clearActiveVariables().

_comm

const Parallel::Communicator& Moose::Kokkos::System::_comm

private

Reference of the libMesh communicator.

Definition at line 507 of file KokkosSystem.h.

Referenced by getComm().

_coupling

Array<Array<unsigned int> > Moose::Kokkos::System::_coupling

private

Off-diagonal coupled variable numbers of each variable.

Definition at line 561 of file KokkosSystem.h.

Referenced by getCoupling().

_dof_map

const libMesh::DofMap& Moose::Kokkos::System::_dof_map

private

Reference of the libMesh DOF map.

Definition at line 503 of file KokkosSystem.h.

_ghost_comm_list

Array<Array<dof_id_type> > Moose::Kokkos::System::_ghost_comm_list

private

Definition at line 591 of file KokkosSystem.h.

Referenced by getGhostCommList().

_local_comm_list

Array<Array<dof_id_type> > Moose::Kokkos::System::_local_comm_list

private

List of DOFs to send and receive.

Definition at line 590 of file KokkosSystem.h.

Referenced by getLocalCommList().

_local_elem_dof_index

Array<Array2D<dof_id_type> > Moose::Kokkos::System::_local_elem_dof_index

private

Local DOF index of each variable.

Definition at line 539 of file KokkosSystem.h.

Referenced by getElemGlobalDofIndex(), and getElemLocalDofIndex().

_local_node_dof_index

Array<Array<dof_id_type> > Moose::Kokkos::System::_local_node_dof_index

private

Definition at line 540 of file KokkosSystem.h.

Referenced by getNodeGlobalDofIndex(), getNodeLocalDofIndex(), and isNodalDefined().

_local_to_global_dof_index

Array<dof_id_type> Moose::Kokkos::System::_local_to_global_dof_index

private

Map from local DOF index to global DOF index.

Definition at line 545 of file KokkosSystem.h.

Referenced by getElemGlobalDofIndex(), getNodeGlobalDofIndex(), getVectorDofADValue(), and localToGlobalDofIndex().

_matrices

Array<Matrix> Moose::Kokkos::System::_matrices

private

Definition at line 526 of file KokkosSystem.h.

Referenced by getMatrix(), and getMatrixValue().

_matrix_tag_active

Array<bool> Moose::Kokkos::System::_matrix_tag_active

private

Definition at line 580 of file KokkosSystem.h.

Referenced by clearActiveMatrixTags(), and isMatrixTagActive().

_max_dofs_per_elem

Array<unsigned int> Moose::Kokkos::System::_max_dofs_per_elem

private

Maximum number of DOFs per element for each variable.

Definition at line 549 of file KokkosSystem.h.

Referenced by getMaxDofsPerElem().

_mesh

const MooseMesh& Moose::Kokkos::System::_mesh

private

Reference of the MOOSE mesh.

Definition at line 499 of file KokkosSystem.h.

_nbc_matrix_tag_dof

Array<Array<bool> > Moose::Kokkos::System::_nbc_matrix_tag_dof

private

Flag whether each DOF is covered by a nodal BC for each matrix tag.

Definition at line 585 of file KokkosSystem.h.

Referenced by hasNodalBCMatrixTag().

_num_ghost_dofs

const dof_id_type Moose::Kokkos::System::_num_ghost_dofs

private

Number of ghost DOFs.

Definition at line 519 of file KokkosSystem.h.

Referenced by getNumGhostDofs().

_num_local_dofs

const dof_id_type Moose::Kokkos::System::_num_local_dofs

private

Number of local DOFs.

Definition at line 515 of file KokkosSystem.h.

Referenced by getNumLocalDofs().

_num_vars

const unsigned int Moose::Kokkos::System::_num_vars

private

Number of variables.

Definition at line 511 of file KokkosSystem.h.

_pg_moose_app

MooseApp& PerfGraphInterface::_pg_moose_app

protectedinherited

The MooseApp that owns the PerfGraph.

Definition at line 135 of file PerfGraphInterface.h.

Referenced by PerfGraphInterface::perfGraph().

_prefix

const std::string PerfGraphInterface::_prefix

protectedinherited

A prefix to use for all sections.

Definition at line 138 of file PerfGraphInterface.h.

Referenced by PerfGraphInterface::timedSectionName().

_qp_solutions

Array<Array2D<Array<Real> > > Moose::Kokkos::System::_qp_solutions

private

Cached elemental quadrature values and gradients.

Definition at line 532 of file KokkosSystem.h.

Referenced by getVectorQpValue().

_qp_solutions_grad

Array<Array2D<Array<Real3> > > Moose::Kokkos::System::_qp_solutions_grad

private

Definition at line 533 of file KokkosSystem.h.

Referenced by getVectorQpGrad().

_residual_tag_active

Array<bool> Moose::Kokkos::System::_residual_tag_active

private

Flag whether each tag is active.

Definition at line 579 of file KokkosSystem.h.

Referenced by clearActiveResidualTags(), and isResidualTagActive().

_sparsity

Sparsity Moose::Kokkos::System::_sparsity

private

Matrix sparsity pattern data.

Definition at line 597 of file KokkosSystem.h.

Referenced by getSparsity().

_system

SystemBase& Moose::Kokkos::System::_system

private

Reference of the MOOSE system.

Definition at line 495 of file KokkosSystem.h.

Referenced by getDofMap(), and getSystem().

_thread

Thread Moose::Kokkos::System::_thread

private

Kokkos thread object.

Definition at line 491 of file KokkosSystem.h.

_var_fe_types

Array<unsigned int> Moose::Kokkos::System::_var_fe_types

private

FE type ID of each variable.

Definition at line 553 of file KokkosSystem.h.

Referenced by getFETypeID(), getVectorQpADGrad(), getVectorQpADGradFace(), getVectorQpADValue(), getVectorQpADValueFace(), getVectorQpGradFace(), and getVectorQpValueFace().

_var_subdomain_active

Array2D<bool> Moose::Kokkos::System::_var_subdomain_active

private

Whether each variable is active on subdomains.

Definition at line 557 of file KokkosSystem.h.

Referenced by isVariableActive().

_vectors

Array<Vector> Moose::Kokkos::System::_vectors

private

Kokkos vectors and matrices on device.

Definition at line 525 of file KokkosSystem.h.

Referenced by getVector(), getVectorDofADValue(), and getVectorDofValue().

---

The documentation for this class was generated from the following file:

• include/kokkos/systems/KokkosSystem.h