The Kokkos base system class. More...
#include <KokkosSystem.h>
Classes | |
| struct | Sparsity |
| CSR format sparsity data. More... | |
Public Member Functions | |
| System (SystemBase &system) | |
| Constructor. | |
| System (const System &src)=default | |
| Defaulted copy constructor Used by FESystem in mixed FE+FV simulations. | |
| void | sync (const MemcpyType dir) |
| Synchronize the active tagged vectors and matrices between host and device. | |
| void | setActiveVariables (const std::set< MooseVariableFieldBase * > &vars) |
| Set the active variables. | |
| void | setActiveSolutionTags (const std::set< TagID > &tags) |
| Set the active solution tags. | |
| void | setActiveResidualTags (const std::set< TagID > &tags) |
| Set the active residual tags. | |
| void | setActiveMatrixTags (const std::set< TagID > &tags) |
| Set the active matrix tags. | |
| void | clearActiveVariables () |
| Clear the cached active variables. | |
| void | clearActiveSolutionTags () |
| Clear the cached active solution tags. | |
| void | clearActiveResidualTags () |
| Clear the cached active residual tags. | |
| void | clearActiveMatrixTags () |
| Clear the cached active matrix tags. | |
| const libMesh::DofMap & | getDofMap () const |
| Get the libMesh DOF map. | |
| const Parallel::Communicator & | getComm () const |
| Get the libMesh communicator. | |
| const Array< Array< dof_id_type > > & | getLocalCommList () const |
| Get the list of local DOF indices to communicate. | |
| const Array< Array< dof_id_type > > & | getGhostCommList () const |
| Get the list of ghost DOF indices to communicate. | |
| const Sparsity & | getSparsity () const |
| Get the sparisty pattern data. | |
| void | buildCooIndices (std::vector< PetscInt > &coo_i, std::vector< PetscInt > &coo_j) const |
| Build the COO index arrays describing the sparsity pattern. | |
| KOKKOS_FUNCTION bool | isScalarVariable (unsigned int var) const |
| Check whether a variable is scalar. | |
| KOKKOS_FUNCTION bool | isVariableActive (unsigned int var, ContiguousSubdomainID subdomain) const |
| Check whether a variable is active on a subdomain. | |
| KOKKOS_FUNCTION bool | isResidualTagActive (TagID tag) const |
| Check whether a residual tag is active. | |
| KOKKOS_FUNCTION bool | isMatrixTagActive (TagID tag) const |
| Check whether a matrix tag is active. | |
| KOKKOS_FUNCTION dof_id_type | getNumLocalDofs () const |
| Get the number of local DOFs. | |
| KOKKOS_FUNCTION dof_id_type | getNumGhostDofs () const |
| Get the number of ghost DOFs. | |
| KOKKOS_FUNCTION unsigned int | getNumScalarDofs (unsigned int var) const |
| Get the number of DOFs of a scalar variable. | |
| KOKKOS_FUNCTION dof_id_type | getElemLocalDofIndex (ContiguousElementID elem, unsigned int i, unsigned int var) const |
| Get the local DOF index of a variable for an element. | |
| KOKKOS_FUNCTION dof_id_type | getScalarLocalDofIndex (unsigned int i, unsigned int var) const |
| Get the local DOF index of a scalar variable. | |
| KOKKOS_FUNCTION dof_id_type | getElemGlobalDofIndex (ContiguousElementID elem, unsigned int i, unsigned int var) const |
| Get the global DOF index of a variable for an element. | |
| KOKKOS_FUNCTION dof_id_type | getScalarGlobalDofIndex (unsigned int i, unsigned int var) const |
| Get the global DOF index of a scalar variable. | |
| KOKKOS_FUNCTION dof_id_type | localToGlobalDofIndex (dof_id_type dof) const |
| Get the global DOF index of a local DOF index. | |
| KOKKOS_FUNCTION Vector & | getVector (TagID tag) const |
| Get a tagged Kokkos vector. | |
| KOKKOS_FUNCTION Matrix & | getMatrix (TagID tag) const |
| Get a tagged Kokkos matrix. | |
| KOKKOS_FUNCTION Real & | getVectorDofValue (const dof_id_type dof, const TagID tag) const |
| Get the DOF value of a tagged vector. | |
| KOKKOS_FUNCTION Real & | getMatrixValue (dof_id_type row, dof_id_type col, TagID tag) const |
| Get an entry from a tagged matrix. | |
| PerfGraph & | perfGraph () |
| Get the PerfGraph. | |
| KOKKOS_FUNCTION const Mesh & | kokkosMesh () const |
| Get the const reference of the Kokkos mesh. | |
| void | sync (const std::set< TagID > &tags, const MemcpyType dir) |
| Synchronize the specified tagged vectors between host and device. | |
| void | sync (const std::vector< TagID > &tags, const MemcpyType dir) |
| void | sync (const TagID tag, const MemcpyType dir) |
| SystemBase & | getSystem () |
| Get the MOOSE system. | |
| const SystemBase & | getSystem () const |
Static Public Member Functions | |
| static InputParameters | validParams () |
Protected Member Functions | |
| PerfID | registerTimedSection (const std::string §ion_name, const unsigned int level) const |
| Call to register a named section for timing. | |
| PerfID | registerTimedSection (const std::string §ion_name, const unsigned int level, const std::string &live_message, const bool print_dots=true) const |
| Call to register a named section for timing. | |
| std::string | timedSectionName (const std::string §ion_name) const |
Protected Attributes | |
| SystemBase & | _system |
| Reference of the MOOSE system. | |
| const MooseMesh & | _mesh |
| Reference of the MOOSE mesh. | |
| const libMesh::DofMap & | _dof_map |
| Reference of the libMesh DOF map. | |
| const Parallel::Communicator & | _comm |
| Reference of the libMesh communicator. | |
| const unsigned int | _num_vars |
| Number of variables. | |
| const dof_id_type | _num_local_dofs |
| Number of local DOFs. | |
| const dof_id_type | _num_ghost_dofs |
| Number of ghost DOFs. | |
| Array< Array2D< dof_id_type > > | _local_elem_dof_index |
| Local element DOF indices of each variable. | |
| Array< Array< dof_id_type > > | _scalar_dof_index |
| DOF indices of each scalar variable. | |
| Array< dof_id_type > | _local_to_global_dof_index |
| Map from local DOF index to global DOF index. | |
| Array< unsigned int > | _max_dofs_per_elem |
| Maximum number of DOFs per element for each variable. | |
| Array< bool > | _var_is_scalar |
| Whether each variable is scalar. | |
| Array2D< bool > | _var_subdomain_active |
| Whether each variable is active on subdomains. | |
| Array< unsigned int > | _active_variables |
| List of active variable numbers. | |
| MooseApp & | _pg_moose_app |
| The MooseApp that owns the PerfGraph. | |
| const std::string | _prefix |
| A prefix to use for all sections. | |
| Array< Vector > | _vectors |
| Kokkos vectors and matrices on device. | |
| Array< Matrix > | _matrices |
| Array< TagID > | _active_solution_tags |
| List of active tags. | |
| Array< TagID > | _active_residual_tags |
| Array< TagID > | _active_matrix_tags |
| Array< bool > | _residual_tag_active |
| Flag whether each tag is active. | |
| Array< bool > | _matrix_tag_active |
| Array< Array< dof_id_type > > | _local_comm_list |
| List of DOFs to send and receive. | |
| Array< Array< dof_id_type > > | _ghost_comm_list |
Private Member Functions | |
| void | setupVariables () |
| Setup variable data. | |
| void | setupDofs () |
| Setup DOF data. | |
| void | setupSparsity () |
| Setup sparsity data. | |
Private Attributes | |
| Sparsity | _sparsity |
| Matrix sparsity pattern data. | |
| const Mesh & | _mesh_host |
| Host reference of the Kokkos mesh. | |
| const Mesh | _mesh_device |
| Device copy of the Kokkos mesh. | |
The Kokkos base system class.
Each system in MOOSE has a corresponding Kokkos base system.
Definition at line 30 of file KokkosSystem.h.
| Moose::Kokkos::System::System | ( | SystemBase & | system | ) |
Constructor.
| system | The associated MOOSE system |
|
default |
Defaulted copy constructor Used by FESystem in mixed FE+FV simulations.
| void Moose::Kokkos::System::buildCooIndices | ( | std::vector< PetscInt > & | coo_i, |
| std::vector< PetscInt > & | coo_j | ||
| ) | const |
Build the COO index arrays describing the sparsity pattern.
PETSc consumes these arrays destructively, so a fresh pair is needed for each matrix preallocated from this system. The arrays are resized to the number of nonzeros and fully overwritten, which lets one allocation be reused across the matrices.
| coo_i | The row index of each nonzero |
| coo_j | The column index of each nonzero |
|
inline |
Clear the cached active matrix tags.
Definition at line 117 of file KokkosSystem.h.
|
inline |
Clear the cached active residual tags.
Definition at line 108 of file KokkosSystem.h.
|
inline |
Clear the cached active solution tags.
Definition at line 103 of file KokkosSystem.h.
|
inline |
Clear the cached active variables.
Definition at line 98 of file KokkosSystem.h.
|
inline |
Get the libMesh communicator.
Definition at line 142 of file KokkosSystem.h.
|
inline |
Get the libMesh DOF map.
Definition at line 136 of file KokkosSystem.h.
|
inline |
Get the global DOF index of a variable for an element.
| elem | The contiguous element ID |
| i | The element-local DOF index |
| var | The variable number |
Definition at line 260 of file KokkosSystem.h.
|
inline |
Get the local DOF index of a variable for an element.
| elem | The contiguous element ID |
| i | The element-local DOF index |
| var | The variable number |
Definition at line 235 of file KokkosSystem.h.
Referenced by getElemGlobalDofIndex(), Moose::Kokkos::FESystem::getVectorQpADGrad(), Moose::Kokkos::FESystem::getVectorQpADGradFace(), Moose::Kokkos::FESystem::getVectorQpADValue(), Moose::Kokkos::FESystem::getVectorQpADValueFace(), Moose::Kokkos::FESystem::getVectorQpGradFace(), Moose::Kokkos::FESystem::getVectorQpValueFace(), Moose::Kokkos::FESystem::getVectorQpVectorGradFace(), and Moose::Kokkos::FESystem::getVectorQpVectorValueFace().
Get the list of ghost DOF indices to communicate.
Definition at line 154 of file KokkosSystem.h.
Get the list of local DOF indices to communicate.
Definition at line 148 of file KokkosSystem.h.
Get a tagged Kokkos matrix.
| tag | The matrix tag |
Definition at line 300 of file KokkosSystem.h.
|
inline |
Get an entry from a tagged matrix.
| row | The local row index |
| col | The global column index |
| tag | The matrix tag |
Definition at line 320 of file KokkosSystem.h.
|
inline |
Get the number of ghost DOFs.
Definition at line 216 of file KokkosSystem.h.
|
inline |
Get the number of local DOFs.
Definition at line 210 of file KokkosSystem.h.
|
inline |
Get the number of DOFs of a scalar variable.
| var | The variable number |
Definition at line 223 of file KokkosSystem.h.
|
inline |
Get the global DOF index of a scalar variable.
| i | The scalar DOF index |
| var | The variable number |
Definition at line 273 of file KokkosSystem.h.
|
inline |
Get the local DOF index of a scalar variable.
| i | The scalar DOF index |
| var | The variable number |
Definition at line 248 of file KokkosSystem.h.
Referenced by Moose::Kokkos::VariableValueTempl< is_ad >::get(), and getScalarGlobalDofIndex().
|
inline |
Get the sparisty pattern data.
Definition at line 160 of file KokkosSystem.h.
|
inline |
Get the MOOSE system.
Definition at line 128 of file KokkosSystem.h.
|
inline |
Definition at line 129 of file KokkosSystem.h.
Get a tagged Kokkos vector.
| tag | The vector tag |
Definition at line 293 of file KokkosSystem.h.
|
inline |
Get the DOF value of a tagged vector.
| dof | The local DOF index |
| tag | The vector tag |
Definition at line 308 of file KokkosSystem.h.
Referenced by Moose::Kokkos::FESystem::getVectorQpGradFace(), Moose::Kokkos::FESystem::getVectorQpValueFace(), Moose::Kokkos::FESystem::getVectorQpVectorGradFace(), and Moose::Kokkos::FESystem::getVectorQpVectorValueFace().
|
inline |
Check whether a matrix tag is active.
| tag | The matrix tag |
Definition at line 204 of file KokkosSystem.h.
|
inline |
Check whether a residual tag is active.
| tag | The residual tag |
Definition at line 197 of file KokkosSystem.h.
|
inline |
Check whether a variable is scalar.
| var | The variable number |
Definition at line 179 of file KokkosSystem.h.
|
inline |
Check whether a variable is active on a subdomain.
| var | The variable number |
| subdomain | The contiguous subdomain ID |
Definition at line 187 of file KokkosSystem.h.
|
inlineinherited |
Get the const reference of the Kokkos mesh.
Definition at line 651 of file KokkosMesh.h.
Referenced by Moose::Kokkos::Assembly::computePhysicalMap(), Moose::Kokkos::Assembly::computePhysicalMap(), Moose::Kokkos::LinearFVFluxKernel::operator()(), Moose::Kokkos::LinearFVBoundaryCondition::operator()(), Moose::Kokkos::LinearFVFluxKernel::operator()(), Moose::Kokkos::LinearFVBoundaryCondition::operator()(), Moose::Kokkos::LinearFVFluxKernel::operator()(), Moose::Kokkos::LinearFVFluxKernel::operator()(), Moose::Kokkos::LinearFVElementalKernel::operator()(), Moose::Kokkos::IntegratedBC::operator()(), Moose::Kokkos::VectorIntegratedBC::operator()(), Moose::Kokkos::Kernel::operator()(), Moose::Kokkos::VectorKernel::operator()(), Moose::Kokkos::IntegratedBC::operator()(), Moose::Kokkos::Kernel::operator()(), Moose::Kokkos::LinearFVElementalKernel::operator()(), Moose::Kokkos::Datum::q_point(), and KokkosBoundNodalKernel::skipOnBoundary().
|
inline |
Get the global DOF index of a local DOF index.
| dof | The local DOF index |
Definition at line 283 of file KokkosSystem.h.
|
inherited |
Get the PerfGraph.
Definition at line 86 of file PerfGraphInterface.C.
Referenced by CommonOutputAction::act(), PerfGraphData::finalize(), PerfGraphReporter::finalize(), and PerfGraphOutput::output().
|
protectedinherited |
Call to register a named section for timing.
| section_name | The name of the code section to be timed |
| level | The importance of the timer - lower is more important (0 will always come out) |
Definition at line 61 of file PerfGraphInterface.C.
|
protectedinherited |
Call to register a named section for timing.
| section_name | The name of the code section to be timed |
| level | The importance of the timer - lower is more important (0 will always come out) |
| live_message | The message to be printed to the screen during execution |
| print_dots | Whether or not progress dots should be printed for this section |
Definition at line 72 of file PerfGraphInterface.C.
| void Moose::Kokkos::System::setActiveMatrixTags | ( | const std::set< TagID > & | tags | ) |
Set the active matrix tags.
| vars | The active matrix tags |
| void Moose::Kokkos::System::setActiveResidualTags | ( | const std::set< TagID > & | tags | ) |
Set the active residual tags.
| tags | The active residual tags |
| void Moose::Kokkos::System::setActiveSolutionTags | ( | const std::set< TagID > & | tags | ) |
Set the active solution tags.
| tags | The active solution tags |
| void Moose::Kokkos::System::setActiveVariables | ( | const std::set< MooseVariableFieldBase * > & | vars | ) |
Set the active variables.
| vars | The active MOOSE variables |
|
private |
Setup DOF data.
|
private |
Setup sparsity data.
|
private |
Setup variable data.
| void Moose::Kokkos::System::sync | ( | const MemcpyType | dir | ) |
Synchronize the active tagged vectors and matrices between host and device.
| dir | Copy direction |
| void Moose::Kokkos::System::sync | ( | const std::set< TagID > & | tags, |
| const MemcpyType | dir | ||
| ) |
Synchronize the specified tagged vectors between host and device.
| tags | The vector tags |
| dir | Copy direction |
| void Moose::Kokkos::System::sync | ( | const std::vector< TagID > & | tags, |
| const MemcpyType | dir | ||
| ) |
| void Moose::Kokkos::System::sync | ( | const TagID | tag, |
| const MemcpyType | dir | ||
| ) |
|
protectedinherited |
section_name.Optionally adds a prefix if one is defined.
Definition at line 55 of file PerfGraphInterface.C.
Referenced by PerfGraphInterface::registerTimedSection(), and PerfGraphInterface::registerTimedSection().
|
staticinherited |
Definition at line 16 of file PerfGraphInterface.C.
Referenced by Convergence::validParams().
Definition at line 412 of file KokkosSystem.h.
Referenced by clearActiveMatrixTags().
Definition at line 411 of file KokkosSystem.h.
Referenced by clearActiveResidualTags().
List of active tags.
Definition at line 410 of file KokkosSystem.h.
Referenced by clearActiveSolutionTags().
List of active variable numbers.
Definition at line 404 of file KokkosSystem.h.
Referenced by clearActiveVariables().
|
protected |
Reference of the libMesh communicator.
Definition at line 346 of file KokkosSystem.h.
Referenced by getComm().
|
protected |
Reference of the libMesh DOF map.
Definition at line 341 of file KokkosSystem.h.
Referenced by getDofMap().
Definition at line 428 of file KokkosSystem.h.
Referenced by getGhostCommList().
List of DOFs to send and receive.
Definition at line 427 of file KokkosSystem.h.
Referenced by getLocalCommList().
Local element DOF indices of each variable.
Definition at line 374 of file KokkosSystem.h.
Referenced by getElemLocalDofIndex().
|
protected |
Map from local DOF index to global DOF index.
Definition at line 384 of file KokkosSystem.h.
Referenced by getElemGlobalDofIndex(), Moose::Kokkos::FESystem::getNodeGlobalDofIndex(), getScalarGlobalDofIndex(), Moose::Kokkos::FESystem::getVectorDofADValue(), and localToGlobalDofIndex().
Definition at line 368 of file KokkosSystem.h.
Referenced by getMatrix(), and getMatrixValue().
|
protected |
Definition at line 420 of file KokkosSystem.h.
Referenced by clearActiveMatrixTags(), and isMatrixTagActive().
Maximum number of DOFs per element for each variable.
Definition at line 389 of file KokkosSystem.h.
|
protected |
Reference of the MOOSE mesh.
Definition at line 336 of file KokkosSystem.h.
|
privateinherited |
Device copy of the Kokkos mesh.
Definition at line 673 of file KokkosMesh.h.
Referenced by Moose::Kokkos::MeshHolder::kokkosMesh().
|
privateinherited |
Host reference of the Kokkos mesh.
Definition at line 669 of file KokkosMesh.h.
Referenced by Moose::Kokkos::MeshHolder::kokkosMesh().
|
protected |
Number of ghost DOFs.
Definition at line 361 of file KokkosSystem.h.
Referenced by getNumGhostDofs().
|
protected |
Number of local DOFs.
Definition at line 356 of file KokkosSystem.h.
Referenced by getNumLocalDofs().
|
protected |
Number of variables.
Definition at line 351 of file KokkosSystem.h.
|
protectedinherited |
The MooseApp that owns the PerfGraph.
Definition at line 135 of file PerfGraphInterface.h.
Referenced by PerfGraphInterface::perfGraph().
|
protectedinherited |
A prefix to use for all sections.
Definition at line 138 of file PerfGraphInterface.h.
Referenced by PerfGraphInterface::timedSectionName().
|
protected |
Flag whether each tag is active.
Definition at line 419 of file KokkosSystem.h.
Referenced by clearActiveResidualTags(), and isResidualTagActive().
DOF indices of each scalar variable.
Definition at line 379 of file KokkosSystem.h.
Referenced by getNumScalarDofs(), and getScalarLocalDofIndex().
|
private |
Matrix sparsity pattern data.
Definition at line 450 of file KokkosSystem.h.
Referenced by getSparsity().
|
protected |
Reference of the MOOSE system.
Definition at line 331 of file KokkosSystem.h.
Referenced by getSystem(), and getSystem().
|
protected |
Whether each variable is scalar.
Definition at line 394 of file KokkosSystem.h.
Referenced by isScalarVariable().
|
protected |
Whether each variable is active on subdomains.
Definition at line 399 of file KokkosSystem.h.
Referenced by isVariableActive().
Kokkos vectors and matrices on device.
Definition at line 367 of file KokkosSystem.h.
Referenced by getVector(), Moose::Kokkos::FESystem::getVectorDofADValue(), and getVectorDofValue().