https://mooseframework.inl.gov
Public Types | Public Member Functions | Protected Member Functions | Protected Attributes | Private Member Functions | List of all members
Moose::Kokkos::Array< T, dimension, index_type, layout > Class Template Reference

The Kokkos array class. More...

#include <KokkosArray.h>

Inheritance diagram for Moose::Kokkos::Array< T, dimension, index_type, layout >:
[legend]

Public Types

using unsigned_index_type = index_type
 
using signed_index_type = typename std::make_signed< index_type >::type
 

Public Member Functions

 Array ()
 Default constructor. More...
 
 Array (const Array< T, dimension, index_type, layout > &array)
 Copy constructor. More...
 
template<typename... size_type>
 Array (size_type... n)
 Constructor Initialize and allocate array with given dimensions This allocates both host and device data. More...
 
auto & operator= (const Array< T, dimension, index_type, layout > &array)
 Shallow copy another Kokkos array. More...
 
KOKKOS_FUNCTION T & operator() (const signed_index_type(&idx)[dimension]) const
 Get an array entry using indices stored in an array. More...
 
void destroy ()
 Free all data and reset. More...
 
void shallowCopy (const ArrayBase< T, dimension, index_type > &array)
 Shallow copy another Kokkos array. More...
 
unsigned int useCount () const
 Get the reference count. More...
 
KOKKOS_FUNCTION bool isAlloc () const
 Get whether the array was allocated either on host or device. More...
 
KOKKOS_FUNCTION bool isHostAlloc () const
 Get whether the array was allocated on host. More...
 
KOKKOS_FUNCTION bool isDeviceAlloc () const
 Get whether the array was allocated on device. More...
 
KOKKOS_FUNCTION bool isHostAlias () const
 Get whether the host array was aliased. More...
 
KOKKOS_FUNCTION bool isDeviceAlias () const
 Get whether the device array was aliased. More...
 
KOKKOS_FUNCTION index_type size () const
 Get the total array size. More...
 
KOKKOS_FUNCTION index_type n (unsigned int dim) const
 Get the size of a dimension. More...
 
KOKKOS_FUNCTION T * data () const
 Get the data pointer. More...
 
KOKKOS_FUNCTION T & first () const
 Get the first element. More...
 
KOKKOS_FUNCTION T & last () const
 Get the last element. More...
 
KOKKOS_FUNCTION T & operator[] (index_type i) const
 Get an array entry. More...
 
T * hostData () const
 Get the host data pointer. More...
 
T * deviceData () const
 Get the device data pointer. More...
 
auto hostView () const
 Get the host unmanaged view. More...
 
auto deviceView () const
 Get the device unmanaged view. More...
 
template<typename... size_type>
void init (size_type... n)
 Initialize array with given dimensions but do not allocate. More...
 
template<bool initialize = true>
void create (const std::vector< index_type > &n)
 Allocate array on host and device. More...
 
template<bool initialize = true, typename... size_type>
void create (size_type... n)
 Allocate array on host and device. More...
 
template<bool initialize = true>
void createHost (const std::vector< index_type > &n)
 Allocate array on host only. More...
 
template<bool initialize = true, typename... size_type>
void createHost (size_type... n)
 Allocate array on host only. More...
 
void createDevice (const std::vector< index_type > &n)
 Allocate array on device only. More...
 
template<typename... size_type>
void createDevice (size_type... n)
 Allocate array on device only. More...
 
void aliasHost (T *ptr)
 Point the host data to an external data instead of allocating it. More...
 
void aliasDevice (T *ptr)
 Point the device data to an external data instead of allocating it. More...
 
void offset (const std::vector< signed_index_type > &d)
 Apply starting index offsets to each dimension. More...
 
template<typename... offset_type>
void offset (offset_type... d)
 Apply starting index offsets to each dimension. More...
 
void copyToDevice ()
 Copy data from host to device. More...
 
void copyToHost ()
 Copy data from device to host. More...
 
void copyIn (const T *ptr, MemcpyType dir, index_type n, index_type offset=0)
 Copy data from an external data to this array. More...
 
void copyOut (T *ptr, MemcpyType dir, index_type n, index_type offset=0)
 Copy data to an external data from this array. More...
 
void copyToDeviceNested ()
 Copy all the nested Kokkos arrays including self from host to device. More...
 
void moveToDevice (bool should_free_host=true)
 Copy data from host to device and deallocate host. More...
 
void moveToHost (bool should_free_device=true)
 Copy data from device to host and deallocate device. More...
 
void deepCopy (const ArrayBase< T, dimension, index_type > &array)
 Deep copy another Kokkos array If ArrayDeepCopy<T>::value is true, it will copy-construct each entry If ArrayDeepCopy<T>::value is false, it will do a memory copy. More...
 
void swap (ArrayBase< T, dimension, index_type > &array)
 Swap with another Kokkos array. More...
 
KOKKOS_FUNCTION iterator begin () const
 Get the beginning iterator. More...
 
KOKKOS_FUNCTION iterator end () const
 Get the end iterator. More...
 
template<typename... indices>
KOKKOS_FUNCTION T & operator() (indices... i) const
 Get an array entry. More...
 

Protected Member Functions

template<bool host, bool device, bool initialize, typename... size_type>
void createInternal (size_type... n)
 The internal method to initialize and allocate this array. More...
 
template<bool host, bool device, bool initialize>
void createInternal (const std::vector< index_type > &n)
 The internal method to initialize and allocate this array. More...
 
template<bool initialize>
void createInternal (const std::vector< index_type > &n, bool host, bool device)
 The internal method to initialize and allocate this array. More...
 
template<typename TargetSpace , typename SourceSpace >
void copyInternal (T *target, const T *source, index_type n)
 The internal method to perform a memory copy. More...
 

Protected Attributes

index_type _n [dimension] = {0}
 Size of each dimension. More...
 
index_type _s [dimension] = {0}
 Stride of each dimension. More...
 
signed_index_type _d [dimension] = {0}
 Offset of each dimension. More...
 
bool _is_offset = false
 Flag whether the array indices are offset. More...
 
bool _is_malloc = false
 Flag whether host data was allocated using malloc. More...
 

Private Member Functions

 usingKokkosArrayBaseMembers (T, dimension, index_type)
 
template<unsigned int... i>
KOKKOS_FUNCTION T & operatorInternal (const signed_index_type(&idx)[dimension], std::integer_sequence< unsigned int, i... >) const
 Internal method for calling operator() with array indices. More...
 

Detailed Description

template<typename T, unsigned int dimension, typename index_type, LayoutType layout>
class Moose::Kokkos::Array< T, dimension, index_type, layout >

The Kokkos array class.

The specialization of the Kokkos array class for each dimension.

All array data that needs to be accessed on device in Kokkos objects should use this class. If the array is populated on host and is to be accessed on device, make sure to call copyToDevice() after populating data. For a nested Kokkos array, either copyToDeviceNested() should be called for the outermost array or copyToDevice() should be called for each instance of Kokkos array from the innermost to the outermost. Do not store this object as reference in your Kokkos object if it is used on device, because the reference refers to a host object and therefore is not accessible on device. If storing it as a reference is required, see ReferenceWrapper.

Template Parameters
TThe data type
dimensionThe array dimension size
index_typeThe array index type
layoutThe memory layout type

Definition at line 64 of file KokkosArray.h.

Member Typedef Documentation

◆ signed_index_type

template<typename T, unsigned int dimension, typename index_type>
using Moose::Kokkos::ArrayBase< T, dimension, index_type >::signed_index_type = typename std::make_signed<index_type>::type
inherited

Definition at line 114 of file KokkosArray.h.

◆ unsigned_index_type

template<typename T, unsigned int dimension, typename index_type>
using Moose::Kokkos::ArrayBase< T, dimension, index_type >::unsigned_index_type = index_type
inherited

Definition at line 113 of file KokkosArray.h.

Constructor & Destructor Documentation

◆ Array() [1/3]

template<typename T, unsigned int dimension, typename index_type, LayoutType layout>
Moose::Kokkos::Array< T, dimension, index_type, layout >::Array ( )
inline

Default constructor.

Definition at line 1362 of file KokkosArray.h.

1362 : ArrayBase<T, dimension, index_type>(layout) {}

◆ Array() [2/3]

template<typename T, unsigned int dimension, typename index_type, LayoutType layout>
Moose::Kokkos::Array< T, dimension, index_type, layout >::Array ( const Array< T, dimension, index_type, layout > &  array)
inline

Copy constructor.

Definition at line 1366 of file KokkosArray.h.

1367  : ArrayBase<T, dimension, index_type>(array)
1368  {
1369  }

◆ Array() [3/3]

template<typename T, unsigned int dimension, typename index_type, LayoutType layout>
template<typename... size_type>
Moose::Kokkos::Array< T, dimension, index_type, layout >::Array ( size_type...  n)
inline

Constructor Initialize and allocate array with given dimensions This allocates both host and device data.

Parameters
nThe size of each dimension

Definition at line 1378 of file KokkosArray.h.

1378  : ArrayBase<T, dimension, index_type>(layout, n...)
1379  {
1380  }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

Member Function Documentation

◆ aliasDevice()

template<typename T, unsigned int dimension, typename index_type >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::aliasDevice ( T *  ptr)
inherited

Point the device data to an external data instead of allocating it.

Parameters
ptrThe pointer to the external device data

Definition at line 760 of file KokkosArray.h.

761 {
762  if (!_is_init)
763  mooseError("Kokkos array error: attempted to alias device data before array initialization.");
764 
766  mooseError("Kokkos array error: cannot alias device data because device data was not aliased.");
767 
768  _device_data = ptr;
769  _is_device_alloc = true;
770  _is_device_alias = true;
771 }
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596
bool _is_device_alias
Flag whether the device data points to an external data.
Definition: KokkosArray.h:604
T * _device_data
Device data.
Definition: KokkosArray.h:612
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:588

◆ aliasHost()

template<typename T, unsigned int dimension, typename index_type >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::aliasHost ( T *  ptr)
inherited

Point the host data to an external data instead of allocating it.

Parameters
ptrThe pointer to the external host data

Definition at line 745 of file KokkosArray.h.

746 {
747  if (!_is_init)
748  mooseError("Kokkos array error: attempted to alias host data before array initialization.");
749 
751  mooseError("Kokkos array error: cannot alias host data because host data was not aliased.");
752 
753  _host_data = ptr;
754  _is_host_alloc = true;
755  _is_host_alias = true;
756 }
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
T * _host_data
Host data.
Definition: KokkosArray.h:608
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592
bool _is_host_alias
Flag whether the host data points to an external data.
Definition: KokkosArray.h:600
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:588

◆ begin()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION iterator Moose::Kokkos::ArrayBase< T, dimension, index_type >::begin ( ) const
inlineinherited

Get the beginning iterator.

Returns
The beginning iterator

Definition at line 479 of file KokkosArray.h.

Referenced by Moose::Kokkos::Array< T, 1, index_type, LayoutType::LEFT >::copySet().

480  {
481  KOKKOS_IF_ON_HOST(return iterator(_host_data);)
482 
483  return iterator(_device_data);
484  }
T * _host_data
Host data.
Definition: KokkosArray.h:608
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ copyIn()

template<typename T, unsigned int dimension, typename index_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::copyIn ( const T *  ptr,
MemcpyType  dir,
index_type  n,
index_type  offset = 0 
)
inherited

Copy data from an external data to this array.

Parameters
ptrThe pointer to the external data
dirThe copy direction
nThe number of entries to copy
offsetThe starting offset of this array

Definition at line 1028 of file KokkosArray.h.

Referenced by Moose::Kokkos::dataLoad().

1032 {
1033  if (n > _size)
1034  mooseError("Kokkos array error: cannot copy in data larger than the array size.");
1035 
1036  if (offset > _size)
1037  mooseError("Kokkos array error: offset cannot be larger than the array size.");
1038 
1039  if (dir == MemcpyType::HOST_TO_HOST)
1040  {
1041  // If host side memory is not allocated, print error
1042  if (!_is_host_alloc)
1043  mooseError(
1044  "Kokkos array error: cannot copy in to the array because host memory was not allocated.");
1045 
1046  // Copy from host to host
1047  copyInternal<::Kokkos::HostSpace, ::Kokkos::HostSpace>(_host_data + offset, ptr, n);
1048  }
1049  else if (dir == MemcpyType::HOST_TO_DEVICE)
1050  {
1051  // If device side memory is not allocated, print error
1052  if (!_is_device_alloc)
1053  mooseError("Kokkos array error: cannot copy in to the array because device memory was not "
1054  "allocated.");
1055 
1056  // Copy from host to device
1057  copyInternal<MemSpace, ::Kokkos::HostSpace>(_device_data + offset, ptr, n);
1058  }
1059  else if (dir == MemcpyType::DEVICE_TO_HOST)
1060  {
1061  // If host side memory is not allocated, print error
1062  if (!_is_host_alloc)
1063  mooseError(
1064  "Kokkos array error: cannot copy in to the array because host memory was not allocated.");
1065 
1066  // Copy from device to host
1067  copyInternal<::Kokkos::HostSpace, MemSpace>(_host_data + offset, ptr, n);
1068  }
1069  else if (dir == MemcpyType::DEVICE_TO_DEVICE)
1070  {
1071  // If device side memory is not allocated, print error
1072  if (!_is_device_alloc)
1073  mooseError("Kokkos array error: cannot copy in to the array because device memory was not "
1074  "allocated.");
1075 
1076  // Copy from device to device
1077  copyInternal<MemSpace, MemSpace>(_device_data + offset, ptr, n);
1078  }
1079 }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596
T * _host_data
Host data.
Definition: KokkosArray.h:608
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592
index_type _size
Total size.
Definition: KokkosArray.h:616
T * _device_data
Device data.
Definition: KokkosArray.h:612
void offset(const std::vector< signed_index_type > &d)
Apply starting index offsets to each dimension.
Definition: KokkosArray.h:915

◆ copyInternal()

template<typename T, unsigned int dimension, typename index_type>
template<typename TargetSpace , typename SourceSpace >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::copyInternal ( T *  target,
const T *  source,
index_type  n 
)
protectedinherited

The internal method to perform a memory copy.

Template Parameters
TargetSpaceThe Kokkos memory space of target data
SourcespaceThe Kokkos memory space of source data
Parameters
targetThe pointer to the target data
sourceThe pointer to the source data
nThe number of entries to copy

Definition at line 907 of file KokkosArray.h.

908 {
909  ::Kokkos::Impl::DeepCopy<TargetSpace, SourceSpace>(target, source, n * sizeof(T));
910  ::Kokkos::fence();
911 }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ copyOut()

template<typename T, unsigned int dimension, typename index_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::copyOut ( T *  ptr,
MemcpyType  dir,
index_type  n,
index_type  offset = 0 
)
inherited

Copy data to an external data from this array.

Parameters
ptrThe pointer to the external data
dirThe copy direction
nThe number of entries to copy
offsetThe starting offset of this array

Definition at line 1083 of file KokkosArray.h.

Referenced by Moose::Kokkos::dataStore().

1087 {
1088  if (n > _size)
1089  mooseError("Kokkos array error: cannot copy out data larger than the array size.");
1090 
1091  if (offset > _size)
1092  mooseError("Kokkos array error: offset cannot be larger than the array size.");
1093 
1094  if (dir == MemcpyType::HOST_TO_HOST)
1095  {
1096  // If host side memory is not allocated, print error
1097  if (!_is_host_alloc)
1098  mooseError("Kokkos array error: cannot copy out from the array because host memory was not "
1099  "allocated.");
1100 
1101  // Copy from host to host
1102  copyInternal<::Kokkos::HostSpace, ::Kokkos::HostSpace>(ptr, _host_data + offset, n);
1103  }
1104  else if (dir == MemcpyType::HOST_TO_DEVICE)
1105  {
1106  // If host side memory is not allocated, print error
1107  if (!_is_host_alloc)
1108  mooseError("Kokkos array error: cannot copy out from the array because host memory was not "
1109  "allocated.");
1110 
1111  // Copy from host to device
1112  copyInternal<MemSpace, ::Kokkos::HostSpace>(ptr, _host_data + offset, n);
1113  }
1114  else if (dir == MemcpyType::DEVICE_TO_HOST)
1115  {
1116  // If device side memory is not allocated, print error
1117  if (!_is_device_alloc)
1118  mooseError("Kokkos array error: cannot copy out from the array because device memory was not "
1119  "allocated.");
1120 
1121  // Copy from device to host
1122  copyInternal<::Kokkos::HostSpace, MemSpace>(ptr, _device_data + offset, n);
1123  }
1124  else if (dir == MemcpyType::DEVICE_TO_DEVICE)
1125  {
1126  // If device side memory is not allocated, print error
1127  if (!_is_device_alloc)
1128  mooseError("Kokkos array error: cannot copy out from the array because device memory was not "
1129  "allocated.");
1130 
1131  // Copy from device to device
1132  copyInternal<MemSpace, MemSpace>(ptr, _device_data + offset, n);
1133  }
1134 }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596
T * _host_data
Host data.
Definition: KokkosArray.h:608
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592
index_type _size
Total size.
Definition: KokkosArray.h:616
T * _device_data
Device data.
Definition: KokkosArray.h:612
void offset(const std::vector< signed_index_type > &d)
Apply starting index offsets to each dimension.
Definition: KokkosArray.h:915

◆ copyToDevice()

template<typename T , unsigned int dimension, typename index_type >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::copyToDevice ( )
inherited

Copy data from host to device.

Definition at line 947 of file KokkosArray.h.

Referenced by Moose::Kokkos::VariableGradientTempl< is_ad >::VariableGradientTempl(), and Moose::Kokkos::VariableValueTempl< is_ad >::VariableValueTempl().

948 {
949  // If host side memory is not allocated, do nothing
950  if (!_is_host_alloc)
951  return;
952 
953  // If device side memory is not allocated,
954  if (!_is_device_alloc)
955  {
956  if (_counter.use_count() == 1)
957  // allocate memory if this array is not shared with other arrays
958  allocDevice();
959  else
960  // print error if this array is shared with other arrays
961  mooseError("Kokkos array error: cannot copy from host to device because device memory "
962  "was not allocated. Cannot allocate device memory for copy because the array is "
963  "being shared.");
964  }
965 
966  // Copy from host to device
967  copyInternal<MemSpace, ::Kokkos::HostSpace>(_device_data, _host_data, _size);
968 }
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
void allocDevice()
Allocate device data for an initialized array that has not allocated device data. ...
Definition: KokkosArray.h:798
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596
T * _host_data
Host data.
Definition: KokkosArray.h:608
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592
index_type _size
Total size.
Definition: KokkosArray.h:616
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:584
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ copyToDeviceNested()

template<typename T , unsigned int dimension, typename index_type >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::copyToDeviceNested ( )
inherited

Copy all the nested Kokkos arrays including self from host to device.

Definition at line 1151 of file KokkosArray.h.

Referenced by Moose::Kokkos::copyToDeviceInner().

1152 {
1153  for (index_type i = 0; i < _size; ++i)
1155 
1156  copyToDevice();
1157 }
T * _host_data
Host data.
Definition: KokkosArray.h:608
index_type _size
Total size.
Definition: KokkosArray.h:616
void copyToDeviceInner(T &)
Definition: KokkosArray.h:1138
void copyToDevice()
Copy data from host to device.
Definition: KokkosArray.h:947

◆ copyToHost()

template<typename T , unsigned int dimension, typename index_type >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::copyToHost ( )
inherited

Copy data from device to host.

Definition at line 972 of file KokkosArray.h.

973 {
974  // If device side memory is not allocated, do nothing
975  if (!_is_device_alloc)
976  return;
977 
978  // If host side memory is not allocated,
979  if (!_is_host_alloc)
980  {
981  if (_counter.use_count() == 1)
982  // allocate memory if this array is not shared with other arrays
983  allocHost<false>();
984  else
985  // print error if this array is shared with other arrays
986  mooseError("Kokkos array error: cannot copy from device to host because host memory "
987  "was not allocated. Cannot allocate host memory for copy because the array is "
988  "being shared.");
989  }
990 
991  // Copy from device to host
992  copyInternal<::Kokkos::HostSpace, MemSpace>(_host_data, _device_data, _size);
993 }
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596
T * _host_data
Host data.
Definition: KokkosArray.h:608
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592
index_type _size
Total size.
Definition: KokkosArray.h:616
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:584
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ create() [1/2]

template<typename T, unsigned int dimension, typename index_type>
template<bool initialize = true>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::create ( const std::vector< index_type > &  n)
inlineinherited

Allocate array on host and device.

Template Parameters
initializeWhether to initialize host data (calls default constructor)
Parameters
nThe vector containing the size of each dimension

Definition at line 303 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< Moose::Kokkos::RPNEvaluator::Instruction, dimension, index_type >::ArrayBase(), Moose::Kokkos::VariableGradientTempl< is_ad >::VariableGradientTempl(), and Moose::Kokkos::VariableValueTempl< is_ad >::VariableValueTempl().

304  {
305  createInternal<true, true, initialize>(n);
306  }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ create() [2/2]

template<typename T, unsigned int dimension, typename index_type>
template<bool initialize = true, typename... size_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::create ( size_type...  n)
inlineinherited

Allocate array on host and device.

Template Parameters
initializeWhether to initialize host data (calls default constructor)
Parameters
nThe size of each dimension

Definition at line 313 of file KokkosArray.h.

314  {
315  createInternal<true, true, initialize>(n...);
316  }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ createDevice() [1/2]

template<typename T, unsigned int dimension, typename index_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::createDevice ( const std::vector< index_type > &  n)
inlineinherited

Allocate array on device only.

Parameters
nThe vector containing the size of each dimension

Definition at line 341 of file KokkosArray.h.

341 { createInternal<false, true, false>(n); }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ createDevice() [2/2]

template<typename T, unsigned int dimension, typename index_type>
template<typename... size_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::createDevice ( size_type...  n)
inlineinherited

Allocate array on device only.

Parameters
nThe size of each dimension

Definition at line 347 of file KokkosArray.h.

348  {
349  createInternal<false, true, false>(n...);
350  }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ createHost() [1/2]

template<typename T, unsigned int dimension, typename index_type>
template<bool initialize = true>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::createHost ( const std::vector< index_type > &  n)
inlineinherited

Allocate array on host only.

Template Parameters
initializeWhether to initialize host data (calls default constructor)
Parameters
nThe vector containing the size of each dimension

Definition at line 323 of file KokkosArray.h.

324  {
325  createInternal<true, false, initialize>(n);
326  }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ createHost() [2/2]

template<typename T, unsigned int dimension, typename index_type>
template<bool initialize = true, typename... size_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::createHost ( size_type...  n)
inlineinherited

Allocate array on host only.

Template Parameters
initializeWhether to initialize host data (calls default constructor)
Parameters
nThe size of each dimension

Definition at line 333 of file KokkosArray.h.

334  {
335  createInternal<true, false, initialize>(n...);
336  }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ createInternal() [1/3]

template<typename T , unsigned int dimension, typename index_type >
template<bool host, bool device, bool initialize, typename... size_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::createInternal ( size_type...  n)
protectedinherited

The internal method to initialize and allocate this array.

Template Parameters
hostWhether host data will be allocated
deviceWhether device data will be allocated
initializeWhether to initialize host data (calls default constructor)
Parameters
nThe size of each dimension

Definition at line 892 of file KokkosArray.h.

893 {
894  static_assert((std::is_convertible<size_type, index_type>::value && ...),
895  "All arguments must be convertible to index_type");
896  static_assert(sizeof...(n) == dimension, "Number of arguments should match array dimension");
897 
898  std::vector<index_type> dims;
899  (dims.push_back(n), ...);
900 
901  createInternal<host, device, initialize>(dims);
902 }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ createInternal() [2/3]

template<typename T , unsigned int dimension, typename index_type>
template<bool host, bool device, bool initialize>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::createInternal ( const std::vector< index_type > &  n)
protectedinherited

The internal method to initialize and allocate this array.

Template Parameters
hostWhether host data will be allocated
deviceWhether device data will be allocated
initializeWhether to initialize host data (calls default constructor)
Parameters
nThe vector containing the size of each dimension

Definition at line 812 of file KokkosArray.h.

813 {
814  if (n.size() != dimension)
815  mooseError("Kokkos array error: the number of dimensions provided (",
816  n.size(),
817  ") must match the array dimension (",
818  dimension,
819  ").");
820 
821  if (_counter)
822  destroy();
823 
824  _counter = std::make_shared<unsigned int>();
825 
826  uint64_t overflow_checker = 1;
827 
828  _size = 1;
829  _s[0] = 1;
830 
831  for (const auto i : make_range(dimension))
832  {
833  overflow_checker *= n[i];
834 
835  _n[i] = n[i];
836  _size *= n[i];
837  }
838 
839  if (overflow_checker > std::numeric_limits<index_type>::max())
840  mooseError("Kokkos array error: the dimensions provided (",
842  ") has the total size of ",
843  overflow_checker,
844  " which exceeds the limit of ",
845  MooseUtils::prettyCppType<index_type>(),
846  ".");
847 
848  if (_layout == LayoutType::LEFT)
849  {
850  _s[0] = 1;
851 
852  for (unsigned int i = 1; i < dimension; ++i)
853  _s[i] = _s[i - 1] * _n[i - 1];
854  }
855  else
856  {
857  _s[dimension - 1] = 1;
858 
859  for (int i = dimension - 2; i >= 0; --i)
860  _s[i] = _s[i + 1] * _n[i + 1];
861  }
862 
863  if constexpr (host)
864  allocHost<initialize>();
865 
866  if constexpr (device)
867  allocDevice();
868 
869  _is_init = true;
870 }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
void allocDevice()
Allocate device data for an initialized array that has not allocated device data. ...
Definition: KokkosArray.h:798
index_type _s[dimension]
Stride of each dimension.
Definition: KokkosArray.h:505
auto max(const L &left, const R &right)
index_type _size
Total size.
Definition: KokkosArray.h:616
const LayoutType _layout
Memory layout type.
Definition: KokkosArray.h:620
void destroy()
Free all data and reset.
Definition: KokkosArray.h:674
std::string stringify(const T &t)
conversion to string
Definition: Conversion.h:64
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:584
IntRange< T > make_range(T beg, T end)
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:588
index_type _n[dimension]
Size of each dimension.
Definition: KokkosArray.h:501

◆ createInternal() [3/3]

template<typename T , unsigned int dimension, typename index_type>
template<bool initialize>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::createInternal ( const std::vector< index_type > &  n,
bool  host,
bool  device 
)
protectedinherited

The internal method to initialize and allocate this array.

Template Parameters
initializeWhether to initialize host data (calls default constructor)
Parameters
nThe vector containing the size of each dimension
hostThe flag whether host data will be allocated
deviceThe flag whether device data will be allocated

Definition at line 875 of file KokkosArray.h.

878 {
879  if (host && device)
880  createInternal<true, true, initialize>(n);
881  else if (host && !device)
882  createInternal<true, false, initialize>(n);
883  else if (!host && device)
884  createInternal<false, true, initialize>(n);
885  else
886  createInternal<false, false, initialize>(n);
887 }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ data()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION T* Moose::Kokkos::ArrayBase< T, dimension, index_type >::data ( ) const
inlineinherited

Get the data pointer.

Returns
The pointer to the underlying data depending on the architecture this function is being called on

Definition at line 217 of file KokkosArray.h.

Referenced by Moose::Kokkos::Matrix::find().

218  {
219  KOKKOS_IF_ON_HOST(return _host_data;)
220 
221  return _device_data;
222  }
T * _host_data
Host data.
Definition: KokkosArray.h:608
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ deepCopy()

template<typename T, unsigned int dimension, typename index_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::deepCopy ( const ArrayBase< T, dimension, index_type > &  array)
inherited

Deep copy another Kokkos array If ArrayDeepCopy<T>::value is true, it will copy-construct each entry If ArrayDeepCopy<T>::value is false, it will do a memory copy.

Parameters
arrayThe Kokkos array to be deep copied

Definition at line 1161 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< Moose::Kokkos::RPNEvaluator::Instruction, dimension, index_type >::ArrayBase().

1162 {
1163  if (_layout != array._layout)
1164  mooseError("Kokkos array error: cannot deep copy arrays with different layouts.");
1165 
1166  if (ArrayDeepCopy<T>::value && !array._is_host_alloc)
1167  mooseError(
1168  "Kokkos array error: cannot deep copy using constructor from array without host data.");
1169 
1170  std::vector<index_type> n(std::begin(array._n), std::end(array._n));
1171 
1172  createInternal<false>(n, array._is_host_alloc, array._is_device_alloc);
1173 
1174  if constexpr (ArrayDeepCopy<T>::value)
1175  {
1176  for (index_type i = 0; i < _size; ++i)
1177  new (_host_data + i) T(array._host_data[i]);
1178 
1179  copyToDevice();
1180  }
1181  else
1182  {
1183  if (_is_host_alloc)
1184  std::memcpy(_host_data, array._host_data, _size * sizeof(T));
1185 
1186  if (_is_device_alloc)
1187  copyInternal<MemSpace, MemSpace>(_device_data, array._device_data, _size);
1188  }
1189 
1190  for (unsigned int i = 0; i < dimension; ++i)
1191  {
1192  _d[i] = array._d[i];
1193  _s[i] = array._s[i];
1194  }
1195 
1196  _is_offset = array._is_offset;
1197 }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211
signed_index_type _d[dimension]
Offset of each dimension.
Definition: KokkosArray.h:509
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
bool _is_offset
Flag whether the array indices are offset.
Definition: KokkosArray.h:513
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596
T * _host_data
Host data.
Definition: KokkosArray.h:608
index_type _s[dimension]
Stride of each dimension.
Definition: KokkosArray.h:505
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592
index_type _size
Total size.
Definition: KokkosArray.h:616
const LayoutType _layout
Memory layout type.
Definition: KokkosArray.h:620
void copyToDevice()
Copy data from host to device.
Definition: KokkosArray.h:947
T * _device_data
Device data.
Definition: KokkosArray.h:612
static constexpr bool value
Definition: KokkosArray.h:92

◆ destroy()

template<typename T , unsigned int dimension, typename index_type >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::destroy ( )
inherited

Free all data and reset.

Definition at line 674 of file KokkosArray.h.

Referenced by Moose::Kokkos::System::clearActiveMatrixTags(), Moose::Kokkos::System::clearActiveResidualTags(), Moose::Kokkos::System::clearActiveSolutionTags(), Moose::Kokkos::System::clearActiveVariables(), and Moose::Kokkos::ArrayBase< Moose::Kokkos::RPNEvaluator::Instruction, dimension, index_type >::~ArrayBase().

675 {
676  if (!_counter)
677  return;
678 
679  if (_counter.use_count() > 1)
680  {
681  _host_data = nullptr;
682  _device_data = nullptr;
683  }
684  else if (_counter.use_count() == 1)
685  {
686  freeHost();
687  freeDevice();
688  }
689 
690  _size = 0;
691 
692  for (unsigned int i = 0; i < dimension; ++i)
693  {
694  _n[i] = 0;
695  _s[i] = 0;
696  _d[i] = 0;
697  }
698 
699  _is_init = false;
700  _is_offset = false;
701  _is_malloc = false;
702  _is_host_alloc = false;
703  _is_device_alloc = false;
704  _is_host_alias = false;
705  _is_device_alias = false;
706 
707  _counter.reset();
708 }
signed_index_type _d[dimension]
Offset of each dimension.
Definition: KokkosArray.h:509
bool _is_offset
Flag whether the array indices are offset.
Definition: KokkosArray.h:513
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596
T * _host_data
Host data.
Definition: KokkosArray.h:608
bool _is_malloc
Flag whether host data was allocated using malloc.
Definition: KokkosArray.h:517
index_type _s[dimension]
Stride of each dimension.
Definition: KokkosArray.h:505
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592
void freeDevice()
Free device data.
Definition: KokkosArray.h:656
index_type _size
Total size.
Definition: KokkosArray.h:616
bool _is_device_alias
Flag whether the device data points to an external data.
Definition: KokkosArray.h:604
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:584
bool _is_host_alias
Flag whether the host data points to an external data.
Definition: KokkosArray.h:600
T * _device_data
Device data.
Definition: KokkosArray.h:612
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:588
void freeHost()
Free host data.
Definition: KokkosArray.h:625
index_type _n[dimension]
Size of each dimension.
Definition: KokkosArray.h:501

◆ deviceData()

template<typename T, unsigned int dimension, typename index_type>
T* Moose::Kokkos::ArrayBase< T, dimension, index_type >::deviceData ( ) const
inlineinherited

Get the device data pointer.

Returns
The pointer to the underlying device data

Definition at line 269 of file KokkosArray.h.

Referenced by Moose::Kokkos::Array< T, 1, index_type, LayoutType::LEFT >::copyVector().

269 { return _device_data; }
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ deviceView()

template<typename T, unsigned int dimension, typename index_type>
auto Moose::Kokkos::ArrayBase< T, dimension, index_type >::deviceView ( ) const
inlineinherited

Get the device unmanaged view.

Returns
The device unmanaged view

Definition at line 283 of file KokkosArray.h.

284  {
285  return ::Kokkos::View<T *, MemSpace, ::Kokkos::MemoryTraits<::Kokkos::Unmanaged>>(_device_data,
286  _size);
287  }
index_type _size
Total size.
Definition: KokkosArray.h:616
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ end()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION iterator Moose::Kokkos::ArrayBase< T, dimension, index_type >::end ( ) const
inlineinherited

Get the end iterator.

Returns
The end iterator

Definition at line 489 of file KokkosArray.h.

Referenced by Moose::Kokkos::Array< T, 1, index_type, LayoutType::LEFT >::copySet().

490  {
491  KOKKOS_IF_ON_HOST(return iterator(_host_data + _size);)
492 
493  return iterator(_device_data + _size);
494  }
T * _host_data
Host data.
Definition: KokkosArray.h:608
index_type _size
Total size.
Definition: KokkosArray.h:616
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ first()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION T& Moose::Kokkos::ArrayBase< T, dimension, index_type >::first ( ) const
inlineinherited

Get the first element.

Returns
The reference of the first element depending on the architecture this function is being called on

Definition at line 228 of file KokkosArray.h.

Referenced by Moose::Kokkos::RPNEvaluator::eval().

229  {
230  KOKKOS_IF_ON_HOST(return _host_data[0];)
231 
232  return _device_data[0];
233  }
T * _host_data
Host data.
Definition: KokkosArray.h:608
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ hostData()

template<typename T, unsigned int dimension, typename index_type>
T* Moose::Kokkos::ArrayBase< T, dimension, index_type >::hostData ( ) const
inlineinherited

Get the host data pointer.

Returns
The pointer to the underlying host data

Definition at line 264 of file KokkosArray.h.

Referenced by Moose::Kokkos::Array< T, 1, index_type, LayoutType::LEFT >::copyVector().

264 { return _host_data; }
T * _host_data
Host data.
Definition: KokkosArray.h:608

◆ hostView()

template<typename T, unsigned int dimension, typename index_type>
auto Moose::Kokkos::ArrayBase< T, dimension, index_type >::hostView ( ) const
inlineinherited

Get the host unmanaged view.

Returns
The host unmanaged view

Definition at line 274 of file KokkosArray.h.

275  {
276  return ::Kokkos::View<T *, ::Kokkos::HostSpace, ::Kokkos::MemoryTraits<::Kokkos::Unmanaged>>(
277  _host_data, _size);
278  }
T * _host_data
Host data.
Definition: KokkosArray.h:608
index_type _size
Total size.
Definition: KokkosArray.h:616

◆ init()

template<typename T, unsigned int dimension, typename index_type>
template<typename... size_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::init ( size_type...  n)
inlineinherited

Initialize array with given dimensions but do not allocate.

Parameters
nThe size of each dimension

Definition at line 293 of file KokkosArray.h.

294  {
295  createInternal<false, false, false>(n...);
296  }
KOKKOS_FUNCTION index_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:211

◆ isAlloc()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension, index_type >::isAlloc ( ) const
inlineinherited

Get whether the array was allocated either on host or device.

Returns
Whether the array was allocated either on host or device

Definition at line 180 of file KokkosArray.h.

Referenced by Moose::Kokkos::dataStore(), Moose::Kokkos::VariableGradientTempl< is_ad >::VariableGradientTempl(), and Moose::Kokkos::VariableValueTempl< is_ad >::VariableValueTempl().

180 { return _is_host_alloc || _is_device_alloc; }
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592

◆ isDeviceAlias()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension, index_type >::isDeviceAlias ( ) const
inlineinherited

Get whether the device array was aliased.

Returns
Whether the device array was aliased

Definition at line 200 of file KokkosArray.h.

200 { return _is_device_alias; }
bool _is_device_alias
Flag whether the device data points to an external data.
Definition: KokkosArray.h:604

◆ isDeviceAlloc()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension, index_type >::isDeviceAlloc ( ) const
inlineinherited

Get whether the array was allocated on device.

Returns
Whether the array was allocated on device

Definition at line 190 of file KokkosArray.h.

Referenced by Moose::Kokkos::dataStore().

190 { return _is_device_alloc; }
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596

◆ isHostAlias()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension, index_type >::isHostAlias ( ) const
inlineinherited

Get whether the host array was aliased.

Returns
Whether the host array was aliased

Definition at line 195 of file KokkosArray.h.

195 { return _is_host_alias; }
bool _is_host_alias
Flag whether the host data points to an external data.
Definition: KokkosArray.h:600

◆ isHostAlloc()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension, index_type >::isHostAlloc ( ) const
inlineinherited

Get whether the array was allocated on host.

Returns
Whether the array was allocated on host

Definition at line 185 of file KokkosArray.h.

Referenced by Moose::Kokkos::dataLoad().

185 { return _is_host_alloc; }
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592

◆ last()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION T& Moose::Kokkos::ArrayBase< T, dimension, index_type >::last ( ) const
inlineinherited

Get the last element.

Returns
The reference of the last element depending on the architecture this function is being called on

Definition at line 239 of file KokkosArray.h.

240  {
241  KOKKOS_IF_ON_HOST(return _host_data[_size - 1];)
242 
243  return _device_data[_size - 1];
244  }
T * _host_data
Host data.
Definition: KokkosArray.h:608
index_type _size
Total size.
Definition: KokkosArray.h:616
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ moveToDevice()

template<typename T , unsigned int dimension, typename index_type >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::moveToDevice ( bool  should_free_host = true)
inherited

Copy data from host to device and deallocate host.

Parameters
should_free_hostWhether the host memory should be freed. Host memory cannot be freed when there are shallow copies of this array that are still alive. If should_free_host is true, and we cannot free for above reason, it will error.

Definition at line 997 of file KokkosArray.h.

Referenced by Moose::Kokkos::JaggedArray< T, inner, outer, index_type, layout >::moveToDevice().

998 {
999  static_assert(!is_kokkos_array<T>::value,
1000  "moveToDevice() not allowed for a nested array whose data type is another array.");
1001 
1002  if (should_free_host && _counter.use_count() > 1)
1003  mooseError("Kokkos array error: cannot move array from host to device because there is at "
1004  "least one shallow copy of this array still alive.");
1005 
1006  copyToDevice();
1007 
1008  if (_counter.use_count() == 1)
1009  freeHost();
1010 }
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
void copyToDevice()
Copy data from host to device.
Definition: KokkosArray.h:947
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:584
void freeHost()
Free host data.
Definition: KokkosArray.h:625

◆ moveToHost()

template<typename T , unsigned int dimension, typename index_type >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::moveToHost ( bool  should_free_device = true)
inherited

Copy data from device to host and deallocate device.

Parameters
should_free_deviceWhether the device memory should be freed. Device memory cannot be freed when there are shallow copies of this array that are still alive. If should_free_device is true, and we cannot free for above reason, it will error.

Definition at line 1014 of file KokkosArray.h.

Referenced by Moose::Kokkos::JaggedArray< T, inner, outer, index_type, layout >::moveToHost().

1015 {
1016  if (should_free_device && _counter.use_count() > 1)
1017  mooseError("Kokkos array error: cannot move array from device to host because there is at "
1018  "least one shallow copy of this array still alive.");
1019 
1020  copyToHost();
1021 
1022  if (_counter.use_count() == 1)
1023  freeDevice();
1024 }
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
void freeDevice()
Free device data.
Definition: KokkosArray.h:656
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:584
void copyToHost()
Copy data from device to host.
Definition: KokkosArray.h:972

◆ n()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION index_type Moose::Kokkos::ArrayBase< T, dimension, index_type >::n ( unsigned int  dim) const
inlineinherited

◆ offset() [1/2]

template<typename T , unsigned int dimension, typename index_type >
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::offset ( const std::vector< signed_index_type > &  d)
inherited

Apply starting index offsets to each dimension.

Parameters
dThe vector containing the offset of each dimension

Definition at line 915 of file KokkosArray.h.

916 {
917  if (d.size() > dimension)
918  mooseError("Kokkos array error: the number of offsets provided (",
919  d.size(),
920  ") cannot be larger than the array dimension (",
921  dimension,
922  ").");
923 
924  for (const auto i : index_range(d))
925  _d[i] = d[i];
926 
927  _is_offset = true;
928 }
signed_index_type _d[dimension]
Offset of each dimension.
Definition: KokkosArray.h:509
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
bool _is_offset
Flag whether the array indices are offset.
Definition: KokkosArray.h:513
auto index_range(const T &sizable)

◆ offset() [2/2]

template<typename T , unsigned int dimension, typename index_type >
template<typename... offset_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::offset ( offset_type...  d)
inherited

Apply starting index offsets to each dimension.

Parameters
dThe offset of each dimension

Definition at line 933 of file KokkosArray.h.

934 {
935  static_assert((std::is_convertible<offset_type, signed_index_type>::value && ...),
936  "All arguments must be convertible to signed_index_type");
937  static_assert(sizeof...(d) == dimension, "Number of arguments should match array dimension");
938 
939  std::vector<signed_index_type> offsets;
940  (offsets.push_back(d), ...);
941 
942  offset(offsets);
943 }
void offset(const std::vector< signed_index_type > &d)
Apply starting index offsets to each dimension.
Definition: KokkosArray.h:915

◆ operator()() [1/2]

template<typename T , unsigned int dimension, typename index_type , LayoutType layout>
template<typename... indices>
KOKKOS_FUNCTION T & Moose::Kokkos::Array< T, dimension, index_type, layout >::operator() ( indices...  i) const

Get an array entry.

Parameters
iThe index of each dimension
Returns
The reference of the entry depending on the architecture this function is being called on

Definition at line 1433 of file KokkosArray.h.

Referenced by Moose::Kokkos::Array< ContiguousElementID >::operatorInternal().

1434 {
1435  static_assert((std::is_convertible<indices, signed_index_type>::value && ...),
1436  "All arguments must be convertible to signed_index_type");
1437  static_assert(sizeof...(i) == dimension, "Number of arguments should match array dimension");
1438 
1439 #ifndef NDEBUG
1440  {
1441  signed_index_type idx[dimension] = {static_cast<signed_index_type>(i)...};
1442 
1443  for (unsigned int d = 0; d < sizeof...(i); ++d)
1444  KOKKOS_ASSERT(idx[d] - _d[d] >= 0 && static_cast<index_type>(idx[d] - _d[d]) < _n[d]);
1445  }
1446 #endif
1447 
1448  index_type idx = 0;
1449  unsigned int d = 0;
1450 
1451  if (_is_offset)
1452  {
1453  if constexpr (layout == LayoutType::LEFT)
1454  (((idx += (d == 0 ? static_cast<signed_index_type>(i) - _d[d]
1455  : (static_cast<signed_index_type>(i) - _d[d]) * _s[d])),
1456  ++d),
1457  ...);
1458  else
1459  (((idx += (d == dimension - 1 ? static_cast<signed_index_type>(i) - _d[d]
1460  : (static_cast<signed_index_type>(i) - _d[d]) * _s[d])),
1461  ++d),
1462  ...);
1463  }
1464  else
1465  {
1466  if constexpr (layout == LayoutType::LEFT)
1467  (((idx +=
1468  (d == 0 ? static_cast<signed_index_type>(i) : static_cast<signed_index_type>(i) * _s[d])),
1469  ++d),
1470  ...);
1471  else
1472  (((idx += (d == dimension - 1 ? static_cast<signed_index_type>(i)
1473  : static_cast<signed_index_type>(i) * _s[d])),
1474  ++d),
1475  ...);
1476  }
1477 
1478  return this->operator[](idx);
1479 }
typename std::make_signed< index_type >::type signed_index_type
Definition: KokkosArray.h:114
signed_index_type _d[dimension]
Offset of each dimension.
Definition: KokkosArray.h:509
KOKKOS_FUNCTION T & operator[](index_type i) const
Get an array entry.
Definition: KokkosArray.h:251
bool _is_offset
Flag whether the array indices are offset.
Definition: KokkosArray.h:513
index_type _s[dimension]
Stride of each dimension.
Definition: KokkosArray.h:505
unsigned int idx(const ElemType type, const unsigned int nx, const unsigned int i, const unsigned int j)
index_type _n[dimension]
Size of each dimension.
Definition: KokkosArray.h:501

◆ operator()() [2/2]

template<typename T, unsigned int dimension, typename index_type, LayoutType layout>
KOKKOS_FUNCTION T& Moose::Kokkos::Array< T, dimension, index_type, layout >::operator() ( const signed_index_type(&)  idx[dimension]) const
inline

Get an array entry using indices stored in an array.

Parameters
idxThe array storing the indices
Returns
The reference of the entry depending on the architecture this function is being called on

Definition at line 1409 of file KokkosArray.h.

1410  {
1411  return operatorInternal(idx, std::make_integer_sequence<unsigned int, dimension>{});
1412  }
KOKKOS_FUNCTION T & operatorInternal(const signed_index_type(&idx)[dimension], std::integer_sequence< unsigned int, i... >) const
Internal method for calling operator() with array indices.
Definition: KokkosArray.h:1421

◆ operator=()

template<typename T, unsigned int dimension, typename index_type, LayoutType layout>
auto& Moose::Kokkos::Array< T, dimension, index_type, layout >::operator= ( const Array< T, dimension, index_type, layout > &  array)
inline

Shallow copy another Kokkos array.

Parameters
arrayThe Kokkos array to be shallow copied

Definition at line 1387 of file KokkosArray.h.

1388  {
1389  this->shallowCopy(array);
1390 
1391  return *this;
1392  }
void shallowCopy(const ArrayBase< T, dimension, index_type > &array)
Shallow copy another Kokkos array.
Definition: KokkosArray.h:712

◆ operator[]()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION T& Moose::Kokkos::ArrayBase< T, dimension, index_type >::operator[] ( index_type  i) const
inlineinherited

Get an array entry.

Parameters
iThe dimensionless index
Returns
The reference of the entry depending on the architecture this function is being called on

Definition at line 251 of file KokkosArray.h.

Referenced by Moose::Kokkos::Array< T, 1, index_type, LayoutType::LEFT >::operator()().

252  {
253  KOKKOS_ASSERT(i < _size);
254 
255  KOKKOS_IF_ON_HOST(return _host_data[i];)
256 
257  return _device_data[i];
258  }
T * _host_data
Host data.
Definition: KokkosArray.h:608
index_type _size
Total size.
Definition: KokkosArray.h:616
T * _device_data
Device data.
Definition: KokkosArray.h:612

◆ operatorInternal()

template<typename T, unsigned int dimension, typename index_type, LayoutType layout>
template<unsigned int... i>
KOKKOS_FUNCTION T& Moose::Kokkos::Array< T, dimension, index_type, layout >::operatorInternal ( const signed_index_type(&)  idx[dimension],
std::integer_sequence< unsigned int, i... >   
) const
inlineprivate

Internal method for calling operator() with array indices.

Definition at line 1421 of file KokkosArray.h.

Referenced by Moose::Kokkos::Array< ContiguousElementID >::operator()().

1423  {
1424  return operator()(idx[i]...);
1425  }
KOKKOS_FUNCTION T & operator()(indices... i) const
Get an array entry.
Definition: KokkosArray.h:1433

◆ shallowCopy()

template<typename T, unsigned int dimension, typename index_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::shallowCopy ( const ArrayBase< T, dimension, index_type > &  array)
inherited

Shallow copy another Kokkos array.

Parameters
arrayThe Kokkos array to be shallow copied

Definition at line 712 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< Moose::Kokkos::RPNEvaluator::Instruction, dimension, index_type >::ArrayBase(), Moose::Kokkos::Array< ContiguousElementID >::operator=(), Moose::Kokkos::Array< T, 1, index_type, LayoutType::LEFT >::operator=(), and Moose::Kokkos::ArrayBase< Moose::Kokkos::RPNEvaluator::Instruction, dimension, index_type >::swap().

713 {
714  if (_layout != array._layout)
715  mooseError("Kokkos array error: cannot shallow copy arrays with different layouts.");
716 
717  destroy();
718 
719  _counter = array._counter;
720 
721  _size = array._size;
722 
723  for (unsigned int i = 0; i < dimension; ++i)
724  {
725  _n[i] = array._n[i];
726  _s[i] = array._s[i];
727  _d[i] = array._d[i];
728  }
729 
730  _is_init = array._is_init;
731  _is_offset = array._is_offset;
732  _is_malloc = array._is_malloc;
733  _is_host_alloc = array._is_host_alloc;
734  _is_device_alloc = array._is_device_alloc;
735  _is_host_alias = array._is_host_alias;
736  _is_device_alias = array._is_device_alias;
737 
738  _host_data = array._host_data;
739  _device_data = array._device_data;
740 }
signed_index_type _d[dimension]
Offset of each dimension.
Definition: KokkosArray.h:509
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:311
bool _is_offset
Flag whether the array indices are offset.
Definition: KokkosArray.h:513
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:596
T * _host_data
Host data.
Definition: KokkosArray.h:608
bool _is_malloc
Flag whether host data was allocated using malloc.
Definition: KokkosArray.h:517
index_type _s[dimension]
Stride of each dimension.
Definition: KokkosArray.h:505
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:592
index_type _size
Total size.
Definition: KokkosArray.h:616
const LayoutType _layout
Memory layout type.
Definition: KokkosArray.h:620
bool _is_device_alias
Flag whether the device data points to an external data.
Definition: KokkosArray.h:604
void destroy()
Free all data and reset.
Definition: KokkosArray.h:674
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:584
bool _is_host_alias
Flag whether the host data points to an external data.
Definition: KokkosArray.h:600
T * _device_data
Device data.
Definition: KokkosArray.h:612
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:588
index_type _n[dimension]
Size of each dimension.
Definition: KokkosArray.h:501

◆ size()

template<typename T, unsigned int dimension, typename index_type>
KOKKOS_FUNCTION index_type Moose::Kokkos::ArrayBase< T, dimension, index_type >::size ( ) const
inlineinherited

◆ swap()

template<typename T, unsigned int dimension, typename index_type>
void Moose::Kokkos::ArrayBase< T, dimension, index_type >::swap ( ArrayBase< T, dimension, index_type > &  array)
inherited

Swap with another Kokkos array.

Parameters
arrayThe Kokkos array to be swapped

Definition at line 1201 of file KokkosArray.h.

1202 {
1203  ArrayBase<T, dimension, index_type> clone(_layout);
1204 
1205  clone.shallowCopy(*this);
1206  this->shallowCopy(array);
1207  array.shallowCopy(clone);
1208 }
void shallowCopy(const ArrayBase< T, dimension, index_type > &array)
Shallow copy another Kokkos array.
Definition: KokkosArray.h:712
const LayoutType _layout
Memory layout type.
Definition: KokkosArray.h:620

◆ useCount()

template<typename T, unsigned int dimension, typename index_type>
unsigned int Moose::Kokkos::ArrayBase< T, dimension, index_type >::useCount ( ) const
inlineinherited

Get the reference count.

Returns
The reference count

Definition at line 173 of file KokkosArray.h.

173 { return _counter.use_count(); }
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:584

◆ usingKokkosArrayBaseMembers()

template<typename T, unsigned int dimension, typename index_type, LayoutType layout>
Moose::Kokkos::Array< T, dimension, index_type, layout >::usingKokkosArrayBaseMembers ( ,
dimension  ,
index_type   
)
private

Member Data Documentation

◆ _d

template<typename T, unsigned int dimension, typename index_type>
signed_index_type Moose::Kokkos::ArrayBase< T, dimension, index_type >::_d[dimension] = {0}
protectedinherited

◆ _is_malloc

template<typename T, unsigned int dimension, typename index_type>
bool Moose::Kokkos::ArrayBase< T, dimension, index_type >::_is_malloc = false
protectedinherited

Flag whether host data was allocated using malloc.

Definition at line 517 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< Moose::Kokkos::RPNEvaluator::Instruction, dimension, index_type >::shallowCopy().

◆ _is_offset

template<typename T, unsigned int dimension, typename index_type>
bool Moose::Kokkos::ArrayBase< T, dimension, index_type >::_is_offset = false
protectedinherited

◆ _n

template<typename T, unsigned int dimension, typename index_type>
index_type Moose::Kokkos::ArrayBase< T, dimension, index_type >::_n[dimension] = {0}
protectedinherited

◆ _s

template<typename T, unsigned int dimension, typename index_type>
index_type Moose::Kokkos::ArrayBase< T, dimension, index_type >::_s[dimension] = {0}
protectedinherited

The documentation for this class was generated from the following file: