https://mooseframework.inl.gov
Classes | Public Member Functions | Protected Member Functions | Protected Attributes | Private Member Functions | Private Attributes | List of all members
Moose::Kokkos::ArrayBase< T, dimension > Class Template Reference

The base class for Kokkos arrays. More...

#include <KokkosArray.h>

Classes

class  iterator
 Array iterator. More...
 

Public Member Functions

 ArrayBase ()=default
 Default constructor. More...
 
 ArrayBase (const ArrayBase< T, dimension > &array)
 Copy constructor. More...
 
 ~ArrayBase ()
 Destructor. More...
 
void destroy ()
 Free all data and reset. More...
 
void shallowCopy (const ArrayBase< T, dimension > &array)
 Shallow copy another Kokkos array. More...
 
unsigned int useCount () const
 Get the reference count. More...
 
KOKKOS_FUNCTION bool isAlloc () const
 Get whether the array was allocated either on host or device. More...
 
KOKKOS_FUNCTION bool isHostAlloc () const
 Get whether the array was allocated on host. More...
 
KOKKOS_FUNCTION bool isDeviceAlloc () const
 Get whether the array was allocated on device. More...
 
KOKKOS_FUNCTION bool isHostAlias () const
 Get whether the host array was aliased. More...
 
KOKKOS_FUNCTION bool isDeviceAlias () const
 Get whether the device array was aliased. More...
 
KOKKOS_FUNCTION dof_id_type size () const
 Get the total array size. More...
 
KOKKOS_FUNCTION dof_id_type n (unsigned int dim) const
 Get the size of a dimension. More...
 
KOKKOS_FUNCTION T * data () const
 Get the data pointer. More...
 
KOKKOS_FUNCTION T & first () const
 Get the first element. More...
 
KOKKOS_FUNCTION T & last () const
 Get the last element. More...
 
KOKKOS_FUNCTION T & operator[] (dof_id_type i) const
 Get an array entry. More...
 
T * hostData () const
 Get the host data pointer. More...
 
T * deviceData () const
 Get the device data pointer. More...
 
void create (const std::vector< dof_id_type > &n)
 Allocate array on host and device. More...
 
void createHost (const std::vector< dof_id_type > &n)
 Allocate array on host only. More...
 
void createDevice (const std::vector< dof_id_type > &n)
 Allocate array on device only. More...
 
void aliasHost (T *ptr)
 Point the host data to an external data instead of allocating it. More...
 
void aliasDevice (T *ptr)
 Point the device data to an external data instead of allocating it. More...
 
void offset (const std::vector< dof_id_signed_type > &d)
 Apply starting index offsets to each dimension. More...
 
void copyToDevice ()
 Copy data from host to device. More...
 
void copyToHost ()
 Copy data from device to host. More...
 
void copyIn (const T *ptr, MemcpyKind dir, dof_id_type n, dof_id_type offset=0)
 Copy data from an external data to this array. More...
 
void copyOut (T *ptr, MemcpyKind dir, dof_id_type n, dof_id_type offset=0)
 Copy data to an external data from this array. More...
 
void copyToDeviceNested ()
 Copy all the nested Kokkos arrays including self from host to device. More...
 
void deepCopy (const ArrayBase< T, dimension > &array)
 Deep copy another Kokkos array If ArrayDeepCopy<T>::value is true, it will copy-construct each entry If ArrayDeepCopy<T>::value is false, it will do a memory copy. More...
 
void swap (ArrayBase< T, dimension > &array)
 Swap with another Kokkos array. More...
 
auto & operator= (const T &scalar)
 Assign a scalar value uniformly. More...
 
KOKKOS_FUNCTION iterator begin () const
 Get the beginning iterator. More...
 
KOKKOS_FUNCTION iterator end () const
 Get the end iterator. More...
 

Protected Member Functions

template<bool host, bool device>
void createInternal (const std::vector< dof_id_type > &n)
 The internal method to initialize and allocate this array. More...
 
void createInternal (const std::vector< dof_id_type > &n, bool host, bool device)
 The internal method to initialize and allocate this array. More...
 
template<typename TargetSpace , typename SourceSpace >
void copyInternal (T *target, const T *source, dof_id_type n)
 The internal method to perform a memory copy. More...
 

Protected Attributes

dof_id_type _n [dimension] = {0}
 Size of each dimension. More...
 
dof_id_type _s [dimension] = {0}
 Stride of each dimension. More...
 
dof_id_signed_type _d [dimension] = {0}
 Offset of each dimension. More...
 

Private Member Functions

void allocHost ()
 Allocate host data for an initialized array that has not allocated host data. More...
 
void allocDevice ()
 Allocate device data for an initialized array that has not allocated device data. More...
 

Private Attributes

std::shared_ptr< unsigned int_counter
 Reference counter. More...
 
bool _is_init = false
 Flag whether array was initialized. More...
 
bool _is_host_alloc = false
 Flag whether host data was allocated. More...
 
bool _is_device_alloc = false
 Flag whether device data was allocated. More...
 
bool _is_host_alias = false
 Flag whether the host data points to an external data. More...
 
bool _is_device_alias = false
 Flag whether the device data points to an external data. More...
 
T * _host_data = nullptr
 Host data. More...
 
T * _device_data = nullptr
 Device data. More...
 
dof_id_type _size = 0
 Total size. More...
 

Detailed Description

template<typename T, unsigned int dimension>
class Moose::Kokkos::ArrayBase< T, dimension >

The base class for Kokkos arrays.

Definition at line 83 of file KokkosArray.h.

Constructor & Destructor Documentation

◆ ArrayBase() [1/2]

template<typename T, unsigned int dimension>
Moose::Kokkos::ArrayBase< T, dimension >::ArrayBase ( )
default

Default constructor.

◆ ArrayBase() [2/2]

template<typename T, unsigned int dimension>
Moose::Kokkos::ArrayBase< T, dimension >::ArrayBase ( const ArrayBase< T, dimension > &  array)
inline

Copy constructor.

Definition at line 94 of file KokkosArray.h.

95  {
96 #ifndef MOOSE_KOKKOS_SCOPE
97  static_assert(!ArrayDeepCopy<T>::value,
98  "Kokkos array cannot be deep copied outside the Kokkos compilation scope");
99 #endif
100 
101  if constexpr (ArrayDeepCopy<T>::value)
102  deepCopy(array);
103  else
104  shallowCopy(array);
105  }
void shallowCopy(const ArrayBase< T, dimension > &array)
Shallow copy another Kokkos array.
Definition: KokkosArray.h:495
static const bool value
Definition: KokkosArray.h:69
void deepCopy(const ArrayBase< T, dimension > &array)
Deep copy another Kokkos array If ArrayDeepCopy<T>::value is true, it will copy-construct each entry ...
Definition: KokkosArray.h:824

◆ ~ArrayBase()

template<typename T, unsigned int dimension>
Moose::Kokkos::ArrayBase< T, dimension >::~ArrayBase ( )
inline

Destructor.

Definition at line 110 of file KokkosArray.h.

110 { destroy(); }
void destroy()
Free all data and reset.
Definition: KokkosArray.h:444

Member Function Documentation

◆ aliasDevice()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::aliasDevice ( T *  ptr)

Point the device data to an external data instead of allocating it.

Parameters
ptrThe pointer to the external device data

Definition at line 538 of file KokkosArray.h.

539 {
540  if (!_is_init)
541  mooseError("Kokkos array error: attempted to alias device data before array initialization.");
542 
544  mooseError("Kokkos array error: cannot alias device data because device data was not aliased.");
545 
546  _device_data = ptr;
547  _is_device_alloc = true;
548  _is_device_alias = true;
549 }
bool _is_device_alias
Flag whether the device data points to an external data.
Definition: KokkosArray.h:427
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:323
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:411
T * _device_data
Device data.
Definition: KokkosArray.h:435

◆ aliasHost()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::aliasHost ( T *  ptr)

Point the host data to an external data instead of allocating it.

Parameters
ptrThe pointer to the external host data

Definition at line 523 of file KokkosArray.h.

524 {
525  if (!_is_init)
526  mooseError("Kokkos array error: attempted to alias host data before array initialization.");
527 
529  mooseError("Kokkos array error: cannot alias host data because host data was not aliased.");
530 
531  _host_data = ptr;
532  _is_host_alloc = true;
533  _is_host_alias = true;
534 }
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:323
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:411
T * _host_data
Host data.
Definition: KokkosArray.h:431
bool _is_host_alias
Flag whether the host data points to an external data.
Definition: KokkosArray.h:423
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ allocDevice()

template<typename T , unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::allocDevice ( )
private

Allocate device data for an initialized array that has not allocated device data.

Definition at line 568 of file KokkosArray.h.

569 {
570  if (_is_device_alloc)
571  return;
572 
573  _device_data =
574  static_cast<T *>(::Kokkos::kokkos_malloc<ExecSpace::memory_space>(_size * sizeof(T)));
575 
576  _is_device_alloc = true;
577 }
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
T * _device_data
Device data.
Definition: KokkosArray.h:435

◆ allocHost()

template<typename T , unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::allocHost ( )
private

Allocate host data for an initialized array that has not allocated host data.

Definition at line 553 of file KokkosArray.h.

554 {
555  if (_is_host_alloc)
556  return;
557 
558  if constexpr (std::is_default_constructible<T>::value)
559  _host_data = new T[_size];
560  else
561  _host_data = static_cast<T *>(std::malloc(_size * sizeof(T)));
562 
563  _is_host_alloc = true;
564 }
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
T * _host_data
Host data.
Definition: KokkosArray.h:431
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ begin()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION iterator Moose::Kokkos::ArrayBase< T, dimension >::begin ( ) const
inline

Get the beginning iterator.

Returns
The beginning iterator

Definition at line 332 of file KokkosArray.h.

333  {
334  KOKKOS_IF_ON_HOST(return iterator(_host_data);)
335 
336  return iterator(_device_data);
337  }
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435

◆ copyIn()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::copyIn ( const T *  ptr,
MemcpyKind  dir,
dof_id_type  n,
dof_id_type  offset = 0 
)

Copy data from an external data to this array.

Parameters
ptrThe pointer to the external data
dirThe copy direction
nThe number of entries to copy
offsetThe starting offset of this array

Definition at line 705 of file KokkosArray.h.

706 {
707  if (n > _size)
708  mooseError("Kokkos array error: cannot copyin data larger than the array size.");
709 
710  if (offset > _size)
711  mooseError("Kokkos array error: offset cannot be larger than the array size.");
712 
713  if (dir == MemcpyKind::HOST_TO_HOST)
714  {
715  // If host side memory is not allocated, do nothing
716  if (!_is_host_alloc)
717  return;
718 
719  // Copy from host to host
720  copyInternal<::Kokkos::HostSpace, ::Kokkos::HostSpace>(_host_data + offset, ptr, n);
721  }
722  else if (dir == MemcpyKind::HOST_TO_DEVICE)
723  {
724  // If device side memory is not allocated, do nothing
725  if (!_is_device_alloc)
726  return;
727 
728  // Copy from host to device
729  copyInternal<MemSpace, ::Kokkos::HostSpace>(_device_data + offset, ptr, n);
730  }
731  else if (dir == MemcpyKind::DEVICE_TO_HOST)
732  {
733  // If host side memory is not allocated, do nothing
734  if (!_is_host_alloc)
735  return;
736 
737  // Copy from device to host
738  copyInternal<::Kokkos::HostSpace, MemSpace>(_host_data + offset, ptr, n);
739  }
740  else if (dir == MemcpyKind::DEVICE_TO_DEVICE)
741  {
742  // If device side memory is not allocated, do nothing
743  if (!_is_device_alloc)
744  return;
745 
746  // Copy from device to device
747  copyInternal<MemSpace, MemSpace>(_device_data + offset, ptr, n);
748  }
749 }
KOKKOS_FUNCTION dof_id_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:165
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:323
void offset(const std::vector< dof_id_signed_type > &d)
Apply starting index offsets to each dimension.
Definition: KokkosArray.h:642
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ copyInternal()

template<typename T, unsigned int dimension>
template<typename TargetSpace , typename SourceSpace >
void Moose::Kokkos::ArrayBase< T, dimension >::copyInternal ( T *  target,
const T *  source,
dof_id_type  n 
)
protected

The internal method to perform a memory copy.

Template Parameters
TargetSpaceThe Kokkos memory space of target data
SourcespaceThe Kokkos memory space of source data
Parameters
targetThe pointer to the target data
sourceThe pointer to the source data
nThe number of entries to copy

Definition at line 634 of file KokkosArray.h.

635 {
636  ::Kokkos::Impl::DeepCopy<TargetSpace, SourceSpace>(target, source, n * sizeof(T));
637  ::Kokkos::fence();
638 }
KOKKOS_FUNCTION dof_id_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:165

◆ copyOut()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::copyOut ( T *  ptr,
MemcpyKind  dir,
dof_id_type  n,
dof_id_type  offset = 0 
)

Copy data to an external data from this array.

Parameters
ptrThe pointer to the external data
dirThe copy direction
nThe number of entries to copy
offsetThe starting offset of this array

Definition at line 753 of file KokkosArray.h.

754 {
755  if (n > _size)
756  mooseError("Kokkos array error: cannot copyout data larger than the array size.");
757 
758  if (offset > _size)
759  mooseError("Kokkos array error: offset cannot be larger than the array size.");
760 
761  if (dir == MemcpyKind::HOST_TO_HOST)
762  {
763  // If host side memory is not allocated, do nothing
764  if (!_is_host_alloc)
765  return;
766 
767  // Copy from host to host
768  copyInternal<::Kokkos::HostSpace, ::Kokkos::HostSpace>(ptr, _host_data + offset, n);
769  }
770  else if (dir == MemcpyKind::HOST_TO_DEVICE)
771  {
772  // If host side memory is not allocated, do nothing
773  if (!_is_host_alloc)
774  return;
775 
776  // Copy from host to device
777  copyInternal<MemSpace, ::Kokkos::HostSpace>(ptr, _host_data + offset, n);
778  }
779  else if (dir == MemcpyKind::DEVICE_TO_HOST)
780  {
781  // If device side memory is not allocated, do nothing
782  if (!_is_device_alloc)
783  return;
784 
785  // Copy from device to host
786  copyInternal<::Kokkos::HostSpace, MemSpace>(ptr, _device_data + offset, n);
787  }
788  else if (dir == MemcpyKind::DEVICE_TO_DEVICE)
789  {
790  // If device side memory is not allocated, do nothing
791  if (!_is_device_alloc)
792  return;
793 
794  // Copy from device to device
795  copyInternal<MemSpace, MemSpace>(ptr, _device_data + offset, n);
796  }
797 }
KOKKOS_FUNCTION dof_id_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:165
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:323
void offset(const std::vector< dof_id_signed_type > &d)
Apply starting index offsets to each dimension.
Definition: KokkosArray.h:642
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ copyToDevice()

template<typename T , unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::copyToDevice ( )

Copy data from host to device.

Definition at line 657 of file KokkosArray.h.

658 {
659  // If host side memory is not allocated, do nothing
660  if (!_is_host_alloc)
661  return;
662 
663  // If device side memory is not allocated,
664  if (!_is_device_alloc)
665  {
666  if (_counter.use_count() == 1)
667  // allocate memory if this array is not shared with other arrays
668  allocDevice();
669  else
670  // print error if this array is shared with other arrays
671  mooseError("Kokkos array error: cannot copy from host to device because device side memory "
672  "was not allocated and array is being shared with other arrays.");
673  }
674 
675  // Copy from host to device
676  copyInternal<MemSpace, ::Kokkos::HostSpace>(_device_data, _host_data, _size);
677 }
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:323
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
T * _host_data
Host data.
Definition: KokkosArray.h:431
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:407
T * _device_data
Device data.
Definition: KokkosArray.h:435
void allocDevice()
Allocate device data for an initialized array that has not allocated device data. ...
Definition: KokkosArray.h:568
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ copyToDeviceNested()

template<typename T , unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::copyToDeviceNested ( )

Copy all the nested Kokkos arrays including self from host to device.

Definition at line 814 of file KokkosArray.h.

815 {
816  for (unsigned int i = 0; i < _size; ++i)
818 
819  copyToDevice();
820 }
void copyToDevice()
Copy data from host to device.
Definition: KokkosArray.h:657
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
void copyToDeviceInner(T &)
Definition: KokkosArray.h:801
T * _host_data
Host data.
Definition: KokkosArray.h:431

◆ copyToHost()

template<typename T , unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::copyToHost ( )

Copy data from device to host.

Definition at line 681 of file KokkosArray.h.

682 {
683  // If device side memory is not allocated, do nothing
684  if (!_is_device_alloc)
685  return;
686 
687  // If host side memory is not allocated,
688  if (!_is_host_alloc)
689  {
690  if (_counter.use_count() == 1)
691  // allocate memory if this array is not shared with other arrays
692  allocHost();
693  else
694  // print error if this array is shared with other arrays
695  mooseError("Kokkos array error: cannot copy from device to host because host side memory "
696  "was not allocated and array is being shared with other arrays.");
697  }
698 
699  // Copy from device to host
700  copyInternal<::Kokkos::HostSpace, MemSpace>(_host_data, _device_data, _size);
701 }
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:323
void allocHost()
Allocate host data for an initialized array that has not allocated host data.
Definition: KokkosArray.h:553
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
T * _host_data
Host data.
Definition: KokkosArray.h:431
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:407
T * _device_data
Device data.
Definition: KokkosArray.h:435
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ create()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::create ( const std::vector< dof_id_type > &  n)
inline

Allocate array on host and device.

Parameters
nThe vector containing the size of each dimension

Definition at line 228 of file KokkosArray.h.

228 { createInternal<true, true>(n); }
KOKKOS_FUNCTION dof_id_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:165

◆ createDevice()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::createDevice ( const std::vector< dof_id_type > &  n)
inline

Allocate array on device only.

Parameters
nThe vector containing the size of each dimension

Definition at line 238 of file KokkosArray.h.

238 { createInternal<false, true>(n); }
KOKKOS_FUNCTION dof_id_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:165

◆ createHost()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::createHost ( const std::vector< dof_id_type > &  n)
inline

Allocate array on host only.

Parameters
nThe vector containing the size of each dimension

Definition at line 233 of file KokkosArray.h.

233 { createInternal<true, false>(n); }
KOKKOS_FUNCTION dof_id_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:165

◆ createInternal() [1/2]

template<typename T , unsigned int dimension>
template<bool host, bool device>
void Moose::Kokkos::ArrayBase< T, dimension >::createInternal ( const std::vector< dof_id_type > &  n)
protected

The internal method to initialize and allocate this array.

Template Parameters
hostWhether host data will be allocated
deviceWhether device data will be allocated
Parameters
nThe vector containing the size of each dimension

Definition at line 582 of file KokkosArray.h.

583 {
584  if (n.size() != dimension)
585  mooseError("Kokkos array error: the number of dimensions provided (",
586  n.size(),
587  ") must match the array dimension (",
588  dimension,
589  ").");
590 
591  if (_counter)
592  destroy();
593 
594  _counter = std::make_shared<unsigned int>();
595 
596  _size = 1;
597  _s[0] = 1;
598 
599  for (const auto i : make_range(dimension))
600  {
601  _n[i] = n[i];
602  _size *= n[i];
603 
604  if (i)
605  _s[i] = _s[i - 1] * _n[i - 1];
606  }
607 
608  if constexpr (host)
609  allocHost();
610 
611  if constexpr (device)
612  allocDevice();
613 
614  _is_init = true;
615 }
KOKKOS_FUNCTION dof_id_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:165
dof_id_type _n[dimension]
Size of each dimension.
Definition: KokkosArray.h:354
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:323
void destroy()
Free all data and reset.
Definition: KokkosArray.h:444
void allocHost()
Allocate host data for an initialized array that has not allocated host data.
Definition: KokkosArray.h:553
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:411
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
dof_id_type _s[dimension]
Stride of each dimension.
Definition: KokkosArray.h:358
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:407
IntRange< T > make_range(T beg, T end)
void allocDevice()
Allocate device data for an initialized array that has not allocated device data. ...
Definition: KokkosArray.h:568

◆ createInternal() [2/2]

template<typename T , unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::createInternal ( const std::vector< dof_id_type > &  n,
bool  host,
bool  device 
)
protected

The internal method to initialize and allocate this array.

Parameters
nThe vector containing the size of each dimension
hostThe flag whether host data will be allocated
deviceThe flag whether device data will be allocated

Definition at line 619 of file KokkosArray.h.

620 {
621  if (host && device)
622  createInternal<true, true>(n);
623  else if (host && !device)
624  createInternal<true, false>(n);
625  else if (!host && device)
626  createInternal<false, true>(n);
627  else
628  createInternal<false, false>(n);
629 }
KOKKOS_FUNCTION dof_id_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:165

◆ data()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION T* Moose::Kokkos::ArrayBase< T, dimension >::data ( ) const
inline

Get the data pointer.

Returns
The pointer to the underlying data depending on the architecture this function is being called on

Definition at line 171 of file KokkosArray.h.

172  {
173  KOKKOS_IF_ON_HOST(return _host_data;)
174 
175  return _device_data;
176  }
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435

◆ deepCopy()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::deepCopy ( const ArrayBase< T, dimension > &  array)

Deep copy another Kokkos array If ArrayDeepCopy<T>::value is true, it will copy-construct each entry If ArrayDeepCopy<T>::value is false, it will do a memory copy.

Parameters
arrayThe Kokkos array to be deep copied

Definition at line 824 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::ArrayBase().

825 {
826  if (ArrayDeepCopy<T>::value && !array._is_host_alloc)
827  mooseError(
828  "Kokkos array error: cannot deep copy using constructor from array without host data.");
829 
830  std::vector<dof_id_type> n(std::begin(array._n), std::end(array._n));
831 
832  createInternal(n, array._is_host_alloc, array._is_device_alloc);
833 
834  if constexpr (ArrayDeepCopy<T>::value)
835  {
836  for (dof_id_type i = 0; i < _size; ++i)
837  new (_host_data + i) T(array._host_data[i]);
838 
839  copyToDevice();
840  }
841  else
842  {
843  if (_is_host_alloc)
844  std::memcpy(_host_data, array._host_data, _size * sizeof(T));
845 
846  if (_is_device_alloc)
847  copyInternal<MemSpace, MemSpace>(_device_data, array._device_data, _size);
848  }
849 
850  for (unsigned int i = 0; i < dimension; ++i)
851  {
852  _d[i] = array._d[i];
853  _s[i] = array._s[i];
854  }
855 }
KOKKOS_FUNCTION dof_id_type n(unsigned int dim) const
Get the size of a dimension.
Definition: KokkosArray.h:165
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:323
void copyToDevice()
Copy data from host to device.
Definition: KokkosArray.h:657
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
static const bool value
Definition: KokkosArray.h:69
dof_id_type _s[dimension]
Stride of each dimension.
Definition: KokkosArray.h:358
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435
void createInternal(const std::vector< dof_id_type > &n)
The internal method to initialize and allocate this array.
Definition: KokkosArray.h:582
dof_id_signed_type _d[dimension]
Offset of each dimension.
Definition: KokkosArray.h:362
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415
uint8_t dof_id_type

◆ destroy()

template<typename T , unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::destroy ( )

Free all data and reset.

Definition at line 444 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::~ArrayBase().

445 {
446  if (!_counter)
447  return;
448 
449  if (_counter.use_count() > 1)
450  {
451  _host_data = nullptr;
452  _device_data = nullptr;
453  }
454  else if (_counter.use_count() == 1)
455  {
457  {
458  if constexpr (std::is_default_constructible<T>::value)
459  // Allocated by new
460  delete[] _host_data;
461  else
462  {
463  // Allocated by malloc
464  for (dof_id_type i = 0; i < _size; ++i)
465  _host_data[i].~T();
466 
468  }
469  }
470 
473  }
474 
475  _size = 0;
476 
477  for (unsigned int i = 0; i < dimension; ++i)
478  {
479  _n[i] = 0;
480  _s[i] = 0;
481  _d[i] = 0;
482  }
483 
484  _is_init = false;
485  _is_host_alloc = false;
486  _is_device_alloc = false;
487  _is_host_alias = false;
488  _is_device_alias = false;
489 
490  _counter.reset();
491 }
bool _is_device_alias
Flag whether the device data points to an external data.
Definition: KokkosArray.h:427
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
dof_id_type _n[dimension]
Size of each dimension.
Definition: KokkosArray.h:354
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:411
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
dof_id_type _s[dimension]
Stride of each dimension.
Definition: KokkosArray.h:358
T * _host_data
Host data.
Definition: KokkosArray.h:431
void free(void *ptr)
bool _is_host_alias
Flag whether the host data points to an external data.
Definition: KokkosArray.h:423
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:407
T * _device_data
Device data.
Definition: KokkosArray.h:435
dof_id_signed_type _d[dimension]
Offset of each dimension.
Definition: KokkosArray.h:362
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415
uint8_t dof_id_type

◆ deviceData()

template<typename T, unsigned int dimension>
T* Moose::Kokkos::ArrayBase< T, dimension >::deviceData ( ) const
inline

Get the device data pointer.

Returns
The pointer to the underlying device data

Definition at line 223 of file KokkosArray.h.

223 { return _device_data; }
T * _device_data
Device data.
Definition: KokkosArray.h:435

◆ end()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION iterator Moose::Kokkos::ArrayBase< T, dimension >::end ( ) const
inline

Get the end iterator.

Returns
The end iterator

Definition at line 342 of file KokkosArray.h.

343  {
344  KOKKOS_IF_ON_HOST(return iterator(_host_data + _size);)
345 
346  return iterator(_device_data + _size);
347  }
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435

◆ first()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION T& Moose::Kokkos::ArrayBase< T, dimension >::first ( ) const
inline

Get the first element.

Returns
The reference of the first element depending on the architecture this function is being called on

Definition at line 182 of file KokkosArray.h.

183  {
184  KOKKOS_IF_ON_HOST(return _host_data[0];)
185 
186  return _device_data[0];
187  }
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435

◆ hostData()

template<typename T, unsigned int dimension>
T* Moose::Kokkos::ArrayBase< T, dimension >::hostData ( ) const
inline

Get the host data pointer.

Returns
The pointer to the underlying host data

Definition at line 218 of file KokkosArray.h.

218 { return _host_data; }
T * _host_data
Host data.
Definition: KokkosArray.h:431

◆ isAlloc()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension >::isAlloc ( ) const
inline

Get whether the array was allocated either on host or device.

Returns
Whether the array was allocated either on host or device

Definition at line 134 of file KokkosArray.h.

134 { return _is_host_alloc || _is_device_alloc; }
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ isDeviceAlias()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension >::isDeviceAlias ( ) const
inline

Get whether the device array was aliased.

Returns
Whether the device array was aliased

Definition at line 154 of file KokkosArray.h.

154 { return _is_device_alias; }
bool _is_device_alias
Flag whether the device data points to an external data.
Definition: KokkosArray.h:427

◆ isDeviceAlloc()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension >::isDeviceAlloc ( ) const
inline

Get whether the array was allocated on device.

Returns
Whether the array was allocated on device

Definition at line 144 of file KokkosArray.h.

144 { return _is_device_alloc; }
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419

◆ isHostAlias()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension >::isHostAlias ( ) const
inline

Get whether the host array was aliased.

Returns
Whether the host array was aliased

Definition at line 149 of file KokkosArray.h.

149 { return _is_host_alias; }
bool _is_host_alias
Flag whether the host data points to an external data.
Definition: KokkosArray.h:423

◆ isHostAlloc()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION bool Moose::Kokkos::ArrayBase< T, dimension >::isHostAlloc ( ) const
inline

Get whether the array was allocated on host.

Returns
Whether the array was allocated on host

Definition at line 139 of file KokkosArray.h.

139 { return _is_host_alloc; }
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ last()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION T& Moose::Kokkos::ArrayBase< T, dimension >::last ( ) const
inline

Get the last element.

Returns
The reference of the last element depending on the architecture this function is being called on

Definition at line 193 of file KokkosArray.h.

194  {
195  KOKKOS_IF_ON_HOST(return _host_data[_size - 1];)
196 
197  return _device_data[_size - 1];
198  }
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435

◆ n()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION dof_id_type Moose::Kokkos::ArrayBase< T, dimension >::n ( unsigned int  dim) const
inline

Get the size of a dimension.

Parameters
dimThe dimension index
Returns
The array size of the dimension

Definition at line 165 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::create(), Moose::Kokkos::ArrayBase< T, 4 >::createDevice(), and Moose::Kokkos::ArrayBase< T, 4 >::createHost().

165 { return _n[dim]; }
dof_id_type _n[dimension]
Size of each dimension.
Definition: KokkosArray.h:354
static constexpr std::size_t dim
This is the dimension of all vector and tensor datastructures used in MOOSE.
Definition: Moose.h:159

◆ offset()

template<typename T , unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::offset ( const std::vector< dof_id_signed_type > &  d)

Apply starting index offsets to each dimension.

Parameters
dThe vector containing the offset of each dimension

Definition at line 642 of file KokkosArray.h.

Referenced by Moose::Kokkos::Array< T, 1 >::offset(), Moose::Kokkos::Array< T, 2 >::offset(), Moose::Kokkos::Array< T, 3 >::offset(), Moose::Kokkos::Array< T, 4 >::offset(), and Moose::Kokkos::Array< T, 5 >::offset().

643 {
644  if (d.size() > dimension)
645  mooseError("Kokkos array error: the number of offsets provided (",
646  d.size(),
647  ") cannot be larger than the array dimension (",
648  dimension,
649  ").");
650 
651  for (const auto i : index_range(d))
652  _d[i] = d[i];
653 }
void mooseError(Args &&... args)
Emit an error message with the given stringified, concatenated args and terminate the application...
Definition: MooseError.h:323
dof_id_signed_type _d[dimension]
Offset of each dimension.
Definition: KokkosArray.h:362
auto index_range(const T &sizable)

◆ operator=()

template<typename T, unsigned int dimension>
auto & Moose::Kokkos::ArrayBase< T, dimension >::operator= ( const T &  scalar)

Assign a scalar value uniformly.

Parameters
scalarThe scalar value to be assigned

Definition at line 870 of file KokkosArray.h.

871 {
872  if (_is_host_alloc)
873  std::fill_n(_host_data, _size, scalar);
874 
875  if (_is_device_alloc)
876  {
877  ::Kokkos::View<T *, MemSpace, ::Kokkos::MemoryTraits<::Kokkos::Unmanaged>> data(_device_data,
878  _size);
879  ::Kokkos::Experimental::fill_n(ExecSpace(), data, _size, scalar);
880  }
881 
882  return *this;
883 }
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
KOKKOS_FUNCTION T * data() const
Get the data pointer.
Definition: KokkosArray.h:171
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ operator[]()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION T& Moose::Kokkos::ArrayBase< T, dimension >::operator[] ( dof_id_type  i) const
inline

Get an array entry.

Parameters
iThe dimensionless index
Returns
The reference of the entry depending on the architecture this function is being called on

Definition at line 205 of file KokkosArray.h.

206  {
207  KOKKOS_ASSERT(i < _size);
208 
209  KOKKOS_IF_ON_HOST(return _host_data[i];)
210 
211  return _device_data[i];
212  }
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
T * _host_data
Host data.
Definition: KokkosArray.h:431
T * _device_data
Device data.
Definition: KokkosArray.h:435

◆ shallowCopy()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::shallowCopy ( const ArrayBase< T, dimension > &  array)

Shallow copy another Kokkos array.

Parameters
arrayThe Kokkos array to be shallow copied

Definition at line 495 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::ArrayBase(), and Moose::Kokkos::ArrayBase< T, 4 >::swap().

496 {
497  destroy();
498 
499  _counter = array._counter;
500 
501  _size = array._size;
502 
503  for (unsigned int i = 0; i < dimension; ++i)
504  {
505  _n[i] = array._n[i];
506  _s[i] = array._s[i];
507  _d[i] = array._d[i];
508  }
509 
510  _is_init = array._is_init;
511  _is_host_alloc = array._is_host_alloc;
512  _is_device_alloc = array._is_device_alloc;
513  _is_host_alias = array._is_host_alias;
514  _is_device_alias = array._is_device_alias;
515 
516  _host_data = array._host_data;
517  _device_data = array._device_data;
518 }
bool _is_device_alias
Flag whether the device data points to an external data.
Definition: KokkosArray.h:427
bool _is_device_alloc
Flag whether device data was allocated.
Definition: KokkosArray.h:419
dof_id_type _n[dimension]
Size of each dimension.
Definition: KokkosArray.h:354
void destroy()
Free all data and reset.
Definition: KokkosArray.h:444
bool _is_init
Flag whether array was initialized.
Definition: KokkosArray.h:411
dof_id_type _size
Total size.
Definition: KokkosArray.h:439
dof_id_type _s[dimension]
Stride of each dimension.
Definition: KokkosArray.h:358
T * _host_data
Host data.
Definition: KokkosArray.h:431
bool _is_host_alias
Flag whether the host data points to an external data.
Definition: KokkosArray.h:423
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:407
T * _device_data
Device data.
Definition: KokkosArray.h:435
dof_id_signed_type _d[dimension]
Offset of each dimension.
Definition: KokkosArray.h:362
bool _is_host_alloc
Flag whether host data was allocated.
Definition: KokkosArray.h:415

◆ size()

template<typename T, unsigned int dimension>
KOKKOS_FUNCTION dof_id_type Moose::Kokkos::ArrayBase< T, dimension >::size ( ) const
inline

Get the total array size.

Returns
The total array size

Definition at line 159 of file KokkosArray.h.

159 { return _size; }
dof_id_type _size
Total size.
Definition: KokkosArray.h:439

◆ swap()

template<typename T, unsigned int dimension>
void Moose::Kokkos::ArrayBase< T, dimension >::swap ( ArrayBase< T, dimension > &  array)

Swap with another Kokkos array.

Parameters
arrayThe Kokkos array to be swapped

Definition at line 859 of file KokkosArray.h.

860 {
861  ArrayBase<T, dimension> clone;
862 
863  clone.shallowCopy(*this);
864  this->shallowCopy(array);
865  array.shallowCopy(clone);
866 }
void shallowCopy(const ArrayBase< T, dimension > &array)
Shallow copy another Kokkos array.
Definition: KokkosArray.h:495

◆ useCount()

template<typename T, unsigned int dimension>
unsigned int Moose::Kokkos::ArrayBase< T, dimension >::useCount ( ) const
inline

Get the reference count.

Returns
The reference count

Definition at line 127 of file KokkosArray.h.

127 { return _counter.use_count(); }
std::shared_ptr< unsigned int > _counter
Reference counter.
Definition: KokkosArray.h:407

Member Data Documentation

◆ _counter

template<typename T, unsigned int dimension>
std::shared_ptr<unsigned int> Moose::Kokkos::ArrayBase< T, dimension >::_counter
private

Reference counter.

Definition at line 407 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::shallowCopy(), and Moose::Kokkos::ArrayBase< T, 4 >::useCount().

◆ _d

template<typename T, unsigned int dimension>
dof_id_signed_type Moose::Kokkos::ArrayBase< T, dimension >::_d[dimension] = {0}
protected

Offset of each dimension.

Definition at line 362 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::deepCopy(), and Moose::Kokkos::ArrayBase< T, 4 >::shallowCopy().

◆ _device_data

template<typename T, unsigned int dimension>
T* Moose::Kokkos::ArrayBase< T, dimension >::_device_data = nullptr
private

◆ _host_data

template<typename T, unsigned int dimension>
T* Moose::Kokkos::ArrayBase< T, dimension >::_host_data = nullptr
private

◆ _is_device_alias

template<typename T, unsigned int dimension>
bool Moose::Kokkos::ArrayBase< T, dimension >::_is_device_alias = false
private

Flag whether the device data points to an external data.

Definition at line 427 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::isDeviceAlias(), and Moose::Kokkos::ArrayBase< T, 4 >::shallowCopy().

◆ _is_device_alloc

template<typename T, unsigned int dimension>
bool Moose::Kokkos::ArrayBase< T, dimension >::_is_device_alloc = false
private

◆ _is_host_alias

template<typename T, unsigned int dimension>
bool Moose::Kokkos::ArrayBase< T, dimension >::_is_host_alias = false
private

Flag whether the host data points to an external data.

Definition at line 423 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::isHostAlias(), and Moose::Kokkos::ArrayBase< T, 4 >::shallowCopy().

◆ _is_host_alloc

template<typename T, unsigned int dimension>
bool Moose::Kokkos::ArrayBase< T, dimension >::_is_host_alloc = false
private

◆ _is_init

template<typename T, unsigned int dimension>
bool Moose::Kokkos::ArrayBase< T, dimension >::_is_init = false
private

Flag whether array was initialized.

Definition at line 411 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::shallowCopy().

◆ _n

template<typename T, unsigned int dimension>
dof_id_type Moose::Kokkos::ArrayBase< T, dimension >::_n[dimension] = {0}
protected

◆ _s

template<typename T, unsigned int dimension>
dof_id_type Moose::Kokkos::ArrayBase< T, dimension >::_s[dimension] = {0}
protected

Stride of each dimension.

Definition at line 358 of file KokkosArray.h.

Referenced by Moose::Kokkos::ArrayBase< T, 4 >::deepCopy(), and Moose::Kokkos::ArrayBase< T, 4 >::shallowCopy().

◆ _size

template<typename T, unsigned int dimension>
dof_id_type Moose::Kokkos::ArrayBase< T, dimension >::_size = 0
private

The documentation for this class was generated from the following file: