MOOSE Newsletter (April 2023)

MOOSE Improvements

Execution ordering printing

Nearly every solve in MOOSE consists of a succession of operations on nodes, quadrature points, elements or elements' faces. These operations may be for example to compute the contribution of a kernel/boundary condition/etc. to the residual, Jacobian, etc.

The MOOSE Debug system now offers the "show_execution_order" parameter to output the order of execution of each of these objects in those loops. This order may depend on local block/boundary restrictions, and local or global dependency resolutions.

This parameter is most helpful to understand if AuxKernels, UserObjects and other systems which can interact in arbitrarily complex ways on a group of variables are indeed executed in the order desired by the modeler. If problematic, object execution may be reordered using various boolean parameters, execute_on flags, and other manual dependency declarations. For example for UserObjects, the execution_order_group parameter lets the modeler select the ordering of executions of user objects.

Support for mesh repartitioning with stateful material properties

We previously could not handle any repartitioning in problems using stateful material properties, and so MOOSE disabled repartitioning in This made adaptive mesh refinement/coarsening (AMR/C) and other mesh modifying simulation types less efficient in parallel.

Now MOOSE will automatically redistribute stateful material property data as part of any mesh repartitioning step, using the same serialization APIs that are also used for restarts, for what should be seamless backwards compatibility.

One note to watch out for: because simulations with stateful material properties and AMR/C will now enable repartitioning+redistribution by default, any other application code used in a such a simulation should be either compatible with those operations, or should itself disable them.

General 2D-XY Mesh Line Cutting Capability

XYMeshLineCutter was added to enable general line cutting for a 2D mesh that only contains TRI3 and QUAD4 elements in the XY-plane. The new mesh generator trims off the fraction of the input mesh that is located on one side of a given straight line. The elements/nodes near the cutting line of the mesh are processed to ensure a smooth cross-section instead of a "zigzag" cross-section as generated by PlaneDeletionGenerator.

Quadrilateral Mesh Generation in Simple Curvilinear Domains

TransfiniteMeshGenerator is a novel mesh generator that allows a range of 2D meshing strategies in curvilinear domains. This mesh generator addresses the need for QUAD element meshes within simple curvilinear domains, which can be flexibly "stitched" to other regions to construct complex templated meshes. The core algorithm is known as transfinite interpolation or the Gordon-Hull algorithm. The only requirement of the mesh generator is that the domain is bounded by pairs of opposing edges with types prescribed by the user. The mesh generator accepts different types of domain edges, LINE, DISCRETE, PARSED, and CIRCARC. The CIRCARC option constructs an arc of a circle given two endpoints and either a third middle point or the deviation from a perfect circle.

Physics module changes

A field split preconditioning example was added to the Navier Stokes module, showing a factor of 3 CPU-time reduction compared to a direct solve on channel cases with various Reynolds numbers. It uses Schur preconditioning splitting between velocities and pressure.

Bug Fixes and Minor Enhancements

  • Compile-time warnings and run-time PETSc errors triggered by use of PETSc 3.19 have been fixed.

  • The boundary_id_offset option to GeneratedMeshGenerator is now compatible with distributed meshes.

  • The verbosity of mesh generator stitching has been reduced to be more manageable for large core geometries.

  • GlobalParams, parameters defined in a single block and applied everywhere they could, are no longer considered for suppressed/private parameters. In the past, they would trigger the error for setting a suppressed parameter from the input file.

libMesh-level Changes

2023.04.19 Update

  • MeshBase subclasses now support operator== tests, to check geometric and subclass type and indexing equality.

  • MeshTools::valid_is_prepared() test, to help debugging code determine whether a mesh modification has left a mesh in an unprepared state without either using prepare_for_use() or set_isnt_prepared() to change or indicate that.

  • APIs for applying constraints to a nonlinear system assembly in a Jacobian-consistent way (as the current PetscNonlinearSolver does), but at an element-by-element level rather than as a post-facto edit of the global residual and Jacobian. This will enable MOOSE to fix a major performance bug affecting solves with adaptive mesh refinement, strongly-enforced periodic boundary conditions, and/or IsoGeometric Analysis meshes.

  • TIMPI updates:

    • TIMPI can now be instructed via a C++ API to use older synchronization algorithms in parallel_sync.h push and pull routines, for easier debugging. libMesh adds a --timpi-sync= command line option for this.

    • Attempts to send empty vectors in sync routines are now assertion failures, since excessive empty sends would be a potential performance issue.

    • Code cleanup, extra assertion in the default NBX sync routine. We think we understand what the MPI standard requires of synchronous-send operations, but this should help catch the problem if any MPI implementations disagree.

  • Assorted fixes:

    • NumericVector::compare() now handles empty local ranges

    • ExodusII edge block reading now handles orientation mismatch

    • isspace() is more robust to polluted global namespaces in libMesh forks

    • Overzealous assertion about mesh state removed from copy_nodes_and_elements

    • LinearPartitioner is now idempotent on distributed meshes

    • PETSc 3.19 compatibility