7. Advanced Topics

note

Under construction

Debugging Numerical Instability

Relaxation
Better initialization (make sure the finicky app does not have to deal with a bad solution from the other app) Plenty of techniques: run uncouple first for example

Reliability of Transfer

How to Handle Single App Crashes

How to handle TransientMultiapp when a child app fails

As previously stated, the default behavior when running TransientMultiApps is that their time state is incremented, e.g. they are auto-advance even the sub-apps solution fails. Despite the obvious disadvantage of this approach, the syncing of main and sub-application states was implemented to enable a clear checkpoint output. The Checkpoint output object is designed to perform restart and recovery of simulations. The output files created contain a complete snapshot of the simulation. These files are required to perform restart or recovery operations with a MOOSE-based application. There could be multiple ways to turn a failed sub-application solve from a warning into an exception that will force corrective behavior by, for example, setting auto_advance = false in the Executioner block of the main application. This will cause the main application to immediately cut its time-step when the sub-application fails.

However, setting this parameter to false also eliminates the possibility of doing restart/recover because the main and sub apps will be out of sync if/when checkpoint output occurs. Or the user may instead set catch_up = true in the TransientMultiApp block. This allows failed sub-app solves to attempt to 'catch up' using smaller timesteps. If catch-up is unsuccessful, then MOOSE registers this as a true failure of the solve, and the main dt will then get cut. This option has the advantage of keeping the main and sub transient states in sync, enabling accurate restart/recover data. In general, if the user wants sub-application failed solves to be treated as exceptions, we recommend using catch_up = true. An example of the MultiApps block for transient multi-physics Griffin-NS (Navier-Stokes) simulation is illustrated below:

Listing 1: An example of using the catch_up option in TransientMultiApps.

[MultiApps]
  [ns]
    type = TransientMultiApp
    input_files = 'run_ns.i'
    execute_on = 'timestep_begin'
    catch_up = true
  []
[]

(msr/msfr/plant/transient/run_neutronics.i)

################################################################################
## Molten Salt Fast Reactor - Euratom EVOL + Rosatom MARS Design              ##
## Griffin Main Application input file                                        ##
## Transient neutronics model                                                 ##
## Neutron diffusion with delayed precursor source, no equivalence            ##
################################################################################
# If using or referring to this model, please cite as explained in
# https://mooseframework.inl.gov/virtual_test_bed/citing.html

[Mesh]
  coord_type = 'RZ'
  [fmg]
    type = FileMeshGenerator
    # when changing restart file, adapt power_scaling postprocessor
    # if not exactly 3e9 -> initial was not completely converged
    file = '../steady/restart/run_neutronics_restart.e'
    use_for_exodus_restart = true
  []
[]

################################################################################
# EQUATION SYSTEM SETUP
################################################################################

[TransportSystems]
  particle = neutron
  equation_type = transient
  restart_transport_system = true

  G = 6

  VacuumBoundary = 'outer'

  [diff]
    scheme = CFEM-Diffusion
    n_delay_groups = 6
    external_dnp_variable = 'dnp'
    family = LAGRANGE
    order = FIRST
    fission_source_aux = true
  []
[]

[Problem]
  allow_initial_conditions_with_restart = true
[]

[AuxVariables]
  [tfuel]
    order = CONSTANT
    family = MONOMIAL
    # initial_condition = 600
    initial_from_file_var = tfuel
    block = 'fuel pump hx'
  []
  # TODO: remove once we have block restricted transfers
  [tfuel_constant]
    initial_condition = 873.15 # in degree K
    block = 'reflector shield'
  []
  [c1]
    order = CONSTANT
    family = MONOMIAL
    initial_from_file_var = c1
    block = 'fuel pump hx'
  []
  [c2]
    order = CONSTANT
    family = MONOMIAL
    initial_from_file_var = c2
    block = 'fuel pump hx'
  []
  [c3]
    order = CONSTANT
    family = MONOMIAL
    initial_from_file_var = c3
    block = 'fuel pump hx'
  []
  [c4]
    order = CONSTANT
    family = MONOMIAL
    initial_from_file_var = c4
    block = 'fuel pump hx'
  []
  [c5]
    order = CONSTANT
    family = MONOMIAL
    initial_from_file_var = c5
    block = 'fuel pump hx'
  []
  [c6]
    order = CONSTANT
    family = MONOMIAL
    initial_from_file_var = c6
    block = 'fuel pump hx'
  []
  [dnp]
    order = CONSTANT
    family = MONOMIAL
    components = 6
    # no need to initalize, initialized by auxkernels
    # initial_from_file_var = dnp
    block = 'fuel pump hx'
  []
  [power_density]
    order = CONSTANT
    family = MONOMIAL
    # no need to initalize, initialized by auxkernels
    # initial_from_file_var = 'power_density'
    block = 'fuel pump hx'
  []
[]

[AuxKernels]
  [build_dnp]
    type = BuildArrayVariableAux
    variable = dnp
    component_variables = 'c1 c2 c3 c4 c5 c6'
    execute_on = 'initial timestep_begin final'
  []
  [power_density]
    type = VectorReactionRate
    variable = power_density
    cross_section = kappa_sigma_fission
    scalar_flux = 'sflux_g0 sflux_g1 sflux_g2 sflux_g3 sflux_g4 sflux_g5'
    scale_factor = power_scaling
    execute_on = 'initial timestep_begin'
    block = 'fuel pump hx'
  []
[]

################################################################################
# CROSS SECTIONS
################################################################################

[Materials]
  [fuel]
    type = CoupledFeedbackNeutronicsMaterial
    library_name = 'msfr_xs'
    library_file = '../../mgxs/msfr_xs_extended.xml'
    grid_names = 'tfuel'
    grid_variables = 'tfuel'
    plus = true
    isotopes = 'pseudo'
    densities = '1.0'
    is_meter = true
    material_id = 2
    block = 'fuel pump hx'
  []
  [shield]
    type = CoupledFeedbackNeutronicsMaterial
    library_name = 'msfr_xs'
    library_file = '../../mgxs/msfr_xs_extended.xml'
    grid_names = 'tfuel'
    grid_variables = 'tfuel_constant'
    isotopes = 'pseudo'
    densities = '1.0'
    is_meter = true
    material_id = 5
    block = 'shield'
  []
  [reflector]
    type = CoupledFeedbackNeutronicsMaterial
    library_name = 'msfr_xs'
    library_file = '../../mgxs/msfr_xs_extended.xml'
    grid_names = 'tfuel'
    grid_variables = 'tfuel_constant'
    isotopes = 'pseudo'
    densities = '1.0'
    is_meter = true
    material_id = 1
    block = 'reflector'
  []
[]

################################################################################
# EXECUTION / SOLVE
################################################################################

[Functions]
  [timestep]
    type = PiecewiseConstant
    x = '0.5  10   20    25   1e6'
    y = '0.2  0.1  0.25  0.5  1.0'
    direction = right
  []
[]

[Executioner]
  type = Transient
  scheme = implicit-euler
  start_time = 0.0
  end_time = 200.0
  dt = 0.25
  solve_type = PJFNK
  petsc_options_iname = '-ksp_gmres_restart -pc_type  -pc_use_amat
                         -pc_gamg_sym_graph -pc_gamg_use_parallel_coarse_grid_solver
                         -mg_levels_1_pc_type -mg_coarse_pc_type
                         -mg_coarse_sub_pc_factor_levels -mg_coarse_sub_pc_factor_mat_ordering_type'
  petsc_options_value = ' 100   gamg  false
                          true  1
                          asm   asm
                          1     rcm'

  line_search = 'none'
  nl_abs_tol = 1e-7
  nl_forced_its = 1
  l_abs_tol = 1e-7
  l_max_its = 200

  # Fixed point iteration parameters
  fixed_point_max_its = 3
  accept_on_max_fixed_point_iteration = true
  fixed_point_abs_tol = 1e-50
[]

################################################################################
# SIMULATION OUTPUT
################################################################################

[Outputs]
  csv = true
  exodus = true
  # Reduce base output
  print_linear_converged_reason = false
  print_linear_residuals = false
  print_nonlinear_converged_reason = false
  hide = 'dnp'
[]

[Postprocessors]
  [power_scaling]
    type = Receiver
    outputs = none
    default = 4.1682608293957647e+18
  []
  [power]
    type = ElementIntegralVariablePostprocessor
    variable = power_density
    execute_on = 'initial timestep_begin timestep_end'
    outputs = all
    block = 'fuel pump hx'
  []
[]

################################################################################
# MULTIAPPS and TRANSFERS for flow simulation
################################################################################

[MultiApps]
  [ns]
    type = TransientMultiApp
    input_files = 'run_ns.i'
    execute_on = 'timestep_begin'
    catch_up = true
  []
[]

[Transfers]
  # NOTE: These four transfers could be condensed down to only two transfers
  [power_density]
    type = MultiAppGeometricInterpolationTransfer
    to_multi_app = ns
    source_variable = power_density
    variable = power_density
  []
  [fission_source]
    type = MultiAppGeneralFieldShapeEvaluationTransfer
    to_multi_app = ns
    source_variable = fission_source
    variable = fission_source
  []
  [T_fluid]
    type = MultiAppGeneralFieldShapeEvaluationTransfer
    from_multi_app = ns
    source_variable = 'T_fluid'
    variable = 'T_fluid'
  []
  [precursors]
    type = MultiAppGeneralFieldShapeEvaluationTransfer
    from_multi_app = ns
    source_variable = 'c1 c2 c3 c4 c5 c6'
    variable = 'c1 c2 c3 c4 c5 c6'
  []
[]

Debugging Numerical Instability
Reliability of Transfer
How to Handle Single App Crashes