Optimization Failure Analysis Report

Introduction

The FAR for Optimization describes the system functional and non-functional requirements that describe the expected interactions that the software shall provide.

Dependencies

The Optimization application is developed using MOOSE and is based on various modules, as such the FAR for Optimization is dependent upon the following documents.

Framework Failure Analysis Report

Failure Analysis Requirements

The following is a complete list for all the requirements related to failure analysis for Optimization.

optimization: Controls
11.1.2The system shall, when the fixed-point iteration cannot converge within the maximum number of iterations and the time step cannot be reduced further, cut the time step and error out.
Specification(s): secant_error_on_max_iteration
Design: SecantInversionControl
Issue(s): #33205
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException
11.1.4The system shall demonstrate that the secant-based inversion Control fails to converge within a fixed iteration budget on a nonlinear problem that the finite-difference Newton Control solves.
Specification(s): secant_fails_on_nonlinear
Design: SecantInversionControl NewtonInversionControl
Issue(s): #33205
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException
11.1.12The system shall report a helpful error, naming the convergence the user actually assigned, when the single-parameter inverse-solve block's generated convergence is not assigned to the executioner's multiapp_fixed_point_convergence.
Specification(s): action_wrong_convergence
Design: SingleParameterInverseSolveAction
Issue(s): #33205
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException
11.1.13The system shall report a helpful error, distinguishing MOOSE's own default convergence from a user-assigned one, when the single-parameter inverse-solve block's generated convergence is not assigned to the executioner's multiapp_fixed_point_convergence.
Specification(s): action_missing_convergence
Design: SingleParameterInverseSolveAction
Issue(s): #33205
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException

optimization: Functions
11.4.2
The system shall report an error if
1. the size of value vector is wrong;
2. x-coordinate data is wrong size;
3. y-coordinate data is wrong size;
4. z-coordinate data is wrong size;
5. time data is wrong size;
Specification(s): error/v, error/x, error/y, error/z, error/t
Design: NearestReporterCoordinatesFunction
Issue(s): #22215
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException
11.4.4
The system shall prevent
1. using second order element meshes
2. with closest point projection;
Specification(s): check_dbgs/create_second, check_dbgs/evaluate_second
Design: ParameterMeshFunction
Issue(s): #30796
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): CheckFilesRunException
11.4.6
The system shall support parsed function expressions with parameter values by
1. computing parameter gradient;
2. erroring if vector is incorrect size;
Specification(s): parsed_function/compute_gradient, parsed_function/wrong_vector_size
Design: ParsedOptimizationFunction
Issue(s): #21885
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunExceptionCSVDiff

optimization: Misc
11.5.2
The system shall throw out an error when
1. scalings are manually assigned to adjoint variables to remind users that no scaling will be performed for adjoint system.
Specification(s): optimization_scaling_errors/scaling_factor_for_adjoint_variables
Design: SteadyAndAdjoint
Issue(s): #25646 #29570
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException

optimization: Optimizationreporter
11.6.10
The system shall throw an error when performing inverse optimization on mesh-based parameters if there is a mismatch between the number of parameters and the number of
1. meshes;
2. finite-element families;
3. finite-element orders;
4. initial conditions;
Specification(s): mismatch_errors/meshes, mismatch_errors/family, mismatch_errors/order, mismatch_errors/initial_condition
Design: ParameterMeshOptimization ParameterMeshFunction
Issue(s): #22995
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException
11.6.11
The system shall throw an error when performing inverse optimization on mesh-based parameters if the parameter function spaces do not match the space defined in the
1. forward problem;
2. adjoint problem;
Specification(s): interpolation_errors/forward, interpolation_errors/adjoint
Design: ParameterMeshOptimization ParameterMeshFunction
Issue(s): #22995
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException
Prerequisite(s): 11.6.9
11.6.13
The system shall correctly read in parameter data defined in the input file
1. measurement data from the input file;
2. with lower bounds, upper bounds and initial conditions provided for every parameter and measurement data from a CSV file;
3. with lower bounds, upper bounds and initial conditions constant for each group and measurement data from a CSV file;
4. with lower bounds, upper bounds and initial conditions are a mix of constant per group and every parameter in a group and measurement data from a CSV file;
5. and error if parameters are provided but do not include a parameter for every group;
6. and error if an incorrect number of parameters are provided for a group;
Specification(s): optimizationReporter/input, optimizationReporter/csv_paramBounds, optimizationReporter/csv_groupBounds, optimizationReporter/csv_mixBounds, optimizationReporter/missing_group_params, optimizationReporter/too_many_params
Design: OptimizationReporter
Issue(s): #22212 #23867
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): JSONDiffRunException
11.6.17
The system shall throw an error when parameter data is incorrectly defined
1. by having input and mesh parameters for the same bound;
2. by having a parameter not on the exodus mesh;
3. by not having the same number of optimization timesteps and timesteps for reading parameter data from the mesh;
4. by specifying a timestep for reading parameter data from the exodus file but not parameter data is being read from the mesh because it is all specified in the input file;
5. by specifying a timestep that is bigger than the timestpes on the exodus the exodus mesh.
Specification(s): errors/errorMixedParams, errors/errorMissingMeshParam, errors/errorMismatchSteps, errors/errorNoSteps, errors/errorBadStep
Design: ParameterMeshOptimization
Issue(s): #24034
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException
11.6.21
The system shall throw an error when regularization parameters are incorrectly specified
1. when the number of regularization coefficients doesn't match the number of regularization types;
2. when an invalid regularization type is specified;
3. when regularization types are specified but no coefficients are provided;
Specification(s): regularization_errors/mismatched_coeffs_types, regularization_errors/invalid_regularization_type, regularization_errors/empty_coeffs_with_types
Design: ParameterMeshOptimization
Issue(s): #30897
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunException

optimization: Userobjects
11.11.1
The system shall be able to load a time-dependent solution from an exodus file while reversing the timesteps by
1. producing the exodus file;
2. loading on to the same mesh;
3. loading on to a coarser mesh;
4. interpolating different time steps;
5. erroring if file is not exodus;
Specification(s): adjoint_solution/forward, adjoint_solution/reverse, adjoint_solution/reverse_space_interp, adjoint_solution/reverse_time_interp, adjoint_solution/error_file_format
Design: AdjointSolutionUserObject
Issue(s): #22215
Collection(s): FUNCTIONALFAILURE_ANALYSIS
Type(s): RunAppCheckFilesRunException


[secant_error_on_max_iteration]
  type = RunException
  input = secant.i
  # A distinct file_base keeps this reuse of secant.i from sharing the secant test's output and
  # recover checkpoint.
  cli_args = 'Convergence/inv_conv/max_iterations=1 Executioner/dtmin=0.3 Outputs/csv=false Outputs/file_base=secant_error_on_max_out'
  expect_err = 'Solve failed and timestep already at or below dtmin'
  design = 'SecantInversionControl.md'
  requirement = "The system shall, when the fixed-point iteration cannot converge within the maximum number of iterations and the time step cannot be reduced further, cut the time step and error out."
[]


[secant_fails_on_nonlinear]
  type = RunException
  input = secant.i
  # The same nonlinear problem the newton_nonlinear test solves: y(p) = p^3 against a target of
  # 1000 from p0 = -3. Secant estimates df/dp from two iterates straddling the inflection at
  # p = 0 and enters a bounded chaotic orbit, so with dtmin = dt the step cannot be cut and the
  # run errors once max_iterations is exhausted.
  cli_args = 'MultiApps/sub/input_files=forward_cubic.i Functions/target_fn/expression=1000 Postprocessors/p/default=-3 Convergence/inv_conv/max_iterations=25 Executioner/num_steps=1 Executioner/dtmin=1 Outputs/csv=false Outputs/file_base=secant_fails_on_nonlinear_out'
  expect_err = 'Solve failed and timestep already at or below dtmin'
  design = 'SecantInversionControl.md NewtonInversionControl.md'
  requirement = "The system shall demonstrate that the secant-based inversion Control fails to converge within a fixed iteration budget on a nonlinear problem that the finite-difference Newton Control solves."
[]


[action_wrong_convergence]
  type = RunException
  input = action.i
  # Points the executioner at a convergence the action did not generate. Distinct from
  # action_missing_convergence below: here the user explicitly named a (wrong) convergence,
  # rather than omitting the line, so the error must report the name they actually wrote.
  cli_args = 'Executioner/multiapp_fixed_point_convergence=not_the_generated_convergence Outputs/csv=false Outputs/file_base=action_wrong_convergence_out'
  expect_err = "currently set to 'not_the_generated_convergence'"
  design = 'SingleParameterInverseSolveAction.md'
  requirement = "The system shall report a helpful error, naming the convergence the user actually assigned, when the single-parameter inverse-solve block's generated convergence is not assigned to the executioner's multiapp_fixed_point_convergence."
[]


[action_missing_convergence]
  type = RunException
  input = action_no_convergence.i
  # Omits the executioner line entirely, rather than assigning a wrong name (see
  # action_wrong_convergence above): MOOSE then silently creates and assigns a default
  # convergence, so the error must say the line was never given rather than naming that default
  # as if the user had written it.
  cli_args = 'Outputs/csv=false Outputs/file_base=action_missing_convergence_out'
  expect_err = "No 'multiapp_fixed_point_convergence' was given, so MOOSE created a default one"
  design = 'SingleParameterInverseSolveAction.md'
  requirement = "The system shall report a helpful error, distinguishing MOOSE's own default convergence from a user-assigned one, when the single-parameter inverse-solve block's generated convergence is not assigned to the executioner's multiapp_fixed_point_convergence."
[]


[v]
  type = RunException
  input = nearest_reporter_point.i
  cli_args = 'Functions/active=errorv AuxKernels/val_aux/function=errorv'
  expect_err = 'Size of value vector \(\d*\) does not match number of coordinates specified \(\d*\).'
  detail = 'the size of value vector is wrong;'
[]


[x]
  type = RunException
  input = nearest_reporter_point.i
  cli_args = 'Functions/active=errorx AuxKernels/val_aux/function=errorx'
  expect_err = 'Number of x coordinates \(\d*\) does not match number of values \(\d*\).'
  detail = 'x-coordinate data is wrong size;'
[]


[y]
  type = RunException
  input = nearest_reporter_point.i
  cli_args = 'Functions/active=errory AuxKernels/val_aux/function=errory'
  expect_err = 'Number of y coordinates \(\d*\) does not match number of values \(\d*\).'
  detail = 'y-coordinate data is wrong size;'
[]


[z]
  type = RunException
  input = nearest_reporter_point.i
  cli_args = 'Functions/active=errorz AuxKernels/val_aux/function=errorz'
  expect_err = 'Number of z coordinates \(\d*\) does not match number of values \(\d*\).'
  detail = 'z-coordinate data is wrong size;'
[]


[t]
  type = RunException
  input = nearest_reporter_point.i
  cli_args = 'Functions/active=errort AuxKernels/val_aux/function=errort'
  expect_err = 'Number of times \(\d*\) does not match number of values \(\d*\).'
  detail = 'time data is wrong size;'
[]


[create_second]
  type = CheckFiles
  input = create_mesh_2d.i
  cli_args = 'Mesh/gmg/elem_type=QUAD8'
  check_files = 'create_mesh_2d_out.e create_mesh_2d_out_param_vec_0001.csv'
  detail = 'using second order element meshes'
  recover = false
[]


[evaluate_second]
  type = RunException
  input = parameter_mesh_2d.i
  prereq = check_dbgs/create_second
  expect_err = "Closet point projection currently does not support second order elements."
  detail = 'with closest point projection;'
  recover = false
[]


[wrong_vector_size]
  type = RunException
  input = parsed_function.i
  allow_test_objects = true
  cli_args = "Reporters/params/real_vector_values='1 2 3'"
  expect_err = "Size of vector \(\d+\) does not match number of specified 'param_names' \(\d+\)\."
  detail = 'erroring if vector is incorrect size;'
[]


[scaling_factor_for_adjoint_variables]
  type = 'RunException'
  input = main.i
  max_threads = 1 # Optimize executioner does not support multiple threads
  cli_args = "MultiApps/forward/cli_args='Variables/T_real_adj/scaling=5'"
  expect_err = 'User cannot supply scaling factors for adjoint variables.   Adjoint system is scaled automatically by the forward system.'
  detail = "scalings are manually assigned to adjoint variables to remind users that no scaling will be performed for adjoint system."
[]


[meshes]
  type = RunException
  input = main.i
  cli_args = 'OptimizationReporter/parameter_meshes=\'parameter_mesh_in.e parameter_mesh_in.e\''
  max_threads = 1 # Optimize executioner does not support multiple threads
  expect_err = 'There must be a mesh associated with each group of parameters\.'
  detail = 'meshes;'
[]


[family]
  type = RunException
  input = main.i
  cli_args = 'OptimizationReporter/parameter_families=\'LAGRANGE LAGRANGE\''
  max_threads = 1 # Optimize executioner does not support multiple threads
  expect_err = 'There must be a family associated with each group of parameters\.'
  detail = 'finite-element families;'
[]


[order]
  type = RunException
  input = main.i
  cli_args = 'OptimizationReporter/parameter_orders=\'FIRST FIRST\''
  max_threads = 1 # Optimize executioner does not support multiple threads
  expect_err = 'There must be an order associated with each group of parameters\.'
  detail = 'finite-element orders;'
[]


[initial_condition]
  type = RunException
  input = main.i
  cli_args = 'OptimizationReporter/initial_condition=\'0; 0\''
  max_threads = 1 # Optimize executioner does not support multiple threads
  expect_err = 'There must be a vector of initial_condition per parameter group\.'
  detail = 'initial conditions;'
[]


[forward]
  type = RunException
  input = main.i
  cli_args = 'forward:Functions/src_func/order=SECOND'
  expect_err = 'Size of parameter vector \(\d+\) does not match number of degrees of freedom in mesh \(\d+\)\.'
  max_threads = 1 # Optimize executioner does not support multiple threads
  prereq = 'mesh_source/mesh'
  detail = 'forward problem;'
[]


[adjoint]
  type = RunException
  input = main.i
  cli_args = 'adjoint:Functions/src_func/order=SECOND'
  expect_err = 'The gradient for parameter \w+ has changed, expected \d+ versus \d+\.'
  max_threads = 1 # Optimize executioner does not support multiple threads
  prereq = 'mesh_source/mesh'
  detail = 'adjoint problem;'
[]


[csv_paramBounds]
  type = JSONDiff
  input = optRep_fromCsv_paramBounds.i
  jsondiff = optRep_fromCsv_paramBounds_out.json
  allow_test_objects = true
  detail = "with lower bounds, upper bounds and initial conditions provided for every parameter and measurement data from a CSV file; "
[]


[csv_groupBounds]
  type = JSONDiff
  input = optRep_fromCsv_groupBounds.i
  jsondiff = optRep_fromCsv_groupBounds_out.json
  allow_test_objects = true
  detail = "with lower bounds, upper bounds and initial conditions constant for each group and measurement data from a CSV file; "
[]


[csv_mixBounds]
  type = JSONDiff
  input = optRep_fromCsv_mixBounds.i
  jsondiff = optRep_fromCsv_mixBounds_out.json
  allow_test_objects = true
  detail = "with lower bounds, upper bounds and initial conditions are a mix of constant per group and every parameter in a group and measurement data from a CSV file; "
[]


[missing_group_params]
  type = RunException
  input = optRep_fromCsv_mixBounds.i
  allow_test_objects = true
  cli_args = 'OptimizationReporter/upper_bounds=\'0\''
  max_threads = 1 # Optimize executioner does not support multiple threads
  expect_err = 'There must be a vector of upper_bounds per parameter group\.  The upper_bounds input format is std::vector<std::vector<Real>> so each vector should be seperated by \";\" even if it is a single value per group for a constant upper_bounds\.'
  detail = ' and error if parameters are provided but do not include a parameter for every group; '
[]


[too_many_params]
  type = RunException
  input = optRep_fromCsv_mixBounds.i
  allow_test_objects = true
  cli_args = 'OptimizationReporter/upper_bounds=\'0;0;0 0\''
  max_threads = 1 # Optimize executioner does not support multiple threads
  expect_err = 'When upper_bounds are given in input file, there must either be a single value per parameter group or a value for every parameter in the group\.'
  detail = ' and error if an incorrect number of parameters are provided for a group; '
[]


[errorMixedParams]
  type = RunException
  input = paramMeshOptRep.i
  cli_args = 'OptimizationReporter/initial_condition=6
                  OptimizationReporter/lower_bound_mesh_variable=lowerBounds
                  OptimizationReporter/lower_bounds=5
                  OptimizationReporter/upper_bound_mesh_variable=upperBounds
                  OptimizationReporter/num_parameter_times=1'
  allow_test_objects = true
  expect_err = 'Lower bounds for all parameter groups can only be defined by lower_bound_mesh_variable or lower_bounds but not both.'
  detail = "by having input and mesh parameters for the same bound; "
[]


[errorMissingMeshParam]
  type = RunException
  input = paramMeshOptRep.i
  cli_args = 'OptimizationReporter/lower_bound_mesh_variable=notOnMesh'
  allow_test_objects = true
  expect_err = 'Exodus file did not contain the parameter name being intitialized.'
  detail = "by having a parameter not on the exodus mesh; "
[]


[errorMismatchSteps]
  type = RunException
  input = paramMeshOptRep.i
  cli_args = 'OptimizationReporter/lower_bound_mesh_variable=lowerBounds
                  OptimizationReporter/num_parameter_times=2
                  OptimizationReporter/exodus_timesteps_for_parameter_mesh_variable=\'2 3 4\''
  allow_test_objects = true
  expect_err = 'Number of timesteps to read mesh data specified by \"exodus_timesteps_for_parameter_mesh_variable\" incorrect.'
  detail = "by not having the same number of optimization timesteps and timesteps for reading parameter data from the mesh; "
[]


[errorNoSteps]
  type = RunException
  input = paramMeshOptRep.i
  cli_args = 'OptimizationReporter/lower_bounds=5
                  OptimizationReporter/num_parameter_times=3
                  OptimizationReporter/exodus_timesteps_for_parameter_mesh_variable=\'2 3 4\''
  allow_test_objects = true
  expect_err = '\"exodus_timesteps_for_parameter_mesh_variable\" should only be specified if reading values from a mesh.'
  detail = "by specifying a timestep for reading parameter data from the exodus file but not parameter data is being read from the mesh because it is all specified in the input file; "
[]


[errorBadStep]
  type = RunException
  input = paramMeshOptRep.i
  cli_args = 'OptimizationReporter/lower_bound_mesh_variable=lowerBounds
      OptimizationReporter/upper_bound_mesh_variable=upperBounds
      UserObjects/optReporterTester/expected_lower_bounds=\'5 5 5 5 5 5\'
      UserObjects/optReporterTester/expected_upper_bounds=\'2 3.5 3.5 2 5 5\'
                  OptimizationReporter/num_parameter_times=3
                  OptimizationReporter/exodus_timesteps_for_parameter_mesh_variable=\'2 3 8\''
  allow_test_objects = true
  expect_err = 'Invalid value passed as "time_step". Expected a valid integer less than 4, received 8'
  detail = "by specifying a timestep that is bigger than the timestpes on the exodus the exodus mesh."
[]


[mismatched_coeffs_types]
  type = 'RunException'
  input = 'main.i optimizationReporterInputIC.i'
  cli_args = 'OptimizationReporter/regularization_types="L2_GRADIENT L2_GRADIENT"
                  OptimizationReporter/regularization_coeffs="0.1"'
  # Optimize executioner does not support multiple threads
  max_threads = 1
  expect_err = 'Number of regularization coefficients \(1\) must match number of regularization types \(2\)'
  detail = "when the number of regularization coefficients doesn't match the number of regularization types;"
[]


[invalid_regularization_type]
  type = 'RunException'
  input = 'main.i optimizationReporterInputIC.i'
  cli_args = 'OptimizationReporter/regularization_types="INVALID_TYPE"
                  OptimizationReporter/regularization_coeffs="0.1"'
  # Optimize executioner does not support multiple threads
  max_threads = 1
  expect_err = 'Invalid option "INVALID_TYPE" in MultiMooseEnum'
  detail = "when an invalid regularization type is specified;"
[]


[empty_coeffs_with_types]
  type = 'RunException'
  input = 'main.i optimizationReporterInputIC.i'
  cli_args = 'OptimizationReporter/regularization_types="L2_GRADIENT"
                  OptimizationReporter/regularization_coeffs=""'
  # Optimize executioner does not support multiple threads
  max_threads = 1
  expect_err = 'Number of regularization coefficients \(0\) must match number of regularization types \(1\)'
  detail = "when regularization types are specified but no coefficients are provided;"
[]


[reverse_time_interp]
  type = RunApp
  input = adjoint.i
  cli_args = 'Executioner/dt=1.25 Functions/u_reverse_fun/expression="(x + y) * (11.25 - t)"'
  prereq = adjoint_solution/forward
  detail = 'interpolating different time steps;'
[]


[error_file_format]
  type = RunException
  input = adjoint.i
  cli_args = 'UserObjects/u_reverse_solution/mesh=forward_out.xda'
  expect_err = 'Performing transient adjoint simulation currently only works if the forward solution is written and read from exodus format.'
  detail = 'erroring if file is not exodus;'
[]

Introduction
Dependencies
Failure Analysis Requirements