Evaluating a Surrogate Model

This example demonstrates how to evaluate a trained surrogate model. NearestPointSurrogate is used as the example surrogate model. See Creating a surrogate model for details on how this surrogate is created. See Training a surrogate model for details on how to train a surrogate model.

Overview

In general, a SurrogateModel object takes in training data in the form of .rd files or the name of the training object itself. The former performs training and evaluating in two separate steps, the latter performs them both in a single step. Since most practical applications perform training once and evaluation in multiple instances, this example focuses on the two step method. Every SurrogateModel object has a public member function, evaluate, that gives the surrogate's estimate to a quantity of interest with the parameters as input. Some specialized surrogates, like PolynomialChaos, have more functions for computing statistics. However, this example will focus on the simple NearestPointSurrogate, which only uses the evaluate function.

Model Problem

This example uses a one-dimensional heat conduction problem as the full-order model which has certain uncertain parameters. The model equation is as follows:

The quantities of interest are the average and maximum temperature:

Parameter Uncertainty

For demonstration, each of these parameters will have two types of probability distributions: Uniform () and Normal (). Where and are the max and minimum bounds of the uniform distribution, respectively. And and are the mean and standard deviation of the normal distribution, respectively.

The uncertain parameters for this model problem are:

ParameterSymbolUniformNormal
Conductivity
Volumetric Heat Source
Domain Size
Right Boundary Temperature

Analytical Solutions

This simple model problem has analytical descriptions for the field temperature, average temperature, and maximum temperature:

With the quadratic feature of the field temperature, using quadratic elements in the discretization will actually yield the exact solution.

Input File

Below is the input file used to solve the one-dimensional heat conduction model.

[Mesh<<<{"href": "../../../syntax/Mesh/index.html"}>>>]
  type = GeneratedMesh
  dim = 1
  nx = 100
  xmax = 1
  elem_type = EDGE3
[]

[Variables<<<{"href": "../../../syntax/Variables/index.html"}>>>]
  [T]
    order<<<{"description": "Specifies the order of the FE shape function to use for this variable (additional orders not listed are allowed)"}>>> = SECOND
    family<<<{"description": "Specifies the family of FE shape functions to use for this variable"}>>> = LAGRANGE
  []
[]

[Kernels<<<{"href": "../../../syntax/Kernels/index.html"}>>>]
  [diffusion]
    type = MatDiffusion<<<{"description": "Diffusion equation Kernel that takes an isotropic Diffusivity from a material property", "href": "../../../source/kernels/MatDiffusion.html"}>>>
    variable<<<{"description": "The name of the variable that this residual object operates on"}>>> = T
    diffusivity<<<{"description": "The diffusivity value or material property"}>>> = k
  []
  [source]
    type = BodyForce<<<{"description": "Demonstrates the multiple ways that scalar values can be introduced into kernels, e.g. (controllable) constants, functions, and postprocessors. Implements the weak form $(\\psi_i, -f)$.", "href": "../../../source/kernels/BodyForce.html"}>>>
    variable<<<{"description": "The name of the variable that this residual object operates on"}>>> = T
    value<<<{"description": "Coefficient to multiply by the body force term"}>>> = 1.0
  []
[]

[Materials<<<{"href": "../../../syntax/Materials/index.html"}>>>]
  [conductivity]
    type = GenericConstantMaterial<<<{"description": "Declares material properties based on names and values prescribed by input parameters.", "href": "../../../source/materials/GenericConstantMaterial.html"}>>>
    prop_names<<<{"description": "The names of the properties this material will have"}>>> = k
    prop_values<<<{"description": "The values associated with the named properties"}>>> = 2.0
  []
[]

[BCs<<<{"href": "../../../syntax/BCs/index.html"}>>>]
  [right]
    type = DirichletBC<<<{"description": "Imposes the essential boundary condition $u=g$, where $g$ is a constant, controllable value.", "href": "../../../source/bcs/DirichletBC.html"}>>>
    variable<<<{"description": "The name of the variable that this residual object operates on"}>>> = T
    boundary<<<{"description": "The list of boundary IDs from the mesh where this object applies"}>>> = right
    value<<<{"description": "Value of the BC"}>>> = 300
  []
[]

[Executioner<<<{"href": "../../../syntax/Executioner/index.html"}>>>]
  type = Steady
  solve_type = PJFNK
  petsc_options_iname = '-pc_type -pc_hypre_type'
  petsc_options_value = 'hypre boomeramg'
[]

[Postprocessors<<<{"href": "../../../syntax/Postprocessors/index.html"}>>>]
  [avg]
    type = AverageNodalVariableValue<<<{"description": "Computes the average value of a field by sampling all nodal solutions on the domain or within a subdomain", "href": "../../../source/postprocessors/AverageNodalVariableValue.html"}>>>
    variable<<<{"description": "The name of the variable that this postprocessor operates on"}>>> = T
  []
  [max]
    type = NodalExtremeValue<<<{"description": "Finds either the min or max elemental value of a variable over the domain.", "href": "../../../source/postprocessors/NodalExtremeValue.html"}>>>
    variable<<<{"description": "The name of the variable that this postprocessor operates on"}>>> = T
    value_type<<<{"description": "Type of extreme value to return. 'max' returns the maximum value. 'min' returns the minimum value. 'max_abs' returns the maximum of the absolute value."}>>> = max
  []
[]

[Outputs<<<{"href": "../../../syntax/Outputs/index.html"}>>>]
[]
(modules/stochastic_tools/examples/surrogates/sub.i)

With this input the uncertain parameters are defined as:

  1. Materials/conductivity/prop_values

  2. Kernels/source/value

  3. Mesh/xmax

  4. BCs/right/value

These values in the sub.i file are arbitrary since the stochastic master app will be modifying them.

Evaluation

This section shows how to set up an input file to load and evaluate a surrogate model. The training data was created with the steps from Training a surrogate model. To demonstrate the usefulness of a surrogate model, the model will be evaluated using Monte Carlo sampling with parameter distributions described in the previous section. The results of this sampling will be used to compute statistical moments and produce probability distributions.

Omitting Solve

Any input file in MOOSE needs to include a Mesh, Variables, and Executioner block. However, the stochastic master app does not actually create or solve a system. So the StochasticToolsAction builds a minimal model to satisfy these requirements:

[StochasticTools<<<{"href": "../../../syntax/StochasticTools/index.html"}>>>]
[]
(modules/stochastic_tools/examples/surrogates/nearest_point_uniform.i)

Surrogate Model

The surrogate model is loaded by inputting the training data file with the "filename" parameter. In this example, two surrogates are loaded with two different training data files for average temperature and maximum temperature.

[Surrogates<<<{"href": "../../../syntax/Surrogates/index.html"}>>>]
  [nearest_point_avg]
    type = NearestPointSurrogate<<<{"description": "Surrogate that evaluates the value from the nearest point from data in [NearestPointTrainer.md]", "href": "../../../source/surrogates/NearestPointSurrogate.html"}>>>
    filename<<<{"description": "The name of the file which will be associated with the saved/loaded data."}>>> = 'nearest_point_training_out_nearest_point_avg.rd'
  []
  [nearest_point_max]
    type = NearestPointSurrogate<<<{"description": "Surrogate that evaluates the value from the nearest point from data in [NearestPointTrainer.md]", "href": "../../../source/surrogates/NearestPointSurrogate.html"}>>>
    filename<<<{"description": "The name of the file which will be associated with the saved/loaded data."}>>> = 'nearest_point_training_out_nearest_point_max.rd'
  []
[]
(modules/stochastic_tools/examples/surrogates/nearest_point_uniform.i)

Defining a Sampler

In this example, the surrogate is evaluated at points given by a sampler. Here we use the MonteCarloSampler to generate random points defined by a Uniform or a Normal distribution for each parameter. See Example 1: Monte Carlo for more details on setting up this sampler.

Uniform distribution for each parameter

[Distributions<<<{"href": "../../../syntax/Distributions/index.html"}>>>]
  [k_dist]
    type = Uniform<<<{"description": "Continuous uniform distribution.", "href": "../../../source/distributions/Uniform.html"}>>>
    lower_bound<<<{"description": "Distribution lower bound"}>>> = 1
    upper_bound<<<{"description": "Distribution upper bound"}>>> = 10
  []
  [q_dist]
    type = Uniform<<<{"description": "Continuous uniform distribution.", "href": "../../../source/distributions/Uniform.html"}>>>
    lower_bound<<<{"description": "Distribution lower bound"}>>> = 9000
    upper_bound<<<{"description": "Distribution upper bound"}>>> = 11000
  []
  [L_dist]
    type = Uniform<<<{"description": "Continuous uniform distribution.", "href": "../../../source/distributions/Uniform.html"}>>>
    lower_bound<<<{"description": "Distribution lower bound"}>>> = 0.01
    upper_bound<<<{"description": "Distribution upper bound"}>>> = 0.05
  []
  [Tinf_dist]
    type = Uniform<<<{"description": "Continuous uniform distribution.", "href": "../../../source/distributions/Uniform.html"}>>>
    lower_bound<<<{"description": "Distribution lower bound"}>>> = 290
    upper_bound<<<{"description": "Distribution upper bound"}>>> = 310
  []
[]
(modules/stochastic_tools/examples/surrogates/nearest_point_uniform.i)

Normal distribution for each parameter

[Distributions<<<{"href": "../../../syntax/Distributions/index.html"}>>>]
  [k_dist]
    type = Normal<<<{"description": "Normal distribution", "href": "../../../source/distributions/Normal.html"}>>>
    mean<<<{"description": "Mean (or expectation) of the distribution."}>>> = 5
    standard_deviation<<<{"description": "Standard deviation of the distribution "}>>> = 2
  []
  [q_dist]
    type = Normal<<<{"description": "Normal distribution", "href": "../../../source/distributions/Normal.html"}>>>
    mean<<<{"description": "Mean (or expectation) of the distribution."}>>> = 10000
    standard_deviation<<<{"description": "Standard deviation of the distribution "}>>> = 500
  []
  [L_dist]
    type = Normal<<<{"description": "Normal distribution", "href": "../../../source/distributions/Normal.html"}>>>
    mean<<<{"description": "Mean (or expectation) of the distribution."}>>> = 0.03
    standard_deviation<<<{"description": "Standard deviation of the distribution "}>>> = 0.01
  []
  [Tinf_dist]
    type = Normal<<<{"description": "Normal distribution", "href": "../../../source/distributions/Normal.html"}>>>
    mean<<<{"description": "Mean (or expectation) of the distribution."}>>> = 300
    standard_deviation<<<{"description": "Standard deviation of the distribution "}>>> = 10
  []
[]
(modules/stochastic_tools/examples/surrogates/nearest_point_normal.i)

Monte Carlo sampler

[Samplers<<<{"href": "../../../syntax/Samplers/index.html"}>>>]
  [sample]
    type = MonteCarlo<<<{"description": "Monte Carlo Sampler.", "href": "../../../source/samplers/MonteCarloSampler.html"}>>>
    num_rows<<<{"description": "The number of rows per matrix to generate."}>>> = 100000
    distributions<<<{"description": "The distribution names to be sampled, the number of distributions provided defines the number of columns per matrix."}>>> = 'k_dist q_dist L_dist Tinf_dist'
    execute_on<<<{"description": "The list of flag(s) indicating when this object should be executed. For a description of each flag, see https://mooseframework.inl.gov/source/interfaces/SetupInterface.html."}>>> = initial
  []
[]
(modules/stochastic_tools/examples/surrogates/nearest_point_uniform.i)

Sampling Surrogate

Evaluating a surrogate model occurs within objects that obtain the surrogate object's reference and call the evaluate function. In this example, we will use EvaluateSurrogate to evaluate the surrogate. EvaluateSurrogate takes in a sampler and the surrogate model as inputs, and evaluates the surrogate at the points given by the sampler.

[Reporters<<<{"href": "../../../syntax/Reporters/index.html"}>>>]
  [samp]
    type = EvaluateSurrogate<<<{"description": "Tool for sampling surrogate models.", "href": "../../../source/reporters/EvaluateSurrogate.html"}>>>
    model<<<{"description": "Name of surrogate models."}>>> = 'nearest_point_avg nearest_point_max'
    sampler<<<{"description": "Sampler to use for evaluating surrogate models."}>>> = sample
    parallel_type<<<{"description": "This parameter will determine how the stochastic data is gathered. It is common for outputting purposes that this parameter be set to ROOT, otherwise, many files will be produced showing the values on each processor. However, if there are lot of samples, gathering on root may be memory restrictive."}>>> = ROOT
  []
[]
(modules/stochastic_tools/examples/surrogates/nearest_point_uniform.i)

The results of evaluating the surrogate can then be used to compute statistics like mean and standard deviation:

[Reporters<<<{"href": "../../../syntax/Reporters/index.html"}>>>]
  [stats]
    type = StatisticsReporter<<<{"description": "Compute statistical values of a given VectorPostprocessor objects and vectors.", "href": "../../../source/reporters/StatisticsReporter.html"}>>>
    reporters<<<{"description": "List of Reporter values to utilized for statistic computations."}>>> = 'samp/nearest_point_avg samp/nearest_point_max'
    compute<<<{"description": "The statistic(s) to compute for each of the supplied vector postprocessors."}>>> = 'mean stddev'
  []
[]
(modules/stochastic_tools/examples/surrogates/nearest_point_uniform.i)

Results

The results of the of inputs from the previous sections produce csv files of the evaluation data. These files can then be used to produce probability distributions like in Figure 1 and Figure 2. Even for this simple model problem, evaluating the surrogate was orders of magnitude faster with significantly less memory consumption.

Figure 1: Temperature distributions with uniform parameter distribution

Figure 2: Temperature distributions with normal parameter distribution