- covariance_functionName of covariance function.
C++ Type:UserObjectName
Controllable:No
Description:Name of covariance function.
- responseReporter value of response results, can be vpp with
/ . C++ Type:ReporterName
Controllable:No
Description:Reporter value of response results, can be vpp with
/ . - samplerSampler used to create predictor and response data.
C++ Type:SamplerName
Controllable:No
Description:Sampler used to create predictor and response data.
GaussianProcessTrainer
"Gaussian Processes for Machine Learning" Rasmussen and Williams (2005) provides a well written discussion of Gaussian Processes, and its reading is highly encouraged. Chapters 1-5 cover the topics presented here with far greater detail, depth, and rigor.
The documentation here is meant to give some practical insight for users to begin creating surrogate models with Gaussian Processes.
Given a set of inputs for which we have made observations of the correspond outputs using the system (). Given another set of inputs we wish to predict the associated outputs without evaluation of , which is presumed costly.
Parameter Covariance
In overly simplistic terms, Gaussian Process Modeling is driven by the idea that trials which are "close" in their input parameter space will be "close" in their output space. Closeness in the parameter space is driven by the covariance function (also called a kernel function, not to be confused with a MOOSE Framework kernel). This covariance function is used to generate a covariance matrix between the complete set of parameters , which can then be interpreted block-wise as various covariance matrices between and .
The Gaussian Process Model consists of an infinite collection of functions, all of which agree with the training/observation data. Importantly the collection has closed forms for 2nd order statistics (mean and variance). When used as a surrogate, the nominal value is chosen to be the mean value. The method can be broken down into two step: definition of the prior distribution then conditioning on observed data.
Gaussian processes
A Gaussian Process is a (potentially infinite) collection of random variables, such that the joint distribution of every finite selection of random variables from the collection is a Gaussian distribution.
In an analogous way that a multivariate Gaussian is completely defined by its mean vector and its covariance matrix, a Gaussian Process is completely defined by its mean function and covariance function.
The (potentially) infinite number of random variables within the Gaussian Process correspond to the (potentially) infinite points in the parameter space our surrogate can be evaluated at.
Prior distribution:
We assume the observations (both training and testing) are pulled from an multivariate Gaussian distribution. The covariance matrix is the result of the choice of covariance function.
Note that and are a vector and matrix respectively, and are a result of the mean and covariance functions applied to the sample points.
Zero Mean Assumption: Discussions of Gaussian Process are typically presented under assumption that . This occurs without loss of generality since any sample can be made by subtracting the sample mean (or a variety of other preprocessing options). Note that in a training\testing paradigm, the testing data is unknown, so determination of what to use as is based on the information from the training data (or some other prior assumption).
Conditioning:
With the prior formed as above, conditioning on the available training data is performed. This alters the mean and variance to new values and , restricting the set of possible functions which agree with the training data.
When used as a surrogate, the nominal value is typically taken as the mean value, with providing variances which can be used to generate confidence intervals.
Common Hyperparameters
While the only apparent decision in the above formulation is the choice of covariance function, most covariance functions will contain hyperparameters of some form which need to be selected in some manner. While each covariance function will have its own set of hyperparameters, a few hyperparameters of specific forms are present in many common covariance functions.
Length Factor or
Frequently Kernels consider the distance between two input parameters and . For system of only a single parameter this distance often takes the form of
In this form the factor set a relevant length scale for the distance measurements.
When multiple input parameters are to be considered, it may be advantageous to specify different length scales for each of the parameters, resulting in a vector . For example distance may be calculated as
When used with standardized parameters, can be interpreted in units of standard deviation for the relevant parameter.
Signal Variance
This serves as an overall scaling parameter. Given a covariance function (which is not a function of ), the multiplication of yields a new valid covariance function.
This multiplication can also be pulled out of the covariance matrix formation, and simply multiply the matrix formed by
Noise Variance
The represents noise in the collected data, and is as a additional factor on the variance terms (when ).
In the matrix representation this adds a factor of to diagonal of the noiseless matrix
Due to the addition of along the diagonal of the matrix, this hyperparameter can aid in the the inversion of the covariance matrix. For this reason adding a small amount of may be preferable, even when you believe the data to be noise free.
Selected Covariance Functions
Table 1: Selected Covariance Functions
| Covariance Function | Description |
|---|---|
| SquaredExponentialCovariance | Also referred to as a radial basis function (RBF) this is a widely used, general purpose covariance function. Serves as a common starting point for many. |
| ExponentialCovariance | A simple exponential covariance function. |
| MaternHalfIntCovariance | Implementation of the Matern class of covariance function, where the parameter takes on half-integer values. |
Input Parameters
- converged_reporterReporter value used to determine if a sample's multiapp solve converged.
C++ Type:ReporterName
Controllable:No
Description:Reporter value used to determine if a sample's multiapp solve converged.
- execute_onTIMESTEP_ENDThe list of flag(s) indicating when this object should be executed, the available options include NONE, INITIAL, LINEAR, NONLINEAR, TIMESTEP_END, TIMESTEP_BEGIN, FINAL, CUSTOM, ALWAYS.
Default:TIMESTEP_END
C++ Type:ExecFlagEnum
Controllable:No
Description:The list of flag(s) indicating when this object should be executed, the available options include NONE, INITIAL, LINEAR, NONLINEAR, TIMESTEP_END, TIMESTEP_BEGIN, FINAL, CUSTOM, ALWAYS.
- predictor_colsSampler columns used as the independent random variables, If 'predictors' and 'predictor_cols' are both empty, all sampler columns are used.
C++ Type:std::vector<unsigned int>
Controllable:No
Description:Sampler columns used as the independent random variables, If 'predictors' and 'predictor_cols' are both empty, all sampler columns are used.
- predictorsReporter values used as the independent random variables, If 'predictors' and 'predictor_cols' are both empty, all sampler columns are used.
C++ Type:std::vector<ReporterName>
Controllable:No
Description:Reporter values used as the independent random variables, If 'predictors' and 'predictor_cols' are both empty, all sampler columns are used.
- prop_getter_suffixAn optional suffix parameter that can be appended to any attempt to retrieve/get material properties. The suffix will be prepended with a '_' character.
C++ Type:MaterialPropertyName
Controllable:No
Description:An optional suffix parameter that can be appended to any attempt to retrieve/get material properties. The suffix will be prepended with a '_' character.
- show_taoFalseSwitch to show TAO solver results
Default:False
C++ Type:bool
Controllable:No
Description:Switch to show TAO solver results
- skip_unconverged_samplesFalseTrue to skip samples where the multiapp did not converge, 'stochastic_reporter' is required to do this.
Default:False
C++ Type:bool
Controllable:No
Description:True to skip samples where the multiapp did not converge, 'stochastic_reporter' is required to do this.
- standardize_dataTrueStandardize (center and scale) training data (y values)
Default:True
C++ Type:bool
Controllable:No
Description:Standardize (center and scale) training data (y values)
- standardize_paramsTrueStandardize (center and scale) training parameters (x values)
Default:True
C++ Type:bool
Controllable:No
Description:Standardize (center and scale) training parameters (x values)
- tao_optionsCommand line options for PETSc/TAO hyperparameter optimization
C++ Type:std::string
Controllable:No
Description:Command line options for PETSc/TAO hyperparameter optimization
- tune_parametersSelect hyperparameters to be tuned
C++ Type:std::vector<std::string>
Controllable:No
Description:Select hyperparameters to be tuned
- tuning_algorithmnoneHyper parameter optimizaton algorithm
Default:none
C++ Type:MooseEnum
Controllable:No
Description:Hyper parameter optimizaton algorithm
- tuning_maxMaximum allowable tuning value
C++ Type:std::vector<double>
Controllable:No
Description:Maximum allowable tuning value
- tuning_minMinimum allowable tuning value
C++ Type:std::vector<double>
Controllable:No
Description:Minimum allowable tuning value
Optional Parameters
- allow_duplicate_execution_on_initialFalseIn the case where this UserObject is depended upon by an initial condition, allow it to be executed twice during the initial setup (once before the IC and again after mesh adaptivity (if applicable).
Default:False
C++ Type:bool
Controllable:No
Description:In the case where this UserObject is depended upon by an initial condition, allow it to be executed twice during the initial setup (once before the IC and again after mesh adaptivity (if applicable).
- control_tagsAdds user-defined labels for accessing object parameters via control logic.
C++ Type:std::vector<std::string>
Controllable:No
Description:Adds user-defined labels for accessing object parameters via control logic.
- enableTrueSet the enabled status of the MooseObject.
Default:True
C++ Type:bool
Controllable:Yes
Description:Set the enabled status of the MooseObject.
- force_postauxFalseForces the UserObject to be executed in POSTAUX
Default:False
C++ Type:bool
Controllable:No
Description:Forces the UserObject to be executed in POSTAUX
- force_preauxFalseForces the UserObject to be executed in PREAUX
Default:False
C++ Type:bool
Controllable:No
Description:Forces the UserObject to be executed in PREAUX
- force_preicFalseForces the UserObject to be executed in PREIC during initial setup
Default:False
C++ Type:bool
Controllable:No
Description:Forces the UserObject to be executed in PREIC during initial setup
- use_displaced_meshFalseWhether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.
Default:False
C++ Type:bool
Controllable:No
Description:Whether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.
Advanced Parameters
Input Files
- (modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_squared_exponential.i)
- (modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_1D_tuned.i)
- (modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_exponential_tuned.i)
- (modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_exponential.i)
- (modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_2D.i)
- (modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_squared_exponential_tuned.i)
- (modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_squared_exponential_training.i)
- (modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_1D.i)
- (modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_Matern_half_int_tuned.i)
- (modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_2D_tuned.i)
- (modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_Matern_half_int.i)
- (modules/stochastic_tools/examples/surrogates/gaussian_process/GP_normal_mc.i)
References
- Carl Edward Rasmussen and Christopher K. I. Williams.
Gaussian Processes for Machine Learning.
The MIT Press, 2005.
ISBN 026218253X.[BibTeX]