GaussianProcessSurrogate

Computes and evaluates Gaussian Process surrogate model.

The theory and use this object is provided within a discussion of the GaussianProcessTrainer training object.

A desirable aspect of Gaussian process modeling is that in addition to returning a predicted value at the evaluation point, it can also provide a measure of uncertainty in the form of a standard deviation. To facilitate this an overloaded evaluate() function which sets the standard deviation by reference is provided.

GaussianProcessSurrogate::evaluate(const std::vector<Real> & x, Real & std_dev) const

Evaluation of multi-output Gaussian Processes

For Gaussian Processes that predict multiple outputs the user can evaluate the the mean and standard deviation estimates using the following function:

void
GaussianProcessSurrogate::evaluate(const std::vector<Real> & x,
                                   std::vector<Real> & y,
                                   std::vector<Real> & std) const

Input Parameters

filenameThe name of the file which will be associated with the saved/loaded data.
C++ Type:FileName
Controllable:No
Description:The name of the file which will be associated with the saved/loaded data.
trainerThe SurrogateTrainer object. If this is specified the trainer data is automatically gathered and available in this SurrogateModel object.
C++ Type:UserObjectName
Controllable:No
Description:The SurrogateTrainer object. If this is specified the trainer data is automatically gathered and available in this SurrogateModel object.

Optional Parameters

control_tagsAdds user-defined labels for accessing object parameters via control logic.
C++ Type:std::vector<std::string>
Controllable:No
Description:Adds user-defined labels for accessing object parameters via control logic.
enableTrueSet the enabled status of the MooseObject.
Default:True
C++ Type:bool
Controllable:No
Description:Set the enabled status of the MooseObject.

Advanced Parameters

Input Files

(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_squared_exponential.i)
(modules/stochastic_tools/test/tests/reporters/BFActiveLearning/main_adam.i)
(modules/stochastic_tools/test/tests/surrogates/multioutput_gp/mogp_lmc_tuned.i)
(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_squared_exponential_tuned_adam.i)
(modules/stochastic_tools/test/tests/surrogates/multioutput_gp/mogp_lmc_load.i)
(modules/stochastic_tools/examples/surrogates/cross_validation/all_trainers_uniform_cv.i)
(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_exponential_tuned_adam.i)
(modules/combined/examples/stochastic/laser_welding_dimred/2d-reconst.i)
(modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_2D.i)
(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_exponential.i)
(modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_2D_tuned.i)
(modules/stochastic_tools/test/tests/surrogates/multioutput_gp/mogp_lmc.i)
(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_squared_exponential_testing.i)
(modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_1D_tuned.i)
(modules/stochastic_tools/examples/surrogates/gaussian_process/GP_normal.i)
(modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_1D.i)
(modules/stochastic_tools/test/tests/reporters/ActiveLearningGP/main_adam.i)
(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_Matern_half_int.i)
(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_Matern_half_int_tuned_adam.i)
(modules/stochastic_tools/test/tests/reporters/AISActiveLearning/ais_al.i)

(modules/stochastic_tools/src/surrogates/GaussianProcessSurrogate.C)

// This file is part of the MOOSE framework
// https://mooseframework.inl.gov
//
// All rights reserved, see COPYRIGHT for full restrictions
// https://github.com/idaholab/moose/blob/master/COPYRIGHT
//
// Licensed under LGPL 2.1, please see LICENSE for details
// https://www.gnu.org/licenses/lgpl-2.1.html

#include "GaussianProcessSurrogate.h"
#include "Sampler.h"

#include "CovarianceFunctionBase.h"

registerMooseObject("StochasticToolsApp", GaussianProcessSurrogate);

InputParameters
GaussianProcessSurrogate::validParams()
{
  InputParameters params = SurrogateModel::validParams();
  params.addClassDescription("Computes and evaluates Gaussian Process surrogate model.");
  return params;
}

GaussianProcessSurrogate::GaussianProcessSurrogate(const InputParameters & parameters)
  : SurrogateModel(parameters),
    CovarianceInterface(parameters),
    _gp(declareModelData<StochasticTools::GaussianProcess>("_gp")),
    _training_params(getModelData<RealEigenMatrix>("_training_params"))
{
}

void
GaussianProcessSurrogate::setupCovariance(UserObjectName covar_name)
{
  if (_gp.getCovarFunctionPtr() != nullptr)
    ::mooseError("Attempting to redefine covariance function using setupCovariance.");
  _gp.linkCovarianceFunction(getCovarianceFunctionByName(covar_name));
}

Real
GaussianProcessSurrogate::evaluate(const std::vector<Real> & x) const
{
  // Overlaod for evaluate to maintain general compatibility. Only returns mean
  Real dummy = 0;
  return this->evaluate(x, dummy);
}

Real
GaussianProcessSurrogate::evaluate(const std::vector<Real> & x, Real & std_dev) const
{
  std::vector<Real> y;
  std::vector<Real> std;
  this->evaluate(x, y, std);
  std_dev = std[0];
  return y[0];
}

void
GaussianProcessSurrogate::evaluate(const std::vector<Real> & x, std::vector<Real> & y) const
{
  // Overlaod for evaluate to maintain general compatibility. Only returns mean
  std::vector<Real> std_dummy;
  this->evaluate(x, y, std_dummy);
}

void
GaussianProcessSurrogate::evaluate(const std::vector<Real> & x,
                                   std::vector<Real> & y,
                                   std::vector<Real> & std) const
{
  const unsigned int n_dims = _training_params.cols();

  mooseAssert(x.size() == n_dims,
              "Number of parameters provided for evaluation does not match number of parameters "
              "used for training.");
  const unsigned int n_outputs = _gp.getCovarFunction().numOutputs();

  y = std::vector<Real>(n_outputs, 0.0);
  std = std::vector<Real>(n_outputs, 0.0);

  RealEigenMatrix test_points(1, n_dims);
  for (unsigned int ii = 0; ii < n_dims; ++ii)
    test_points(0, ii) = x[ii];

  _gp.getParamStandardizer().getStandardized(test_points);

  RealEigenMatrix K_train_test(_training_params.rows() * n_outputs, n_outputs);

  _gp.getCovarFunction().computeCovarianceMatrix(
      K_train_test, _training_params, test_points, false);
  RealEigenMatrix K_test(n_outputs, n_outputs);
  _gp.getCovarFunction().computeCovarianceMatrix(K_test, test_points, test_points, true);

  // Compute the predicted mean value (centered)
  RealEigenMatrix pred_value = (K_train_test.transpose() * _gp.getKResultsSolve()).transpose();
  // De-center/scale the value and store for return
  _gp.getDataStandardizer().getDestandardized(pred_value);

  RealEigenMatrix pred_var =
      K_test - (K_train_test.transpose() * _gp.getKCholeskyDecomp().solve(K_train_test));

  // Vairance computed, take sqrt for standard deviation, scale up by training data std and store
  RealEigenMatrix std_dev_mat = pred_var.array().sqrt();
  _gp.getDataStandardizer().getDescaled(std_dev_mat);

  for (const auto output_i : make_range(n_outputs))
  {
    y[output_i] = pred_value(0, output_i);
    std[output_i] = std_dev_mat(output_i, output_i);
  }
}

(modules/stochastic_tools/src/surrogates/GaussianProcessSurrogate.C)

// This file is part of the MOOSE framework
// https://mooseframework.inl.gov
//
// All rights reserved, see COPYRIGHT for full restrictions
// https://github.com/idaholab/moose/blob/master/COPYRIGHT
//
// Licensed under LGPL 2.1, please see LICENSE for details
// https://www.gnu.org/licenses/lgpl-2.1.html

#include "GaussianProcessSurrogate.h"
#include "Sampler.h"

#include "CovarianceFunctionBase.h"

registerMooseObject("StochasticToolsApp", GaussianProcessSurrogate);

InputParameters
GaussianProcessSurrogate::validParams()
{
  InputParameters params = SurrogateModel::validParams();
  params.addClassDescription("Computes and evaluates Gaussian Process surrogate model.");
  return params;
}

GaussianProcessSurrogate::GaussianProcessSurrogate(const InputParameters & parameters)
  : SurrogateModel(parameters),
    CovarianceInterface(parameters),
    _gp(declareModelData<StochasticTools::GaussianProcess>("_gp")),
    _training_params(getModelData<RealEigenMatrix>("_training_params"))
{
}

void
GaussianProcessSurrogate::setupCovariance(UserObjectName covar_name)
{
  if (_gp.getCovarFunctionPtr() != nullptr)
    ::mooseError("Attempting to redefine covariance function using setupCovariance.");
  _gp.linkCovarianceFunction(getCovarianceFunctionByName(covar_name));
}

Real
GaussianProcessSurrogate::evaluate(const std::vector<Real> & x) const
{
  // Overlaod for evaluate to maintain general compatibility. Only returns mean
  Real dummy = 0;
  return this->evaluate(x, dummy);
}

Real
GaussianProcessSurrogate::evaluate(const std::vector<Real> & x, Real & std_dev) const
{
  std::vector<Real> y;
  std::vector<Real> std;
  this->evaluate(x, y, std);
  std_dev = std[0];
  return y[0];
}

void
GaussianProcessSurrogate::evaluate(const std::vector<Real> & x, std::vector<Real> & y) const
{
  // Overlaod for evaluate to maintain general compatibility. Only returns mean
  std::vector<Real> std_dummy;
  this->evaluate(x, y, std_dummy);
}

void
GaussianProcessSurrogate::evaluate(const std::vector<Real> & x,
                                   std::vector<Real> & y,
                                   std::vector<Real> & std) const
{
  const unsigned int n_dims = _training_params.cols();

  mooseAssert(x.size() == n_dims,
              "Number of parameters provided for evaluation does not match number of parameters "
              "used for training.");
  const unsigned int n_outputs = _gp.getCovarFunction().numOutputs();

  y = std::vector<Real>(n_outputs, 0.0);
  std = std::vector<Real>(n_outputs, 0.0);

  RealEigenMatrix test_points(1, n_dims);
  for (unsigned int ii = 0; ii < n_dims; ++ii)
    test_points(0, ii) = x[ii];

  _gp.getParamStandardizer().getStandardized(test_points);

  RealEigenMatrix K_train_test(_training_params.rows() * n_outputs, n_outputs);

  _gp.getCovarFunction().computeCovarianceMatrix(
      K_train_test, _training_params, test_points, false);
  RealEigenMatrix K_test(n_outputs, n_outputs);
  _gp.getCovarFunction().computeCovarianceMatrix(K_test, test_points, test_points, true);

  // Compute the predicted mean value (centered)
  RealEigenMatrix pred_value = (K_train_test.transpose() * _gp.getKResultsSolve()).transpose();
  // De-center/scale the value and store for return
  _gp.getDataStandardizer().getDestandardized(pred_value);

  RealEigenMatrix pred_var =
      K_test - (K_train_test.transpose() * _gp.getKCholeskyDecomp().solve(K_train_test));

  // Vairance computed, take sqrt for standard deviation, scale up by training data std and store
  RealEigenMatrix std_dev_mat = pred_var.array().sqrt();
  _gp.getDataStandardizer().getDescaled(std_dev_mat);

  for (const auto output_i : make_range(n_outputs))
  {
    y[output_i] = pred_value(0, output_i);
    std[output_i] = std_dev_mat(output_i, output_i);
  }
}

(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_squared_exponential.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 10
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
  [test_sample]
    type = MonteCarlo
    num_rows = 100
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Materials/conductivity/prop_values Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
    parallel_type = ROOT
  []
  [samp_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = test_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [train_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[VectorPostprocessors]
  [hyperparams]
    type = GaussianProcessData
    gp_name = 'GP_avg'
    execute_on = final
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'covar' #Choose a squared exponential for the kernel
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
    sampler = train_sample
    response = results/data:avg:value
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    trainer = GP_avg_trainer
  []
[]

[Covariance]
  [covar]
    type = SquaredExponentialCovariance
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-6 #A small amount of noise can help with numerical stability
    length_factor = '0.38971 0.38971' #Select a length factor for each parameter (k and q)
  []
[]

[Outputs]
  [out]
    type = CSV
    execute_on = FINAL
  []
[]

(modules/stochastic_tools/test/tests/reporters/BFActiveLearning/main_adam.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 5
    upper_bound = 20
  []
  [q_dist]
    type = Uniform
    lower_bound = 7000
    upper_bound = 13000
  []
  [Tinf_dist]
    type = Uniform
    lower_bound = 250
    upper_bound = 350
  []
[]

[Samplers]
  [mc]
    type = ActiveLearningMonteCarloSampler
    num_batch = 1
    distributions = 'k_dist q_dist Tinf_dist'
    flag_sample = 'conditional/flag_sample'
    seed = 5
    num_samples = 10
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[MultiApps]
  [sub_lf]
    type = SamplerFullSolveMultiApp
    sampler = mc
    input_files = 'sub_lf.i'
  []
  [sub]
    type = SamplerFullSolveMultiApp
    sampler = mc
    input_files = 'sub.i'
    mode = batch-reset
    should_run_reporter = conditional/need_sample
    execute_on = TIMESTEP_END
  []
[]

[Transfers]
  [sub]
    type = SamplerParameterTransfer
    to_multi_app = sub
    sampler = mc
    parameters = 'Materials/conductivity/prop_values Kernels/source/value BCs/right/value'
    check_multiapp_execute_on = false
  []
  [sub_lf]
    type = SamplerParameterTransfer
    to_multi_app = sub_lf
    sampler = mc
    parameters = 'Materials/conductivity/prop_values Kernels/source/value BCs/right/value'
    check_multiapp_execute_on = false
  []
  [reporter_transfer_lf]
    type = SamplerReporterTransfer
    from_reporter = 'avg/value'
    stochastic_reporter = 'constant'
    from_multi_app = sub_lf
    sampler = mc
  []
  [reporter_transfer]
    type = SamplerReporterTransfer
    from_reporter = 'avg/value'
    stochastic_reporter = 'conditional'
    from_multi_app = sub
    sampler = mc
  []
[]

[Reporters]
  [constant]
    type = StochasticReporter
  []
  [conditional]
    type = BiFidelityActiveLearningGPDecision
    sampler = mc
    parallel_type = ROOT
    execute_on = 'timestep_begin'
    flag_sample = 'flag_sample'
    inputs = 'inputs'
    gp_mean = 'gp_mean'
    gp_std = 'gp_std'
    n_train = 8
    al_gp = GP_al_trainer
    gp_evaluator = GP_eval
    learning_function = 'Ufunction'
    learning_function_parameter = 349.345
    learning_function_threshold = 2.0
    outputs_lf = constant/reporter_transfer_lf:avg:value
  []
[]

[Trainers]
  [GP_al_trainer]
    type = ActiveLearningGaussianProcess
    covariance_function = 'covar'
    standardize_params = 'true'
    standardize_data = 'true'
    tune_parameters = 'covar:signal_variance covar:length_factor'
    num_iters = 5000
    learning_rate = 0.001
    show_every_nth_iteration = 1
    batch_size = 200
  []
[]

[Surrogates]
  [GP_eval]
    type = GaussianProcessSurrogate
    trainer = GP_al_trainer
  []
[]

[Covariance]
  [covar]
    type = SquaredExponentialCovariance
    signal_variance = 1.0
    noise_variance = 1e-8
    length_factor = '1.0 1.0 1.0'
  []
[]

[Executioner]
  type = Transient
[]

[Outputs]
  file_base = 'single_proc_single_row_ufunction'
  [out]
    type = JSON
    execute_system_information_on = none
  []
[]

(modules/stochastic_tools/test/tests/surrogates/multioutput_gp/mogp_lmc_tuned.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Normal
    mean = 15.0
    standard_deviation = 2.0
  []
  [bc_dist]
    type = Normal
    mean = 1000.0
    standard_deviation = 100.0
  []
[]

[Samplers]
  [train]
    type = LatinHypercube
    num_rows = 10
    distributions = 'k_dist bc_dist'
    execute_on = PRE_MULTIAPP_SETUP
    seed = 100
  []
  [test]
    type = LatinHypercube
    num_rows = 5
    distributions = 'k_dist bc_dist'
    seed = 101
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    mode = batch-reset
    sampler = train
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train
    param_names = 'Materials/conductivity/prop_values BCs/right/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train
    stochastic_reporter = results
    from_reporter = 'T_vec/T'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
    outputs = none
  []
  [eval_test]
    type = EvaluateSurrogate
    model = mogp
    response_type = vector_real
    parallel_type = ROOT
    execute_on = timestep_end
    sampler = test
    evaluate_std = true
  []
[]

[Trainers]
  [mogp_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'lmc'
    standardize_params = 'true'
    standardize_data = 'true'
    sampler = train
    response_type = vector_real
    response = results/data:T_vec:T
    tune_parameters = 'lmc:acoeff_0 lmc:lambda_0 covar:signal_variance covar:length_factor'
    tuning_min = '1e-9 1e-9 1e-9 1e-9'
    tuning_max = '1e16 1e16 1e16  1e16'
    num_iters = 1000
    batch_size = 10
    learning_rate = 0.0005
    show_every_nth_iteration = 1
  []
[]

[Covariance]
  [covar]
    type = SquaredExponentialCovariance
    signal_variance = 2.76658083
    noise_variance = 0.0
    length_factor = '3.67866381 2.63421705'
  []
  [lmc]
    type = LMC
    covariance_functions = covar
    num_outputs = 2
    num_latent_funcs = 1
  []
[]

[Surrogates]
  [mogp]
    type = GaussianProcessSurrogate
    trainer = mogp_trainer
  []
[]

[VectorPostprocessors]
  [train_params]
    type = SamplerData
    sampler = train
    execute_on = final
  []
  [test_params]
    type = SamplerData
    sampler = test
    execute_on = final
  []
  [hyperparams]
    type = GaussianProcessData
    gp_name = mogp
    execute_on = final
  []
[]

[Outputs]
  [out]
    type = JSON
    execute_on = final
    vectorpostprocessors_as_reporters = true
    execute_system_information_on = NONE
  []
  [surr]
    type = SurrogateTrainerOutput
    execute_on = FINAL
    trainers = "mogp_trainer"
  []
[]

(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_squared_exponential_tuned_adam.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 20
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
    seed = 100
  []
  [test_sample]
    type = MonteCarlo
    num_rows = 100
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
    seed = 100
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Materials/conductivity/prop_values Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
    parallel_type = ROOT
  []
  [samp_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = test_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [train_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[VectorPostprocessors]
  [hyperparams]
    type = GaussianProcessData
    gp_name = 'GP_avg'
    execute_on = final
  []
  [data]
    type = SamplerData
    sampler = test_sample
    execute_on = 'initial timestep_end'
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'covar' #Choose a squared exponential for the kernel
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
    sampler = train_sample
    response = results/data:avg:value
    tune_parameters = 'covar:signal_variance covar:length_factor'
    num_iters = 1000
    batch_size = 20
    learning_rate = 0.005
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    trainer = GP_avg_trainer
  []
[]

[Covariance]
  [covar]
    type = SquaredExponentialCovariance
    signal_variance = 1.0 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-6 #A small amount of noise can help with numerical stability
    length_factor = '1.0 1.0' #Select a length factor for each parameter (k and q)
  []
[]

[Outputs]
  [out]
    type = CSV
    execute_on = FINAL
  []
[]

(modules/stochastic_tools/test/tests/surrogates/multioutput_gp/mogp_lmc_load.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Normal
    mean = 15.0
    standard_deviation = 2.0
  []
  [bc_dist]
    type = Normal
    mean = 1000.0
    standard_deviation = 100.0
  []
[]

[Samplers]
  [test]
    type = LatinHypercube
    num_rows = 5
    distributions = 'k_dist bc_dist'
    seed = 101
  []
[]

[Reporters]
  [eval_test]
    type = EvaluateSurrogate
    model = mogp
    response_type = vector_real
    parallel_type = ROOT
    execute_on = timestep_end
    sampler = test
    evaluate_std = true
  []
[]

[Surrogates]
  [mogp]
    type = GaussianProcessSurrogate
    filename = "mogp_lmc_tuned_surr_mogp_trainer.rd"
  []
[]

[VectorPostprocessors]
  [test_params]
    type = SamplerData
    sampler = test
    execute_on = 'final'
  []
  [hyperparams]
    type = GaussianProcessData
    gp_name = mogp
    execute_on = final
  []
[]

[Outputs]
  [out]
    type = JSON
    execute_on = final
    vectorpostprocessors_as_reporters = true
    execute_system_information_on = NONE
  []
[]

(modules/stochastic_tools/examples/surrogates/cross_validation/all_trainers_uniform_cv.i)

[StochasticTools]
[]

[GlobalParams]
  sampler = cv_sampler
  response = results/response_data:max:value
  cv_type = "k_fold"
  cv_splits = 5
  cv_n_trials = 100
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
  [L_dist]
    type = Uniform
    lower_bound = 0.01
    upper_bound = 0.05
  []
  [Tinf_dist]
    type = Uniform
    lower_bound = 290
    upper_bound = 310
  []
[]

[Samplers]
  [cv_sampler]
    type = LatinHypercube
    distributions = 'k_dist q_dist L_dist Tinf_dist'
    num_rows = 1000
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[MultiApps]
  [cv_sub]
    type = SamplerFullSolveMultiApp
    input_files = all_sub.i
    mode = batch-reset
  []
[]

[Controls]
  [pr_cmdline]
    type = MultiAppSamplerControl
    multi_app = cv_sub
    param_names = 'Materials/conductivity/prop_values Kernels/source/value Mesh/xmax BCs/right/value'
  []
[]

[Transfers]
  [response_data]
    type = SamplerReporterTransfer
    from_multi_app = cv_sub
    stochastic_reporter = results
    from_reporter = 'max/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
    outputs = none
  []
  [cv_scores]
    type = CrossValidationScores
    models = 'pr_surr pc_surr np_surr gp_surr ann_surr'
    execute_on = FINAL
  []
[]

[Trainers]
  [pr_max]
    type = PolynomialRegressionTrainer
    regression_type = "ols"
    max_degree = 3
    cv_surrogate = "pr_surr"
    execute_on = timestep_end
  []
  [pc_max]
    type = PolynomialChaosTrainer
    order = 3
    distributions = "k_dist q_dist L_dist Tinf_dist"
    cv_surrogate = "pc_surr"
    execute_on = timestep_end
  []
  [np_max]
    type = NearestPointTrainer
    cv_surrogate = "np_surr"
    execute_on = timestep_end
  []
  [gp_max]
    type = GaussianProcessTrainer
    covariance_function = 'rbf'
    standardize_params = 'true'
    standardize_data = 'true'
    cv_surrogate = "gp_surr"
    execute_on = timestep_end
  []
  [ann_max]
    type = LibtorchANNTrainer
    num_epochs = 100
    num_batches = 5
    num_neurons_per_layer = '64'
    learning_rate = 1e-2
    rel_loss_tol = 1e-4
    filename = mynet.pt
    read_from_file = false
    print_epoch_loss = 0
    activation_function = 'relu'
    cv_surrogate = "ann_surr"
    standardize_input = false
    standardize_output = false
  []
[]

[Covariance]
  [rbf]
    type = SquaredExponentialCovariance
    noise_variance = 3.79e-6
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    length_factor = '5.34471 1.41191 5.90721 2.83723' #Select a length factor for each parameter
  []
[]

[Surrogates]
  [pr_surr]
    type = PolynomialRegressionSurrogate
    trainer = pr_max
  []
  [pc_surr]
    type = PolynomialChaos
    trainer = pc_max
  []
  [np_surr]
    type = NearestPointSurrogate
    trainer = np_max
  []
  [gp_surr]
    type = GaussianProcessSurrogate
    trainer = gp_max
  []
  [ann_surr]
    type = LibtorchANNSurrogate
    trainer = ann_max
  []
[]

[Outputs]
  [out]
    type = JSON
    execute_on = FINAL
  []
[]

(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_exponential_tuned_adam.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 20
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
  [test_sample]
    type = MonteCarlo
    num_rows = 100
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Materials/conductivity/prop_values Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
    parallel_type = ROOT
  []
  [samp_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = test_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [train_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[VectorPostprocessors]
  [hyperparams]
    type = GaussianProcessData
    gp_name = 'GP_avg'
    execute_on = final
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'covar' #Choose a squared exponential for the kernel
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
    sampler = train_sample
    response = results/data:avg:value
    tune_parameters = 'covar:signal_variance covar:length_factor'
    num_iters = 1000
    batch_size = 20
    learning_rate = 0.005
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    trainer = GP_avg_trainer
  []
[]

[Covariance]
  [covar]
    type = ExponentialCovariance
    gamma = 2 #Define the exponential factor
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-6 #A small amount of noise can help with numerical stability
    length_factor = '1.0 1.0' #Select a length factor for each parameter (k and q)
  []
[]

[Outputs]
  [out]
    type = CSV
    execute_on = FINAL
  []
[]

(modules/combined/examples/stochastic/laser_welding_dimred/2d-reconst.i)

!include parameters.i

!include mesh.i

[Variables]
  [vel]
    family = LAGRANGE_VEC
  []
  [T]
  []
  [p]
  []
  [disp_x]
  []
  [disp_y]
  []
[]

[AuxVariables]
  [T_reconst]
  []
[]

!include physics_objects.i

[UserObjects]
  [inverse]
    type = InverseMapping
    mapping = pod
    variable_to_fill = "T_reconst"
    variable_to_reconstruct = "T"
    surrogate = mogp
    parameters = '${R} ${power}'
    execute_on = TIMESTEP_END
  []
[]

[Surrogates]
  [mogp]
    type = GaussianProcessSurrogate
    filename = "train_mogp_out_mogp.rd"
  []
[]

[VariableMappings]
  [pod]
    type = PODMapping
    filename = "train_pod_out_pod.rd"
  []
[]

[Executioner]
  type = Transient
  end_time = ${endtime}
  dtmin = 1e-10
  dtmax = 1e-5
  petsc_options_iname = '-pc_type -pc_factor_shift_type'
  petsc_options_value = 'lu       NONZERO'
  petsc_options = '-snes_converged_reason -ksp_converged_reason -options_left'
  solve_type = 'NEWTON'
  line_search = 'none'
  nl_max_its = 16
  l_max_its = 100
  [TimeStepper]
    type = IterationAdaptiveDT
    optimal_iterations = 10
    iteration_window = 4
    dt = ${timestep}
    linear_iteration_ratio = 1e6
    growth_factor = 1.25
  []
[]

[Postprocessors]
  [l2error]
    type = ElementL2Difference
    variable = T
    other_variable = T_reconst
  []
[]

[Debug]
  show_var_residual_norms = true
[]

[Postprocessors]
[]

(modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_2D.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 50
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
  [cart_sample]
    type = CartesianProduct
    linear_space_items = '1 0.09 100
                          9000 20 100 '
    execute_on = initial
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Materials/conductivity/prop_values Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
  []
  [train_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [cart_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = cart_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'rbf'
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
    sampler = train_sample
    response = results/data:avg:value
  []
[]

[Covariance]
  [rbf]
    type = SquaredExponentialCovariance
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-6 #A small amount of noise can help with numerical stability
    length_factor = '0.38971 0.38971' #Select a length factor for each parameter (k and q)
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    trainer = 'GP_avg_trainer'
  []
[]

[VectorPostprocessors]
  [hyperparams]
    type = GaussianProcessData
    gp_name = 'GP_avg'
    execute_on = final
  []
[]

[Outputs]
  [out]
    type = CSV
    execute_on = FINAL
  []
[]

(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_exponential.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 10
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
  [test_sample]
    type = MonteCarlo
    num_rows = 100
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Materials/conductivity/prop_values Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
    parallel_type = ROOT
  []
  [samp_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = test_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [train_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[VectorPostprocessors]
  [hyperparams]
    type = GaussianProcessData
    gp_name = 'GP_avg'
    execute_on = final
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'covar' #Choose an exponential for the kernel
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
    sampler = train_sample
    response = results/data:avg:value
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    trainer = GP_avg_trainer
  []
[]

[Covariance]
  [covar]
    type = ExponentialCovariance
    gamma = 1 #Define the exponential factor
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-6 #A small amount of noise can help with numerical stability
    length_factor = '0.551133 0.551133' #Select a length factor for each parameter (k and q)
  []
[]

[Outputs]
  [out]
    type = CSV
    execute_on = FINAL
  []
[]

(modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_2D_tuned.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 50
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
  [cart_sample]
    type = CartesianProduct
    linear_space_items = '1 0.09 10
                          9000 20 10 '
    execute_on = initial
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Materials/conductivity/prop_values Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'rbf'
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
    sampler = train_sample
    response = results/data:avg:value
    tune_parameters = 'rbf:signal_variance rbf:length_factor'
    tuning_min = '1e-9 1e-9'
    tuning_max = '1e16  1e16'
    batch_size = 50
    num_iters = 5000
    learning_rate = 5e-3
  []
[]

[Covariance]
  [rbf]
    type = SquaredExponentialCovariance
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-3 #A small amount of noise can help with numerical stability
    length_factor = '0.38971 0.38971' #Select a length factor for each parameter (k and q)
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    trainer = 'GP_avg_trainer'
  []
[]

[Reporters]
  [train_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [cart_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = cart_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[VectorPostprocessors]
  [hyperparams]
    type = GaussianProcessData
    gp_name = 'GP_avg'
    execute_on = final
  []
[]

[Outputs]
  [out]
    type = CSV
    execute_on = FINAL
  []
[]

(modules/stochastic_tools/test/tests/surrogates/multioutput_gp/mogp_lmc.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Normal
    mean = 15.0
    standard_deviation = 2.0
  []
  [bc_dist]
    type = Normal
    mean = 1000.0
    standard_deviation = 100.0
  []
[]

[Samplers]
  [train]
    type = LatinHypercube
    num_rows = 10
    distributions = 'k_dist bc_dist'
    execute_on = PRE_MULTIAPP_SETUP
    seed = 100
  []
  [test]
    type = LatinHypercube
    num_rows = 5
    distributions = 'k_dist bc_dist'
    seed = 101
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    mode = batch-reset
    sampler = train
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train
    param_names = 'Materials/conductivity/prop_values BCs/right/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train
    stochastic_reporter = results
    from_reporter = 'T_vec/T'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
    outputs = none
  []
  [eval_test]
    type = EvaluateSurrogate
    model = mogp
    response_type = vector_real
    parallel_type = ROOT
    execute_on = timestep_end
    sampler = test
    evaluate_std = true
  []
[]

[Trainers]
  [mogp_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'lmc'
    standardize_params = 'true'
    standardize_data = 'true'
    sampler = train
    response_type = vector_real
    response = results/data:T_vec:T
  []
[]

[Covariance]
  [covar]
    type = SquaredExponentialCovariance
    signal_variance = 2.76658083
    noise_variance = 0.0
    length_factor = '3.67866381 2.63421705'
  []
  [lmc]
    type = LMC
    covariance_functions = covar
    num_outputs = 2
    num_latent_funcs = 1
  []
[]

[Surrogates]
  [mogp]
    type = GaussianProcessSurrogate
    trainer = mogp_trainer
  []
[]

[VectorPostprocessors]
  [train_params]
    type = SamplerData
    sampler = train
    execute_on = final
  []
  [test_params]
    type = SamplerData
    sampler = test
    execute_on = final
  []
  [hyperparams]
    type = GaussianProcessData
    gp_name = mogp
    execute_on = final
  []
[]

[Outputs]
  [out]
    type = JSON
    execute_on = final
    vectorpostprocessors_as_reporters = true
    execute_system_information_on = NONE
  []
[]

(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_squared_exponential_testing.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
[]

[Samplers]
  [test_sample]
    type = MonteCarlo
    num_rows = 100
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[Reporters]
  [samp_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = test_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    filename = 'gauss_process_training_GP_avg_trainer.rd'
  []
[]

[Outputs]
  [out]
    type = CSV
    execute_on = FINAL
  []
[]

(modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_1D_tuned.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
  [L_dist]
    type = Uniform
    lower_bound = 0.01
    upper_bound = 0.05
  []
  [Tinf_dist]
    type = Uniform
    lower_bound = 290
    upper_bound = 310
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 6
    distributions = 'q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
  [cart_sample]
    type = CartesianProduct
    linear_space_items = '9000 20 100'
    execute_on = initial
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    response = results/data:avg:value
    covariance_function = 'rbf'
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
    sampler = train_sample
    tune_parameters = 'rbf:signal_variance rbf:length_factor'
    tuning_min = ' 1e-9 1e-9'
    tuning_max = ' 1e16  1e16'
    num_iters = 10000
    batch_size = 6
    learning_rate = 0.0005
    show_every_nth_iteration = 1
  []
[]

[Covariance]
  [rbf]
    type = SquaredExponentialCovariance
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-3 #A small amount of noise can help with numerical stability
    length_factor = '0.38971' #Select a length factor for each parameter (k and q)
  []
[]

[Surrogates]
  [gauss_process_avg]
    type = GaussianProcessSurrogate
    trainer = 'GP_avg_trainer'
  []
[]

# # Computing statistics
[VectorPostprocessors]
  [hyperparams]
    type = GaussianProcessData
    gp_name = 'gauss_process_avg'
    execute_on = final
  []
[]

[Reporters]
  [cart_avg]
    type = EvaluateSurrogate
    model = gauss_process_avg
    sampler = cart_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [train_avg]
    type = EvaluateSurrogate
    model = gauss_process_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[Outputs]
  csv = true
  execute_on = FINAL
[]

(modules/stochastic_tools/examples/surrogates/gaussian_process/GP_normal.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = TruncatedNormal
    mean = 5
    standard_deviation = 2
    lower_bound = 0
  []
  [q_dist]
    type = TruncatedNormal
    mean = 10000
    standard_deviation = 500
    lower_bound = 0
  []
  [L_dist]
    type = TruncatedNormal
    mean = 0.03
    standard_deviation = 0.01
    lower_bound = 0
  []
  [Tinf_dist]
    type = TruncatedNormal
    mean = 300
    standard_deviation = 10
    lower_bound = 0
  []
[]

[Samplers]
  [sample]
    type = MonteCarlo
    num_rows = 100000
    distributions = 'k_dist q_dist L_dist Tinf_dist'
    execute_on = initial
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    filename = 'GP_training_normal_GP_avg.rd'
  []
[]

# Computing statistics
[VectorPostprocessors]
  [GP_avg_hyperparams]
    type = GaussianProcessData
    gp_name = 'GP_avg'
  []
[]

[Outputs]
  csv = true
[]

(modules/stochastic_tools/examples/surrogates/gaussian_process/gaussian_process_uniform_1D.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
  [L_dist]
    type = Uniform
    lower_bound = 0.01
    upper_bound = 0.05
  []
  [Tinf_dist]
    type = Uniform
    lower_bound = 290
    upper_bound = 310
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 6
    distributions = 'q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
  [cart_sample]
    type = CartesianProduct
    linear_space_items = '9000 20 100'
    execute_on = initial
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
  []
  [cart_avg]
    type = EvaluateSurrogate
    model = gauss_process_avg
    sampler = cart_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [train_avg]
    type = EvaluateSurrogate
    model = gauss_process_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    sampler = train_sample
    response = results/data:avg:value
    covariance_function = 'rbf'
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
  []
[]

[Covariance]
  [rbf]
    type = SquaredExponentialCovariance
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-3 #A small amount of noise can help with numerical stability
    length_factor = '0.38971' #Select a length factor for each parameter (k and q)
  []
[]

[Surrogates]
  [gauss_process_avg]
    type = GaussianProcessSurrogate
    trainer = 'GP_avg_trainer'
  []
[]

[Outputs]
  csv = true
  execute_on = FINAL
[]

(modules/stochastic_tools/test/tests/reporters/ActiveLearningGP/main_adam.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 5
    upper_bound = 20
  []
  [q_dist]
    type = Uniform
    lower_bound = 7000
    upper_bound = 13000
  []
  [Tinf_dist]
    type = Uniform
    lower_bound = 250
    upper_bound = 350
  []
[]

[Samplers]
  [mc]
    type = ActiveLearningMonteCarloSampler
    num_batch = 1
    distributions = 'k_dist q_dist Tinf_dist'
    flag_sample = 'conditional/flag_sample'
    seed = 5
    num_samples = 20
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    sampler = mc
    input_files = 'sub.i'
    mode = batch-reset
    should_run_reporter = conditional/need_sample
    execute_on = TIMESTEP_END
  []
[]

[Transfers]
  [sub]
    type = SamplerParameterTransfer
    to_multi_app = sub
    sampler = mc
    parameters = 'Materials/conductivity/prop_values Kernels/source/value BCs/right/value'
    to_control = 'stochastic'
    check_multiapp_execute_on = false
  []
  [reporter_transfer]
    type = SamplerReporterTransfer
    from_reporter = 'avg/value'
    stochastic_reporter = 'conditional'
    from_multi_app = sub
    sampler = mc
  []
[]

[Reporters]
  [conditional]
    type = ActiveLearningGPDecision
    sampler = mc
    parallel_type = ROOT
    execute_on = 'timestep_begin'
    flag_sample = 'flag_sample'
    inputs = 'inputs'
    gp_mean = 'gp_mean'
    gp_std = 'gp_std'
    n_train = 6
    al_gp = GP_al_trainer
    gp_evaluator = GP_eval
    learning_function = 'Ufunction'
    learning_function_parameter = 349.345
    learning_function_threshold = 2.0
  []
[]

[Trainers]
  [GP_al_trainer]
    type = ActiveLearningGaussianProcess
    covariance_function = 'covar'
    standardize_params = 'true'
    standardize_data = 'true'
    tune_parameters = 'covar:signal_variance covar:length_factor'
    num_iters = 1000
    learning_rate = 0.005
  []
[]

[Surrogates]
  [GP_eval]
    type = GaussianProcessSurrogate
    trainer = GP_al_trainer
  []
[]

[Covariance]
  [covar]
    type = SquaredExponentialCovariance
    signal_variance = 1.0
    noise_variance = 1e-4
    length_factor = '1.0 1.0 1.0'
  []
[]

[Executioner]
  type = Transient
[]

[Outputs]
  file_base = 'single_proc_single_row_ufunction'
  [out]
    type = JSON
    execute_system_information_on = none
  []
[]

(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_Matern_half_int.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 10
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
  [test_sample]
    type = MonteCarlo
    num_rows = 100
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Materials/conductivity/prop_values Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
    parallel_type = ROOT
  []
  [samp_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = test_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [train_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[VectorPostprocessors]
  [hyperparams]
    type = GaussianProcessData
    gp_name = 'GP_avg'
    execute_on = final
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'covar' #Choose a Matern with half-integer argument for the kernel
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
    sampler = train_sample
    response = results/data:avg:value
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    trainer = GP_avg_trainer
  []
[]

[Covariance]
  [covar]
    type = MaternHalfIntCovariance
    p = 2 #Define the exponential factor
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-6 #A small amount of noise can help with numerical stability
    length_factor = '0.551133 0.551133' #Select a length factor for each parameter (k and q)
  []
[]

[Outputs]
  [out]
    type = CSV
    execute_on = FINAL
  []
[]

(modules/stochastic_tools/test/tests/surrogates/gaussian_process/GP_Matern_half_int_tuned_adam.i)

[StochasticTools]
[]

[Distributions]
  [k_dist]
    type = Uniform
    lower_bound = 1
    upper_bound = 10
  []
  [q_dist]
    type = Uniform
    lower_bound = 9000
    upper_bound = 11000
  []
[]

[Samplers]
  [train_sample]
    type = MonteCarlo
    num_rows = 20
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
  [test_sample]
    type = MonteCarlo
    num_rows = 100
    distributions = 'k_dist q_dist'
    execute_on = PRE_MULTIAPP_SETUP
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = train_sample
  []
[]

[Controls]
  [cmdline]
    type = MultiAppSamplerControl
    multi_app = sub
    sampler = train_sample
    param_names = 'Materials/conductivity/prop_values Kernels/source/value'
  []
[]

[Transfers]
  [data]
    type = SamplerReporterTransfer
    from_multi_app = sub
    sampler = train_sample
    stochastic_reporter = results
    from_reporter = 'avg/value'
  []
[]

[Reporters]
  [results]
    type = StochasticReporter
    parallel_type = ROOT
  []
  [samp_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = test_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
  [train_avg]
    type = EvaluateSurrogate
    model = GP_avg
    sampler = train_sample
    evaluate_std = 'true'
    parallel_type = ROOT
    execute_on = final
  []
[]

[VectorPostprocessors]
  [hyperparams]
    type = GaussianProcessData
    gp_name = 'GP_avg'
    execute_on = final
  []
[]

[Trainers]
  [GP_avg_trainer]
    type = GaussianProcessTrainer
    execute_on = timestep_end
    covariance_function = 'covar' #Choose a squared exponential for the kernel
    standardize_params = 'true' #Center and scale the training params
    standardize_data = 'true' #Center and scale the training data
    sampler = train_sample
    response = results/data:avg:value
    tune_parameters = 'covar:signal_variance covar:length_factor'
    num_iters = 1000
    batch_size = 20
    learning_rate = 0.005
  []
[]

[Surrogates]
  [GP_avg]
    type = GaussianProcessSurrogate
    trainer = GP_avg_trainer
  []
[]

[Covariance]
  [covar]
    type = MaternHalfIntCovariance
    p = 2 #Define the exponential factor
    signal_variance = 1 #Use a signal variance of 1 in the kernel
    noise_variance = 1e-6 #A small amount of noise can help with numerical stability
    length_factor = '1.0 1.0' #Select a length factor for each parameter (k and q)
  []
[]

[Outputs]
  [out]
    type = CSV
    execute_on = FINAL
    file_base = 'GP_Matern_half_int_tuned_adam'
  []
[]

(modules/stochastic_tools/test/tests/reporters/AISActiveLearning/ais_al.i)

[StochasticTools]
[]

[Distributions]
  [mu1]
    type = Normal
    mean = 0.0
    standard_deviation = 0.5
  []
  [mu2]
    type = Normal
    mean = 1
    standard_deviation = 0.5
  []
[]

[Samplers]
  [sample]
    type = AISActiveLearning
    distributions = 'mu1 mu2'
    proposal_std = '1.0 1.0'
    output_limit = 0.65
    num_samples_train = 15
    num_importance_sampling_steps = 5
    std_factor = 0.9
    initial_values = '-0.103 1.239'
    inputs_reporter = 'adaptive_MC/inputs'
    use_absolute_value = true
    flag_sample = 'conditional/flag_sample'
    seed = 9874
  []
[]

[MultiApps]
  [sub]
    type = SamplerFullSolveMultiApp
    input_files = sub.i
    sampler = sample
    mode = batch-reset
    should_run_reporter = conditional/need_sample
    execute_on = TIMESTEP_END
  []
[]

[Transfers]
  [param]
    type = SamplerParameterTransfer
    to_multi_app = sub
    sampler = sample
    parameters = 'BCs/left/value BCs/right/value'
    to_control = 'stochastic'
  []
  [reporter_transfer]
    type = SamplerReporterTransfer
    from_reporter = 'average/value'
    stochastic_reporter = 'conditional'
    from_multi_app = sub
    sampler = sample
  []
[]

[Reporters]
  [conditional]
    type = ActiveLearningGPDecision
    sampler = sample
    parallel_type = ROOT
    execute_on = 'initial timestep_begin'
    flag_sample = 'flag_sample'
    inputs = 'inputs'
    gp_mean = 'gp_mean'
    gp_std = 'gp_std'
    n_train = 5
    al_gp = GP_al_trainer
    gp_evaluator = GP_eval
    learning_function = 'Ufunction'
    learning_function_parameter = 0.65
    learning_function_threshold = 2.0
  []
  [adaptive_MC]
    type = AdaptiveMonteCarloDecision
    output_value = conditional/gp_mean
    inputs = 'inputs'
    sampler = sample
    gp_decision = conditional
  []
  [ais_stats]
    type = AdaptiveImportanceStats
    output_value = conditional/gp_mean
    sampler = sample
    flag_sample = 'conditional/flag_sample'
  []
[]

[Trainers]
  [GP_al_trainer]
    type = ActiveLearningGaussianProcess
    covariance_function = 'covar'
    standardize_params = 'true'
    standardize_data = 'true'
    tune_parameters = 'covar:signal_variance covar:length_factor'
    num_iters = 2000
    learning_rate = 0.005
  []
[]

[Surrogates]
  [GP_eval]
    type = GaussianProcessSurrogate
    trainer = GP_al_trainer
  []
[]

[Covariance]
  [covar]
    type = SquaredExponentialCovariance
    signal_variance = 1.0
    noise_variance = 1e-8
    length_factor = '1.0 1.0'
  []
[]

[Executioner]
  type = Transient
[]

[Outputs]
  file_base = 'ais_al'
  [out]
    type = JSON
    execute_system_information_on = NONE
  []
[]

Evaluation of multi - output Gaussian Processes
Input Parameters
Input Files