LibtorchNeuralNetControlTransfer

Copies a neural network from a trainer object on the main app to a LibtorchNeuralNetControl object on the subapp.

Overview

Transfer which copies a LibtorchArtificialNeuralNet from a neural net control trainer object (say LibtorchDRLControlTrainer.md]) on the main app to a [LibtorchNeuralNetControl object on the sub app.

Input Parameters

control_nameController object name.
C++ Type:std::string
Controllable:No
Description:Controller object name.
trainer_nameTrainer object that contains the neural networks. for different samples.
C++ Type:UserObjectName
Controllable:No
Description:Trainer object that contains the neural networks. for different samples.

Required Parameters

check_multiapp_execute_onTrueWhen false the check between the multiapp and transfer execute on flags is not performed.
Default:True
C++ Type:bool
Controllable:No
Description:When false the check between the multiapp and transfer execute on flags is not performed.
displaced_source_meshFalseWhether or not to use the displaced mesh for the source mesh.
Default:False
C++ Type:bool
Controllable:No
Description:Whether or not to use the displaced mesh for the source mesh.
displaced_target_meshFalseWhether or not to use the displaced mesh for the target mesh.
Default:False
C++ Type:bool
Controllable:No
Description:Whether or not to use the displaced mesh for the target mesh.
execute_onSAME_AS_MULTIAPPThe list of flag(s) indicating when this object should be executed. For a description of each flag, see https://mooseframework.inl.gov/source/interfaces/SetupInterface.html.
Default:SAME_AS_MULTIAPP
C++ Type:ExecFlagEnum
Options:XFEM_MARK, FORWARD, ADJOINT, HOMOGENEOUS_FORWARD, ADJOINT_TIMESTEP_BEGIN, ADJOINT_TIMESTEP_END, NONE, INITIAL, LINEAR, LINEAR_CONVERGENCE, NONLINEAR, NONLINEAR_CONVERGENCE, POSTCHECK, TIMESTEP_END, TIMESTEP_BEGIN, MULTIAPP_FIXED_POINT_END, MULTIAPP_FIXED_POINT_BEGIN, MULTIAPP_FIXED_POINT_CONVERGENCE, FINAL, CUSTOM, SAME_AS_MULTIAPP, POST_ADAPTIVITY
Controllable:No
Description:The list of flag(s) indicating when this object should be executed. For a description of each flag, see https://mooseframework.inl.gov/source/interfaces/SetupInterface.html.
to_multi_appThe name of the MultiApp to transfer the data to
C++ Type:MultiAppName
Controllable:No
Description:The name of the MultiApp to transfer the data to

Optional Parameters

_called_legacy_paramsTrue
Default:True
C++ Type:bool
Controllable:No
control_tagsAdds user-defined labels for accessing object parameters via control logic.
C++ Type:std::vector<std::string>
Controllable:No
Description:Adds user-defined labels for accessing object parameters via control logic.
enableTrueSet the enabled status of the MooseObject.
Default:True
C++ Type:bool
Controllable:Yes
Description:Set the enabled status of the MooseObject.
skip_coordinate_collapsingTrueWhether to skip coordinate collapsing (translation and rotation are still performed, only XYZ, RZ etc collapsing is skipped) when performing mapping and inverse mapping coordinate transformation operations. This parameter should only be set by users who really know what they're doing.
Default:True
C++ Type:bool
Controllable:No
Description:Whether to skip coordinate collapsing (translation and rotation are still performed, only XYZ, RZ etc collapsing is skipped) when performing mapping and inverse mapping coordinate transformation operations. This parameter should only be set by users who really know what they're doing.
use_displaced_meshFalseWhether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.
Default:False
C++ Type:bool
Controllable:No
Description:Whether or not this object should use the displaced mesh for computation. Note that in the case this is true but no displacements are provided in the Mesh block the undisplaced mesh will still be used.

Advanced Parameters

Input Files

(modules/stochastic_tools/examples/libtorch_drl_control/libtorch_drl_control_trainer.i)
(modules/stochastic_tools/test/tests/transfers/libtorch_nn_transfer/libtorch_drl_control_trainer.i)

(modules/stochastic_tools/examples/libtorch_drl_control/libtorch_drl_control_trainer.i)

[StochasticTools]
[]

[Samplers]
  [dummy]
    type = CartesianProduct
    linear_space_items = '0 0.01 1'
  []
[]

[MultiApps]
  [runner]
    type = SamplerFullSolveMultiApp
    sampler = dummy
    input_files = 'libtorch_drl_control_sub.i'
  []
[]

[Transfers]
  [nn_transfer]
    type = LibtorchNeuralNetControlTransfer
    to_multi_app = runner
    trainer_name = nn_trainer
    control_name = src_control
  []
  [r_transfer]
    type = MultiAppReporterTransfer
    from_multi_app = runner
    to_reporters = 'results/center_temp results/env_temp results/reward results/top_flux results/log_prob_top_flux'
    from_reporters = 'T_reporter/center_temp_tend:value T_reporter/env_temp:value T_reporter/reward:value T_reporter/top_flux:value T_reporter/log_prob_top_flux:value'
  []
[]

[Trainers]
  [nn_trainer]
    type = LibtorchDRLControlTrainer
    response = 'results/center_temp results/env_temp'
    control = 'results/top_flux'
    log_probability = 'results/log_prob_top_flux'
    reward = 'results/reward'

    num_epochs = 1000
    update_frequency = 10
    decay_factor = 0.0

    loss_print_frequency = 10

    critic_learning_rate = 0.0001
    num_critic_neurons_per_layer = '64 27'

    control_learning_rate = 0.0005
    num_control_neurons_per_layer = '16 6'

    # keep consistent with LibtorchNeuralNetControl
    input_timesteps = 2
    response_scaling_factors = '0.03 0.03'
    response_shift_factors = '290 290'
    action_standard_deviations = '0.02'

    standardize_advantage = true

    read_from_file = false
  []
[]

[Reporters]
  [results]
    type = ConstantReporter
    real_vector_names = 'center_temp env_temp reward top_flux log_prob_top_flux'
    real_vector_values = '0; 0; 0; 0; 0'
    outputs = csv
    execute_on = timestep_begin
  []
  [reward]
    type = DRLRewardReporter
    drl_trainer_name = nn_trainer
  []
[]

[Executioner]
  type = Transient
  num_steps = 440
[]

[Outputs]
  file_base = output/train_out
  csv = true
  time_step_interval = 10
[]

(modules/stochastic_tools/test/tests/transfers/libtorch_nn_transfer/libtorch_drl_control_trainer.i)

[StochasticTools]
[]

[Samplers]
  [dummy]
    type = CartesianProduct
    linear_space_items = '0 0.01 1'
  []
[]

[MultiApps]
  [runner]
    type = SamplerFullSolveMultiApp
    sampler = dummy
    input_files = 'libtorch_drl_control_sub.i'
  []
[]

[Transfers]
  [nn_transfer]
    type = LibtorchNeuralNetControlTransfer
    to_multi_app = runner
    trainer_name = nn_trainer
    control_name = src_control
  []
  [r_transfer]
    type = MultiAppReporterTransfer
    from_multi_app = runner
    to_reporters = 'results/center_temp results/env_temp results/reward results/left_flux results/log_prob_left_flux'
    from_reporters = 'T_reporter/center_temp_tend:value T_reporter/env_temp:value T_reporter/reward:value T_reporter/left_flux:value T_reporter/log_prob_left_flux:value'
  []
[]

[Trainers]
  [nn_trainer]
    type = LibtorchDRLControlTrainer
    response = 'results/center_temp results/env_temp'
    control = 'results/left_flux'
    log_probability = 'results/log_prob_left_flux'
    reward = 'results/reward'

    num_epochs = 10
    update_frequency = 2
    decay_factor = 0.0

    loss_print_frequency = 3

    critic_learning_rate = 0.0005
    num_critic_neurons_per_layer = '4 2'

    control_learning_rate = 0.0005
    num_control_neurons_per_layer = '4 2'

    # keep consistent with LibtorchNeuralNetControl
    input_timesteps = 2
    response_scaling_factors = '0.03 0.03'
    response_shift_factors = '270 270'
    action_standard_deviations = '0.1'

    read_from_file = false
  []
[]

[Reporters]
  [results]
    type = ConstantReporter
    real_vector_names = 'center_temp env_temp reward left_flux log_prob_left_flux'
    real_vector_values = '0; 0; 0; 0; 0'
    outputs = 'csv_out'
    execute_on = timestep_begin
  []
  [nn_parameters]
    type = DRLControlNeuralNetParameters
    trainer_name = nn_trainer
    outputs = json_out
  []
[]

[Executioner]
  type = Transient
  num_steps = 1
[]

[Outputs]
  file_base = train_out
  [json_out]
    type = JSON
    execute_on = TIMESTEP_BEGIN
    execute_system_information_on = NONE
  []
[]

Overview
Input Parameters
Input Files