DRLControlNeuralNetParameters

Overview

Converts the parameters of a LibtorchArtificialNeuralNet within a LibtorchDRLControlTrainer into a vector which can be used for the visualization of the evolution of different parameters over the training process.

Example Syntax

[Reporters<<<{"href": "../../../syntax/Reporters/index.html"}>>>]
  [storage]
    type = StochasticReporter<<<{"description": "Storage container for stochastic simulation results coming from Reporters.", "href": "../../reporters/StochasticReporter.html"}>>>
    parallel_type<<<{"description": "This parameter will determine how the stochastic data is gathered. It is common for outputting purposes that this parameter be set to ROOT, otherwise, many files will be produced showing the values on each processor. However, if there are lot of samples, gathering on root may be memory restrictive."}>>> = ROOT
    outputs<<<{"description": "Vector of output names where you would like to restrict the output of variables(s) associated with this object"}>>> = none
  []
  [nn_parameters]
    type = DRLControlNeuralNetParameters
    trainer_name = nn_trainer
    outputs = json_out
  []
[]

warning

The detailed documentation of this object is only available when Moose is compiled with Libtorch. For instructions on how to compile Moose with Libtorch, visit the general installation webpage or click here.

(moose/modules/stochastic_tools/test/tests/transfers/libtorch_nn_transfer/libtorch_drl_control_trainer.i)

[StochasticTools]
[]

[Samplers]
  [dummy]
    type = CartesianProduct
    linear_space_items = '0 0.01 1'
  []
[]

[MultiApps]
  [runner]
    type = SamplerFullSolveMultiApp
    sampler = dummy
    input_files = 'libtorch_drl_control_sub.i'
  []
[]

[Transfers]
  [nn_transfer]
    type = LibtorchNeuralNetControlTransfer
    to_multi_app = runner
    trainer_name = nn_trainer
    control_name = src_control
  []
  [r_transfer]
    type = SamplerReporterTransfer
    from_multi_app = runner
    sampler = dummy
    stochastic_reporter = storage
    from_reporter = 'T_reporter/center_temp_tend:value T_reporter/env_temp:value T_reporter/reward:value T_reporter/left_flux:value T_reporter/log_prob_left_flux:value'
  []
[]

[Trainers]
  [nn_trainer]
    type = LibtorchDRLControlTrainer
    observation = 'storage/r_transfer:T_reporter:center_temp_tend:value storage/r_transfer:T_reporter:env_temp:value'
    control = 'storage/r_transfer:T_reporter:left_flux:value'
    log_probability = 'storage/r_transfer:T_reporter:log_prob_left_flux:value'
    reward = 'storage/r_transfer:T_reporter:reward:value'

    num_epochs = 10
    update_frequency = 2
    decay_factor = 0.0

    loss_print_frequency = 3

    critic_learning_rate = 0.0005
    num_critic_neurons_per_layer = '4 2'

    control_learning_rate = 0.0005
    num_control_neurons_per_layer = '4 2'

    # keep consistent with LibtorchNeuralNetControl
    input_timesteps = 2
    observation_scaling_factors = '0.03 0.03'
    observation_shift_factors = '270 270'
    action_scaling_factors = 100

    read_from_file = false
    shift_outputs = false
  []
[]

[Reporters]
  [storage]
    type = StochasticReporter
    parallel_type = ROOT
    outputs = none
  []
  [nn_parameters]
    type = DRLControlNeuralNetParameters
    trainer_name = nn_trainer
    outputs = json_out
  []
[]

[Executioner]
  type = Transient
  num_steps = 1
[]

[Outputs]
  file_base = train_out
  [json_out]
    type = JSON
    execute_on = TIMESTEP_BEGIN
    execute_system_information_on = NONE
  []
[]

Overview
Example Syntax

Theory

Usage

Development

DRLControlNeuralNetParameters

Overview

Example Syntax