INL HPC Pre-Built MOOSE
While operating on one of the INL HPC clusters, there exists the option of using pre-built versions of MOOSE. To request access to these clusters, please follow the instructions on INL's Nuclear Computational Resource Center website.
Once access has been granted, log into Sawtooth, Bitterroot, or Windriver using either HPC OnDemand Interactive Shell services, or directly by following our SSH Primer.
At the time of this writing, the following INL HPC clusters are available for use:
Cluster | Login Nodes | NCPUS | NODES | Max Hours |
---|---|---|---|---|
WindRiver | windriver1 , windriver2 | 56 | 843 | 168 |
Bitterroot | bitterroot1 , bitterroot2 | 56 | 336 | 168 |
Sawtooth | sawtooth1 , sawtooth2 | 48 | 2052 | 168 |
Additional information for each cluster can be found at the INL HPC website.
If you are waiting to be granted access, you can instead follow our Conda Pre-Built MOOSE in the meantime.
Load Modules
Load the following modules:
module load use.moose moose-openmpi
If you receive an error about modules not being known, please make sure you are logged into either WindRiver, Bitterroot, or Sawtooth.
Once loaded, moose
becomes available. You need now only provide input files to run simulations. Example input files are also available while this module is loaded.
Run an Example
A MOOSE installation binary comes with several examples you can run to make sure everything is sound, as well as moving some of the example inputs into a safe location you can play with.
You may query the executable for the available inputs with the following:
moose-opt --show-copyable-inputs
which will show something like this:
The following directories are installable into a user-writeable directory:
chemical_reactions combined contact electromagnetics external_petsc_solver fluid_properties framework fsi functional_expansion_tools geochemistry heat_transfer level_set misc navier_stokes optimization peridynamics phase_field porous_flow ray_tracing rdg reactor reactor_tutorial richards scalar_transport solid_mechanics solid_properties stochastic_tools tensor_mechanics thermal_hydraulics xfem
To install one or more directories of inputs, execute the binary with the "--copy-inputs" flag. e.g.:
$ moose-opt --copy-inputs chemical_reactions
For now, lets copy heat_transfer
into a safe location for editing:
mkdir -p ~/projects/examples
cd ~/projects/examples
moose-opt --copy-inputs heat_transfer
<output trimmed>
Directory successfully copied into ./moose/heat_transfer/
With heat_transfer's examples and inputs copied, move into the heat_transfer directory and instruct moose
to run the tests:
cd ./moose/heat_transfer
moose-opt --run -j 6
Testing will commence and take a few moments to finish. There may be several skipped tests for one reason or another. This is normal. However none of the tests should fail.
Next, we will run a single input file manually, to demonstrate how you will ultimately use moose-opt
. Peruse the subdirectories and find an input file you wish to run:
You can list all available input files by running:
# be sure to be in the heat_transfer's directory
find . -name '*.i'
# cd into the directory containing the input file you wish to run
moose-opt -i <the input file you chose>.i --mesh-only
You will see some information scroll by, and ultimately end back at your prompt. If you perform a directory listing (ls
) you should see an exodus file was generated in the process (a file with the same input filename but with _in.e
suffix).
View Results
You can use HPC OnDemand to view the results file remotely. Head on over to HPC OnDemand Dashboard, and select: Interactive Apps
and then Linux Desktop with Visualization
. Next, select your cluster (such as Sawtooth), the amount of time you believe you need, and then click Launch
.
It may take some time before your 'Visualization Job' becomes available. When it does, simply click on it, and you will be presented a GUI desktop within your web browser. From here, you can open visualization applications (such as Paraview), and open your results file.
To use Paraview, open a terminal by clicking Applications
at the top left, then click Terminal Emulator
. A terminal window will open. Enter the following commands:
module load paraview
paraview
Paraview should open. From here, you can select File
, Open
, and navigate to the directory containing your exodus file, and open it.
In many cases it is more desirable to view results using your local machine. This is done by copying result files from the remote machine to your local machine using scp
or rsync
.
Copying files from an HPC cluster to your machine first requires that you follow the instructions: Remote Access Primer. This workflow is for advanced users, comfortable with modifying their SSH settings, and familiar with their terminal in general.
Example commands you would use with a terminal to copy files from HPC to your local machine's Downloads folder:
cd ~/Downloads
scp <your hpc user id>@hpclogin:<path to where your exodus file is> .
Perform the above command while on your machine. Not while on an HPC machine.
Examples
Continue on to see more examples and tutorials using MOOSE! However, most of the next section is geared towards developing your own application.