HPC Cluster
The following instructions aims at setting up a baseline single-user environment for building MOOSE based applications in a job scheduling capable environment.
If instead, you are here interested in allowing MOOSE-based development to be made available to multiple users, please see our Multi-User setup instructions (requires administrative rights). Normal users: please ignore the previous sentence, and continue on...
Pre-Reqs
What ever compiler you choose to use on your cluster (GCC/Clang, MPICH/OpenMPI), the minimum requirement, is that it must be C++17 compatible. If you are unsure, please consult with your system admins for your cluster on which compiler to use (and how to use it).
CMake. A modern version of CMake (>3.5) is required to build some of the meta packages we need to include in PETSc.
Python 3.x Development libraries.
Your cluster will most likely have these two requirements available via some form of environment management software. If you are unfamiliar with how to manage your environment, please consult with your cluster administrators.
Create MOOSE Profile
Use an editor to add the following contents to $HOME/.moose_profile
export CC=mpicc
export CXX=mpicxx
export F90=mpif90
export F77=mpif77
export FC=mpif90
Source the MOOSE Profile
source $HOME/.moose_profile
By sourcing the above file, you are now ready to begin MOOSE-based development.
You will need to perform the above (source $HOME/.moose_profile) for every new terminal session for which you perform work with MOOSE. If you want this to be automatic, add the above to your ~/.bash_profile (or ~/.bashrc or, which ever profile you use on your system)
Obtaining and Building MOOSE
Cloning MOOSE
MOOSE is hosted on GitHub and should be cloned directly from there using git. We recommend creating a directory named projects to put all of your MOOSE related work.
To install MOOSE run the following commands in a terminal.
mkdir ~/projects
cd ~/projects
git clone https://github.com/idaholab/moose.git
cd moose
git checkout master
The master branch of MOOSE is the stable branch that will only be updated after all tests are passing. This protects you from the day-to-day changes in the MOOSE repository.
Compile PETSc
By default, we use PETSc submodule inside of MOOSE as our nonlinear/linear solvers. PETSc can be built using the following script:
cd ~/projects/moose
unset PETSC_DIR PETSC_ARCH
./scripts/update_and_rebuild_petsc.sh
If you prefer to install PETSc into a specified location, use the following:
cd ~/projects/moose
unset PETSC_DIR PETSC_ARCH
./scripts/update_and_rebuild_petsc.sh --prefix=/where/you/want/to/put/petsc
During the follow-up libMesh compile, you need to set PETSC_DIR, that is,
export PETSC_DIR=/where/you/want/to/put/petsc
If PETSc is built successfully, you should see some output like the following:
Now to check if the libraries are working do:
make PETSC_DIR=/your/home/projects/moose/scripts/../petsc PETSC_ARCH=arch-moose check
You could optionally check if PETSc works as follows:
cd ./petsc
make PETSC_DIR=/your/home/projects/moose/scripts/../petsc PETSC_ARCH=arch-moose check
PETSc should produce the output like this:
Running check examples to verify correct installation
Using PETSC_DIR=/your/home/projects/moose/scripts/../petsc and PETSC_ARCH=arch-moose
C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process
C/C++ example src/snes/tutorials/ex19 run successfully with 2 MPI processes
C/C++ example src/snes/tutorials/ex19 run successfully with hypre
C/C++ example src/snes/tutorials/ex19 run successfully with mumps
C/C++ example src/snes/tutorials/ex19 run successfully with superlu_dist
Completed test examples
Compile libMesh
MOOSE directly relies on the libMesh finite-element framework. Because of this strong tie MOOSE contains a particular version of libMesh that we have vetted for our users. To pull down and compile this version of libMesh you simply need to run a script in MOOSE:
cd ~/projects/moose
./scripts/update_and_rebuild_libmesh.sh
Do not use sudo when running update_and_rebuild_libmesh.sh.
Compile and Test MOOSE
cd ~/projects/moose/test
make -j 4
./run_tests -j 4
If the installation was successful you should see most of the tests passing (some tests will be skipped depending on your system environment).
Head back over to the Getting Started page to continue your tour of MOOSE.