MOOSE provides the ability to pre-split a mesh into several chunks for use with distributed mesh. This can be useful if the whole mesh is too big to fit into memory. A split-mesh workflow involves generating the mesh split configuration(s) and then telling moose to use them when you run simulations. This is described in the sections below.
Generating Split Configurations
You can run MOOSE with the
--split-mesh <split1>[,<split2>]... flag which takes a comma-separated list of describing each split with an integer number of chunks to split the mesh into.
$ moose-app-opt -i your_input.i --split-mesh 13,42,1000 Splitting 13 ways... - writing 13 files per process... Splitting 42 ways... - writing 42 files per process... ...
This will create three split configurations for your mesh that include 13, 42, and 1000 chunks each. If your input file uses the mesh file
foo.e, this process will generate a directory named
foo.cpr. In general, you should neither rename this directory nor modify its contents. All split configurations for the mesh will be stored in this directory (even across independent splitting operations).
For non-file-based mesh cases (e.g. GeneratedMesh), you will need to tell MOOSE what name to use for the split mesh files it will generate using the
--split-file <file_name> flag; in general, you should use the
.cpr file extension here. For example:
$ moose-app-opt -i your_input.i --split-mesh 42 --split-file foo.cpr
You can also run the splitting operation itself in parallel on several processors to speed up the operation:
$ mpiexec -n 9 -i moose-app-opt your_input.i --split-mesh 42 Splitting 42 ways... - writing 5 files per process...
This will generate the same split configuration as the 42 chunk split generated by the first command but just generates it in parallel.
Using Split Meshes
To use a mesh split configuration use the
--use-split flag (which takes no arguments):
$ mpiexec -n 42 moose-app-opt -i your_input.i --use-split
This will cause MOOSE to look for a mesh split configuration with 42 chunks. If one exists, MOOSE will use it running in a distributed mesh mode, otherwise an error will occur. Note that you do not need to modify your input file - MOOSE automatically switches to using the pre-split distributed mesh regardless of the mesh type specified in the input file. If your input file does not use a file-based mesh, you will need to specify a split mesh file name using the
--split-file <file_name> flag just as you did when splitting the mesh:
$ mpiexec -n 42 moose-app-opt -i your_input.i --use-split --split-file foo.cpr
The mesh splitter commands do not work with DistributedMesh. You must only split with a ReplicatedMesh.