MOOSE Tools Requirements Traceability Matrix

This template follows INL template TEM-214, "IT System Requirements Traceability Matrix."

Introduction

Minimum System Requirements

In general, the following is required for MOOSE-based development:

GCC/Clang C++17 compliant compiler (GCC @ 7.5.0, Clang @ 10.0.1 or greater)
- Note: Intel compilers are not supported.
Memory: 8 GBs of RAM for optimized compilation (16 GBs for debug compilation), 2 GB per core execution
Processor: 64-bit x86 or ARM64 (specifically, Apple Silicon)
Disk: 30GB
A POSIX compliant Unix-like operating system, including the two most recent versions of MacOS and most current versions of Linux.
Git version control system
Python @ 3.7 or greater

System Purpose

MOOSE Tools is a set of Python utilities designed to support the MOOSE framework. MOOSE Tools contains two main utilities – MOOSE Documentation System (MooseDocs) and TestHarness – that facilitate MOOSE documentation (e.g., website and presentation building) and code testing. While tied to the framework most directly (and housed within the MOOSE code repository), MOOSE Tools can be used in a more general sense to support other codes. More specific information on the design and structure of MOOSE Tools utilities can be found on the main documentation page.

The design goal of MOOSE Tools is to give code developers access to common tools and resources to help them verify, test, and document their code and simulations, as well as make best use of the input and output data obtained in the process of performing research. To this end, many of the Tools utilities are extensible and flexible, like MOOSE itself, and therefore are designed to be expanded and reconfigured to meet the needs of the end-user.

System Scope

The scope of MOOSE Tools is to provide a set of utilities to support MOOSE development. Namely, verification, testing, documentation, and data analysis and manipulation of MOOSE code and I/O. Tools code is written in a general way, so that it can be extended and tuned to the needs of the user/developer. The major two systems of the Tools utilities are described in the following sections, but more information on the support utiltities also contained within MOOSE tools can be found on the main documentation page.

MooseDocs

The MOOSE Documentation System (MooseDocs) facilitates documentation of the MOOSE and MOOSE Tools code bases, as well as supports the MOOSE SQA practices. It contains many extensions for website rendering, navigation, linking, bibliographic references, and support for integration of code and input file snippets (among many more features), as well as capabilities for development of training and presentation slides and reports.

TestHarness

The TestHarness system is responsible for finding tests and running them. The extended philosophy behind MOOSE testing can be found in the MOOSE Test System documentation page, and this philsophy has driven the creation and design choices of the TestHarness system.

Within MOOSE there are three different testing ideas:

The "tests": which are typically "Regression Tests" consisting of input files and known good outputs ("gold" files).
Unit tests that test the funcationality of small seperable pieces
The TestHarness: a piece of software that was written to _run_ tests and aggregate the results.

The TestHarness integrates with the MOOSE continuous integration (CI) and continuous deployment (CD) workflows to facilitate testing across multiple operating systems and hardware architectures.

Assumptions and Dependencies

The software should be designed with the fewest possible constraints. Ideally the software should run on a wide variety of evolving hardware so it should follow well-adopted standards and guidelines. The software should run on any POSIX compliant system containing the Python language and interpreter. The main interface for the software will be command line based with no assumptions requiring advanced terminal capabilities such as coloring and line control.

Pre-test Instructions/Environment/Setup

Ideally all testing should be performed on a clean test machine following one of the supported configurations set up by the test system engineer. Testing may be performed on local workstations and cluster systems containing supported operating systems.

The repository should be clean prior to building and testing. When using "git" this can be done by doing a force clean in the main repository and each one of the submodules:


git clean -xfd
git submodule foreach 'git clean -xfd'

All tests must pass in accordance with the type of test being performed. This list can be found in the Software Test Plan.

System Requirements Traceability

Functional Requirements

python: Test
2.1.1The system shall include a tool for creating spatial convergence plots using method of manufactured solutions.
Specification(s): spatial
Design: Method of Manufactured Solutions (MMS)
Issue(s): #13181
Collection(s): FUNCTIONAL
Type(s): CheckFiles
2.1.2The system shall include a tool for writing spatial convergence data to a file using method of manufactured solutions.
Specification(s): spatial_csv
Design: Method of Manufactured Solutions (MMS)
Issue(s): #13181
Collection(s): FUNCTIONAL
Type(s): CSVDiff
Prerequisite(s): 2.1.1
2.1.3The system shall include a tool for creating temporal convergence plots using method of manufactured solutions.
Specification(s): temporal
Design: Method of Manufactured Solutions (MMS)
Issue(s): #13181
Collection(s): FUNCTIONAL
Type(s): CheckFiles
2.1.4The system shall include a tool for writing temporal convergence data to a file using method of manufactured solutions.
Specification(s): temporal_csv
Design: Method of Manufactured Solutions (MMS)
Issue(s): #13181
Collection(s): FUNCTIONAL
Type(s): CSVDiff
Prerequisite(s): 2.1.3
2.1.5The system shall include a tool to output solution and forcing functions.
Specification(s): exact
Design: Method of Manufactured Solutions (MMS)
Issue(s): #13181
Collection(s): FUNCTIONAL
Type(s): RunApp
2.1.6The system shall include a tool for computing forcing functions for the method of manufactured solutions.
Specification(s): evaluate
Design: Method of Manufactured Solutions (MMS)
Issue(s): #13181
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.7The system shall include a TokenComponent and RenderComponent objects for creating MooseDocs extensions.
Specification(s): components
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.8The system shall include lexer object capable of recursively processing text.
Specification(s): lexer
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.9The system shall include a reader object for reading files.
Specification(s): reader
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.10The system shall include a translator for converting documentation formats.
Specification(s): translator
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.12The system shall include a utility for generating documentation stub pages.
Specification(s): generate
Design: MOOSE Tools
Issue(s): #16155
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.13The system shall include a utility for displaying the application syntax tree.
Specification(s): syntax
Design: MOOSE Tools
Issue(s): #16155
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.14The system shall include a utility for converting markdown documentation to other formats.
Specification(s): build
Design: MOOSE Tools
Issue(s): #16807 #18137
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.15The system shall include a utility to initialize documentation items.
Specification(s): init
Design: MOOSE Tools
Issue(s): #16868
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.16The system shall have a storage structure for holding ordered items by name.
Specification(s): storage
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.17The system shall include utility for parsing key, value pairs from a string.
Specification(s): parse_settings
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.18The system shall include special exceptions for use during tokenization and rendering.
Specification(s): exceptions
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.19The system shall include a utility for creating text boxes suitable for terminal output.
Specification(s): box
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.20The system shall include mixin classes for creating objects that require configuration data, translator objects, or component objects.
Specification(s): mixins
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.21The system shall include the ability to load extensions dynamically.
Specification(s): load_extensions
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.22The system shall include a YAML based configuration.
Specification(s): load_config
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.23The system shall include a database object for storing information regarding source files.
Specification(s): class_database
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.24The system shall include the ability to build a file tree from markdown files.
Specification(s): get_pages
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.25The system shall include a utility for locating filenames using a fuzzy search.
Specification(s): project_find
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.26The system shall include a utility to locate a filename using a fuzzy search and error if none or multiple are located.
Specification(s): check_filenames
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.27The system shall include a utility to conditionally add dependencies to output of documentation based on the existence of certain content.
Specification(s): has_tokens
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #16789
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.28
The system shall include documentation tools that provide support for
1. markdown-like syntax,
2. special characters,
3. comments within markdown syntax,
4. acronyms within markdown syntax,
5. building software quality documents,
6. automatically cross-linking files,
7. linking to continuous integration testing results,
8. listing and linking to content within and across documents,
9. automatically retrieving application syntax,
10. creating alert text boxes,
11. displaying images and videos,
12. creating source code listings,
13. displaying image galleries,
14. template files,
15. creating syntax examples,
16. conditional operations,
17. automatically retrieving python syntax,
18. create modal website content,
19. support equations,
20. displaying algorithmic pseudo-code,
21. include date/time operations, and
22. include git repository information.
Specification(s): all/core, all/special, all/comment, all/acronym, all/sqa, all/autolink, all/civet, all/content, all/appsyntax, all/alert, all/media, all/listing, all/gallery, all/template, all/devel, all/ifelse, all/pysyntax, all/modal, all/katex, all/algorithm, all/datetime, all/gitutils
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699 #9638 #12049 #13567 #13611 #15232 #15927 #16622
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.29The system shall include a test that ensures that all python files within the tests directories have an associated test specification.
Specification(s): spec
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.30The system shall be able to successfully translate the standalone form of the subsite test configuration by using the default build command.
Specification(s): build
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #18137
Collection(s): FUNCTIONAL
Type(s): RunCommand
2.1.31
The system shall be able to use each of the available Executioner types (serial, queue-based parallel, barrier-based parallel, and pipe-based parallel schemes) to translate markdown syntax to:
1. HTML;
2. HTML with Materialize support; and
3. LaTeX.
Specification(s): verify/html, verify/materialize, verify/latex
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
2.1.32
The system shall be able to successfully translate multiple configurations by using the build command with the --config option such that
1. the build routines for the main test site precede those of the subsite, as well as
2. in the reverse order but with the --fast option added for testing efficiency.
Specification(s): multiconfig_build/site_subsite, multiconfig_build/subsite_site
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #18137
Collection(s): FUNCTIONAL
Type(s): RunCommand
2.1.33The system shall include a tree node object with arbitrary property creation.
Specification(s): base
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.34The system shall include tree object suitable for abstract syntax trees.
Specification(s): tokens
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.35The system shall include tree object suitable for HTML.
Specification(s): html
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.36The system shall include tree object suitable for latex.
Specification(s): latex
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.37The system shall include tree object suitable for file structures.
Specification(s): page
Design: MOOSE Documentation System (MooseDocs)
Issue(s): #6699
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.38The system shall include data structures for reporting requirement information.
Specification(s): Requirement
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.39The system shall include a tool for checking application syntax for software quality documentation.
Specification(s): check_syntax
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.40The system shall include a tool for gathering requirement information for software quality documentation.
Specification(s): get_requirements
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.41The system shall include a tool for testing requirement information for software quality documentation.
Specification(s): check_requirements
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.42The system shall have reporting tools for software quality.
Specification(s): SQAReport
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.43
The system shall have tools for reporting software quality status that includes reports for monitoring
1. files and websites;
2. tests and requirements; and
3. design content.
Specification(s): reports/SQADocumentReport, reports/SQARequirementReport, reports/SQAMooseAppReport
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.44The system shall include a tool for gathering multiple software quality reports.
Specification(s): get_reports
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.45The system shall include a utility to enable the recording of logging messages without display.
Specification(s): silent_logging
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.46The system shall include a utility to enable control the of logging messages for software quality reports.
Specification(s): LogHelper
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.47The system shall include documented requirement collections and SQA template files.
Specification(s): documents
Design: MOOSE Tools
Issue(s): #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.48The system shall include the necessary components for building a syntax tree from application information.
Specification(s): base
Design: MOOSE Tools
Issue(s): #6699 #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.1.49The system shall include a utility for generating a complete syntax tree from an application executable.
Specification(s): tree
Design: MOOSE Tools
Issue(s): #6699 #12049
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest

python: Tests
2.2.1
The system shall include a utility for building tree structures using a scripting language that includes
1. a node object,
2. a tree iteration function, and
3. search algorithms.
Specification(s): tree/Node, tree/iterator, tree/search
Design: moosetree package
Issue(s): #13657
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.2The system shall contain python utilities that include a messaging interface.
Specification(s): mooseMessage
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.3The system shall contain python utilities that include a messaging interface capable of creating a dialog window.
Specification(s): mooseMessageDialog
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.4The system shall contain python utilities for reading CSV data via pandas.DataFrame.
Specification(s): moose_data_frame
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.5The system shall contain python utilities for reading postprocessor data.
Specification(s): postprocessors
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.6The system shall contain python utilities for reading vector postprocessor data.
Specification(s): vector_postprocessors
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.7The system shall contain python utilities for reading reporter data.
Specification(s): reporters
Design: MOOSE Tools
Issue(s): #17391
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.8The system shall contain python utilities for converting camel case text to underscore separated text.
Specification(s): camel
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.9The system shall contain python utilities for reading YAML files.
Specification(s): yaml_load
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.10The system shall contain python utilities for breaking a list of items into a specified number of chunks.
Specification(s): make_chunks
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.11The system shall include a utility for locating a MOOSE-based application executable.
Specification(s): find_moose_executable
Design: MOOSE Tools
Issue(s): #15017
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.12The system shall include utilities for executing version control system commands.
Specification(s): gitutils
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.13The system shall include a utility running an executable.
Specification(s): run_executable
Design: MOOSE Tools
Issue(s): #15996
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.14The system shall include a tool for accessing CIVET testing results.
Specification(s): civet_results
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.15The system shall contain python utilities for reading PerfGraphReporter data.
Specification(s): perfgraph
Design: MOOSE Tools
Issue(s): #16256
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.16The system shall contain python utilities for reading compared csv files.
Specification(s): csvdiff
Design: MOOSE Tools
Issue(s): #20032
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.17The system shall contain a python postprocessor class to combine CSV files together.
Specification(s): combine_csv
Design: MOOSE Tools
Issue(s): #13988
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.18The system shall contain a python utility for parsing hierarchical input text (HIT) files.
Specification(s): parser
Design: MOOSE Tools
Issue(s): #11189
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.19The system shall contain a python utility for tokenizing hierarchical input text (HIT) files.
Specification(s): tokenize
Design: MOOSE Tools
Issue(s): #15889
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.20The system shall include examples for reading, writing, and manipulating input file syntax using python.
Specification(s): examples
Design: MOOSE Tools
Issue(s): #16622
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.21The system shall report a non-failing status after a predetermined time of no activity
Specification(s): long_running
Design: TestHarness
Issue(s): #9280
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.22The system shall support the output of the longest running jobs
Specification(s): longest_jobs
Design: TestHarness
Issue(s): #16752
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.26The system shall restrict tests based on the available dual number derivative vector size
Specification(s): min_ad_size
Design: TestHarness
Issue(s): #427
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.28The system shall not perform a test if said test has a skipped dependency
Specification(s): dependency_skip
Design: TestHarness
Issue(s): #427
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.32The system shall report deleted tests as failures when specified with additional –extra-info optionsIn all other cases, deleted tests will be treated as skipped tests
Specification(s): deleted
Design: TestHarness
Issue(s): #427
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.33The system shall perform all operations required of the TestHarness except executing a test
Specification(s): dry_run
Design: TestHarness
Issue(s): #8637
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.34The system shall run only tests designated with display_required.
Specification(s): dislpay_required
Design: TestHarness
Issue(s): #8700 #8701
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.35The system shall allow users to ignore and override specified prerequisites
Specification(s): ignore
Design: TestHarness
Issue(s): #427
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.40The system shall skip a test if required application is unavailable
Specification(s): required_apps
Design: TestHarness
Issue(s): #11095
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.41The system shall only perform the validation of test results without executing the test itself
Specification(s): should_execute
Design: TestHarness
Issue(s): #9932
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.42The system shall skip syntax only tests if instructed to do so
Specification(s): report_skipped
Design: TestHarness
Issue(s): #9359
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.43The system shall properly run tests using distributed mesh options
Specification(s): distributed_mesh
Design: TestHarness
Issue(s): #9181
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.44The system shall supply the necessary resources a test requires, and report when these resources are insufficient to run said test
Specification(s): allocations
Design: TestHarness
Issue(s): #10272
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.45The system shall print all caveats pertaining to the test involved
Specification(s): extra_info
Design: TestHarness
Issue(s): #10272
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.47The system shall allow the test scheduler to gracefully skip tests having the same name contained within the same tests file for further checks against the test file parser.
Specification(s): duplicate_test_names
Design: TestHarness
Issue(s): #10424
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.48The system shall perform normal operating procedures on a single provided test spec file
Specification(s): arbitrary_tests
Design: TestHarness
Issue(s): #11076
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.49The system shall write the output (stdout|stderr) that an executed test generated to a file as designated by user supplied arguments
Specification(s): write_results
Design: TestHarness
Issue(s): #11116
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
Prerequisite(s): 2.2.24
2.2.50The system shall be able to perform recovery of a test
Specification(s): recover_tests
Design: TestHarness
Issue(s): #11492
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.51The system shall be able to submit jobs to a PBS third party scheduler
Specification(s): pbs_tests
Design: TestHarness
Issue(s): #12138
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.52The system shall trim output once threshold has exceeded
Specification(s): trim_output
Design: TestHarness
Issue(s): #12167
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.53The system shall detect and report race conditions that exist in the supplied tests
Specification(s): race_conditions
Design: TestHarness
Issue(s): #13186
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.54The system shall detect and report unreadable output in executed commands
Specification(s): unreadable_output
Design: TestHarness
Issue(s): #14370
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.55The system shall run only tests which previously have failed
Specification(s): failed_tests
Design: TestHarness
Issue(s): #14512
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.56The system shall support restrictions based on the python version available.
Specification(s): python_version
Design: TestHarness
Issue(s): #13903
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.57The system shall be able to compare computed values against measured data using mean value and standard deviation
Specification(s): csvvalidationtester
Design: TestHarness
Issue(s): #14511
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.58The system shall be able to run tests in relative path directories supplied by the spec file
Specification(s): working_directory
Design: TestHarness
Issue(s): #14962
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.60The system shall perform a test after all other tests have passed if specified to do so
Specification(s): do_last
Design: TestHarness
Issue(s): #15230
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.62The system shall be able to replay last results
Specification(s): test_show_last_results
Design: TestHarness
Issue(s): #22545
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.63The system shall be able to evaluate a given test with a user-supplied evaluation function.
Specification(s): custom_eval
Design: TestHarness
Issue(s): #22946
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.64The system shall test for a valid json output dump
Specification(s): test_failed_json
Design: TestHarness
Issue(s): #21967
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest
2.2.65The system shall skip tests not capable of being run depending on binary installation type
Specification(s): test_install_type
Design: TestHarness
Issue(s): #24195
Collection(s): FUNCTIONAL
Type(s): PythonUnitTest

Usability Requirements

No requirements of this type exist for this application, beyond those of its dependencies.

Performance Requirements

No requirements of this type exist for this application, beyond those of its dependencies.

System Interface Requirements

No requirements of this type exist for this application, beyond those of its dependencies.

References

No citations exist within this document.


[spatial]
  type = CheckFiles
  command = mms_spatial.py
  check_files = 'mms_spatial.png'
  requirement = "The system shall include a tool for creating spatial convergence plots using method of manufactured solutions."
  valgrind = none
  required_python_packages = 'sympy'
[]


[spatial_csv]
  type = CSVDiff
  command = mms_spatial.py
  recover = false # this is not running a MooseApp, recover doesn't make any sense
  csvdiff = 'mms_spatial_first.csv mms_spatial_second.csv'
  requirement = "The system shall include a tool for writing spatial convergence data to a file using method of manufactured solutions."
  prereq = spatial
  valgrind = none
  required_python_packages = 'sympy'
[]


[temporal]
  type = CheckFiles
  command = mms_temporal.py
  check_files = 'mms_temporal.png'
  requirement = "The system shall include a tool for creating temporal convergence plots using method of manufactured solutions."
  valgrind = none
  required_python_packages = 'sympy'
[]


[temporal_csv]
  type = CSVDiff
  command = mms_temporal.py
  recover = false # this is not running a MooseApp, recover doesn't make any sense
  csvdiff = 'mms_temporal_first.csv mms_temporal_second.csv'
  requirement = "The system shall include a tool for writing temporal convergence data to a file using method of manufactured solutions."
  prereq = temporal
  valgrind = none
  required_python_packages = 'sympy'
[]


[exact]
  type = RunApp
  command = mms_exact.py
  expect_out = 'type = ParsedFunction'
  requirement = "The system shall include a tool to output solution and forcing functions."
  valgrind = none
  required_python_packages = 'sympy'
[]


[evaluate]
  type = PythonUnitTest
  input = test_mms.py
  requirement = "The system shall include a tool for computing forcing functions for the method of manufactured solutions."
  required_python_packages = 'sympy'
[]


[components]
  type = PythonUnitTest
  input = test_components.py
  requirement = "The system shall include a TokenComponent and RenderComponent objects for creating MooseDocs extensions."
  required_python_packages = 'mock'
[]


[mixins]
  type = PythonUnitTest
  input = test_mixins.py
  requirement = "The system shall include mixin classes for creating objects that require configuration data, translator objects, or component objects."
[]


[check_filenames]
  type = PythonUnitTest
  input = test_check_filenames.py
  requirement = "The system shall include a utility to locate a filename using a fuzzy search and error if none or multiple are located."
[]


[has_tokens]
  type = PythonUnitTest
  input = test_has_tokens.py
  requirement = "The system shall include a utility to conditionally add dependencies to output of documentation based on the existence of certain content."
  issues = '#16789'
[]


[./spec]
  type = PythonUnitTest
  input = test_spec_files.py
  requirement = "The system shall include a test that ensures that all python files within the tests directories have an associated test specification."
[../]


[build]
  type = RunCommand
  command = 'python moosedocs.py build'
  requirement = "The system shall be able to successfully translate the standalone form of the subsite test configuration by using the default build command."
[]


[html]
  detail = "HTML;"

  [serial]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form html --executioner MooseDocs.base.Serial'
  []
  [queue]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form html --executioner MooseDocs.base.ParallelQueue'
    prereq = verify/html/serial
  []
  [barrier]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form html --executioner MooseDocs.base.ParallelBarrier'
    prereq = verify/html/queue
  []
  [pipe]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form html --executioner MooseDocs.base.ParallelPipe'
    prereq = verify/html/barrier
  []
[]


[materialize]
  detail = "HTML with Materialize support; and"

  [serial]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form materialize --executioner MooseDocs.base.Serial'
  []
  [queue]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form materialize --executioner MooseDocs.base.ParallelQueue'
    prereq = verify/materialize/serial
  []
  [barrier]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form materialize --executioner MooseDocs.base.ParallelBarrier'
    prereq = verify/materialize/queue
  []
  [pipe]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form materialize --executioner MooseDocs.base.ParallelPipe'
    prereq = verify/materialize/barrier
  []
[]


[latex]
  detail = "LaTeX."

  [serial]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form latex --executioner MooseDocs.base.Serial'
  []
  [queue]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form latex --executioner MooseDocs.base.ParallelQueue'
    prereq = verify/latex/serial
  []
  [barrier]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form latex --executioner MooseDocs.base.ParallelBarrier'
    prereq = verify/latex/queue
  []
  [pipe]
    type = RunCommand
    command = 'python3 moosedocs.py verify --form latex --executioner MooseDocs.base.ParallelPipe'
    prereq = verify/latex/barrier
  []
[]


[site_subsite]
  type = RunCommand
  command = 'python3 moosedocs.py build --config config.yml subsite_config.yml'
  detail = "the build routines for the main test site precede those of the subsite, as well as"
[]


[subsite_site]
  type = RunCommand
  command = 'python3 moosedocs.py build --fast --config subsite_config.yml config.yml'
  prereq = multiconfig_build/site_subsite
  detail = "in the reverse order but with the `--fast` option added for testing efficiency."
[]


[get_requirements]
  type = PythonUnitTest
  input = test_get_requirements.py
  requirement = "The system shall include a tool for gathering requirement information for software quality documentation."
[]


[check_requirements]
  type = PythonUnitTest
  input = test_check_requirements.py
  requirement = "The system shall include a tool for testing requirement information for software quality documentation."
[]


[mooseMessageDialog]
  type = PythonUnitTest
  input = test_mooseMessageDialog.py
  display_required = True
  requirement = "The system shall contain python utilities that include a messaging interface capable of creating a dialog window."
[]


[vector_postprocessors]
  type = PythonUnitTest
  input = test_VectorPostprocessorReader.py
  separate = True
  requirement = "The system shall contain python utilities for reading vector postprocessor data."
[]


[find_moose_executable]
  type = PythonUnitTest
  input = test_find_moose_executable.py
  issues = "#15017"
  requirement = "The system shall include a utility for locating a MOOSE-based application executable."
[]


[perfgraph]
  type = PythonUnitTest
  input = test_PerfGraphReporterReader.py
  separate = True
  requirement = "The system shall contain python utilities for reading PerfGraphReporter data."
  issues = "#16256"
[]


[examples]
  type = PythonUnitTest
  input = test_examples.py

  issues = '#16622'
  requirement = "The system shall include examples for reading, writing, and manipulating input file syntax using python."
[]


[deleted]
  type = PythonUnitTest
  input = test_Deleted.py
  requirement = "The system shall report deleted tests as failures when specified with additional --extra-info optionsIn all other cases, deleted tests will be treated as skipped tests"
[]


[should_execute]
  type = PythonUnitTest
  input = test_ShouldExecute.py
  requirement = "The system shall only perform the validation of test results without executing the test itself"
  issues = '#9932'
[]


[allocations]
  type = PythonUnitTest
  input = test_Allocations.py
  requirement = "The system shall supply the necessary resources a test requires, and report when these resources are insufficient to run said test"
  issues = '#10272'
[]


[duplicate_test_names]
  type = PythonUnitTest
  input = test_DuplicateTestNames.py
  requirement = "The system shall allow the test scheduler to gracefully skip tests having the same name contained within the same tests file for further checks against the test file parser."
  issues = '#10424'
[]


[arbitrary_tests]
  type = PythonUnitTest
  input = test_ArbitrarySpecFile.py
  requirement = "The system shall perform normal operating procedures on a single provided test spec file"
  issues = '#11076'
[]


[write_results]
  type = PythonUnitTest
  input = test_WriteResults.py
  prereq = 'diff'
  requirement = "The system shall write the output (stdout|stderr) that an executed test generated to a file as designated by user supplied arguments"
  issues = '#11116'
[]


[race_conditions]
  type = PythonUnitTest
  input = test_RaceConditions.py
  requirement = "The system shall detect and report race conditions that exist in the supplied tests"
  issues = '#13186'
[]


[csvvalidationtester]
  type = PythonUnitTest
  input = test_CSVValidationTester.py
  requirement = "The system shall be able to compare computed values against measured data using mean value and standard deviation"
  issues = '#14511'
[]


[working_directory]
  type = PythonUnitTest
  input = test_WorkingDirectory.py
  requirement = "The system shall be able to run tests in relative path directories supplied by the spec file"
  issues = '#14962'
[]


[custom_eval]
  type = PythonUnitTest
  input = test_CustomEval.py
  issues = '#22946'
  requirement = "The system shall be able to evaluate a given test with a user-supplied evaluation function."
[]


[test_install_type]
  type = PythonUnitTest
  input = test_InstallType.py
  requirement = "The system shall skip tests not capable of being run depending on binary installation type"
  issues = '#24195'
[]

Introduction
System Requirements Traceability
References