Setup MooseDocs for SQA
Create CIVET recipes for SQA
Enforce reviews for merges
Performing a release
MooseDocs is utilized to minimize the effort necessary for a MOOSE-based application
These instructions assume that you have documentation setup within your application, which should be the case for most applications.
To test run the following commands, if you have problems please use the MOOSE Discussion forum for assistance.
cd doc
./moosedocs.py build --serve
To allow for the design documentation to be complete, the content of the Multiphysics Object Oriented Simulation Environment (MOOSE) framework and all modules enabled for an application must be added to the "config.yml" within the application "doc" directory.
A basic application that uses the heat conduction module, should have a "Contents" section in the "config.yml" should be as follows.
Content:
- ${ROOT_DIR}/doc/content
- ${MOOSE_DIR}/framework/doc/content
- ${MOOSE_DIR}/modules/heat_conduction/doc/content
To correctly populate MooseObject inheritance lists, the list of include files should also be updated within the MooseDocs "appsyntax" extension section of the "config.yml" file as follows.
Extensions:
MooseDocs.extensions.appsyntax:
executable: ${ROOT_DIR}
includes:
- ${ROOT_DIR}/include
- ${MOOSE_DIR}/framework/include
- ${MOOSE_DIR}/modules/heat_conduction/include
This is not necessary for NQA-1 compliance, but it is good practice for complete documentation.
The MooseDocs "template" extension is leveraged to allow for sharing of Software Quality Assurance (SQA) related documentation to minimize the effort necessary to build NQA-1 compliant software.
To enable the extension, add the following to the "config.yml" file, within the "Extensions" section.
MooseDocs.extensions.template:
active: True
PLN-4005 includes a list of "repository documentation" that are necessary to be compliant with the Idaho National Laboratory (INL) SQA process. Templates exist for a majority of these documents.
Within a MOOSE-based application (e.g., "blackbear"), the following files should be created with in the "doc/content/sqa" directory. The prefix should be a reflection of the application name.
blackbear_cci.md, blackbear_far.md, blackbear_rtm.md, blackbear_scs.md, blackbear_sdd.md, blackbear_sll.md, blackbear_srs.md, blackbear_stp.md, blackbear_vvr.md
The "blackbear_vvr.md" is an optional file. This is the Verification and Validation Report (VVR) that can be used to detail verification and validation tests (in the scientific sense) for an application.
Within each of the template files, load the corresponding "app" template file. For example, within the "blackbear_sdd.md" file the following should be included.
!template load file=app_sdd.md.template app=BlackBear category=blackbear
The "category" should match the prefix of the file names and the "app" be the registered MOOSE-based application name.
Certain templates have required sections and others have optional sections. The required sections will be detailed upon rendering the website with the "build" command.
To aid in monitoring the status of SQA documentation a command-line reporting system exists. To use this system a report configuration file ("sqa_reports.yml") must be created in the "doc" directory of the MOOSE-based application.
This file should contain three sections: "Applications", "Documents", and "Requirements". In the following slides the configuration from the "blackbear" application is presented. The file for any application will be nearly identical, with the application name being the major difference.
The application reports provide feedback regarding the existence of the application design pages, which are associated with the registered objects.
Applications:
blackbear:
exe_directory: ${ROOT_DIR}
exe_name: blackbear
app_types:
- BlackbearApp
content_directory: doc/content
The document reports provide assurance that the necessary SQA documents are in place for conformance with PLN-4005.
Documents:
working_dirs:
- ${BLACKBEAR_DIR}/doc/content
- ${MOOSE_DIR}/framework/doc/content
software_requirements_specification: sqa/blackbear_srs.md
software_design_description: sqa/blackbear_sdd.md
software_test_plan: sqa/blackbear_stp.md
requirements_traceablity_matrix: sqa/blackbear_rtm.md
verification_validation_report: sqa/blackbear_vvr.md
failure_analysis_report: sqa/blackbear_far.md
safety_software_determination: sqa/inl_records.md#safety_software_determination
quality_level_determination: sqa/inl_records.md#quality_level_determination
enterprise_architecture_entry: sqa/inl_records.md#enterprise_architecture_entry
software_quality_plan: sqa/inl_records.md#software_quality_plan
configuration_management_plan: sqa/inl_records.md#configuration_management_plan
asset_management_plan: sqa/inl_records.md#asset_management_plan
verification_validation_plan: sqa/inl_records.md#verification_and_validation_plan
software_library_list: sqa/blackbear_sll.md
communication_and_contact_information: sqa/blackbear_cci.md
software_coding_standards: sqa/blackbear_scs.md
user_manual: syntax/blackbear_index.md # symlink to avoid duplicate file errors
theory_manual: syntax/blackbear_index.md
show_warning: falseThe document reports provide assurance that the test specifications are properly documented with a requirement, design, and issues as described in PLN-4005.
Requirements:
blackbear:
directories:
- test/tests
log_testable: WARNING
show_warning: false
include_non_testable: true
After building an "sqa_reports.yml" configuration file for an application, the reports can be produced by running the MooseDocs executable with the "check" command.
cd docs
./moosedocs.py check
The SQA documentation and report information is added to the "config.yml" using the MooseDocs "sqa" extension. This is accomplished by adding the extension to the "Extensions" section. For example, the following is what should be included for an application that includes the heat conduction module.
MooseDocs.extensions.sqa:
active: True
reports: !include ${ROOT_DIR}/doc/sqa_reports.yml
repos:
default: https://github.com/idaholab/blackbear
moose: https://github.com/idaholab/moose
categories:
framework: !include ${MOOSE_DIR}/framework/doc/sqa_framework.yml
heat_conduction: !include ${MOOSE_DIR}/modules/heat_conduction/doc/sqa_heat_conduction.yml
blackbear: !include ${ROOT_DIR}/doc/sqa_blackbear.yml
The above configuration includes an "sqa_blackbear.yml" file. It provides a connection between the report configuration and the MooseDocs website configuration. This is done in a separate file to allow for applications to share this information.
The included file simply indicates where to locate the tests for the application: in the "tests" spec with the "test/tests" directories.
specs:
- tests
directories:
- ${BLACKBEAR_DIR}/test/tests
The testing results from Continuous Integration for Verification, Enhancement, and Testing (CIVET) can be displayed by enabling the MooseDocs "civet" extension within the "Extensions" sections of the "config.yml" file. The following is an example from the "blackbear" application.
MooseDocs.extensions.civet:
test_results_cache: '/tmp/civet/jobs'
remotes:
blackbear:
url: https://civet.inl.gov
repo: idaholab/blackbear
moose:
url: https://civet.inl.gov
repo: idaholab/moose
location: ${MOOSE_DIR}
To build the complete documentation, including the SQA pages, the following method is required when running the build command.
cd doc
./moosedocs.py build --with-sqa --serve
Necessary CIVET recipes for SQA including building the documentation, running reports, and enabling code coverage.
The following blocks should be added to a recipe that includes the following to ensure that all documentation and SQA checks execute on on "Merge/Pull Requests" and the development branch.
trigger_pull_request = True
trigger_push_branch = devel
[Documentation: SQA]
script = scripts/run_cmd.sh
abort_on_failure = False
allowed_to_fail = False
APP_SUBDIR = doc
RUN_CMD = ./moosedocs.py check
[Documentation: build]
script = scripts/moosedocs.sh
abort_on_failure = False
allowed_to_fail = False
MOOSEDOCS_ARGS = --with-sqa
[Documentation: sync]
script = scripts/copy_to_remote.sh
abort_on_failure = False
allowed_to_fail = False
FROM_DIRECTORY = site
APP_SUBDIR = doc
CIVET_SERVER_POST_COMMENT=1
CHECK_EXISTS = site/index.html
CIVET_SERVER_POST_EDIT_EXISTING = 1
Code coverage can be added by building the the application with coverage enabled, initializing the coverage, running desired tests, and reporting the coverage results.
[build]
script = scripts/build.sh
abort_on_failure = True
COVERAGE = 1
[coverage: initialize]
script = scripts/coverage_initialize.sh
abort_on_failure = False
# Add testing steps here
[coverage: report]
script = scripts/coverage_extract_and_store.sh
COVERAGE_TAG = main
The MOOSE website includes additional information regarding code coverage. Please use the MOOSE Discussion forum for assistance.
PLN-4005 requires that all changes to MOOSE-based application be reviewed by an independent reviewer.
GitHub/GitLab should be configured to enforce the reviews.
PLN-4005 details what must be performed during a release of MOOSE and MOOSE-based applications, and it should be utilized as a reference. This presentation is an introduction to aid in following the required process that is documented in PLN-4005
Select a stable (main/master branch) for release.
Download testing results.
Complete second review.
Document the review by creating "release log."
Update MooseDocs config.yml to point to downloaded results.
Commit the results and log.
Use git tag to label the release.
Push the tag to the main repository.
Any revision of MOOSE or a MOOSE-based application can be selected for a release as long as all the testing is complete and passed.
A MOOSE-based application release must use a released version of MOOSE and dependent applications. When releasing an application a Merge/Pull Request should be created with the MOOSE and application submodules updated to the release tags. The following process is then followed after merging and testing of this change is complete.
Checkout the branch for release and execute the git log command.
$ git log
commit b657ae26aae14fbc91ad2651a364cfad80c2b9ac (HEAD -> master, upstream/master)
Merge: b107961dbd 4812111857
Author: moosetest <[email protected]>
Date: Wed Sep 29 11:22:23 2021 -0600
Merge commit '4812111857205877df7bb943958f9cd1f32b0d27'
Use these hashes to download results from CIVET (see links below). The tag.gz files should be placed in the "release" directory.
PLN-4005 details the necessary steps for performing the release which includes performing a final review of MOOSE or a MOOSE-based application. This review must be performed by the Project Lead and is intended to ensure that all requirements are complete, accurate, and are satisifed and that all SQA documentation is correct and complete.
The "release log" provides evidence that the final review of software was conducted. This log is a text file ("RELEASE_LOG.md") that must be added to the "release" directory of the repository.
Please refer to the current version of PLN-4005 to determine what is necessary. Do not rely on a log from a previous release. The necessary components of the log might have been altered since the previous release.
To ensure that the results displayed on the generated website are correct and pulled from the stored results, the "config.yml" must be updated to disable downloading and enable using the results in the "release" directory. The following are the changes to the "config.yml" for a release of "blackbear".
$ git diff
diff --git a/doc/config.yml b/doc/config.yml
index b46b6c19..a97a9828 100644
--- a/doc/config.yml
+++ b/doc/config.yml
@@ -46,17 +46,21 @@ Extensions:
active: True
MooseDocs.extensions.civet:
- test_results_cache: '/tmp/civet/jobs'
remotes:
blackbear:
+ download_test_results: false
+ test_results_cache: ../release
url: https://civet.inl.gov
repo: idaholab/blackbear
location: ${ROOT_DIR}
moose:
+ download_test_results: false
+ test_results_cache: ../moose/release
url: https://civet.inl.gov
repo: idaholab/moose
location: ${MOOSE_DIR}
After the testing results are downloaded, the release log created, and the "config.yml" is created, the additions must be committed to the repository. It is suggested that the following commit message pattern be followed.
git commit release -m "2021-09-15 release"
The release is completed by tagging it with git and pushing the tag to the main remote repository. The tag must follow the YYYY-MM-DD pattern. The following is an example of creating the tag and pushing the change to the main remote repository (upstream).
git tag 2021-09-15
git push upstream
Depending on your configuration the main repository may be named something other than "upstream."
Although not required by PLN-4005 the website for the release should be uploaded for consumption by users of MOOSE or the MOOSE-based application. Please contact the MOOSE developers for assistance with this process.