tests: make Fortran DType tests assert an explicit success marker (fix loose "10" pass regex) - #189
Merged
Conversation
Replace the loose PASS_REGULAR_EXPRESSION "10" on the five OpenMP_Language_Constructs_Fortran_DType_* tests with an explicit FORTRAN-DTYPE-RESULT marker emitted from the real executable output, add set -euo pipefail so build/run failures fail visibly, and drop a stray cd derived_types in the pointer test (no such directory).
dannybaths
changed the title
Collaborator
|
Making this success marker a little more rigorous -- good |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The five
OpenMP_Language_Constructs_Fortran_DType_*CTest entries gate on a loosePASS_REGULAR_EXPRESSION "10". A bare10can be matched by output unrelated to the test result, for example a build path or dashboard build name that contains10such asmoduleset10, so a test can report PASS without the executable ever producing the correct answer.This change makes the pass criterion honest:
tests/openmp_language_constructs_fortran_dtype_*.shnow runs the real executable, prints its output, and emits an explicit marker:FORTRAN-DTYPE-RESULT: <last non-empty output line>.tests/CMakeLists.txtmatchesFORTRAN-DTYPE-RESULT: +10instead of10. The+tolerates Fortran list-directed leading spaces, for exampleRESULT: 10.set -euo pipefailafter the existing module-detection block, so a failed build or a nonzero executable exit fails the test visibly instead of silently.cd derived_types; there is no7_derived_types/derived_typesdirectory, and underset -ethat failedcdmust not remain.Why
A pass regex of
10is trivially satisfied by unrelated text and hides real build/run failures. Matching an explicit marker printed from the executable output makes these tests trustworthy.Validation
AAC7: MI300A / gfx942, focused no-submit run:
Result: 5/5 pass, each via the explicit marker
FORTRAN-DTYPE-RESULT: 10;100% tests passed, 0 tests failed out of 5. Confirmed the new regex does not match amoduleset10-style path while the old10did.AAC6 no-regression: PASS, Slurm job 19708 on ppac-pl1-s24-16 / PPAC_MI300A_SPX, 5/5 Fortran DType tests passed with
FORTRAN-DTYPE-RESULT: 10.