Skip to content

Fix invalid integration comparisons and reference-generation failures - #8138

Open
cheerly-pku wants to merge 2 commits into
deepmodeling:developfrom
cheerly-pku:integration-reference-hardening-review
Open

cheerly-pku wants to merge 2 commits into
deepmodeling:developfrom
cheerly-pku:integration-reference-hardening-review

Conversation

@cheerly-pku

@cheerly-pku cheerly-pku commented Oct 13, 2026 •

Copy link
Copy Markdown

What's changed?

Integration comparisons can currently accept extra unmatched data lines or
non-finite values. Reference generation can also record a failed comparison as
an expected result, or report success after a collector failed. This change
rejects those invalid comparisons and makes reference generation fail with a
nonzero status while removing incomplete result.ref files.

The collectors preserve failed checks as status entries in ordinary
result.out output. Numerical tolerances and the existing Wannier90 entry-wise
comparison policy are unchanged. Trailing blank lines are ignored; missing or
shifted data and interior blank lines remain significant.

Linked Issue

No separate issue. Extracted from #8042 following review so changes affecting
all integration tests can be reviewed independently of PPCG. This PR contains
no solver implementation or reference-value changes.

Unit Tests and/or Case Tests for my changes

Commands rerun for merge commit 657582441 (merging develop at
8b546bdf3, without rebasing or force-pushing):

cd tests/integrate
PATH=/usr/bin:/bin OMP_NUM_THREADS=1 python3 -m unittest -v validation_tools/test_compare_file.py
PATH=/usr/bin:/bin OMP_NUM_THREADS=1 bash validation_tools/test_catch_properties.sh
cd ../..
PATH=/usr/bin:/bin OMP_NUM_THREADS=1 python3 -m unittest discover -s tests/integrate/validation_tools -p test_librpa_wavefunctions.py -v
bash -n tests/integrate/Autotest.sh tests/integrate/validation_tools/*.sh
python3 tools/03_code_analysis/agent_governance_check.py --base deepmodeling/develop --head HEAD --format text
git diff --check deepmodeling/develop...HEAD

Results: 7/7 comparison tests and 8/8 upstream LibRPA parser tests pass;
the expanded shell regression and syntax check exit 0; the PR-diff governance
check reports no findings; diff whitespace check passes. The shell tests
exercise missing outputs, failed comparisons, missing/malformed descriptors,
cleanup of partial references, and run_check.sh/Autotest.sh failure status.
System Python with NumPy was selected for the malformed-descriptor test.

Conflict resolution preserves upstream vxc.ref / OUT.autotest/vxc.txt
naming while retaining failed-comparison handling. New shell coverage checks
both Gamma-only and multi-k VXC success/mismatch/missing-output cases, plus
upstream's -p dedicated-validator mode, custom MPI launcher and pre/post
flags, OFML single-rank override, and commented/blank case-list entries.
No reference values, tolerances, or PPCG solver changes are included.

The actual tests/integrate/CMakeLists.txt was also exercised through a
temporary CMake driver (project(... NONE), enable_testing(),
ABACUS_TEST_DIR=<checkout>/tests, and add_subdirectory of integrate):

PATH=/usr/bin:/bin cmake -S /tmp/pr8138-conflict-check.5hibZt -B /tmp/pr8138-conflict-check.5hibZt/with-python -DPython3_EXECUTABLE=/usr/bin/python3
PATH=/usr/bin:/bin OMP_NUM_THREADS=1 ctest --test-dir /tmp/pr8138-conflict-check.5hibZt/with-python -V -R '^(compare_file_test|catch_properties_test)$'
PATH=/usr/bin:/bin cmake -S /tmp/pr8138-conflict-check.5hibZt -B /tmp/pr8138-conflict-check.5hibZt/no-python -DCMAKE_DISABLE_FIND_PACKAGE_Python3=TRUE
ctest --test-dir /tmp/pr8138-conflict-check.5hibZt/no-python -N

Both registered regression tests pass (CTest 2/2). Python-disabled configuration
also succeeds and explicitly skips those two tests, preserving upstream's
removal of the hard Python configure dependency. The ordinary integration-test
entry remains registered; it was not run by this temporary driver.

The full ABACUS integration suite was not rerun on this standalone branch;
these focused regressions exercise the validation scripts without an ABACUS
build. CI should additionally verify their behavior with the existing cases.

Governance Notes

  • INPUT/docs changes: none; no INPUT parameter behavior changes.
  • Core module impact: none; changes are confined to tests/integrate.
  • Exceptions requested: none.

Reminder

  • I have read AGENTS.md and docs/developers_guide/agent_governance.md.
  • I have linked an issue or explained why this PR does not need one.
  • I have added adequate unit tests and/or case tests, or explained why not.
  • I have listed the exact verification commands run and their results.
  • I have described user-visible behavior changes, including INPUT parameter changes.
  • I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other source/ changes.
  • I have requested any needed governance exception below.

@mohanchen mohanchen added Bugs Bugs that only solvable with sufficient knowledge of DFT Refactor Refactor ABACUS codes Tests/Examples Issues/PR related to unit tests and integrate tests labels Oct 13, 2026
@mohanchen

Copy link
Copy Markdown
Collaborator

some conflicts need to be solved. @cheerly-pku

Resolve validation-script conflicts with LibRPA producer changes: retain vxc.ref/vxc.txt naming and comparison failure propagation; preserve upstream MPI launcher flags, dedicated-validator mode and case-list filtering. Register script regressions when a Python interpreter is available without restoring a hard configure dependency.

Verified compare_file 7/7, expanded catch_properties shell regression, CTest 2/2 via the actual tests/integrate CMake subdirectory, upstream LibRPA parser 8/8, Python-disabled configuration, shell syntax and whitespace checks. No numerical references or tolerances changed; no PPCG source commits included. Full ABACUS build/runtime integration not rerun for this script-only resolution.
@cheerly-pku

cheerly-pku commented Oct 13, 2026 •

Copy link
Copy Markdown
Author

some conflicts need to be solved. @cheerly-pku

Thanks for the reminder! The conflicts have been resolved. @mohanchen

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bugs Bugs that only solvable with sufficient knowledge of DFT Refactor Refactor ABACUS codes Tests/Examples Issues/PR related to unit tests and integrate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants