Skip to content

Stabilize CI before the JEFF updates #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shimwell merged 1 commit into from
Oct 20, 2026
Merged

Stabilize CI before the JEFF updates #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/test_package.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,10 @@ jobs:
cd ..
pip install .
cd ..
pip install .
pip install '.[tests]'

- name: Test offline unit and CLI behavior
run: python -m pytest -q -rs tests -m 'not network and not integration'

# this just check the scripts are accessible after the install
# the scripts are not run or checked for correct behaviour
Expand Down
85 changes: 32 additions & 53 deletions .github/workflows/test_processing.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,49 +1,39 @@

name: test_convert_scripts
on:
on:
workflow_dispatch:
pull_request:
branches:
- main
paths-ignore:
- "**.md"
- "**.txt"
- "**.cff"
- 'docs/**'
- '.gitignore'
push:
branches:
- main
inputs:
script:
description: Script recipe to run (downloads and processes large datasets)
required: true
type: choice
options:
- convert_fendl
- convert_jeff32
- convert_jeff33
- convert_endf
- convert_tendl
- convert_tendl_rand
- download_endf
- download_tendl
- download_chain
- add_branching_ratios
- reduce_chain


jobs:
run_scripts:
runs-on: ubuntu-latest
timeout-minutes: 360
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0 # OR "2" -> To retrieve the preceding commit.

- name: Get specific changed files
id: changed-files-specific
uses: tj-actions/changed-files@v47
with:
files: |
src/openmc_data/convert/convert_*.py
src/openmc_data/other/convert_tendl_rand.py
src/openmc_data/download/download_*.py
src/openmc_data/depletion/add_branching_ratios.py
src/openmc_data/depletion/reduce_chain.py
src/openmc_data/utils.py

- name: Set up Python
if: steps.changed-files-specific.outputs.any_changed == 'true'
uses: actions/setup-python@v6
with:
python-version: 3.12

- name: Install testing dependencies
if: steps.changed-files-specific.outputs.any_changed == 'true'
run: |
sudo apt-get update
sudo apt-get install libhdf5-serial-dev
Expand All @@ -59,66 +49,57 @@ jobs:
cd ..
pip install .

# convert scripts that require mcnp nuclear data are not tested when the
# contents change as the CI doesn't contain that nuclear data.
# Convert scripts that require MCNP nuclear data are not tested because
# the CI doesn't contain that nuclear data.
# convert_lib80x.py convert_mcnp70.py convert_mcnp71.py files are not tested here

- name: test default convert_fendl
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/convert/convert_fendl.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'convert_fendl'
run: |
echo "convert_fendl.py file has been modified."
convert_fendl --cleanup

# this running convert_jeff32 in with default arguments produces 49GB of
# data. Hence the use of arguments to process just one temperature
- name: test default convert_jeff32
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/convert/convert_jeff32.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'convert_jeff32'
run: |
echo "convert_jeff32.py file has been modified."
convert_jeff32 -r 3.2 -t 293 --cleanup

- name: test default convert_jeff33
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/convert/convert_jeff33.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'convert_jeff33'
run: |
echo "convert_jeff33.py file has been modified."
convert_jeff33 -r 3.3 -t 293 --cleanup

- name: test default convert_endf
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/convert/convert_endf.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'convert_endf'
run: |
echo "convert_endf.py file has been modified."
convert_endf --cleanup

- name: test default convert_tendl
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/convert/convert_tendl.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'convert_tendl'
run: |
echo "convert_tendl.py file has been modified."
convert_tendl --cleanup

- name: test default convert_tendl_rand
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/other/convert_tendl_rand.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'convert_tendl_rand'
run: |
echo "convert_tendl_rand.py file has been modified."
convert_tendl_rand --nuclides O16 --cleanup

# download scripts
- name: test default download_endf
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/download/download_endf.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'download_endf'
run: |
echo "download_endf.py file has been modified."
download_endf

- name: test default download_tendl
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/download/download_tendl.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'download_tendl'
run: |
echo "download_tendl.py file has been modified."
download_tendl -r 2019
download_tendl -r 2021

- name: test default download_chain
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/download/download_chain.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'download_chain'
run: |
echo "download_chain.py file has been modified."
download_chain -l endf -r b7.1 -b None
download_chain -l endf -r b8.0 -b None
download_chain -l endf -r b7.1 -b SFR
Expand All @@ -128,16 +109,14 @@ jobs:
download_chain -l endf -r b8.0 -b PWR

- name: test default add_branching_ratios
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/depletion/add_branching_ratios.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'add_branching_ratios'
run: |
echo "add_branching_ratios.py file has been modified."
download_chain -l endf -r b8.0
add_branching_ratios -i chain-endf-b8.0.xml -o chain_endf_b8.0_sfr.xml -b sfr
add_branching_ratios -i chain-endf-b8.0.xml -o chain_endf_b8.0_pwr.xml -b pwr

- name: test default reduce_chain
if: contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/depletion/reduce_chain.py') || contains(steps.changed-files-specific.outputs.modified_files, 'src/openmc_data/utils.py')
if: inputs.script == 'reduce_chain'
run: |
echo "reduce_chain.py file has been modified."
download_chain -l endf -r b8.0
reduce_chain -i chain-endf-b8.0.xml -o chain_endf_b8.0_sfr.xml -hl 1e15
43 changes: 23 additions & 20 deletions .github/workflows/test_urls.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,23 +30,26 @@ jobs:
python-version: 3.12

- name: Install testing dependencies
run: |
sudo apt-get update
sudo apt-get install libhdf5-serial-dev
git clone --single-branch --branch develop --depth 1 https://github.com/openmc-dev/openmc.git
cd openmc
mkdir build
cd build
cmake ..
make -j2
sudo make -j2 install
cd ..
pip install .
cd ..
pip install .[tests]

# the -s allows normal print statements in the url testing to be seen in
# the pytest output
- name: test urls
run: |
pytest -v -s tests/test_urls.py
run: python -m pip install '.[tests]'

- name: Test URL registry and HTTP helpers offline
run: python -m pytest -q -rs tests/test_urls.py -m 'not network and not integration'

live_urls:
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
timeout-minutes: 180
steps:
- uses: actions/checkout@v5

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: 3.12

- name: Install testing dependencies
run: python -m pip install '.[tests]'

# External failures remain failures; this health check is not a PR gate.
- name: Test live URLs with explicit skip reasons
run: python -m pytest -v -rs tests/test_urls.py -m network
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ coverage.xml
*.py,cover
.hypothesis/
.pytest_cache/
.pytest-tmp/

# Translations
*.mo
Expand Down
28 changes: 28 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,34 @@ The package is distributed on [PYPI](https://pypi.org/project/openmc-data/) and
pip install openmc_data
```

# Testing

Install the test dependencies with `python -m pip install '.[tests]'`, then run
the offline suite with `python -m pytest -q -rs tests -m 'not network and not integration'`.
Tests stay in the flat [tests](tests) directory: [test_utils.py](tests/test_utils.py)
covers shared utilities, [test_urls.py](tests/test_urls.py) covers registry
validation, HTTP helper regressions and marked live URL checks, and script-specific
tests belong in files named test_<script>.py alongside them. Use pytest's
`tmp_path` for generated test artifacts rather than writing into the repository.

The `network` marker identifies live external URL checks; `integration` identifies
data download and processing tests. Pull requests and pushes run offline tests
and CLI help checks in [test_package.yml](.github/workflows/test_package.yml), and
offline URL syntax and repository-file checks in
[test_urls.yml](.github/workflows/test_urls.yml), without compiling OpenMC for the
URL jobs. Live URL checks run nightly or manually in a separate job of that same
workflow. Full, multi-GB processing runs only through the required script choice
in the manual [test_processing.yml](.github/workflows/test_processing.yml) workflow.

Exactly one URL is explicitly **not tested over HTTP** because of its Cloudflare
challenge:
<https://nds.iaea.org/fendl/data/neutron/fendl-FENDL-3.2c-neutron-endf.zip>.
Its syntax is still checked offline. Other HTTP errors, including IAEA and
Cloudflare 403 responses, remain failures, not skips or expected failures.
Nightly/manual external health checks may remain red until blocked IAEA and
obsolete OECD URLs are restored; passing offline checks does not establish that
downloads are available.

# Usage

Once installed several scripts are available in your terminal that are able to
Expand Down
7 changes: 7 additions & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,13 @@ tests = [
"requests"
]

[tool.pytest.ini_options]
testpaths = ["tests"]
markers = [
"network: live external URL checks (scheduled or manual, not a pull request gate)",
"integration: data download and processing tests (manual only)",
]

[project.urls]
"Homepage" = "https://github.com/fusion-energy/openmc_data"
"Bug Tracker" = "https://github.com/fusion-energy/openmc_data/issues"
Expand Down
Loading
Loading