Conversation
Replace clean_osm_data.py/build_osm_network.py and the build.smk/clean.smk rules with clean.py/build_network.py/network.smk, moving output paths from osm/retrieve|clean|build to retrieve|clean|build directly. Also fixes a crash in relation-to-line merging for route relations whose members are all nodes (no way geometry at all) - previously a raw KeyError, now correctly dropped as having nothing to merge.
The "m"/"medium"/"low"/etc. voltage corrections matched as substrings, so any unrelated text containing those letters got mangled - one Philippines line's freeform garbage tag ballooned into a 30+ digit number and crashed the pipeline. Match those rules on the whole value instead, and drop any voltage that's still implausibly long as a defensive backstop.
New build_interactive_map rule renders buses/lines/transformers/stations as a standalone HTML map (pydeck), with layer toggles, voltage and text search, multi-circuit line rendering, light/dark theming, and click-through OSM links - now the workflow's default target. Also builds it in the integration test.
include_relations is a modelling decision, not a retrieval one, so it now lives under network instead of retrieve. Also adds an interactive_map config section (geometry simplification tolerances, coordinate rounding) and a clearer error for an unrecognised country code, which used to surface as a raw traceback from earth-osm instead of the intended message.
README and INTERFACE.yaml didn't mention map.html at all, and INTERFACE.yaml's old description pointed at a "release" directory that doesn't exist anywhere in the actual rules.
Regional defaults (60 Hz, lower minimum voltage) plus runnable example configs, matching the existing BE-NL example.
Local Claude Code settings and a test-run PBF cache, neither of which belong in the repo.
Collaborator
Author
ExamplesBrasil (BR)
Germany (DE)
Egypt (EG)
Japan (JP)
Korea (KR)
Mexico (MX)
USA (US)
South Africa (ZA)
|
# Conflicts: # tests/integration_test.py
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Fixes CI lint failure from ruff format --check on the re.split call that grew a re.IGNORECASE flag; no logic change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes in this pull request
Rename
clean_osm_data.py/build_osm_network.pyand their rules toclean.py/build_network.py/network.smk, with output paths moved fromosm/retrieve|clean|buildtoretrieve|clean|build;Fix crash on route relations with no way geometry, and a voltage tag-correction rule that was corrupting invalid, long tags into number
Add
build_interactive_map.py/plot.smk, a self-contained interactive HTML map of the built network (layer toggles, voltage/text filtering, multi-circuit line rendering, light/dark theming, click-through OSM links), now the workflow's default target; documented inREADME.md/INTERFACE.yamland exercised by the integration test. Ported over from PyPSA-Eur'sprepare_osm_network_releaseMove
include_relationstonetworkconfig, and add aninteractive_mapconfig section for geometry simplification and coordinate rounding: The higher the numbers chosen, the longer the distance between geometry points, the lower the resolution, the smaller the file of the htmlAdd regional defaults and example configs for Brazil, Mexico, the Philippines, and the US (work-in-progress)
There are no
pipdependencies in the module's environment files (workflow/envs/).All rules use
pathvars(e.g.,<results>) in their inputs and outputs.The integration test-suite is successful, including:
pre-commit.citests pass.Module documentation is up-to-date, including:
INTERFACE.yamlmentions all relevantpathvarsandwildcards.README.mddescribes how to use the module and has the necessary citations.