Repository navigation
fix(ModelIO): repair corrupted model files and harden model load/save - #1369
Conversation
The uuid -> graph vertex map was read from the file and trusted: an inconsistent map silently returned the relations of another component, and an out of range index was read out of bounds while removing isolated vertices. The map is now rebuilt in O(V) from the component ids stored on the graph vertices (still read for format compatibility). Vertices with an undefined or duplicated id are removed with their relations, with a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Loading threw when a component mesh had no (or several) "unique vertices" attribute, and unique vertices kept component vertices of components missing from the model or with an out of range vertex index. A missing attribute is now recreated and restored from the unique vertices, with a warning. After loading a BRep or a Section, component vertices of unknown components, with an invalid vertex or linked to another unique vertex are removed, with a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A missing components file was silently ignored although it is always saved: it is now reported, except for collections which may be missing from files written before they existed. After deserialization, null components are removed and components stored under another uuid than their own are stored back under their id, with a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only relations of unsupported component types were removed after loading a BRep or a Section. Relations of a component of a supported type that does not exist in the model are now removed too, with a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h type A missing mesh file made the load fail with a bare std::out_of_range, and a mesh implementation not registered in the MeshFactory made it throw. A component whose mesh file is missing now gets an empty mesh, and an unknown implementation is replaced by the default implementation of the mesh type matching the file extension, with a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saving a model whose surface (2D) or block mesh is a RegularGrid threw "Cannot find the explicit SurfaceMesh/SolidMesh type", and loading would have read it as a polygonal surface or polyhedral solid. RegularGrid meshes are now saved and loaded with the RegularGrid IO. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saving the components, relationships, unique vertices and physical properties files never checked the output stream: a file that could not be written was silently skipped. The stream is now checked after opening and after flushing, like Identifier::save. These files are also opened with UTF-8 paths (std::filesystem::u8path), consistently with the archive directory, and the mesh files mapping uses UTF-8 file names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first Component::Impl version read its name with no size limit (max_size()). It now uses MAX_SERIALIZED_NAME_SIZE like other names, so a corrupted size is rejected instead of allocating and reading gigabytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The native BRep and Section input and output lower the Logger level to warning and restore it afterwards, but an exception skipped the restore and left the level stuck. A new detail::ScopedLoggerLevel restores it on scope exit. The level is still global: concurrent loads may race on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
load_brep, load_section and load_brep_time_series only caught OpenGeodeException: std exceptions raised by an input escaped. They now also catch std::exception and rethrow an OpenGeodeModelException with the usual "Cannot load ... from file" message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cpp-Linter Report
|
…ession Clang and MSVC reject a fold expression over qualified member calls (model.Components::has_component). The check now goes through a helper function template. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
geode_object_input_impl checked the file existence and geode_object_output_impl created the parent directories with narrow paths. On Windows, a non-ASCII (UTF-8) file name was reported as missing when loading and directories got a wrong name when saving. Both now use std::filesystem::u8path, consistently with ZipFile and Identifier. Test: saving and loading a BRep with a non-ASCII archive name and directory, which failed on Windows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Model mesh files are written in a temporary directory next to the archive by mesh outputs that still open narrow paths, so a non-ASCII directory fails on Windows. That is a mesh IO limitation outside this change; the UTF-8 test now only covers non-ASCII archive names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pairing them repair_component_vertices ran on every model load, serially over all unique vertices, with one allocation per unique vertex. A read-only parallel scan now flags the unique vertices to repair, and only those are repaired sequentially, so valid models skip all allocations and writes. The repair result is unchanged: it only fills NO_ID links, which never belong to unflagged unique vertices. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| if( nb_unknown_components > 0 ) | ||
| { | ||
| Logger::warning( "[VertexIdentifier::load] Removed ", | ||
| nb_unknown_components, | ||
| " component vertices of unknown components" ); | ||
| } | ||
| if( nb_invalid_vertices > 0 ) | ||
| { | ||
| Logger::warning( "[VertexIdentifier::load] Removed ", | ||
| nb_invalid_vertices, | ||
| " component vertices with invalid vertex index" ); | ||
| } | ||
| if( nb_inconsistent_vertices > 0 ) | ||
| { | ||
| Logger::warning( "[VertexIdentifier::load] Removed ", | ||
| nb_inconsistent_vertices, | ||
| " component vertices linked to another unique vertex" ); | ||
| } | ||
| if( nb_restored_vertices > 0 ) | ||
| { | ||
| Logger::warning( "[VertexIdentifier::load] Restored ", | ||
| nb_restored_vertices, | ||
| " component vertex links to their unique vertex" ); | ||
| } |
There was a problem hiding this comment.
put this warning raisors if-branches into a dedicated function
There was a problem hiding this comment.
Done: the warnings are now grouped in ComponentVerticesRepair::log().
| { | ||
| if( !component ) | ||
| { | ||
| Logger::warning( "[ComponentsStorage::load_components] " |
There was a problem hiding this comment.
I wonder if it should be a simple warning, or an exception raising?
There was a problem hiding this comment.
Kept as a warning on purpose: this runs on load, and our policy for corrupted files is to repair what can be repaired and warn, rather than reject the whole model. Here the invalid component is dropped, so the loaded model stays consistent.
| else if( !existence_checker.apply( model, component_id.id ) ) | ||
| { | ||
| Logger::warning( "[filter_unsupported_components] " | ||
| "Removing relations of missing ", |
There was a problem hiding this comment.
Same rationale: the relations pointing to missing components are removed during load, so the model is repaired and usable; a warning reports it instead of failing the whole load.
# Conflicts: # src/geode/model/mixin/core/vertex_identifier.cpp
# Conflicts: # include/geode/model/mixin/core/detail/components_storage.hpp
|
🎉 This PR is included in version 19.1.0-rc.7 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
|
🎉 This PR is included in version 19.1.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Robustness fixes for native BRep and Section IO. Corrupted or incomplete model files are now repaired with a warning wherever a sensible repair exists. When none exists, loading throws an
OpenGeodeExceptioninstead of a bare std exception, out of bounds reads or silently inconsistent data. Saving checks its output streams, RegularGrid component meshes can be saved, and the Logger level is restored when a load fails. The serialized format is unchanged, so no new backward IO files are needed. The newtest-model-corrupted-filesbuilds every corrupted file from a saved model: it patches, removes or swaps files inside the archive. Each test fails before its fix and passes after it.UuidToIndex::update(its NO_ID check is debug-only), anderase(index_t)scanned the whole map for each isolated vertex. The map is still read, for format compatibility, but it is now rebuilt in O(V) from theidattribute of the graph vertices. Vertices with an undefined or duplicated id are removed with their relations, with a warning. Test: two map entries of a saved file are swapped; the loaded relations used to be wrong.VertexIdentifierBuilder::repair_component_vertices, called after loading a BRep or a Section, removes component vertices of unregistered components, with an invalid vertex or linked to another unique vertex. It also restores missing links. Each repair logs a warning. Tests: the attribute is renamed in each mesh kind and checked to be restored; theverticesfile of a model is used with the same model minus one corner, and that corner's entries are dropped.linesfile is reported; a corner stored under a wrong key is found under its own id with its mesh.filter_unsupported_componentsnow also removes the relations of supported-type components that don't exist in the model, with a warning. Test: therelationshipsfile of a model is used with the same model minus one corner.std::out_of_range(mapping.at). A mesh implementation not registered in theMeshFactorymadeMeshFactory::typethrow. Now a missing file gives an empty mesh of the component's implementation, or of the default one if it is unknown; this also fixesmesh_type. An unknown implementation is replaced by the default implementation of the mesh type that matches the file extension. Both cases log a warning. Tests: a surface and a line mesh file are removed; the surface implementation name is replaced by an unknown one.dynamic_cast. Loading would have read the grid as a polygonal mesh. RegularGrid meshes now use the RegularGrid IO, for 2D surfaces and for blocks. Test: save and reload a Section with a grid surface and a BRep with a grid block.Identifier::Impl::save. They are opened withstd::filesystem::u8path, andfile_mappingusesu8string. Tests: saving each of these files into a missing directory now throws (it used to return silently); a save and reload round trip with a non-ASCII archive name. It already passed on Linux and failed on Windows until the UTF-8 commit below.Component::Implv1 reader now usesMAX_SERIALIZED_NAME_SIZEinstead ofmax_size(). Test: a component name longer than 1 MiB in an old file (layers.og_brep) is rejected; it used to load.detail::ScopedLoggerLevel(ingeode/basic/logger.hpp) restores the level on scope exit. Test: loading a model with a truncatedrelationshipsfile throws and the level is restored. Known limitation, not fixed: the level is global, so concurrent loads or saves in different threads can still race on it.load_brep,load_sectionandload_brep_time_seriesnow also catchstd::exceptionand rethrow anOpenGeodeModelExceptionwith the usual "Cannot load ... from file" message, as fix(MeshIO): native IO and deserialization robustness #1361 did for meshes. Test: an input that throwsstd::runtime_erroris registered for each of the three factories.( model.Components::has_component( id ) || ... ), so the check now goes through a helper function template.geode_object_input_implchecked the file with a narrow path, andgeode_object_output_implalso created parent directories with a narrow path. Both now usestd::filesystem::u8path, likeZipFileandIdentifier. Test: the non-ASCII archive name round trip, which failed on Windows before.BITSERY_WRITE...) still open narrow paths. Fixing that is mesh IO work outside this PR, so the test only covers archive names.Notes
BITSERY_WRITE/BITSERY_READ, LightRegularGrid) still uses narrow paths, so models in non-ASCII directories cannot be saved or loaded on Windows.UuidToIndex::updateanderase(index_t)are left unchanged: they are on the removal paths that another PR works on, and loading no longer depends on them.ComponentsStorage::load_componentsgets a defaultedrequiredparameter, so downstream callers (OpenGeode-Geosciences horizons, faults...) build unchanged.RelationshipsImpl::delete_isolated_vertices(protected, detail) is renamedrepair_after_load. No downstream code uses it;StratigraphicRelationshipsgets the repair through its base class.🤖 Generated with Claude Code