From 09f57bd9fe77a2c25f8ef5704ed6bfb2eb507eb4 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 20 Jun 2026 21:27:21 +0000 Subject: [PATCH 1/5] docs: add per-page SEO metadata and fix grammar errors MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add YAML frontmatter (title + description) to all documentation pages so search engines generate accurate per-page snippets rather than falling back to the global site description. Also update site_name to "xml2db" for cleaner browser tab titles, add explicit nav labels in mkdocs.yml, and fix several grammar errors (a/an before XML/XSD, "Hash are" → "Hashes are", "is call" → "is called", "corrsponding" typo, extra space). Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_013pzr3Q9XGumr7WVfHGGtXV --- docs/api/data_model.md | 5 +++++ docs/api/document.md | 5 +++++ docs/api/overview.md | 7 ++++++- docs/api/xml_converter.md | 5 +++++ docs/configuring.md | 7 ++++++- docs/getting_started.md | 9 ++++++++- docs/how_it_works.md | 17 +++++++++++------ docs/index.md | 3 ++- mkdocs.yml | 20 ++++++++++---------- 9 files changed, 58 insertions(+), 20 deletions(-) diff --git a/docs/api/data_model.md b/docs/api/data_model.md index f91d6f8..2e03956 100644 --- a/docs/api/data_model.md +++ b/docs/api/data_model.md @@ -1,3 +1,8 @@ +--- +title: "DataModel" +description: "API reference for the xml2db DataModel class: constructor parameters, methods for parsing XML files, visualizing the schema as ERD or tree, and generating DDL statements." +--- + # DataModel ::: xml2db.model.DataModel diff --git a/docs/api/document.md b/docs/api/document.md index bfb46f3..aef5567 100644 --- a/docs/api/document.md +++ b/docs/api/document.md @@ -1,3 +1,8 @@ +--- +title: "Document" +description: "API reference for the xml2db Document class: methods for inserting parsed XML data into database tables and converting flat data back to an XML file." +--- + # Document ::: xml2db.document.Document diff --git a/docs/api/overview.md b/docs/api/overview.md index 3fb19f2..9a20f79 100644 --- a/docs/api/overview.md +++ b/docs/api/overview.md @@ -1,3 +1,8 @@ +--- +title: "API Overview" +description: "Overview of the xml2db Python API: building DataModel objects, parsing and loading XML files into a database, and extracting data back to XML — including multiprocessing patterns." +--- + # API Overview ## Building a data model from an XSD file @@ -11,7 +16,7 @@ ## Inspecting the data model * [`DataModel.source_tree`](data_model.md): see the data model in tree format before any transformation -* [`DataModel.target_tree`](data_model.md): see the data model in tree format after simplification (corrsponding to the data model +* [`DataModel.target_tree`](data_model.md): see the data model in tree format after simplification (corresponding to the data model which will be created in the database) * [`DataModel.get_entity_rel_diagram`](data_model.md/#xml2db.model.DataModel.get_entity_rel_diagram): get a visual representation of the data model using Mermaid diff --git a/docs/api/xml_converter.md b/docs/api/xml_converter.md index 07f9fe1..4e0ee50 100644 --- a/docs/api/xml_converter.md +++ b/docs/api/xml_converter.md @@ -1,3 +1,8 @@ +--- +title: "XMLConverter" +description: "API reference for the xml2db XMLConverter class: methods for parsing an XML document into a document tree and converting a document tree back to XML." +--- + # XMLConverter ::: xml2db.xml_converter.XMLConverter diff --git a/docs/configuring.md b/docs/configuring.md index 645362c..16c17a5 100644 --- a/docs/configuring.md +++ b/docs/configuring.md @@ -1,3 +1,8 @@ +--- +title: "Configuring your data model" +description: "Configure xml2db's data model via model_config: override column types, control field elevation, adjust deduplication, simplify XSD choice groups, and add custom indexes." +--- + # Configuring your data model The data model in the database is derived automatically from a XML schema definition file (XSD) you provide. It is a set @@ -51,7 +56,7 @@ access to the underlying tree data structure just before it is extracted to be l for instance, to prune or modify some parts of the document tree before loading it into the database. The document tree should of course stay compatible with the data model. * `document_tree_node_hook` (`Callable`): sets a hook function which can modify the data extracted from the XML. It is -similar with `document_tree_hook`, but it is call as soon as a node is completed, not waiting for the entire parsing to +similar with `document_tree_hook`, but it is called as soon as a node is completed, not waiting for the entire parsing to finish. It is especially useful if you intend to filter out some nodes and reduce memory footprint while parsing. * `row_numbers` (`bool`): adds `xml2db_row_number` columns either to `n-n` relationships tables, or directly to data tables when deduplication of rows is opted out. This allows recording the original order of elements in the source XML, which is not diff --git a/docs/getting_started.md b/docs/getting_started.md index b69958b..0c51624 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -1,5 +1,12 @@ +--- +title: "Getting started" +description: "Step-by-step guide to installing xml2db, creating a DataModel from an XSD file, importing XML into a relational database, and visualizing the resulting data model." +--- + # Getting started +This guide walks you through installing xml2db, creating a data model from an XSD schema, loading XML files into a relational database, and exporting data back to XML. + ## Installation The package can be installed, preferably in a virtual environment, using `pip`: @@ -102,7 +109,7 @@ It is useful to visualize your data model in order to [configure it](configuring Once you are happy with the data model created from previous steps, you are now ready to actually process XML files and load their content to your database. It goes like this: -``` py title="Parse a XML file" linenums="1" +``` py title="Parse an XML file" linenums="1" document = data_model.parse_xml( xml_file="path/to/file.xml", ) diff --git a/docs/how_it_works.md b/docs/how_it_works.md index fc86a4a..dc73a37 100644 --- a/docs/how_it_works.md +++ b/docs/how_it_works.md @@ -1,3 +1,8 @@ +--- +title: "How it works" +description: "Deep dive into how xml2db builds a relational data model from an XSD schema, performs hash-based deduplication, models 1-1/1-n/n-n relationships, and loads XML data into a database." +--- + # How it works This page covers more advanced topics to understand in depth how data models are created and how data is loaded to @@ -5,7 +10,7 @@ the database. This can help with troubleshooting or for advanced use cases. ## Building a data model -A XML document is a tree-like structure where every element is either a simple type (i.e. a scalar value) or a complex +An XML document is a tree-like structure where every element is either a simple type (i.e. a scalar value) or a complex type which has children which can be simple types or complex types. Elements can also have attributes, which are also scalar values, which `xml2db` will handle just as simple type children. In this page we will call "properties" the simple type children of an element or its attributes. @@ -36,11 +41,11 @@ Taking advantage of the initial tree structure, after parsing the XML document i each node, which includes all its properties and all its children hash, recursively. Two nodes with the same hash are thus identical, so only one of them needs to be stored, even if they appear under different parent nodes. -Hash are stored in the database, with a unique constraint (as a column with binary type named `xml2db_record_hash`). The +Hashes are stored in the database, with a unique constraint (as a column with binary type named `xml2db_record_hash`). The primary key of all databases is an auto-incremented integer column, always named `pk_table_name`, `table_name` being the name of the table. -In some cases, especially when only few duplicates are expected, it may be more efficient to allow duplicated nodes in +In some cases, especially when only a few duplicates are expected, it may be more efficient to allow duplicated nodes in order to avoid extra tables to store relationships. This can be configured for each table of the data model. ### Modeling relationships @@ -141,10 +146,10 @@ elements. This section gives more detailed explanations on how parsing and loading data work. The integration of an XML file can be decomposed in lower level steps described below. -### Parsing a XML document +### Parsing an XML document -First, we load all the XML document in memory using `lxml` and extract the data as a nested `dict`, where each node -keeps a reference of its type, and store its data content. This task is achieved by the function +First, we load the entire XML document into memory using `lxml` and extract the data as a nested `dict`, where each node +keeps a reference of its type and stores its data content. This task is achieved by the function [`XMLConverter.parse_xml`](api/xml_converter.md#xml2db.xml_converter.XMLConverter.parse_xml). This limits the size of files that can be loaded, due to memory limitations. The merging database transaction also diff --git a/docs/index.md b/docs/index.md index 5254f4a..aabad47 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,12 +1,13 @@ --- title: "Loading XML into a relational database" +description: "xml2db is a Python package that automatically maps an XSD schema to relational database tables and loads XML files into them — no custom mapping rules required." --- # Loading XML into a relational database `xml2db` is a Python package which allows parsing and loading XML files into a relational database: -* it maps automatically a XSD schema with a set of tables in the database +* it automatically maps an XSD schema to a set of tables in the database * it can handle complex XML files which cannot be denormalized into flat tables * it works out of the box, without any custom mapping rules. diff --git a/mkdocs.yml b/mkdocs.yml index e234fb1..5df29c8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,21 +1,21 @@ -site_name: Xml2db Docs +site_name: xml2db site_url: https://cre-dev.github.io/xml2db repo_url: https://github.com/cre-dev/xml2db repo_name: cre-dev/xml2db copyright: Copyright © 2024 Commission de régulation de l'énergie -site_description: Xml2db is a Python package to load XML files into a relational database. It handles complex XML and +site_description: xml2db is a Python package to load XML files into a relational database. It handles complex XML and works out of the box, without any custom mapping rules. nav: - Introduction: "index.md" - - "getting_started.md" - - "configuring.md" - - "how_it_works.md" - - Api: - - "api/overview.md" - - "api/data_model.md" - - "api/document.md" - - "api/xml_converter.md" + - Getting started: "getting_started.md" + - Configuring: "configuring.md" + - How it works: "how_it_works.md" + - API: + - Overview: "api/overview.md" + - DataModel: "api/data_model.md" + - Document: "api/document.md" + - XMLConverter: "api/xml_converter.md" theme: name: material From 0274e7581e6aa46efa8dfd3089820e0c057cb4c0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 20 Jun 2026 21:35:55 +0000 Subject: [PATCH 2/5] docs: tighten prose for clarity and conciseness MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rewrite wordy or awkward sentences across all four main documentation pages — especially the home page. Changes include splitting overly long sentences, removing filler phrases ("in order to", "if need be", "the simplest solution will be to"), fixing subject-verb agreement ("follow" → "follows", "Columns corresponds" → "Columns correspond"), correcting a few remaining article errors ("a XML" → "an XML"), and removing double spaces left by earlier edits. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_013pzr3Q9XGumr7WVfHGGtXV --- docs/configuring.md | 48 ++++++++---------- docs/getting_started.md | 41 +++++++--------- docs/how_it_works.md | 105 ++++++++++++++++++---------------------- docs/index.md | 35 ++++++-------- 4 files changed, 99 insertions(+), 130 deletions(-) diff --git a/docs/configuring.md b/docs/configuring.md index 16c17a5..04e071f 100644 --- a/docs/configuring.md +++ b/docs/configuring.md @@ -5,11 +5,10 @@ description: "Configure xml2db's data model via model_config: override column ty # Configuring your data model -The data model in the database is derived automatically from a XML schema definition file (XSD) you provide. It is a set -of tables linked by foreign keys relationships. Basically, each `complexType` of the XML schema definition corresponds -to a table in the target database data model. Each table is named after the first element name of this type, with -de-duplication if needed. Columns in a table corresponds to `simpleType` elements within a complex type and its -attributes. Columns are named after the names of children XML elements or attributes. +The data model is derived automatically from an XSD file you provide. It is a set of tables linked by foreign key +relationships. Each `complexType` in the XSD corresponds to a table, named after the first element of that type (with +deduplication if needed). Columns correspond to `simpleType` elements and attributes within the complex type, named +after the XML element or attribute. `xml2db` applies a few simplifications to the original data model by default, but they can also be opted-out or forced through the configuration `dict` provided to the `DataModel` constructor. @@ -21,8 +20,7 @@ The column types can also be configured to override the default type mapping, us diagram (see the [Getting started](getting_started.md) page for directions on how to visualize data models) and then adapt the configuration if need be. -Configuration options are described below. Some options can be set at the model level, others at the table level and -others at the field level. The general structure of the configuration dict is the following: +Options apply at three levels: model, table, and field. The general structure of the configuration dict is: ```py title="Model config general structure" linenums="1" { @@ -49,7 +47,7 @@ others at the field level. The general structure of the configuration dict is th ## Model configuration -The following options can be passed as a top-level keys of the model configuration `dict`: +The following options can be passed as top-level keys of the model configuration `dict`: * `document_tree_hook` (`Callable`): sets a hook function which can modify the data extracted from the XML. It gives direct access to the underlying tree data structure just before it is extracted to be loaded to the database. This can be used, @@ -152,20 +150,17 @@ automatically applied `join`, as it would require a complex process of adding a ### Elevate children to upper level -If a complex child element has a minimum and maximum occurrences number of 1 and 1 respectively, it can be "pulled" up -to its parent element. This behaviour will always be applied by default. +A complex child that occurs exactly once (`[1, 1]`) is always elevated to its parent by default. -If a complex child element has a minimum and maximum occurrences number of 0 and 1 respectively, it can also be "pulled" -up to its parent element fields. This is applied by default if the child has less than 5 fields, because otherwise it -could clutter the parent element with many columns that will often be all `NULL`. +An optional child (`[0, 1]`) is also elevated by default if it has fewer than 5 fields, to avoid cluttering the parent +with mostly-NULL columns. -This simplification can be opted out using a configuration option, and forced in the case of a child with more than 5 -fields, using the following option: +This behaviour can be disabled, or forced for children with more than 5 fields, using: `"transform":` `"elevate"` (default) or `"elevate_wo_prefix"` or `False` (disable). -By default, the elevated field name is prefixed with the name of the complex child so its origin is clear and to prevent -duplicated names, but this prefixing can be avoided with the value `"elevate_wo_prefix"`. +By default, the elevated field is prefixed with the child's name to clarify its origin and avoid name collisions. +Use `"elevate_wo_prefix"` to skip the prefix. For example, complex child `timeInterval` with 2 fields of max occurrence 1, before elevation... ```shell @@ -202,11 +197,10 @@ timeInterval_end[1, 1]: string ### Simplify "choice groups" -In XML schemas, choice groups are quite frequent. It means that only one of its possible children types should be -present. +In XML schemas, choice groups are common: only one of the possible children may be present at a time. -Here we consider only choice groups of simple elements (not complex types). The naive way to convert this to a table is -to create one column for each possible choice, of which only one will have a non `NULL` value for each record. +This section covers choice groups of simple elements only (not complex types). The straightforward conversion creates +one column per option, of which only one will be non-`NULL` for each record. If there are more than 2 possible choice options and the simple elements are of the same type, they can be transformed into two columns: @@ -249,9 +243,8 @@ in or out otherwise, with the following option: ### Deduplication -By default, `xml2db` will try to deduplicate elements (store identical element only once in the database) in order to -reduce storage footprint, which is particularly relevant for "feature" fields in XML schemas, meaning when a XML element -specify a feature as a child element, which is shared with many other elements. +By default, `xml2db` deduplicates elements — identical elements are stored only once — which is particularly useful +when an XML element specifies a feature shared by many other elements. This is done using a hash of each node in the XML file, which includes recursively all its children. The detailed process is described in the [how it works](how_it_works.md) page. @@ -260,10 +253,9 @@ The implication is that relationships with 1-1 or 1-n cardinality in the XML sch n-1 and n-n relationships in the database. For n-n, relationships, it means that there is an additional relationship table which has foreign keys relations to both tables in the relationship. -This behaviour can be opted-out, for instance if you know that there will be mostly unique elements and you prefer not -having the additional relationship table. The 1-n relationship will be modelled using only a foreign key to the parent, -without an intermediate table holding the relationship, which makes the data model simpler, and maybe some queries -faster, but stores more records in case of duplicated records. +Disable deduplication if you expect mostly unique elements and want to avoid the extra join table. The `1-n` +relationship is then modelled with a plain foreign key to the parent, which simplifies the schema but stores duplicate +records. Configuration: `"reuse":` `True` (default) or `False` (disable) diff --git a/docs/getting_started.md b/docs/getting_started.md index 0c51624..3138e2a 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -38,21 +38,19 @@ data_model = DataModel( ) ``` -At this stage, it is not required to provide a connection string and have an actual database set up, but it will be -necessary if you want to use this [`DataModel`](api/data_model.md) to actually import data. You may need to install the -Python package of the connector you use in your `sqlalchemy` connection string (e.g. `psycopg2` or `psycopg` for -PostgreSQL, `pymysql` or `mysqlclient` for MySQL, `pyodbc` for SQL Server, `duckdb-engine` for DuckDB). See +A connection string and database are not required at this stage, but both are needed to actually import data. You +may need to install a connector package (e.g. `psycopg2` or `psycopg` for PostgreSQL, `pymysql` or `mysqlclient` for +MySQL, `pyodbc` for SQL Server, `duckdb-engine` for DuckDB). See [How it works](how_it_works.md#bulk-loading) for which drivers enable native bulk loading. -You can provide an optional model configuration, which will allow forcing or preventing some schema simplification, set -columns types manually, etc. By default, some simplifications will be applied when possible, in order to limit the -resulting data model complexity. +The optional `model_config` controls schema simplifications, column types, and more. By default, some simplifications +are applied to reduce data model complexity. ## Visualizing the data model -When you start from a new XML schema, we recommend that you first visualize the resulting data model and decide whether -it needs some tweaking. The simplest solution will be to generate a markdown page which contains a visual representation -of your schema (`data_model` being the [`DataModel`](api/data_model.md) object previously created): +When starting from a new XML schema, we recommend visualizing the data model first to decide whether any tweaking is +needed. Generate a Markdown file with a visual representation of your schema (`data_model` being the +[`DataModel`](api/data_model.md) object previously created): ``` py title="Write an Entity Relationship Diagram to a file" linenums="1" with open(f"target_data_model_erd.md", "w") as f: @@ -61,10 +59,9 @@ with open(f"target_data_model_erd.md", "w") as f: You can see an example of these diagrams on the [Introduction page](index.md). -The data model visualization uses [Mermaid](https://mermaid.js.org/syntax/entityRelationshipDiagram.html) to create an -"entity relationship diagram" which will show the tables created and the relationships between them. To visualize this, -you can for instance rely on [Pycharm IDE mermaid support](https://www.jetbrains.com/help/pycharm/markdown.html#diagrams). -GitHub will also natively support those. +The diagram uses [Mermaid](https://mermaid.js.org/syntax/entityRelationshipDiagram.html) to show the tables and their +relationships. [PyCharm](https://www.jetbrains.com/help/pycharm/markdown.html#diagrams) and GitHub both render Mermaid +diagrams natively. You can also visualize your model in a tree-like text mode. In this format, you can visualize the raw, untouched XML schema, as well as the simplified one (we call it "target" model): @@ -106,8 +103,7 @@ It is useful to visualize your data model in order to [configure it](configuring ## Importing XML files -Once you are happy with the data model created from previous steps, you are now ready to actually process XML files and -load their content to your database. It goes like this: +Once the data model looks right, load XML files as follows: ``` py title="Parse an XML file" linenums="1" document = data_model.parse_xml( @@ -116,14 +112,12 @@ document = data_model.parse_xml( document.insert_into_target_tables() ``` -By default, the validity of your XML file will not be checked against the XML schema used to create your `data_model` -object, which can be enabled if you are unsure that your XML files will be valid. +By default, XML files are not validated against the schema. Enable validation if you need to verify file integrity. -The [`Document.insert_into_target_tables`](api/document.md#xml2db.document.Document.insert_into_target_tables) method is -then all you need to load your data to the database. +[`Document.insert_into_target_tables`](api/document.md#xml2db.document.Document.insert_into_target_tables) is all you +need to load data into the database. -Please read the [How it works](how_it_works.md) page to learn more about the process, which could help -troubleshooting if need be. +See the [How it works](how_it_works.md) page for a deeper explanation of the loading process. !!! note `xml2db` can save metadata for each loaded XML file. These can be configured using the @@ -160,8 +154,7 @@ troubleshooting if need be. ## Getting back the data into XML -You can extract the data from the database into XML files. This was implemented primarily to be able to test the package -using "round trip" tests to and from the database. +Data can be extracted from the database back to XML — primarily useful for round-trip testing. ``` py title="Extract data back to XML" linenums="1" document = data_model.extract_from_database( diff --git a/docs/how_it_works.md b/docs/how_it_works.md index dc73a37..967711f 100644 --- a/docs/how_it_works.md +++ b/docs/how_it_works.md @@ -5,23 +5,21 @@ description: "Deep dive into how xml2db builds a relational data model from an X # How it works -This page covers more advanced topics to understand in depth how data models are created and how data is loaded to -the database. This can help with troubleshooting or for advanced use cases. +This page explains in depth how data models are built and how XML data is loaded — useful for troubleshooting or +advanced use cases. ## Building a data model -An XML document is a tree-like structure where every element is either a simple type (i.e. a scalar value) or a complex -type which has children which can be simple types or complex types. Elements can also have attributes, which are also -scalar values, which `xml2db` will handle just as simple type children. In this page we will call "properties" the -simple type children of an element or its attributes. +An XML document is a tree where every element is either a simple type (a scalar value) or a complex type with children +that are themselves simple or complex types. Attributes are also scalar values and are treated the same as simple type +children. Throughout this page, "properties" refers to the simple type children of an element or its attributes. -The general idea of `xml2db` is to convert complex types to a database tables, and simple types or XML attributes into -columns in these tables. When a complex type has complex type children, they are themselves stored in other tables, and -related to their parents with a foreign key constraint, or a relationship table which holds foreign key constraints to -both related tables. +`xml2db` converts complex types to database tables and simple types or XML attributes to columns. When a complex type +has complex type children, they are stored in their own tables and linked to their parents via a foreign key or a +join table. -The data model created by `xml2db` is mostly bijective with the original XML document, which can in many cases be -extracted and converted back to a XML document as it was loaded into the database. +The resulting data model is mostly bijective with the source XML — in most cases, data can be extracted from the +database and converted back to XML. !!! note XSD specification allows multiple-root schemas. This means that XML documents conforming to this schema can have @@ -37,9 +35,9 @@ By default, `xml2db` tries to reduce storage footprint in the database by storin original XML document, if it is already present in the database. This deduplication process takes into account the whole subtree starting from a given element, and not only the direct children of an element. -Taking advantage of the initial tree structure, after parsing the XML document into a python dict, we compute a hash for -each node, which includes all its properties and all its children hash, recursively. Two nodes with the same hash are -thus identical, so only one of them needs to be stored, even if they appear under different parent nodes. +After parsing the XML document into a Python dict, we compute a recursive hash for each node covering all its +properties and children. Two nodes with the same hash are identical, so only one needs to be stored, even if they +appear under different parents. Hashes are stored in the database, with a unique constraint (as a column with binary type named `xml2db_record_hash`). The primary key of all databases is an auto-incremented integer column, always named `pk_table_name`, `table_name` being the @@ -50,13 +48,12 @@ order to avoid extra tables to store relationships. This can be configured for e ### Modeling relationships -Within a tree, parent-child relationship can have `1-1` or `1-n` cardinality. As we want to reuse child nodes, we -convert these relationships to `n-1` or `n-n`, respectively. Besides, some children are optional. This does not affect -the representation of relationships in any way. +Within a tree, parent-child relationships have `1-1` or `1-n` cardinality. To allow reuse of child nodes, these +become `n-1` or `n-n` respectively. Optional children are handled the same way. -A same child node (same hash) which is used under different parents will have several parents after the "recycling" -process, while it had only one parent (because of the tree structure) in the initial dataset. This example illustrates -a `1-1` relationship converted to `n-1` after reusing nodes: +A child node (identified by its hash) that appears under multiple parents will have several parents after +deduplication — whereas it had only one in the original tree. This example illustrates a `1-1` relationship converted +to `n-1` after reuse: ```mermaid erDiagram @@ -81,9 +78,8 @@ erDiagram UNIQUE_CONTRACT }|--|{ UNIQUE_DELIVERY_PROFILE : delivers ``` -In a SQL relational data model, `1-n` relationships can be easily represented with foreign keys. `n-n` relationships -require however an additional table holding the relationship, which gives, for the last example with contracts and -delivery profiles: +`1-n` relationships are represented with foreign keys. `n-n` relationships require an additional join table, which +gives, for the last example with contracts and delivery profiles: ```mermaid erDiagram @@ -100,31 +96,26 @@ erDiagram ### Duplicated elements -As explained previously, deduplication can be opted out to avoid the complexity of an intermediary table holding a `n-n` -relationship. In that case, the relationship will stay a `1-n` relationship, which will be modeled with the child -element holding a foreign key relationship to its parent. +Deduplication can be disabled to avoid the extra join table. The relationship then stays `1-n`, with the child +holding a foreign key to its parent. -In that case, no hash is stored for the children as there are effectively duplicated elements. The column -`fk_parent_tablename` in the child table holds the primary keys of associated parent rows in the parent table -`tablename`. +In that case, no hash is stored for the children since rows may be duplicated. The column `fk_parent_tablename` in the +child table holds the primary keys of associated rows in the parent table `tablename`. This choice is made for each table individually, and children of a duplicated element can effectively be themselves stored as deduplicated elements. -This means that as the end, there can be both `1-n` and `n-1` relationships involved to represent a tree structure, -which means that the order of dependencies of the resulting tables will not be the same as the original tree structure. -For instance, when we want to make sure to process all dependent tables before processing a table, we won't likely -start with the root table of the tree, as it would be expected without the de-duplication process. +In the end, the schema can have both `1-n` and `n-1` relationships, so table dependency order differs from the +original tree structure. When processing tables in dependency order, the root table is no longer necessarily first. ## Caveats `xml2db` handles a variety of data models, but does not cover all possible schemas allowed by the [XML schema documents specification](https://en.wikipedia.org/wiki/XML_Schema_(W3C)). -Some known cases which are not supported by `xml2db` are described below. Other cases can fail and may require some -adjustments to work. We recommend thorough testing for more "exotic" schemas; for instance it is possible to implement -"round-trip" tests from sample XML files to database and back to XML, and compare the resulting XML file with the -original one. +Known unsupported cases are described below. Other edge cases may also fail and require adjustments. We recommend +thorough testing for unusual schemas — for example, you can implement round-trip tests (XML → database → XML) and +compare the output against the original. ### Recursive XSD @@ -143,8 +134,7 @@ elements. ## Loading process -This section gives more detailed explanations on how parsing and loading data work. The integration of an XML file can -be decomposed in lower level steps described below. +This section explains in detail how parsing and loading work. Loading an XML file breaks down into the following steps. ### Parsing an XML document @@ -152,15 +142,14 @@ First, we load the entire XML document into memory using `lxml` and extract the keeps a reference of its type and stores its data content. This task is achieved by the function [`XMLConverter.parse_xml`](api/xml_converter.md#xml2db.xml_converter.XMLConverter.parse_xml). -This limits the size of files that can be loaded, due to memory limitations. The merging database transaction also -limits the size of the files that can be loaded, depending on the server performance. On the other hand, handling data -in memory makes the processing way simpler and faster. We handle files with a size around 500 MB without any issue. +This constrains the maximum file size — in-memory parsing has memory limits, and the merge transaction adds a +server-performance constraint. In practice, `xml2db` handles files around 500 MB without issue. ### Computing hashes -We compute tree hashes recursively by adding to each node's hash the hashes of its children element, be it simple -types, attributes or complex types. Children are processed in the specific order they appeared in the XSD schema, -so that hashing is really deterministic. +Tree hashes are computed recursively by combining each node's hash with the hashes of its children — simple types, +attributes, and complex types alike. Children are processed in the order they appear in the XSD schema, making +hashing fully deterministic. Right after this step, a hook function is called if provided in the configuration (top-level `document_tree_hook` option in the configuration `dict`), which gives direct access to the underlying tree data structure just before it is @@ -179,8 +168,8 @@ only the data from the XML file we just parsed, and the final primary keys and f ### Loading the data -The data we converted is then loaded to the database in a separate set of tables, which have the same names that the -target tables, but prefixed with `temp_XXX` (with `XXX` being a random 8 characters `uuid` string, by default). +The converted data is loaded into a separate set of tables with the same names as the target tables, but prefixed +with `temp_XXX` (a random 8-character UUID string, by default). We keep the primary keys from the flat data model created at the previous stage, as temporary keys. @@ -234,22 +223,20 @@ The process boils down to: ### Summing up -The whole loading process can be achieved by the high level functions -[`DataModel.parse_xml`](api/data_model.md#xml2db.model.DataModel.parse_xml) and -[`Document.insert_into_target_tables`](api/document.md#xml2db.document.Document.insert_into_target_tables). However, -the later can be decomposed in lower level function calls, for instance if you want to separate the loading to the -temporary tables and the merge operation into the target tables. You can have a look at -[`Document.insert_into_target_tables`](api/document.md#xml2db.document.Document.insert_into_target_tables) -source code to see how the lower level steps are stitched together. +The full loading process is exposed via +[`DataModel.parse_xml`](api/data_model.md#xml2db.model.DataModel.parse_xml) and +[`Document.insert_into_target_tables`](api/document.md#xml2db.document.Document.insert_into_target_tables). The latter +can be broken down into lower-level calls if you need to separate the temporary-table load from the merge step. See the +[`Document.insert_into_target_tables`](api/document.md#xml2db.document.Document.insert_into_target_tables) source code +for how these steps fit together. ## Extracting the data back to XML -Extracting the data from the database and converting it back to XML follow similar steps, in reverse order. +Extracting data from the database and converting it back to XML follows similar steps, in reverse order. !!! info - Extracting the data from the database is not very optimized and is actually currently quite slow, mostly due to - complex join queries to retrieve data based on a filter only on the top node. This feature is currently useful - for roundtrip test, but has limited value otherwise, because of its poor performance compared to loading. + Extraction is currently slow due to complex join queries filtered only at the root level. It is mainly useful + for round-trip testing and has limited practical value otherwise. ### Querying data from the database diff --git a/docs/index.md b/docs/index.md index aabad47..0ac862d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,15 +5,15 @@ description: "xml2db is a Python package that automatically maps an XSD schema t # Loading XML into a relational database -`xml2db` is a Python package which allows parsing and loading XML files into a relational database: +`xml2db` is a Python package that parses and loads XML files into a relational database: * it automatically maps an XSD schema to a set of tables in the database * it can handle complex XML files which cannot be denormalized into flat tables * it works out of the box, without any custom mapping rules. -`xml2db` fits well within an [Extract, Load, Transform](https://docs.getdbt.com/terms/elt) data pipeline pattern: it -loads XML files into a relational data model which is very close to the source data, yet easy to work with, being flat -database tables. +`xml2db` fits naturally into an [ELT (Extract, Load, Transform)](https://docs.getdbt.com/terms/elt) pipeline. It loads +XML files into a relational model that stays close to the source data while remaining easy to query as flat database +tables. ## How to load XML files into a database @@ -35,26 +35,23 @@ document = data_model.parse_xml(xml_file="path/to/file.xml") document.insert_into_target_tables() ``` -The resulting data model will be very similar with the XSD schema. However, `xml2db` will perform automatically a few -simplifications aimed at limiting the complexity of the resulting data model and the storage footprint. The data model -can be configured, but the above code will work out of the box for most schemas, with reasonable defaults. +The resulting data model closely follows the XSD schema. By default, `xml2db` applies a few simplifications to reduce +complexity and storage footprint. The above code works out of the box for most schemas. -The raw data loaded into the database can then be processed if need be, using for instance [DBT](https://www.getdbt.com/), -SQL views or stored procedures aimed at extracting, correcting and formatting the data into more user-friendly tables. +The raw data can then be transformed using [DBT](https://www.getdbt.com/), SQL views, or stored procedures to produce +more user-friendly tables. -This package uses `sqlalchemy` to interact with the database, so it should work with different database backends. -Automated integration tests run against PostgreSQL, MySQL, MS SQL Server and DuckDB. You may have to install additional -packages to connect to your database (e.g. `psycopg2` or `psycopg` for PostgreSQL, `pymysql` or `mysqlclient` for -MySQL, `pyodbc` for MS SQL Server, or `duckdb-engine` for DuckDB). See [How it works](how_it_works.md#bulk-loading) -for which drivers enable native bulk loading. +Built on `sqlalchemy`, `xml2db` supports multiple database backends. Integration tests cover PostgreSQL, MySQL, +MS SQL Server, and DuckDB. You may need to install a connector package (e.g. `psycopg2` or `psycopg` for PostgreSQL, +`pymysql` or `mysqlclient` for MySQL, `pyodbc` for MS SQL Server, or `duckdb-engine` for DuckDB). See +[How it works](how_it_works.md#bulk-loading) for which drivers enable native bulk loading. ## How to visualize your data model -`xml2db` can also generate automatically beautiful visualisations of your data models extracted from an XSD file. It -uses [Mermaid](https://mermaid.js.org/syntax/entityRelationshipDiagram.html) to represent database tables and their -relationships. +`xml2db` can also generate visual diagrams of your data model directly from an XSD file, using +[Mermaid](https://mermaid.js.org/syntax/entityRelationshipDiagram.html) to represent tables and their relationships. -It is useful to visualize your data model in order to further [configure it](./configuring.md) if need be. +This is useful to review the data model before deciding whether any [configuration](./configuring.md) is needed. It looks like this: @@ -127,7 +124,7 @@ erDiagram `xml2db` is developed and used at the [French energy regulation authority (CRE)](https://www.cre.fr/) to process complex XML data. -Contributions are welcome, as well as bug reports, starting on the project's +Contributions and bug reports are welcome on the project's [issue page](https://github.com/cre-dev/xml2db/issues). If you find this package useful, you can give it a star on [`xml2db`'s GitHub repo](https://github.com/cre-dev/xml2db)! From f90c56c3d99e1e7075b483fe6ab06b8de3e4aa6a Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 21 Jun 2026 06:15:43 +0000 Subject: [PATCH 3/5] docs: replace em-dashes with natural punctuation Swap all em-dash asides introduced in the previous rewrite for commas, colons, full stops, or parentheses as appropriate. Covers all five edited files plus the api/overview.md multiprocessing section. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_013pzr3Q9XGumr7WVfHGGtXV --- docs/api/overview.md | 12 ++++++------ docs/configuring.md | 2 +- docs/getting_started.md | 2 +- docs/how_it_works.md | 16 ++++++++-------- docs/index.md | 2 +- 5 files changed, 17 insertions(+), 17 deletions(-) diff --git a/docs/api/overview.md b/docs/api/overview.md index 9a20f79..cfd15d9 100644 --- a/docs/api/overview.md +++ b/docs/api/overview.md @@ -1,6 +1,6 @@ --- title: "API Overview" -description: "Overview of the xml2db Python API: building DataModel objects, parsing and loading XML files into a database, and extracting data back to XML — including multiprocessing patterns." +description: "Overview of the xml2db Python API: building DataModel objects, parsing and loading XML files into a database, and extracting data back to XML, including multiprocessing patterns." --- # API Overview @@ -64,13 +64,13 @@ XML parsing is CPU-bound and scales well across processes. Loading into the database, however, must be coordinated to avoid conflicts on shared tables. The right level of synchronisation depends on the backend: -* **DuckDB (file-based)** — only one active writer is allowed at a time, so +* **DuckDB (file-based)**: only one active writer is allowed at a time, so all database I/O must be serialised. -* **PostgreSQL, MS SQL Server, …** — concurrent writes to *different* temp +* **PostgreSQL, MS SQL Server, …**: concurrent writes to *different* temp tables are safe (each process gets a unique temp-table prefix), but the final merge into the shared target tables should be serialised. -The simplest approach — and the one shown below — is to serialise the entire +The simplest approach (and the one shown below) is to serialise the entire database phase with a `multiprocessing.Lock`, keeping only the parsing step parallel. This works correctly for all backends. @@ -121,8 +121,8 @@ if __name__ == "__main__": [`Document.insert_into_target_tables`](document.md/#xml2db.document.Document.insert_into_target_tables) into separate calls to [`Document.insert_into_temp_tables`](document.md/#xml2db.document.Document.insert_into_temp_tables) - (run concurrently — each process has a unique temp-table prefix so there - are no collisions) and + (run concurrently, since each process has a unique temp-table prefix, so + there are no collisions) and [`Document.merge_into_target_tables`](document.md/#xml2db.document.Document.merge_into_target_tables) (serialised via lock). diff --git a/docs/configuring.md b/docs/configuring.md index 04e071f..2199d10 100644 --- a/docs/configuring.md +++ b/docs/configuring.md @@ -243,7 +243,7 @@ in or out otherwise, with the following option: ### Deduplication -By default, `xml2db` deduplicates elements — identical elements are stored only once — which is particularly useful +By default, `xml2db` deduplicates elements (storing each unique element only once), which is particularly useful when an XML element specifies a feature shared by many other elements. This is done using a hash of each node in the XML file, which includes recursively all its children. The detailed diff --git a/docs/getting_started.md b/docs/getting_started.md index 3138e2a..b6e6b83 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -154,7 +154,7 @@ See the [How it works](how_it_works.md) page for a deeper explanation of the loa ## Getting back the data into XML -Data can be extracted from the database back to XML — primarily useful for round-trip testing. +Data can be extracted from the database back to XML, primarily for round-trip testing. ``` py title="Extract data back to XML" linenums="1" document = data_model.extract_from_database( diff --git a/docs/how_it_works.md b/docs/how_it_works.md index 967711f..cb07b10 100644 --- a/docs/how_it_works.md +++ b/docs/how_it_works.md @@ -5,8 +5,8 @@ description: "Deep dive into how xml2db builds a relational data model from an X # How it works -This page explains in depth how data models are built and how XML data is loaded — useful for troubleshooting or -advanced use cases. +This page explains in depth how data models are built and how XML data is loaded, which is useful for troubleshooting +and advanced use cases. ## Building a data model @@ -18,7 +18,7 @@ children. Throughout this page, "properties" refers to the simple type children has complex type children, they are stored in their own tables and linked to their parents via a foreign key or a join table. -The resulting data model is mostly bijective with the source XML — in most cases, data can be extracted from the +The resulting data model is mostly bijective with the source XML. In most cases, data can be extracted from the database and converted back to XML. !!! note @@ -52,7 +52,7 @@ Within a tree, parent-child relationships have `1-1` or `1-n` cardinality. To al become `n-1` or `n-n` respectively. Optional children are handled the same way. A child node (identified by its hash) that appears under multiple parents will have several parents after -deduplication — whereas it had only one in the original tree. This example illustrates a `1-1` relationship converted +deduplication, whereas it had only one in the original tree. This example illustrates a `1-1` relationship converted to `n-1` after reuse: ```mermaid @@ -114,7 +114,7 @@ original tree structure. When processing tables in dependency order, the root ta specification](https://en.wikipedia.org/wiki/XML_Schema_(W3C)). Known unsupported cases are described below. Other edge cases may also fail and require adjustments. We recommend -thorough testing for unusual schemas — for example, you can implement round-trip tests (XML → database → XML) and +thorough testing for unusual schemas. For example, you can implement round-trip tests (XML → database → XML) and compare the output against the original. ### Recursive XSD @@ -142,13 +142,13 @@ First, we load the entire XML document into memory using `lxml` and extract the keeps a reference of its type and stores its data content. This task is achieved by the function [`XMLConverter.parse_xml`](api/xml_converter.md#xml2db.xml_converter.XMLConverter.parse_xml). -This constrains the maximum file size — in-memory parsing has memory limits, and the merge transaction adds a +This constrains the maximum file size: in-memory parsing has memory limits, and the merge transaction adds a server-performance constraint. In practice, `xml2db` handles files around 500 MB without issue. ### Computing hashes -Tree hashes are computed recursively by combining each node's hash with the hashes of its children — simple types, -attributes, and complex types alike. Children are processed in the order they appear in the XSD schema, making +Tree hashes are computed recursively by combining each node's hash with the hashes of its children: simple types, +attributes, and complex types. Children are processed in the order they appear in the XSD schema, making hashing fully deterministic. Right after this step, a hook function is called if provided in the configuration (top-level `document_tree_hook` option diff --git a/docs/index.md b/docs/index.md index 0ac862d..249e484 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,6 +1,6 @@ --- title: "Loading XML into a relational database" -description: "xml2db is a Python package that automatically maps an XSD schema to relational database tables and loads XML files into them — no custom mapping rules required." +description: "xml2db is a Python package that automatically maps an XSD schema to relational database tables and loads XML files into them, with no custom mapping rules required." --- # Loading XML into a relational database From 64ad39c9f759f146504a0bbf7b447e45e779beba Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 21 Jun 2026 06:23:12 +0000 Subject: [PATCH 4/5] docs: mention ETL alongside ELT on home page ETL is searched far more often than ELT and is what many teams call this pattern regardless of load order. Mentioning both is accurate and improves discoverability without keyword stuffing. Co-Authored-By: Claude Sonnet 4.6 Claude-Session: https://claude.ai/code/session_013pzr3Q9XGumr7WVfHGGtXV --- docs/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.md b/docs/index.md index 249e484..7fe8a6d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -11,7 +11,7 @@ description: "xml2db is a Python package that automatically maps an XSD schema t * it can handle complex XML files which cannot be denormalized into flat tables * it works out of the box, without any custom mapping rules. -`xml2db` fits naturally into an [ELT (Extract, Load, Transform)](https://docs.getdbt.com/terms/elt) pipeline. It loads +`xml2db` fits naturally into an ETL or [ELT (Extract, Load, Transform)](https://docs.getdbt.com/terms/elt) pipeline. It loads XML files into a relational model that stays close to the source data while remaining easy to query as flat database tables. From feb6d6a646607e02354d7d1b10c7130b44c50aaf Mon Sep 17 00:00:00 2001 From: Martin Vergier Date: Mon, 22 Jun 2026 09:50:05 +0000 Subject: [PATCH 5/5] docs: fix docstring and configuring.md inaccuracies MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Fix ngroup type annotation (Union[int, None] → Union[str, None]) - Add missing name_chain and has_suffix to DataModelColumn Args docstring; clarify ngroup and model_config descriptions - Fix add_relation_n docstring: "1-1" → "1-n" - Replace stale save_db references with insert_into_target_tables in get_merge_temp_records_statements docstrings - Fix >>> continuation lines to ... in DataModel doctest example - Remove empty Examples: section from extract_from_database - Fix "indentin" typo in XMLConverter.to_xml - Fix metadata_columns default: None → [] in configuring.md - Add base64Binary and decimal to join-eligible types list - Fix extra_args example: wrap bare Index in a list - Fix choice_transform label: "True (default)" → "True (force on)" - Correct elevation rules: 4-column threshold, columns-only count, and missing "no n-n parents" constraint Co-Authored-By: Claude Sonnet 4.6 --- docs/configuring.md | 21 +++++++++++---------- src/xml2db/model.py | 9 +++------ src/xml2db/table/column.py | 9 ++++++--- src/xml2db/table/duplicated_table.py | 7 ++++--- src/xml2db/table/reused_table.py | 8 ++++---- src/xml2db/table/table.py | 2 +- src/xml2db/xml_converter.py | 2 +- 7 files changed, 30 insertions(+), 28 deletions(-) diff --git a/docs/configuring.md b/docs/configuring.md index 2199d10..86b767b 100644 --- a/docs/configuring.md +++ b/docs/configuring.md @@ -28,7 +28,7 @@ Options apply at three levels: model, table, and field. The general structure of "document_tree_node_hook": None, "row_numbers": False, "as_columnstore": False, - "metadata_columns": None, + "metadata_columns": [], "tables": { "table1": { "reuse": True, @@ -125,7 +125,7 @@ defined as `sqlalchemy` types and will be passed to the `sqlalchemy.Column` cons ### Joining values for simple types -By default, XML simple type elements with types in `["string", "date", "dateTime", "NMTOKEN", "time"]` and max +By default, XML simple type elements with types in `["string", "date", "dateTime", "NMTOKEN", "time", "base64Binary", "decimal"]` and max occurrences >= 1 are joined in one column as comma separated values and optionally wrapped in double quotes if they contain commas (an Excel-like csv format, which can be queried with `LIKE` statements in SQL). @@ -150,12 +150,13 @@ automatically applied `join`, as it would require a complex process of adding a ### Elevate children to upper level -A complex child that occurs exactly once (`[1, 1]`) is always elevated to its parent by default. +A mandatory child (min occurrences = 1, i.e. `[1, 1]`) is always elevated to its parent by default, as long as it +is not involved in a 1-n relationship elsewhere in the schema. -An optional child (`[0, 1]`) is also elevated by default if it has fewer than 5 fields, to avoid cluttering the parent -with mostly-NULL columns. +An optional child (`[0, 1]`) is also elevated by default if it has 4 or fewer simple-type columns (relation fields +are not counted), and again only when it is not involved in a 1-n relationship elsewhere. -This behaviour can be disabled, or forced for children with more than 5 fields, using: +This behaviour can be disabled, or forced for larger children (more than 4 simple-type columns), using: `"transform":` `"elevate"` (default) or `"elevate_wo_prefix"` or `False` (disable). @@ -224,10 +225,10 @@ idOfMarketParticipant[1, 1] (choice): value[1, 1]: string ``` -This simplification is applied by default when there are more than 2 options of the same data type, but it can be opted -in or out otherwise, with the following option: +This simplification is applied automatically when there are more than 2 options of the same data type. It can be +forced on or disabled explicitly with the following option: -`"choice_transform":` `True` (default) or `False` (disable) +`"choice_transform":` `True` (force on) or `False` (disable) !!! example Disable choice group simplification for a choice group: @@ -290,7 +291,7 @@ Configuration: `"extra_args": []` (default) model_config = { "tables": { "my_table": { - "extra_args": sqlalchemy.Index("my_index", "my_column1", "my_column2"), + "extra_args": [sqlalchemy.Index("my_index", "my_column1", "my_column2")], } } } diff --git a/src/xml2db/model.py b/src/xml2db/model.py index 89bc1ab..05e70cf 100644 --- a/src/xml2db/model.py +++ b/src/xml2db/model.py @@ -63,9 +63,9 @@ class DataModel: Examples: Create a `DataModel` like this: >>> data_model = DataModel( - >>> xsd_file="path/to/file.xsd", - >>> connection_string="postgresql+psycopg2://testuser:testuser@localhost:5432/testdb", - >>> ) + ... xsd_file="path/to/file.xsd", + ... connection_string="postgresql+psycopg2://testuser:testuser@localhost:5432/testdb", + ... ) """ @@ -731,9 +731,6 @@ def extract_from_database( Returns: A [`Document`](document.md) object containing extracted data - - Examples: - """ doc = Document(self) doc.extract_from_database(self.root_table, root_select_where, force_tz=force_tz) diff --git a/src/xml2db/table/column.py b/src/xml2db/table/column.py index 7504c11..9579360 100644 --- a/src/xml2db/table/column.py +++ b/src/xml2db/table/column.py @@ -14,15 +14,18 @@ class DataModelColumn: Args: name: column name + name_chain: list of (field_name, type_name) pairs tracing the path from the root to this column, + used to reconstruct elevated fields when converting back to XML data_type: column data type occurs: min and max occurrences of the field min_length: min length max_length: max length is_attr: does the column value come from an xml attribute? + has_suffix: does the column name carry an ``_attr`` suffix to disambiguate it from a same-named element? is_content: is the column used to store the content value of a mixed complex type? allow_empty: is nullable ? - ngroup: a key used to handle nested sequences - model_config: data model config, may contain column type information + ngroup: a string key used to handle nested sequences (``str(hash(parent_node))``) or ``None`` + model_config: the table-level config dict; may contain a ``fields`` entry with a custom column type data_model: the DataModel object it belongs to Attributes: @@ -43,7 +46,7 @@ def __init__( has_suffix: bool, is_content: bool, allow_empty: bool, - ngroup: Union[int, None], + ngroup: Union[str, None], model_config: dict[str, Any], data_model: "DataModel", ): diff --git a/src/xml2db/table/duplicated_table.py b/src/xml2db/table/duplicated_table.py index e67ca7e..8131783 100644 --- a/src/xml2db/table/duplicated_table.py +++ b/src/xml2db/table/duplicated_table.py @@ -130,9 +130,10 @@ def get_merge_temp_records_statements(self) -> Iterable[Any]: into the target tables (unprefixed). As this kind of node can be duplicated, no unique constraint \ is used, but a record is inserted only if its parent record is inserted too. - This method should not be called directly but through the save_db method in the :class:`xml2db.Document` \ - object holding the parsed XML document data, which will ensure that merge queries are issued in the \ - correct order, and which will encapsulated all queries in a transaction in order to rollback changes on failure. + This method should not be called directly but through + :meth:`~xml2db.document.Document.insert_into_target_tables`, which ensures that merge + queries are issued in the correct order and wraps them in a transaction so that changes + are rolled back on failure. """ # update foreign keys and temp_exists based on parent table diff --git a/src/xml2db/table/reused_table.py b/src/xml2db/table/reused_table.py index 754cc38..a0a9fe3 100644 --- a/src/xml2db/table/reused_table.py +++ b/src/xml2db/table/reused_table.py @@ -147,10 +147,10 @@ def get_merge_temp_records_statements(self): looking up first existing records with the same hash in order to reuse already existing records when the new record is identical. - This method should not be called directly but through the save_db method in the Document - class, which will ensure that merge queries are issued in the correct order for all the - data flow, and which will encapsulated all queries in a transaction in order to rollback - changes on failure. + This method should not be called directly but through + :meth:`~xml2db.document.Document.insert_into_target_tables`, which ensures that merge + queries are issued in the correct order and wraps them in a transaction so that changes + are rolled back on failure. """ # find matching records hash in target table diff --git a/src/xml2db/table/table.py b/src/xml2db/table/table.py index f2a1972..db64a99 100644 --- a/src/xml2db/table/table.py +++ b/src/xml2db/table/table.py @@ -202,7 +202,7 @@ def add_relation_n(self, name, other_table, occurs, ngroup): """Helper to add a 1-to-many relationship Args: - name: name of the 1-1 relationship + name: name of the 1-n relationship other_table: the child table of the relationship occurs: min and max occurs for this relationship ngroup: a string id signaling that the relation belongs to a nested sequence diff --git a/src/xml2db/xml_converter.py b/src/xml2db/xml_converter.py index 1dbbf8a..a16f401 100644 --- a/src/xml2db/xml_converter.py +++ b/src/xml2db/xml_converter.py @@ -401,7 +401,7 @@ def to_xml( Args: out_file: If provided, write output to a file. nsmap: An optional namespace mapping. - indent: A string used as indentin XML output. + indent: A string used as indent in XML output. Returns: The etree object corresponding to the root XML node.