Skip to content

fix: Ensure that prod/dev interval and physical model logic consistently uses Deployability - #5988

Open
davem-bis wants to merge 5 commits into
SQLMesh:mainfrom
davem-bis:feature/DRM/resolve-indirect-non-breaking-deployable-representative-mismatch
Open

fix: Ensure that prod/dev interval and physical model logic consistently uses Deployability#5988
davem-bis wants to merge 5 commits into
SQLMesh:mainfrom
davem-bis:feature/DRM/resolve-indirect-non-breaking-deployable-representative-mismatch

Conversation

@davem-bis

@davem-bis davem-bis commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #5793.

Changes the logic determining whether to use the prod or dev instance of intervals or physical model to rely solely on deployability. Previously this was inconsistently using either deployability and representativeness, leading to non-idempotent behaviour in non-prod environments.

Test Plan

  • Expansion of existing unit tests for changed logic.
  • Run through of INDIRECT_BREAKING_CHANGES snapshots to prove idempotence.
    • Run plan:
      1. sqlmesh init

        Option 1 throughout - basic DuckDB implementation with the demo models.

      2. sqlmesh plan

      3. Add 'ABC' AS extra_column to incremental_model.sql

      4. sqlmesh plan indirect_non_breaking

      5. sqlmesh run

      6. sqlmesh run indirect_non_breaking

      7. sqlmesh fetchdf "UPDATE sqlmesh._intervals SET end_ts = end_ts - 86400000, start_ts = start_ts - 86400000, last_altered_ts = last_altered_ts - 86400000"

        86400000 is the number of milliseconds in a day (246060*1000)

      8. sqlmesh run indirect_non_breaking

      9. sqlmesh run indirect_non_breaking

        Previously this would result in the intervals being re-run, it now correctly does nothing.

    • Comments:
      • Model Name Change Type Deployability Representativeness Version Used In indirect_non_breaking
        seed_model None True True Same as prod
        incremental_model NonBreaking True True Separate physical model
        full_model IndirectNonBreaking False True Separate physical model with dev extension
      • full_model now uses the __dev physical model, and derives missing intervals from there, and points at that physical model in the virtual layer.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

…deployability.

Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
…ndirect-non-breaking-deployable-representative-mismatch
…ty instead of representativeness for determining whether dev or prod instance of each should be used.

Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
@davem-bis
davem-bis marked this pull request as ready for review August 26, 2026 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: INDIRECT_NON_BREAKING snapshots in dev environments write to a table nobody reads, causing repeated re-backfills

1 participant