Repository navigation
fix(dora): stream the diff-based deployment fetch in calculateChangeLeadTime - #9281
Merged
Merged
Conversation
…eadTime Signed-off-by: mdelacruz-avantio <mdelacruz@avantio.com>
klesh
approved these changes
Oct 30, 2026
klesh
left a comment
Contributor
There was a problem hiding this comment.
Good job.
Thank you for your contribution.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
batchFetchDeploymentsloaded the wholecicd_deployment_commitsxcommits_diffsjoin into a slice of fullCicdDeploymentCommitrows before keeping the first row per merge commit. The join has one row per commit between consecutive production deployments for every merge commit in that range, so its size has no relation to the result. On a project with 14k merge commits it returned 936k rows and the lake container was OOMKilled on every nightly run.db.Cursoranddb.Fetchone row at a time into a struct withid,finished_dateandmerge_sha, keeping the first row per merge commit. Joins, filters andORDER BY cd.commit_sha, dc.started_date ASC, dc.id ASCare unchanged, so the first row seen is still the earliest deploymentdc.id, dc.finished_date, dc.commit_shainstead ofdc.*, the fields the conversion readscursor.Err()is checked after the loop, so a failedNextreturns an error instead of a silently truncated mapIdandFinishedDatepr8whose merge commit is in the diff range of two production deployments, where the deployment with the lower id started later. The expected deployment is the earlier-started one, which an order-by-id or last-wins implementation gets wrongMemory of the second phase on the instance above goes from about 1.6 GiB to a few MiB. The database work is the same as before.
Does this close any open issues?
Closes #9280
Screenshots
No UI change.
Other Information