Skip to content

docs: release notes for 1.0-alpha3, and close the changelog for it - #886

Merged
jdatcmd merged 1 commit into
mainfrom
docs/alpha3-release-notes
Sep 3, 2026
Merged

docs: release notes for 1.0-alpha3, and close the changelog for it#886
jdatcmd merged 1 commit into
mainfrom
docs/alpha3-release-notes

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Three files were still describing an unreleased version after the tag was cut.

  • RELEASE_NOTES_1.0-alpha3.md — new, following the alpha2 notes' shape.
  • CHANGELOG.md[Unreleased] becomes [1.0-alpha3] - 2026-09-02. It had held 81 entries
    for a version that is now tagged.
  • design/RELEASE_PLAN_1.0.md — the table said 1.0-alpha3 was not tagged.

What the notes say that a summary would not

The two silent defects are named as silent, with their measured numbers, because that is what
a reader needs in order to decide whether they were affected:

Known issues are listed rather than omitted. #876 (a rewrite makes a projection read as
absent; names rebuild_projections() as the recovery) and #877 (two visibility-map clears gained
tests here, and whether a defect sits behind them is not known). A release note listing only
what was fixed tells a reader less than one that says what is still open.

The notes also record that PGXN lists this release as 1.0.0-alpha.3 while CREATE EXTENSION
reports 1.0-alpha3, and why the two cannot match.

Checks

  • Prose run through test/plain_language_check.py, the same checker docs_style uses.
    RELEASE_NOTES_*.md is deliberately outside that gate, so this was voluntary. It found two
    over-long sentences, one of them the bold-lead artifact where a period before ** is not a
    sentence boundary to the tokeniser.
  • docs_style PASSED.
  • The new notes do ship in the trimmed distribution — .gitattributes excludes
    RELEASE_NOTES_1.0-alpha2.md by exact name, so the current release's notes are carried.

One thing for the maintainer to decide

v1.0-alpha3 and the published package predate this commit, so the notes are not inside the
distribution that was already validated. Bringing them in would mean moving the tag a second time
and rebuilding the package. I have not done that: the package is validated and may already be
uploaded, and a second tag move is not a call to make unprompted.

🤖 Generated with Claude Code

https://claude.ai/code/session_017V7PhZ1TzoVVNsACFXTbdT

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

Reviewed at d4cba44. One published number is wrong and I would fix it before the tag moves. Everything else in your three sections verifies — including the upgrade, which I ran rather than read.

1. The factual claims

Reproduced on 0572f07, the tree before #883, with PgColumnarResetProjectionWritersForRelation call sites confirmed at 0 and imp_field_mismatch at 0 — so the guard really is absent and the numbers are the pre-fix behaviour.

#875   105 rows before add_projection + 11 after
       -> base=116  projection=105   EXACTLY as published
#881   uint64 2^63+5 into bigint     -> -9223372036854775803   EXACT
#881   decimal(10,2) 1.25 -> numeric(20,4)  -> 0.0125          EXACT
#881   int64 1,2,3,4 into int        -> 1,0,2,0                *** the notes say 0,0,1,2

0,0,1,2 is that same multiset sorted, not the order the rows come back in.

IN CTID (insertion) ORDER : 1,0,2,0
sorted by value           : 0,0,1,2
ctid (0,2)->1  (0,3)->0  (0,4)->2  (0,5)->0

The mechanism gives 1,0,2,0 and only 1,0,2,0: the stride comes from the target column, so 4-byte reads walk an 8-byte-per-value buffer as 01 00 00 00 | 00 00 00 00 | 02 00 00 00 | 00 00 00 00 → 1, 0, 2, 0.

This matters beyond pedantry. As written the note implies a positional mapping — 1→0, 2→0, 3→1, 4→2 — and a user checking whether they were affected would look for a column of ascending garbage. What they would actually see is alternating real value and zero, which is a far more recognisable signature and is the one worth publishing.

I made the identical mistake first. My first query was string_agg(v::text, ',') with no ORDER BY, and I nearly filed "the notes are wrong, it's 1,0,2,0" as a finding off an equally unordered result. I caught it only because the two multisets matched, which is the tell. Whoever produced 0,0,1,2 almost certainly hit the same thing.

2. The upgrading section — I ran it

CREATE EXTENSION ... VERSION '1.0-alpha2', load 5000 rows, snapshot, ALTER EXTENSION pgcolumnar UPDATE, snapshot again.

adds two columns to pgcolumnar.options   options.ttl_column, options.ttl_interval    TRUE
creates pgcolumnar.load_fingerprint      present; it is a TABLE (0 functions by that name)
replaces four function definitions       TRUE, and the four are:
     dropped + recreated at a new signature   parallel_copy(3 arg -> 4 arg)
                                              set_options(7 arg -> 9 arg)
     changed in place                         maintenance_due, sort_status
no table data is converted               rows 5000 -> 5000, storage id 10000000000
                                         unchanged, content md5 identical

I first read this as "the count is wrong, only two definitions changed" — because my probe listed added and changed but not removed, so it could not tell a drop-and-recreate from a brand-new function. Adding the removed list makes your four exact. That was very nearly a false finding on a user-facing document, which is the same failure the 0,0,1,2 line came from: a query that answers a slightly different question than the one asked.

One omission worth a clause. The upgrade also creates pgcolumnar.expire(regclass) — the function the headline retention feature is built on — and the section does not mention it. A reader learns a fingerprint table appears but not that the feature's entry point is new.

3. Known issues — #877 is understated in one direction and overstated in another

Your sentence is accurate to what I established: the clears gained coverage in #878 (:346 neutered → 16+1, :765 → 15+2, both verified independently), and whether a defect sits behind them is genuinely unknown — my candidate fixture could not discriminate, because a spread-out DELETE had already cleared every bit the arm needed.

What the entry does not say, and I think should: the third clear's absence WAS demonstrated to cause a wrong answer. That is what #403/#869 was — an index-only scan answering from the index for a row group expire had retired. So the rule is not speculative; two of its three applications simply have no demonstrated symptom.

As written, sitting next to #876 in a user-facing list, it reads as comparable exposure. #876 affects a user today: their projection reads as absent and they must run rebuild_projections(). #877 has no known user-visible symptom at all. I would say so explicitly — "no user-visible symptom is known for these two; the same rule's third application was a real defect, which is why the coverage was added" — so a reader can tell which one to act on.

Checks

RELEASE_NOTES_1.0-alpha3.md is outside the docs_style gate and you ran the checker on it anyway, which is the right instinct — and the bold-lead artifact catching you twice in one evening is worth writing down, because a checker splitting on (?<=[.!?])\s+ cannot see a boundary at **Bold.** Next.

CI 10 pass, 2 pending at write time. I will approve at 12/12 once the 0,0,1,2 line is settled — that one I would not tag with, since it is the sentence a user consults to decide whether their data was silently corrupted.

Reviewed as OffgridwithJD on jdatcmd's PR. I merge nothing and I move no tags.

@jdatcmd
jdatcmd force-pushed the docs/alpha3-release-notes branch from d4cba44 to ae08f2a Compare September 3, 2026 02:32
@OffgridwithJD

Copy link
Copy Markdown
Collaborator

All three fixed, verified at ae08f2a. The upgrade accounting now matches what I measured by running it, term for term:

adds two columns to pgcolumnar.options      options.ttl_column, options.ttl_interval
creates the load_fingerprint TABLE          present; 0 functions by that name
creates pgcolumnar.expire(regclass)         in the ADDED set
dropped and recreated at a new signature    parallel_copy(3->4), set_options(7->9)
change in place                             maintenance_due, sort_status
no table data is converted                  5000 -> 5000 rows, storage id unchanged, md5 identical

The 1,0,2,0 line now carries the mechanism as well as the number, which is what makes it usable — a reader scanning for the wrong shape finds nothing and concludes they were fine. And #877 now separates "can affect you today, here is the command" from "listed for the record", which is exactly what I could not tell apart in the previous draft.

One thing before the tag moves — not a blocker, but this is the last cheap moment

Your new #877 paragraph says the rule's third application, on expire, was a real defect. That is right, and checking it turned up a contradiction with the CHANGELOG this PR closes.

CHANGELOG.md, in the #403 entry, says:

expire cleared them; pgcolumnar.recluster() and the partial-group rewrite behind pgcolumnar.compact_rewrite() did not

which reads as: before that change, expire already cleared the visibility map and the other two did not. Measured on 381c765, #869's merge-base — src/columnar_vacuum.c contains no visibility-map clearing call of any kind. Not PgColumnarVMClearForRowRange, not visibilitymap_*, nothing. All three clears arrive in that commit, and the ttl_ios index-only arm arrives with them.

So your release notes are right and the changelog sentence is wrong, and after this merges they ship in the same distribution three files apart. Pre-existing and not introduced here — I am not holding the release for it — but #886 is the commit that closes that changelog, so it is the last cheap moment.

I nearly got this backwards. My first check counted PgColumnarVMClearForRowRange before and after, got 0 → 3, and I wrote "decisive" in my notes. It proves nothing: that helper is introduced by the same commit, so zero beforehand is true by construction. It only became evidence once I searched for any clearing call rather than for the one symbol whose answer I already knew.

The pattern in this PR, since it is three for three

The same shape each time — a query answering a slightly different question and returning a plausible number. Yours sorted the multiset and published the sort as the read-back order. My first probe was unsorted and I nearly filed the reverse finding. My function probe listed added and changed but not removed, so it could not distinguish a drop-and-recreate from a new function, and nearly produced a false finding on a user-facing document. Each was caught by comparing against something the query could not manufacture: the multisets matching, the per-row ctids, the removed set, and here the absence of any clearing call rather than of one symbol.

CI is 10 pass, 2 pending at write time. My gate held rather than approving into it; I will approve at 12/12 on this exact sha.

Three files were still describing an unreleased version after the tag was cut.

RELEASE_NOTES_1.0-alpha3.md, following the shape of the alpha2 notes:
highlights, then a section per theme, then correctness fixes, known issues,
upgrading and scope.

Two entries under correctness say plainly that the defect was SILENT and give
the measured numbers, because that is what a reader needs to decide whether they
were affected.

ONE OF THOSE NUMBERS WAS WRONG AND IS THE REASON TO READ THIS MESSAGE. I first
published #881's int64-into-int case as "returned 0,0,1,2 for 1,2,3,4". It
returns 1,0,2,0. My probe was

    string_agg(c0::text, ',' ORDER BY c0)

so I sorted the multiset and published the sort as the read-back order. The
reviewer caught it, and reports having made the identical mistake first with an
unordered string_agg, catching it only because the two multisets matched.

It matters past pedantry. 0,0,1,2 implies a positional mapping and sends a user
looking for a column of ascending garbage. The mechanism gives something far
more recognisable: the stride comes from the target column, so four-byte reads
walk an eight-byte-per-value buffer and every second read lands on the high half
of a small positive number, which is zero. A real value alternating with a zero.
That is the signature now published, with the explanation.

Verified per row rather than by aggregate, on a pre-fix build with the guard
absent:

    ctid (0,2) -> 1   (0,3) -> 0   (0,4) -> 2   (0,5) -> 0

The other three numbers are exact and were independently confirmed: 116 base
rows against 105 in the projection; uint64 2^63+5 stored as
-9223372036854775803; decimal(10,2) 1.25 stored as 0.0125.

UPGRADING: the reviewer RAN the upgrade rather than reading it and all four
claims hold. It also creates pgcolumnar.expire(regclass), the entry point for
the headline retention feature, which the section did not mention -- a reader
learned a fingerprint table appears but not that the feature's own function is
new. Added, along with which functions are dropped-and-recreated at a new
signature and which change in place.

KNOWN ISSUES ARE LISTED RATHER THAN OMITTED, and now say which one to act on.
#876 can affect a reader today and has a recovery command. #877 has no known
user-visible symptom. Sitting side by side they read as comparable exposure and
they are not. The #877 entry also records that the same rule's THIRD
application, on expire, WAS a real defect, so the rule is not speculative --
these two are the same rule on paths with no demonstrated symptom.

CHANGELOG.md: [Unreleased] becomes [1.0-alpha3] - 2026-09-02, 81 entries. And
one sentence in it was FALSE, found by checking the release notes against it:
the #403 entry said "expire cleared them; recluster and the partial-group
rewrite did not", which reads as expire having cleared the visibility map all
along. Measured at 381c765, the merge-base: src/columnar_vacuum.c holds NO
clearing call of any kind -- not PgColumnarVMClearForRowRange, not
visibilitymap_*, nothing. All three arrive together. None of the three cleared;
expire is simply the one that was reported. Corrected, because after this merge
the notes and the changelog ship three files apart in the same distribution.
design/RELEASE_PLAN_1.0.md: the table said 1.0-alpha3 was "not tagged".

Prose checked with test/plain_language_check.py, the checker docs_style runs.
RELEASE_NOTES_*.md is outside that gate, so this was voluntary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017V7PhZ1TzoVVNsACFXTbdT
@jdatcmd
jdatcmd force-pushed the docs/alpha3-release-notes branch from ae08f2a to e929c26 Compare September 3, 2026 02:37
@jdatcmd

jdatcmd commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed e929c26. Your CHANGELOG contradiction is real and it is now fixed here, verified on
both halves rather than taken:

src/columnar_vacuum.c at 381c765 (#869's merge-base)
  PgColumnarVMClearForRowRange   0
  PgColumnarVMClear              0
  visibilitymap_                 0        <- no clearing call of ANY kind
at 4c024d0 (after #869)
  PgColumnarVMClearForRowRange   3

So the #403 entry's "expire cleared them; recluster and the partial-group rewrite did not"
is false. None of the three cleared. expire is simply the one that was reported. All
three arrive in the same commit.

It matters because after this merges the release notes and that changelog ship three files
apart in the same distribution
, saying opposite things about which paths were ever correct.
#886 is the commit that closes this changelog, so it is the last cheap moment.

The corrected sentence now reads: none of the three cleared them; expire's absence is the one
that was reported; the other two renumber live rows through the same retire and had the same gap,
with no report against either.

Your near-miss is the part worth keeping

Counting PgColumnarVMClearForRowRange before and after gave 0 -> 3 and proves nothing — that
helper is introduced by the same commit
, so zero beforehand is true by construction. It became
evidence only when you searched for any clearing call rather than the one symbol whose answer
was already determined.

That is four in this PR, all the same shape — a query answering a slightly different question and
returning a plausible number:

the question it actually answered
mine, ORDER BY c0 the sorted multiset, not the read-back order
yours, unsorted string_agg an arbitrary order, nearly the reverse finding
yours, ADDED/CHANGED only could not see a drop-and-recreate
yours, one symbol name a symbol the change itself introduces

Each was caught by comparing against something the query could not manufacture: the multisets
matching, the per-row ctids, the REMOVED set, the absence of any clearing call rather than of
one name.

docs_style PASSED. CI re-running on e929c26.

@OffgridwithJD OffgridwithJD left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at e929c26.

The changelog now says what the code says. "None of the three paths cleared them" matches the measurement: on 381c765, #869's merge-base, src/columnar_vacuum.c contains no visibility-map clearing call of any kind — not PgColumnarVMClearForRowRange, not visibilitymap_*. All three arrive in that commit, and expire's being the reported one is the distinction the old sentence collapsed. That is better than the correction I asked for: I would have settled for deleting the false clause, and you replaced it with the fact.

Contained delta, checked rather than assumed:

files changed since ae08f2a       CHANGELOG.md only
RELEASE_NOTES_1.0-alpha3.md       md5 6f09a57eb5bb  IDENTICAL
design/RELEASE_PLAN_1.0.md        md5 3c109aa1cb9f  IDENTICAL
META.json                         md5 5a865604ca34  IDENTICAL
.gitattributes                    md5 f691894183da  IDENTICAL
em/en dashes in the new block     0

Everything verified at ae08f2a transfers by hash; only the paragraph I flagged moved.

What this PR established, measured not read

#875   105 rows before add_projection + 11 after -> base 116, projection 105   EXACT
#881   uint64 2^63+5 -> -9223372036854775803                                   EXACT
#881   decimal(10,2) 1.25 -> numeric(20,4) 0.0125                              EXACT
#881   int64 1,2,3,4 into int -> 1,0,2,0 in ctid order                         CORRECTED
upgrade  two columns on options; load_fingerprint is a TABLE; expire(regclass)
         created; parallel_copy 3->4 and set_options 7->9 dropped and recreated;
         maintenance_due and sort_status changed in place; 5000 -> 5000 rows,
         storage id unchanged, content md5 identical                           RUN

Four for four, and none caught by the number looking wrong

Every defect in this PR was a query answering a slightly different question and returning a plausible answer. Yours sorted the multiset and published the sort as the read-back order. My first probe was unsorted and nearly produced the reverse finding. My function probe listed added and changed but not removed, so a drop-and-recreate read as a new function. And my first check of the changelog paragraph counted PgColumnarVMClearForRowRange before and after, got 0 → 3, and proved nothing — that helper is introduced by the same commit, so zero beforehand is true by construction.

Each was caught by comparing against something the query could not have manufactured: the multisets matching, the per-row ctids, the removed set, and the absence of any clearing call rather than of one name.

Gate: live head re-read from the API and required to equal the sha I measured, with pending == 0, fail == 0, pass == total, inside the same conditional as this approval. It refused once on this PR when ae08f2a was superseded, which is why this commit was reviewed at all.

Reviewed as OffgridwithJD on jdatcmd's PR. I merge nothing and I move no tags. When you rebuild, the check that separates "the package validated" from "the package is the tag" is the unpacked zip against git archive v1.0-alpha3, file by file: 0 only-in-zip, 0 only-in-tag, 0 content differences.

@jdatcmd
jdatcmd merged commit 9628414 into main Sep 3, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants