Repository navigation
Render BOOLEAN keys as booleans when selecting a whole OPEN_STRUCT column - #20337
Open
raghavyadav01 wants to merge 1 commit into
Open
raghavyadav01 wants to merge 1 commit into
raghavyadav01 wants to merge 1 commit into
Conversation
…lumn
A BOOLEAN key is stored as INT 0/1, and the reconstruction behind SELECT
open_struct_col hands back the stored number, so the document read
{"flag":1} where col['flag'] returns true.
ProjectionBlock now reads the materialized keys declared BOOLEAN from the
per-key data sources and renders their 0/1 as false/true. A boolean that lives
in the sparse blob is already a JSON boolean and is unchanged. The
reconstruction shared with the seal path is not touched.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #20337 +/- ##
============================================
+ Coverage 71.23% 71.25% +0.01%
Complexity 1639 1639
============================================
Files 3594 3594
Lines 242224 242234 +10
Branches 39244 39248 +4
============================================
+ Hits 172560 172600 +40
+ Misses 56741 56705 -36
- Partials 12923 12929 +6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SELECT open_struct_colrendered a BOOLEAN key as1or0, whilecol['flag']returnstrue. The whole-document text now saystrueorfalse.Where it breaks
Take an OPEN_STRUCT column with a BOOLEAN key
flag(materialized) and an INT keycount, and a row withflag = true,count = 5:SELECT col['flag']truetrueSELECT col{"flag":1,"count":5}{"flag":true,"count":5}Pinot stores a BOOLEAN as INT 0/1.
OpenStructDataSource#openMapValueReader()hands back the stored number, andProjectionBlock#openStructDocumentsrendered it as is. A BOOLEAN that lives in the sparse blob is already a JSON boolean, so the same value read differently depending on whether the key was materialized.Fix
ProjectionBlockcollects the materialized keys declared BOOLEAN from the per-key data sources and renders their 0/1 asfalse/truein the top-level document. The reconstruction shared with the seal path is not touched. Nested objects are the source's own JSON and are unchanged.User-visible change
SELECT colon an OPEN_STRUCT column with a materialized BOOLEAN key now returnstrue/falseinstead of1/0. Worth a release-note line.Validation
ProjectionBlockOpenStructTest.testBooleanKeyRendersAsBoolean(new): read{"flag":1,"off":0,"count":5}before the fix.testSparseBooleanStaysBooleanpins the sparse side.pinot-coreOPEN_STRUCT, projection and item-transform tests: 583 run, 0 failures.