Skip to content

Render BOOLEAN keys as booleans when selecting a whole OPEN_STRUCT column - #20337

Open
raghavyadav01 wants to merge 1 commit into
apache:masterfrom
raghavyadav01:open-struct-document-boolean
Open

raghavyadav01 wants to merge 1 commit into
apache:masterfrom
raghavyadav01:open-struct-document-boolean

Conversation

@raghavyadav01

Copy link
Copy Markdown
Contributor

Summary

SELECT open_struct_col rendered a BOOLEAN key as 1 or 0, while col['flag'] returns true. The whole-document text now says true or false.

Where it breaks

Take an OPEN_STRUCT column with a BOOLEAN key flag (materialized) and an INT key count, and a row with flag = true, count = 5:

query before after
SELECT col['flag'] true true
SELECT col {"flag":1,"count":5} {"flag":true,"count":5}

Pinot stores a BOOLEAN as INT 0/1. OpenStructDataSource#openMapValueReader() hands back the stored number, and ProjectionBlock#openStructDocuments rendered it as is. A BOOLEAN that lives in the sparse blob is already a JSON boolean, so the same value read differently depending on whether the key was materialized.

Fix

ProjectionBlock collects the materialized keys declared BOOLEAN from the per-key data sources and renders their 0/1 as false/true in the top-level document. The reconstruction shared with the seal path is not touched. Nested objects are the source's own JSON and are unchanged.

User-visible change

SELECT col on an OPEN_STRUCT column with a materialized BOOLEAN key now returns true/false instead of 1/0. Worth a release-note line.

Validation

  • ProjectionBlockOpenStructTest.testBooleanKeyRendersAsBoolean (new): read {"flag":1,"off":0,"count":5} before the fix. testSparseBooleanStaysBoolean pins the sparse side.
  • Related pinot-core OPEN_STRUCT, projection and item-transform tests: 583 run, 0 failures.
  • Checkstyle, license and spotless clean.

…lumn

A BOOLEAN key is stored as INT 0/1, and the reconstruction behind SELECT
open_struct_col hands back the stored number, so the document read
{"flag":1} where col['flag'] returns true.

ProjectionBlock now reads the materialized keys declared BOOLEAN from the
per-key data sources and renders their 0/1 as false/true. A boolean that lives
in the sparse blob is already a JSON boolean and is unchanged. The
reconstruction shared with the seal path is not touched.
@codecov-commenter

codecov-commenter commented Nov 21, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.25%. Comparing base (9cb4ef8) to head (aa173ea).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #20337      +/-   ##
============================================
+ Coverage     71.23%   71.25%   +0.01%     
  Complexity     1639     1639              
============================================
  Files          3594     3594              
  Lines        242224   242234      +10     
  Branches      39244    39248       +4     
============================================
+ Hits         172560   172600      +40     
+ Misses        56741    56705      -36     
- Partials      12923    12929       +6     
Flag Coverage Δ
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 ?
java-25 71.25% <100.00%> (+0.01%) ⬆️
lane-a 100.00% <ø> (ø)
lane-b 0.00% <ø> (ø)
temurin 71.25% <100.00%> (+0.01%) ⬆️
unittests 71.25% <100.00%> (+0.01%) ⬆️
unittests1 60.48% <100.00%> (+<0.01%) ⬆️
unittests2 43.01% <0.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants