Repository navigation

[WIP] CLI program to write ALP encode example parquet files - #49154

Closed

alamb wants to merge 116 commits into

apache:mainfrom

alamb:alamb/example_encoding_writer

Closed

[WIP] CLI program to write ALP encode example parquet files#49154
alamb wants to merge 116 commits into
apache:mainfrom
alamb:alamb/example_encoding_writer

Commits on Dec 8, 2025
- Add alp code
  
  sfc-gh-pgaur
  and
  dhirhank
  committed
  0d442fd
- Integrate ALP with arrow
  sfc-gh-pgaur
  committed
  06d1e19
- Add alp benchmark
  sfc-gh-pgaur
  committed
  a98c594
- Add datasets for alp benchmarking
  sfc-gh-pgaur
  committed
  c297f97
- Update cmake file
  sfc-gh-pgaur
  committed
  ab928e8
- Move hpp files to h
  sfc-gh-pgaur
  committed
  6a95a59
- Update flow digram and layout digram to use ASCII and not unicode characters
  sfc-gh-pgaur
  committed
  865e46a
- Rename cpp files to cc
  sfc-gh-pgaur
  committed
  cb6d0b6
- Update documentation to align with arrow's doxygen style
  sfc-gh-pgaur
  committed
  496e23b
- Adapt methods and variable names to arrow style
  sfc-gh-pgaur
  committed
  8803b52
- Update the tests to adhere to arrow style code
  sfc-gh-pgaur
  committed
  31e94ec
- Update callers
  sfc-gh-pgaur
  committed
  46c0ecc
- Fuse FOR and decode loop
  sfc-gh-pgaur
  committed
  a70b08f
- Reduce memory allocation in the decompress call
  sfc-gh-pgaur
  committed
  ccbb1dd
- Attempt at making decoding faster with SIMD
  sfc-gh-pgaur
  committed
  6a01df2
- Revert "Attempt at making decoding faster with SIMD"
  sfc-gh-pgaur
  committed
  4ced783
- Move cpp files to cc
  sfc-gh-pgaur
  committed
  4fac73c
- Move data file to parquet-testing submodule
  sfc-gh-pgaur
  committed
  1cb0852
Commits on Dec 9, 2025
- Update path to the data file
  sfc-gh-pgaur
  committed
  8d307a6
Commits on Dec 15, 2025
- Adapt files names to arrow convention
  sfc-gh-pgaur
  committed
  0908342
- File rename
  sfc-gh-pgaur
  committed
  e56c877
- Obtain compressed size and number of elements from page header
  sfc-gh-pgaur
  committed
  cfa00ba
Commits on Dec 16, 2025
- Fix namespace depth
  sfc-gh-pgaur
  committed
  a1d11ee
- Better pack the compression block header
  sfc-gh-pgaur
  committed
  719468b
- Rename class
  sfc-gh-pgaur
  committed
  69b4e07
- Rearrage field for vector metadata for better packing
  sfc-gh-pgaur
  committed
  193a808
Commits on Dec 31, 2025
- Add spec files
  sfc-gh-pgaur
  committed
  11aaab3
Commits on Jan 2, 2026
- Add more tests
  sfc-gh-pgaur
  committed
  c7e7569
Commits on Jan 12, 2026
- Move number of elements metadata info to uber header
  sfc-gh-pgaur
  committed
  acbc8ba
- Keep version as 1
  sfc-gh-pgaur
  committed
  6b3e2ed
- Tests pass
  sfc-gh-pgaur
  committed
  506088f
- Get rid of numElements and derive it for each vector
  sfc-gh-pgaur
  committed
  f5f5011
- Update the md files
  sfc-gh-pgaur
  committed
  e0e367c
Commits on Jan 13, 2026
- Fix ubsan error
  sfc-gh-pgaur
  committed
  81bcc81
- Use safe copy
  sfc-gh-pgaur
  committed
  71b244a
- Add more unit tests
  sfc-gh-pgaur
  committed
  18e3c59
Commits on Jan 14, 2026
- Templatize AlpEncodedVectorInfo
  sfc-gh-pgaur
  committed
  687e1af
- Fix
  sfc-gh-pgaur
  committed
  3a538cc
Commits on Jan 16, 2026
- [SpecReview] Num values in page is at max int32
  sfc-gh-pgaur
  committed
  8b942af
- [SpecReview] Rename layout to IntegerEncoding
  sfc-gh-pgaur
  committed
  82a3c3b
- [SpecReview] Rename enum
  sfc-gh-pgaur
  committed
  5b4d16d
- [SpecReview] Make AlpHeader exact 8 bytes
  sfc-gh-pgaur
  committed
  ac2d679
- [SpecReview] Move all vector data towards the start
  sfc-gh-pgaur
  committed
  f66fe95
- [SpecReview][Metadata] Introduce wrapper to read metadata
  sfc-gh-pgaur
  committed
  c67c745
- [SpecReview] Separate out ALP and FOR metadata
  sfc-gh-pgaur
  committed
  37ac31d
- [SpecReview] Remove unnecessary field
  sfc-gh-pgaur
  committed
  58df7a3
- [FutureFlexibility] Add the FOR specific code behind switch
  sfc-gh-pgaur
  committed
  50f380c
Commits on Jan 20, 2026
- Add unit test
  sfc-gh-pgaur
  committed
  283de64
Commits on Jan 26, 2026
- ALP: Implement offset-based layout for O(1) random access
  sfc-gh-pgaur
  committed
  777c9a8
- ALP: Update documentation for offset-based layout
  sfc-gh-pgaur
  committed
  63cbec9
Commits on Jan 27, 2026
- ALP: Add CreateSamplingPreset and EncodeWithPreset APIs
  sfc-gh-pgaur
  committed
  b813a6d
- ALP: Update all comments and docs for offset-based layout
  sfc-gh-pgaur
  committed
  bdb28ec
Commits on Mar 7, 2026
Commits on Mar 9, 2026
- ALP: Add float32 cross-language tests and generator support
  sfc-gh-pgaur
  committed
  8cd7402
Commits on Apr 22, 2026
- Address code review comments
  sfc-gh-pgaur
  committed
  1eaa75d
Commits on Apr 27, 2026
Commits on May 6, 2026
- Address review nits: uint64_t→int64_t, spell out ALP, use uint8_t for byte spans
  sfc-gh-pgaur
  committed
  6c65848
- Move decoder creation outside benchmark loop
  sfc-gh-pgaur
  committed
  9fdea7a
- Rename alp_wrapper.h/.cc to alp_codec.h/.cc
  sfc-gh-pgaur
  committed
  cb5acfc
- Revert CMake flag changes (restore -O2 and armv8-a)
  sfc-gh-pgaur
  committed
  54b86b7
- Convert ARROW_CHECK to Status returns on decode/load paths
  sfc-gh-pgaur
  committed
  d39cc66
- Add precondition docs to Store methods
  sfc-gh-pgaur
  committed
  3e6b27d
- Use signed integer types consistently per Arrow style guide
  sfc-gh-pgaur
  committed
  c61484d
Commits on May 7, 2026
- Move out parameters to last position and remove unused enforce_mode
  sfc-gh-pgaur
  committed
  f8ccabd
Commits on May 13, 2026
Commits on May 14, 2026
- Address reviewer feedback: use int64_t, reorder out-params last, clean up naming
  sfc-gh-pgaur
  committed
  e0dd429
Commits on Jun 3, 2026
- Add sign branching to FastRound for correct rounding of negative values
  sfc-gh-pgaur
  committed
  347517c
Commits on Jun 8, 2026
Commits on Jun 25, 2026
Commits on Jun 26, 2026
Commits on Jun 28, 2026
- Code review changes
  sfc-gh-pgaur
  committed
  658aa56
Commits on Jun 30, 2026
Commits on Aug 3, 2026
- [WIP] test driver for creating ALP encoded files
  alamb
  committed
  7ad79b7
- example
  alamb
  committed
  d873373
Commits on Aug 4, 2026
- Update for synthetic files
  alamb
  committed
  e41c4c5
Commits on Aug 7, 2026
- Add some nan variety and a large FOR value
  alamb
  committed
  195ebd5

Commits

Commits on Dec 8, 2025

Commits on Dec 9, 2025

Commits on Dec 15, 2025

Commits on Dec 16, 2025

Commits on Dec 31, 2025

Commits on Jan 2, 2026

Commits on Jan 12, 2026

Commits on Jan 13, 2026

Commits on Jan 14, 2026

Commits on Jan 16, 2026

Commits on Jan 20, 2026

Commits on Jan 26, 2026

Commits on Jan 27, 2026

Commits on Mar 7, 2026

Commits on Mar 9, 2026

Commits on Apr 22, 2026

Commits on Apr 27, 2026

Commits on May 6, 2026

Commits on May 7, 2026

Commits on May 13, 2026

Commits on May 14, 2026

Commits on Jun 3, 2026

Commits on Jun 8, 2026

Commits on Jun 25, 2026

Commits on Jun 26, 2026

Commits on Jun 28, 2026

Commits on Jun 30, 2026

Commits on Aug 3, 2026

Commits on Aug 4, 2026

Commits on Aug 7, 2026