Repository navigation

ARROW-11362:[Rust][DataFusion] Use iterator APIs in to_array_of_size to improve performance - #9305

Closed

Dandandan wants to merge 62 commits into

apache:masterfrom

Dandandan:to_array_of_size_perf

Commits on Jan 18, 2021
- ARROW-11303: [Release][C++] Enable mimalloc in the windows verification script
  
  kszucs
  authored and
  kou
  committed
  69a9a1c
Commits on Jan 19, 2021
- ARROW-11309: [Release][C#] Use .NET 3.1 for verification
  kou
  committed
  903b41c
- ARROW-11315: [Packaging][APT][arm64] Add missing gir1.2 files
  
  kou
  authored and
  kszucs
  committed
  19e9559
- ARROW-11314: [Release][APT][Yum] Add support for verifying arm64 packages
  
  kou
  authored and
  kszucs
  committed
  17a3fab
- ARROW-7633: [C++][CI] Create fuzz targets for tensors and sparse tensors
  
  mrkn
  and
  pitrou
  committed
  275fda1
- ARROW-11246: [Rust] Add type to Unexpected accumulator state error
  
  ovr
  authored and
  alamb
  committed
  2d3e8f9
- ARROW-11254: [Rust][DataFusion] Add SIMD and snmalloc flags as options to benchmarks
  
  Dandandan
  authored and
  alamb
  committed
  e20f439
- ARROW-11074: [Rust][DataFusion] Implement predicate push-down for parquet tables
  
  yordan-pavlov
  authored and
  alamb
  committed
  18dc62c
- ARROW-10489: [C++] Add Intel C++ compiler options for different warning levels
  
  jcmuel
  authored and
  pitrou
  committed
  127961a
- ARROW-9128: [C++] Implement string space trimming kernels: trim, ltrim, and rtrim
  
  maartenbreddels
  authored and
  pitrou
  committed
  0e5d646
- ARROW-11305 Skip first argument (which is the program name) in parquet-rowcount binary
  
  jhorstmann
  authored and
  jorgecarleitao
  committed
  f63cffa
- ARROW-11108: [Rust] Fixed performance issue in mutableBuffer.
  jorgecarleitao
  committed
  7e0cb0a
- ARROW-11216: [Rust] add doc example for StringDictionaryBuilder
  
  alamb
  authored and
  jorgecarleitao
  committed
  b448de7
- ARROW-11268: [Rust][DataFusion] MemTable::load output partition support
  
  Dandandan
  authored and
  jorgecarleitao
  committed
  4a6eb19
- ARROW-11321: [Rust][DataFusion] Fix DataFusion compilation error
  
  Dandandan
  authored and
  alamb
  committed
  a4266a1
- ARROW-11156: [Rust][DataFusion] Create hashes vectorized in hash join
  
  Dandandan
  authored and
  alamb
  committed
  bbc9029
- ARROW-11313: [Rust] Fixed size_hint
  
  jorgecarleitao
  authored and
  alamb
  committed
  8e218e0
- ARROW-11222: [Rust] Catch up with flatbuffers 0.8.1 which had some UB problems fixed
  
  mqy
  authored and
  alamb
  committed
  35053fe
- ARROW-11277: [C++] Workaround macOS 10.11: don't default construct consts
  
  bkietz
  authored and
  nealrichardson
  committed
  50ba534
Commits on Jan 20, 2021
- ARROW-11322: [Rust] Re-opening `memory` module as public
  
  maxburke
  authored and
  jorgecarleitao
  committed
  a7633c7
- ARROW-11269: [Rust] [Parquet] Preserve timezone in int96 reader
  nevi-me
  committed
  555643a
- ARROW-11279: [Rust][Parquet] ArrowWriter Definition Levels Memory Usage
  
  Ryan Jennings
  authored and
  nevi-me
  committed
  e7c69e6
- ARROW-11318: [Rust] Support pretty printing timestamp, date, and timestamp types
  alamb
  committed
  71572bd
- ARROW-11311: [Rust] Fixed unset_bit
  
  jorgecarleitao
  authored and
  alamb
  committed
  ed709e0
- ARROW-11265: [Rust] Made bool not ArrowNativeType
  
  jorgecarleitao
  authored and
  alamb
  committed
  01c5aec
- ARROW-11290: [Rust][DataFusion] Address hash aggregate performance issue with high number of groups
  
  Dandandan
  authored and
  alamb
  committed
  6912869
- ARROW-11149: [Rust] DF Support List/LargeList/FixedSizeList in create_batch_empty
  
  ovr
  authored and
  alamb
  committed
  23550c2
- ARROW-11329: [Rust] Don't rerun build.rs on every file change
  
  mbrubeck
  authored and
  jorgecarleitao
  committed
  a0e1244
Commits on Jan 21, 2021
- ARROW-11220: [Rust] Implement GROUP BY support for Boolean
  
  ovr
  authored and
  alamb
  committed
  8b56f85
- ARROW-11330: [Rust][DataFusion] add ExpressionVisitor to encode expression walking
  alamb
  committed
  4601c02
- ARROW-11323: [Rust][DataFusion] Allow sort queries to return no results
  alamb
  committed
  84126d5
- ARROW-10831: [C++][Compute] Implement quantile kernel
  
  cyb70289
  and
  pitrou
  committed
  bd90043
- ARROW-11334: [Python][CI] Fix failing pandas nightly tests
  jorisvandenbossche
  committed
  72bf95a
- ARROW-11320: [C++] Try to strengthen temporary dir creation
  pitrou
  committed
  bc5d8bf
- ARROW-11141: [Rust] Add basic Miri checks to CI pipeline
  
  vertexclique
  authored and
  alamb
  committed
  c413566
Commits on Jan 22, 2021
- ARROW-11337: [C++] Compilation error with ThreadSanitizer
  
  westonpace
  authored and
  kou
  committed
  6959e46
- ARROW-11333: [Rust] Generalized creation of empty arrays.
  
  jorgecarleitao
  authored and
  nevi-me
  committed
  499b6d0
- ARROW-10299: [Rust] Use IPC Metadata V5 as default
  
  nevi-me
  authored and
  jorgecarleitao
  committed
  629a6fd
- ARROW-11343: [Rust][DataFusion] Simplified example with UDF.
  
  jorgecarleitao
  authored and
  alamb
  committed
  457fa91
- ARROW-10766: [Rust] [Parquet] Compute nested list definitions
  
  nevi-me
  authored and
  alamb
  committed
  251ecac
- ARROW-11332: [Rust] Use MutableBuffer in take_string instead of Vec
  
  Dandandan
  authored and
  alamb
  committed
  262bbdc
- Add from_iter_values to create arrays from (non null) values
  Dandandan
  committed
  37c70fb
- Remove borrow (they are primitive types anyway)
  Dandandan
  committed
  8cd118d
- Fix comment
  Dandandan
  committed
  3a63974
- ARROW-11299: [Python] Fix invalid-offsetof warnings
  
  cyb70289
  authored and
  bkietz
  committed
  b44a4ad
Commits on Jan 23, 2021
- ARROW-11291: [Rust] Add extend to MutableBuffer (-20% for arithmetic, -97% for length)
  
  authored and
  alamb
  committed
  13e2134
- ARROW-11319: [Rust] [DataFusion] Improve test comparisons to record batch, remove test::format_batch
  alamb
  committed
  67d0c2e
- Merge branch 'master' of github.com:apache/arrow into array_iter_non_null
  Dandandan
  committed
  79c92aa
- Use extend
  Dandandan
  committed
  941ee5d
- Use .collect() api
  Dandandan
  committed
  a37941c
- Use iterators in `to_array_of_size`
  Dandandan
  committed
  e448fcc
- Add benchmark
  Dandandan
  committed
  69c298e
Commits on Jan 24, 2021
- ARROW-11317: [Rust] Include the prettyprint feature in CI Coverage
  
  alamb
  authored and
  jorgecarleitao
  committed
  10f4ada
- Use none
  Dandandan
  committed
  d612b0f
- Use None for Microsecond as well
  Dandandan
  committed
  6144a23
- Use `None`
  Dandandan
  committed
  f2c4e26
Commits on Jan 25, 2021
- ARROW-11349: [Rust] Add from_iter_values to create arrays from (non null) values
  
  Dandandan
  authored and
  jorgecarleitao
  committed
  cf7638f
- Merge remote-tracking branch 'upstream/master' into to_array_of_size_perf
  Dandandan
  committed
  e07f7e5
- Use None for strings
  Dandandan
  committed
  eddf021
- fmt
  Dandandan
  committed
  555eb1d
Commits on Jan 26, 2021
- Merge remote-tracking branch 'upstream/master' into to_array_of_size_perf
  Dandandan
  committed
  8a20338
Commits on Jan 29, 2021
- Add license
  Dandandan
  committed
  32a9e0e

Commits

Commits on Jan 18, 2021

Commits on Jan 19, 2021

Commits on Jan 20, 2021

Commits on Jan 21, 2021

Commits on Jan 22, 2021

Commits on Jan 23, 2021

Commits on Jan 24, 2021

Commits on Jan 25, 2021

Commits on Jan 26, 2021

Commits on Jan 29, 2021