chore(): Added benchmark tests for parquet and hdf5

[Ichbinkiana]

No description provided.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.56%. Comparing base (34a19f0) to head (359c381).
⚠️ Report is 72 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2509      +/-   ##
============================================
+ Coverage     71.38%   71.56%   +0.17%     
- Complexity    48756    49047     +291     
============================================
  Files          1571     1574       +3     
  Lines        188912   189565     +653     
  Branches      37067    37188     +121     
============================================
+ Hits         134858   135663     +805     
+ Misses        43603    43430     -173     
- Partials      10451    10472      +21     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[janniklinde]

Thanks for the first PR @Ichbinkiana. I am missing concrete results and interpretations. Did you get some insights from the benchmarks that you could share?

Please move those benchmarks to the performance test package and do not use JUnit to run those, because it adds overhead which may distort the results.

[Ichbinkiana]

Summary of changes

Latest commits moved the HDF5 benchmark away from JUNIT and also introduced two HDF5 I/O optimizations as below:

• Added HDF5 benchmarks which by default disabled
• My Benchmark records write/read time, file size, number of files, heap usage, GC stats, parallelism, and correctness status in both Json and CSV format
• Added batched HDF5 writing for both dense and sparse inputs
• Cleaned up HDF5 write to avoid unnecessary boxed Double and used Java primitive types instead.
• Now the write and read are sparse-aware and this can be enabled for sparse matrices with below args:

• Dsysds.hdf5.write.sparse.layout=coo
• Dsysds.hdf5.read.sparse.layout=coo

• Added a fallback so ReaderHDF5Parallel uses the sparse-aware sequential reader when Coordinate layout is enabled for sparse test cases.

---

Results

For dense sequential writing:

• Baseline: ~7.1 s
• After batching: ~4.4 s
• Improvement: ~37% faster

For sparse matrix writing with 125000 x 1000 and 125000 nonzeros:

• Baseline dense-layout file size: ~1.0 GB

• optimized file size: ~3.0 MB

• Baseline sparse write: several seconds

• Optimized sparse write: ~23 ms

• Optimized sparse read: ~18 ms

---

Interpretations/ Reasoning

• Batching improved dense write performance by calling write less and in batches instead of row-by-row.
• The large sparse improvement came from writing only nonzero entries instead of all entries.

Why did i use synthetic Data

• SystemDS HDF5 implementation works with matrices, which only support doubles. Real HDF5 files can contain non-double entries that are outside the current matrix reader/writer scope.
  I also used synthetic dense and sparse matrices to isolate and validate my optimized code behavior under controlled conditions.

[Ichbinkiana]

I'll also push the optimization with results and interpretation for Parquet reader and writer by tomorrow EoD.
And i'll finalise the comparison to external benchmarks before deadline.
Many thanks for your patience and review.