Skip to content

feat: FlatList.Parallel module with the vocabulary of the FSharp.Core Array.Parallel module - #27

Merged
xperiandri merged 14 commits into
mainfrom
feat/flat-list-parallel
Oct 19, 2026
Merged

xperiandri merged 14 commits into
mainfrom
feat/flat-list-parallel

Conversation

@xperiandri

@xperiandri xperiandri commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Proposed Changes

Adds the nested module FlatList.Parallel of ADR 0001 D11 (#23). #26 is merged, and so is #40, which split Helper.fs into root modules and moved InternalsVisibleTo into the project file, as the review of this pull request asked.

  • A nested [<RequireQualifiedAccess>] module Parallel at the end of the root module FlatList in FlatList.fsi/.fs, as Array.Parallel lives inside array.fs: the 32 functions of Array.Parallel except the three sortInPlace* (average, averageBy, choose, collect, exists, filter, forall, groupBy, init, iter, iteri, map, mapi, max, maxBy, min, minBy, partition, partitionWith, reduce, reduceBy, sort, sortBy, sortByDescending, sortDescending, sortWith, sum, sumBy, tryFind, tryFindIndex, tryPick, zip). The module has its own category banners, the compiled names of Array.Parallel and full XML documentation.
  • Shapes as in the main module: tryFind, tryFindIndex and tryPick return voption, choose and tryPick take voption choosers, and zip, groupBy, partition and partitionWith return struct tuples. Nested Parallel modules in the root modules Option.FlatList and Tuple.FlatList repeat them with options and reference tuples.
  • Most functions unwrap the array without a copy and call Array.Parallel. Six functions use new shared algorithms in the file Helper/ParallelArrays.fs (the root module ParallelArrays, after Arrays.fs in the project file), so that nothing is allocated per element:
    • choose maps to value options in parallel and compacts them sequentially;
    • tryFind and tryFindIndex run the same parallel loop with Break as FSharp.Core;
    • tryPick keeps the value of the lowest index under a lock;
    • zip uses the chunks of Array.Parallel.zip, computed without Int32 overflow for every array length;
    • collect maps in parallel and concatenates through a builder.
  • groupBy computes the keys in parallel and groups them sequentially through ToLookup, so null keys work. Keys compare in F# structural equality, except float and float32 keys, which compare with GenericEqualityER as in Array.Parallel.groupBy, so all NaN keys are in one group. The order of its result is unspecified, as for Array.Parallel.groupBy.
  • The tests reach the internal chunk bounds of the parallel loops through the InternalsVisibleTo item that the base pull request puts into the project file of the library (this pull request has no assembly attribute in a source file any more), so the bounds are tested for arrays that do not fit into the memory of a test run.

Tests (2760 per target framework, 313 of them for Parallel): every function is compared with its sequential FlatList counterpart on the contract inputs and on 10 000 shuffled elements; groupBy, iter and iteri are compared after sorting, and reduceBy with Array.Parallel.reduceBy. Exceptions for empty, mismatched and default inputs are compared with Array.Parallel. Every case is a data row: the inputs are the cases of a ParallelInput union (Empty, Singleton, Several, Large, whose elements the test builds, so that a row prints a short name) and the functions are the cases of predicate unions, so each of the 312 cases of the former loops runs as a test of its own. The Option and Tuple tests prove that the nested overrides shadow the default functions. The test classes carry the Parallel category (D8).

Types of changes

What types of changes does your code introduce to FSharp.Collections.Immutable?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • Build and tests pass locally
  • I have added tests that prove my fix is effective or that my feature works (if appropriate)
  • I have added necessary documentation (if appropriate)

Further comments

Deliberate details, each documented on the function:

  • iter and iteri are in the Folding category, as in the main module; ADR Appendix A lists the Parallel row under Transformation.
  • init with a negative count throws OverflowException, exactly as Array.Parallel.init does (the sequential init throws ArgumentException).
  • sum and sumBy throw ArgumentNullException for a default flat list, as D6 requires. Array.Parallel throws NullReferenceException for a null array, although its documentation names ArgumentNullException.
  • collect throws ArgumentNullException for a default flat list from the mapping, as the sequential collect does. Array.Parallel.collect dereferences a null result.
  • maxBy and minBy return the last of several elements with the greatest or smallest key, as Array.Parallel does.
  • Array.Parallel.groupBy throws AggregateException for other keys that contain NaN, such as tuples: its second pass cannot find the key that its first pass added. Here such keys compare in F# structural equality, so each NaN is a group of its own, as in Array.groupBy.

🤖 Generated with Claude Code

Copilot AI balanced review requested due to automatic review settings October 18, 2026 10:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Chunk arithmetic can overflow for large zip inputs, and NaN grouping differs from Array.Parallel.

2 open issues
What changed in this PR

Adds FlatList.Parallel, implementing ADR 0001’s parallel collection API alongside the sequential FlatList module.

Changes:

  • Adds 32 parallel functions with documented signatures.
  • Provides shared algorithms and Option/Tuple overrides.
  • Adds behavior, exception, and namespace-resolution tests.
File Description
tests/​FSharp.Collections.Immutable.Tests/​FSharp.Collections.Immutable.Tests.fsproj Registers parallel tests.
tests/​FSharp.Collections.Immutable.Tests/​FlatList/​FlatListParallelTupleTests.fs Tests Tuple overrides.
tests/​FSharp.Collections.Immutable.Tests/​FlatList/​FlatListParallelTests.fs Tests parallel behavior and exceptions.
tests/​FSharp.Collections.Immutable.Tests/​FlatList/​FlatListParallelOptionTests.fs Tests Option overrides.
src/​FSharp.Collections.Immutable/​Helper.fs Adds shared parallel algorithms.
src/​FSharp.Collections.Immutable/​FlatList/​FlatListTuple.fsi Documents parallel Tuple signatures.
src/​FSharp.Collections.Immutable/​FlatList/​FlatListTuple.fs Implements parallel Tuple overrides.
src/​FSharp.Collections.Immutable/​FlatList/​FlatListOption.fsi Documents parallel Option signatures.
src/​FSharp.Collections.Immutable/​FlatList/​FlatListOption.fs Implements parallel Option overrides.
src/​FSharp.Collections.Immutable/​FlatList/​FlatList.fsi Defines the documented parallel API.
src/​FSharp.Collections.Immutable/​FlatList/​FlatList.fs Implements the parallel module.
CHANGELOG.md Records the new API.

Review effort: Balanced

Comment thread src/FSharp.Collections.Immutable/Helper.fs Outdated
Comment thread src/FSharp.Collections.Immutable/Helper.fs Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Static review found no unresolved blockers, and the prior overflow and NaN defects have fixes with regression coverage; builds and tests were not run.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The custom concurrent algorithms warrant human validation across both target frameworks; builds and tests were not run during this review.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

No blocking defects were identified, but the custom concurrent algorithms need final human review and runtime validation.

No open issues 🎉

Review effort: Balanced

@xperiandri
xperiandri force-pushed the feat/flat-list-parallel branch from 98056aa to 165c14e Compare October 18, 2026 22:09
Base automatically changed from feat/flat-list to main October 18, 2026 22:34
@xperiandri
xperiandri force-pushed the feat/flat-list-parallel branch from 165c14e to 7510544 Compare October 18, 2026 22:36
Comment thread src/FSharp.Collections.Immutable/Helper.fs Outdated
Comment thread src/FSharp.Collections.Immutable/Helper.fs Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Static review found no unresolved blocking issues, and previously reported defects have regression coverage; builds and tests were not run.

No open issues 🎉

Review effort: Balanced

@xperiandri
xperiandri force-pushed the feat/flat-list-parallel branch from 69c276a to b60b630 Compare October 19, 2026 12:07
@xperiandri
xperiandri force-pushed the feat/flat-list-parallel branch from 310204b to 4a75706 Compare October 19, 2026 14:40
@xperiandri
xperiandri force-pushed the feat/flat-list-parallel branch from 4a75706 to 1acc70b Compare October 19, 2026 14:46
@xperiandri
xperiandri force-pushed the feat/flat-list-parallel branch from 1acc70b to e7cf503 Compare October 19, 2026 14:48
Base automatically changed from chore/split-helper to main October 19, 2026 15:18
@xperiandri
xperiandri force-pushed the feat/flat-list-parallel branch 2 times, most recently from a39db38 to 9bb2f82 Compare October 19, 2026 16:16
xperiandri and others added 14 commits October 19, 2026 19:29
…re `Array.Parallel` module

Adds the nested module `Parallel` at the end of `FlatList` in
`FlatList/FlatList.fsi` and `FlatList/FlatList.fs`, as `Array.Parallel`
lives inside `Array`: 32 functions in the canonical category order, each
documented in the signature file and compiled under the name that
`Array.Parallel` gives it. The `sortInPlace*` functions are left out.

- Most functions unwrap the underlying array without a copy, call the
  `Array.Parallel` function and wrap its fresh result, so that they keep
  its semantics and exceptions, including the `OverflowException` of
  `init` for a negative count. `sum` and `sumBy` throw
  `ArgumentNullException` for a default flat list, which
  `Array.Parallel` documents but does not check.
- `tryFind`, `tryFindIndex` and `tryPick` return `voption`, `choose` and
  `tryPick` take choosers that return `voption`, and `zip`, `groupBy`,
  `partition` and `partitionWith` return struct tuples. Where the
  `Array.Parallel` function takes or returns options or builds a tuple
  per element or per group, a new `ParallelArrays` algorithm in
  `Helper.fs` runs instead and allocates nothing per element: `choose`
  maps to value options in parallel and compacts them, `tryPick` keeps
  the value of the lowest index under a lock, `groupBy` computes the
  keys in parallel and groups them with `ToLookup`, and `zip` pairs the
  elements in the chunks of `Array.Parallel.zip`.
- The `Parallel` modules of the `Option` and `Tuple` namespaces repeat
  `choose`, `tryFind`, `tryFindIndex` and `tryPick` with options, and
  `groupBy`, `partition`, `partitionWith` and `zip` with reference
  tuples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
….Parallel`

- Every function of `FlatList.Parallel` runs on the inputs of the
  contracts and on 10 000 scrambled elements, enough to split the work
  among threads, and is compared with its sequential `FlatList`
  counterpart. `groupBy`, `iter` and `iteri`, whose order is open, are
  compared after sorting, and `reduceBy`, which has no sequential
  counterpart, with `Array.Parallel.reduceBy`.
- Exceptions for empty, mismatched and default inputs are compared with
  those of `Array.Parallel`, and so is the choice of `maxBy` and `minBy`
  among equal keys.
- The tests of the `Option` and `Tuple` namespaces prove that their
  nested `Parallel` modules shadow the default functions and fall
  through to them otherwise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…harp.Core `Array.Parallel` module

Parallel.collect rejects a default flat list from the mapping with ArgumentNullException, as the sequential collect does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… `Array.Parallel`

Parallel.collect is expected to throw ArgumentNullException naming the mapping for a default flat list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…harp.Core `Array.Parallel` module

The chunk count of the parallel loops divides the length minus one, and the end of a chunk adds only the remaining elements, so neither overflows Int32 for arrays near the maximum length; the chunks are a pure function of the processor count and the length, which the tests reach through InternalsVisibleTo.

Parallel.groupBy compares float and float32 keys with GenericEqualityER as Array.Parallel.groupBy does, so all NaN keys are in one group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… `Array.Parallel`

The chunks of zip are checked for 1.5 billion elements and for the maximum array length on 64 processors, and groupBy of both modules for NaN keys of type float and float32 against Array.Parallel.groupBy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…harp.Core `Array.Parallel` module

The nested `Parallel` modules follow their root modules (ADR 0001, D2) one level less indented; Fantomas joins the
lines that now fit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… `Array.Parallel`

Test methods carry no categories: the classes carry them (ADR 0001, D8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… `Array.Parallel`

The tests compare arrays and take their cases as data rows, so every case runs and
reports as a test of its own (313 tests in the three classes instead of 64):

- The inputs are the cases of `ParallelInput`: MSTest names a row after its values, and
  the larger input would write ten thousand numbers into the name of each of its rows.
  `Elements` gives the elements of a case, and each class builds its own larger input.
- The predicates and their choosers are the cases of `ParallelPredicate`; `forall` runs
  each of them as it is and negated. The tables of the default flat lists give their
  names as rows; the two cases of `zip` are named after the parameter that is default.
- The chunks of the parallel loop of `zip` are rows; the check of the longest array on 64
  processors is a test of its own instead of a check repeated by every row.
- Groupings are compared by their keys and their elements, which a pair would compare by
  reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…harp.Core `Array.Parallel` module

A `<summary>` and a `<remarks>` stay on one line each only when both are short (up to 60 characters of text);
otherwise both span several lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… `Array.Parallel`

A `<summary>` and a `<remarks>` stay on one line each only when both are short; otherwise both span several lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… `Array.Parallel`

The assertion helpers of `main` are static classes with tupled arguments, so the calls of
the `FlatList.Parallel` tests use them: `ExceptionAssert.Same` and
`OverrideAssert.OptionMatches`.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…harp.Core `Array.Parallel` module

The comments of ParallelArrays are the same kind of documentation as the ones of
Arrays: the summary of the FSharp.Core function word for word, the parameters, the
result, and remarks that give the complexity and say why the algorithm exists
(the options or the reference tuples that FSharp.Core takes or returns, the loop
that calls no delegate per element, the comparer of the keys). The callers of each
algorithm are seealso. The functions without a counterpart in FSharp.Core say so.
The code does not change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xperiandri
xperiandri force-pushed the feat/flat-list-parallel branch from 9bb2f82 to 46729a2 Compare October 19, 2026 17:31
@xperiandri
xperiandri merged commit 1daa7bd into main Oct 19, 2026
7 checks passed
@xperiandri
xperiandri deleted the feat/flat-list-parallel branch October 19, 2026 17:34
xperiandri added a commit that referenced this pull request Oct 20, 2026
…re `Array.Parallel` module (#27)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants