Repository navigation
OptunaSharp 1.1.0 - Performance & Parity Release - #6
Conversation
…a logs nan
Found by diffing the INFO/WARNING log lines of a real-world-shaped run (completed,
pruned, failed-by-exception, NaN and multi-objective trials) against Optuna 5.0.0.
Every line matched except two: the exception type name (InvalidOperationException
vs RuntimeError - language, expected) and the NaN failure line.
optuna/study/_optimize.py::_log_failed_trial warns
f"Trial {trial_number} failed with value {repr(value_or_values)}." with the
objective's own return value - a float for a single-objective study, so
"Trial 0 failed with value nan.". The single-objective Study.Optimize overload
wraps the scalar in a one-element array before the loop, and
Internal/Optimize.cs::LogFailedTrial prints it with ListRepr: "[nan]". Same
scalar-wrapping root cause as QA-12 (the skipped-tell message), different message.
New [OpenBugs] test Optimize_NaNScalarObjective_LogsScalarValueLikePython in
test/OptunaSharp.Tests/QA/RealWorldExampleBugTests.cs (fails today).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…_exists) + stop() at a provable bound A tunable bin-packing heuristic (size^p ordering with a seeded tie-break jitter, first/best/worst fit, repair passes that try to empty the least-full trailer) loads Falkenauer-style "triplet" shipments into 13.6 m trailers. The load planners' rules of thumb (first-fit and best-fit decreasing) are enqueued twice each with skip_if_exists=True (the duplicates are skipped), and a callback calls study.Stop() once a plan reaches the lower bound ceil(sum/capacity). Here FFD and BFD need 46 trailers, the tuned heuristic 45, the bound is 40 - so the budget is used up and the output says so. Checked while writing it (not bugs, faithful to Optuna): when stop() fires right after the first trial, the second enqueued trial stays WAITING with no params and study.trials includes it, as Python's does; the example summarizes COMPLETE trials only and explains why. Also checked in this round (no defect): sampler-fallback and enqueue warnings match Optuna verbatim (ExperimentalWarning aside - removed on purpose earlier); multi-objective trials_dataframe columns (flat and multi_index) match pandas; 3,000 TPE trials take 11.6 s. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…2 on one journal file Optuna's shared-file-system recipe: the parent creates the study in a JournalFileBackend file and starts three worker processes (the same program with --worker), each joining via load_study and running 16 trials on two threads with HyperbandPruner; then it audits the file. Objective: mini-batch SGD ridge regression (optional pairwise-product features) on a deterministic house-price set, validation RMSE reported every epoch so Hyperband's brackets compare trials from different processes at the same epoch. Run here: 48 trials, numbers 0..47 each exactly once, 34 pruned / 14 complete, 546 of 1,296 epochs trained (58 % saved), best RMSE 0.098 at the 0.1 noise floor (mean predictor 0.676). No defect found. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ted default recipe Every trial of an L2 logistic churn scorecard (optional quadratic features, gradient descent) reports its 5 CV fold accuracies with TerminatorFunctions.ReportCrossValidationScores, and the default TerminatorCallback (GP regret bound vs CrossValidationErrorEvaluator) ends the study. The same seeded search is also run without the terminator for comparison: it stopped at trial 20 of 120, and the remaining 100 trials would have added +0.05 percentage points - inside the +-2 point fold noise it reads from terminator:cv_scores. The per-trial `optuna.terminator` FutureWarning is Optuna's own behavior (deprecated in v4.9). No defect found. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ython) through the gRPC storage proxy The database host runs GrpcProxyServer.MakeServer over an RDBStorage SQLite file on a free loopback port (the hosted, stoppable form of run_grpc_proxy_server); the study is created through a GrpcStorageProxy like any remote client; four .NET workers (own proxy connection each, TPE seeds 1-4) and - when python with optuna is on PATH - a Python Optuna worker using optuna.storages.GrpcStorageProxy optimize it together. Objective: a quarter's gross profit under constant- elasticity demand with a decaying promo lift and a post-promo dip. Run here: 70 trials (4 x 15 .NET + 10 Python) in one study, best $222.5k found by a .NET worker; the C# server and the Python client interoperate over the wire. No defect found. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…the managed random forest The scikit-learn-style integration as a real job: OptunaSearchCV cross-validates Core's byte-exact RandomForestRegressor port (SklearnEstimator, KFold(5, shuffle)) over n_estimators / max_depth / min_samples_leaf / max_features to predict a building's heating load from geometry and glazing (a deterministic 768-design table shaped like the classic energy-efficiency study); the refitted best forest is scored on 200 held-out designs: CV R^2 0.9972, held-out R^2 0.9972, MAE 0.33. The per-trial terminator FutureWarning is Optuna's own behavior. No defect found. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…int/categorical space with constraints Concrete mix design as Bayesian optimization: cement content (5 kg steps), water-cement ratio (0.01 steps), slag and fly-ash replacement (int, step 5), superplasticizer dose (continuous) and the aggregate type (categorical), maximizing 28-day strength from Abrams' law with SCM and aggregate factors, under SetConstraint cost <= $95/m^3 and slump >= 75 mm. 12 random pours then 28 GP-planned: 57.1 MPa (both constraints nearly active) vs 44.7 MPa best random. Live parity probe (40 trials): every stepped, integer and categorical choice is identical to Optuna 5.0.0's GPSampler; the continuous dose differs by <= 2.1e-4 relative from trial 24 on - the documented GP / L-BFGS-B tolerance class. No defect found. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s as the only path, opt-in switches where results would change The owner asked for a plan to integrate Rustuna's optimizations, with an evaluation of where a dual implementation is warranted (a user choosing Optuna's path or Rustuna's). Rustuna 0.2.1, the Optuna team's Rust port, runs its announcement's workloads 3-60x faster than OptunaSharp (TPE 7x, CMA-ES 100k 60x, NSGA-II 3-4x), while OptunaSharp is 3-5x faster than Optuna 5.0 and byte-identical to it. Decisions recorded (owner, 2026-11-08, through two rounds of design questions): - Rule: "prefer rustuna unless it affects results". A Rustuna technique that keeps every observable result becomes the only implementation; one that changes results is measured and decided per item. - CMA-ES reusing the optimizer it just told (Rustuna's live object): opt-in, Optuna's path the default. - TPE fast numerics: decided after the byte-identical fused kernel exists and can be measured. - Separate trial queue and trial discarding: deferred. The owner asked whether Rustuna has a TrialQueue (it does: an in-memory default plus SQLite3 and directory backends, Redis planned). For discarding the owner asked for "what's best + don't break Optuna compatibility", so the plan proposes compact storage of finished trials first. Evidence in the plan, all measured in session 5417a82e on the owner's i9-13900K (Release, throttling opt-out): - A scratch prototype of the CMA-ES part (a patched copy of d0558f5) took 10k trials from 18 s to 2.6 s and 100k from 812 s to 26 s (single-threaded OpenBLAS), byte-identical: five seeded studies hashed over every trial and its stored pickle, 86 unit and 29 live-Python interop CMA tests, mixed C#/Python fleet included. Its three changes: an optimizer cache dropped after each tell, a folded history, zero-copy reads. - How different the result-changing paths are. Keeping the told CMA-ES optimizer diverges from generation 3 (eigenvector sign flips make the first difference large), with the same quality over 10 seeds and about 15 % more speed. TPE scoring noise up to 1e-6 changed no decision in three 1,000-trial runs (smallest gap 3.4e-4); MOTPE stayed identical up to 1e-9. The queue is identical in one process; discarding changes study.Trials. - Profiles behind each item: copy-on-write storage clones 66 % of NSGA-II and 23 % of MOTPE; the MOTPE split, recomputed per parameter, 24 %; TPE scoring 58 %. The plan: an inventory of 12 same-result items (S1-S12), 4 two-path items (D1-D4) and 6 rejected ones, each with Rustuna's code reference, our current code and the evidence. Then the rule for one path or two, the switch design (on the component, named for the behavior), phases 0-8, gates (Python as the oracle on every CI OS, concurrency suites, Release benchmarks, both paths tested), targets and risks. docs/plans/README.md lists plan 13 and describes the performance workstream.
…istory incrementally, same results as Optuna; opt-in reuseToldOptimizer Plan 13 (docs/plans/13-rustuna-derived-performance.md), phase 1 (items S1-S3) and item D1. Optuna's CmaEsSampler restores its optimizer from the pickled state and rescans every trial on every ask, which makes a run quadratic in its trial count (40-D, seed 0: 18.3 s for 10k trials, 812 s for 100k; Rustuna 0.2.1 takes 1.6 s and 13.6 s). The scratch prototype of session 5417a82e took 100k trials to about 26 s with byte-identical results; this is that prototype, rewritten for production. What changed, none of it observable: - Samplers/CmaEs/CmaTrialHistory.cs (new): Optuna's per-ask _get_trials, _restore_optimizer backward scan and _get_solution_trials, kept up to date incrementally. Finished trials cannot change, so the leading run of finished trials is folded once, in trial order, into a completed count, a by-generation index and the latest state holder. Every refresh classifies only the trials after the first unfinished one, which keeps Optuna's scan order when trials finish out of order. A history that is not the one folded (another storage, study, or trials at the folded positions) starts over and tells the owner, so the sampler drops anything keyed by trial id. - CmaEsSampler: the optimizer deserialized from the latest stored state is cached under the id of the trial holding that state and reused by the generation's remaining asks, eigendecomposition included. It is dropped after every tell: Optuna's pickle stores only C's upper triangle and a restore mirrors it, while the told C keeps the update's floating-point asymmetry. Measured in the prototype: keeping it moved seeded 40-D results from generation 3. GetTrials/GetSolutionTrials are replaced by the history; the per-trial filter is AsSamplerTrial, so a pruned trial's value-rewritten copy is made once when folded. SampleIndependent's warning check reads the history too. - BaseStorage.HasZeroCopyTrialsView (internal, true for InMemoryStorage) and Study.GetTrialsForSampler: samplers that read the history every ask use the zero-copy view where it is one, and Optuna's read (including the per-trial cache) elsewhere, so a database-backed storage makes no extra round trip. - IntersectionSearchSpace.Calculate reads through GetTrialsForSampler: its walk already stopped at the cached trial number; the full copy of the history was what grew with the study. D1, opt-in (owner's decision: Optuna's path stays the default): CmaEsSampler(..., reuseToldOptimizer: false). When true, the optimizer that just told a generation keeps answering the asks after its trial completes, as Rustuna keeps its live optimizer: about 15 % faster on a 40-D study, and seeded runs stop matching Python Optuna from about the third generation (same quality: 40-D sphere, 10 seeds, best-value ratio median 0.79, range 0.51-1.48). The state is still stored every generation exactly as Optuna stores it, so resuming and Python workers keep working. The parameter comes after every Optuna parameter. Tests: - CmaTrialHistoryTests: randomly evolving studies (600 steps, 5 seeds; out-of-order completion, failures, pruned trials with and without intermediate values, attributes written while running) compared at every step with a naive re-implementation of Optuna's scans; a stuck running trial; the pruned copy made once; reset on another storage, study, folded trial or a shorter history; a holder in the tail; null arguments. - CmaEsSamplerCacheTests: one restore per tell in a sequential run (RestoreCount, internal); a failed teller's generation told again from the previous state, the stored state identical byte for byte; a second study on another in-memory storage (colliding trial ids) never reuses the first study's optimizer; GetTrialsForSampler is the zero-copy view in memory and Optuna's read on a journal file; 8 parallel workers, shared and per-worker. D1: off by default (identical trials, one restore per tell), one restore per run when on, a failed teller restores the previous state, a default sampler resumes a study the reusing sampler wrote, and both paths reach 1e-6 on a 10-D sphere for three seeds. - Interop, live Python: long runs (10-D 400 trials, 6-D 300), 40-D 1,200 trials ([Slow]), failing tellers (every other teller fails, both languages failing the same trials), consider_pruned_trials, SepCMA and lr-adapt over 30+ generations: every trial and every stored pickle equal Optuna's. The mixed C#/Python fleet with the C# worker reusing its told optimizer completes with the reference generations. Results: the CI unit filter (OpenBugs/HighMemory/RequiresPython/Slow excluded) 1998/1998 on net8.0 and on net10.0; unit CMA-ES/pickle/search-space 103 pass with no filter (the 3 failures are known [OpenBugs]); concurrency suites 243/243; live-Python CMA-ES suite 41/41 (engine required); documentation gate 0 issues. Phase 2 of the plan (in-place writes to a running trial nobody holds, S4; shared distributions and parameter names, S7) follows in its own commit; InMemoryStorage.cs carries only the HasZeroCopyTrialsView override here.
…t, share identical distributions and names; same results as Optuna
Plan 13 (docs/plans/13-rustuna-derived-performance.md), phase 2: items S4 and S7.
S4. Optuna's _InMemoryStorage copies a trial's record and the dictionary it writes on every write
(set_trial_param, set_trial_user_attr, set_trial_system_attr, set_trial_intermediate_value), so a FrozenTrial a
caller holds never changes. The port did the same, which made a trial's suggestions O(d^2): the profile of an
NSGA-II study with 40 parameters spent ~66 % in SetTrialParam's dictionary clones (plan 13 section 2.2). Rustuna
inserts in place and clones only for readers. Now:
- FrozenTrial.StorageExclusive (internal): true while the storage alone holds a record. CreateNewTrial sets it
(both branches build dictionaries nobody else holds: a new running trial or a deep copy of the template).
- InMemoryStorage.WritableTrialUnlocked: a write changes the stored record in place while it is exclusive;
otherwise it first replaces it with a copy that owns all five dictionaries (any later write may touch any of
them), so the reader's record keeps its contents.
- HandOut: every read that returns the stored record itself clears the flag under the lock: GetTrial,
GetBestTrial, GetAllTrials(deepcopy: false) (unfinished records only; finished ones are never written again),
and through them the attribute reads and GetNTrials. Deep copies and GetTrialParam hand nothing out.
- GetTrialsPrefixView passes TrialsPrefixView a read for unfinished records (new internal constructor): taken
under the storage lock with HandOut. Finished records stay lock-free, as before: a state change still publishes a
new record whole (SetTrialStateValues keeps Optuna's copy), so a record that reads as finished is final.
- InMemoryStorage.WriteCopyCount (internal diagnostic, like CmaEsSampler.RestoreCount): writes that had to copy.
In an ask/suggest/tell loop the trial's own read at Ask is the only hand-out between its suggestions: about one
copy per trial instead of one per write.
S7. SetTrialParam stores one distribution instance per parameter across the study's trials when the new one is
identical to the bit (Distribution.IsBitwiseIdentical: the same runtime type, bounds/step compared by bits, log
flag), and one string instance per parameter name (StudyInfo.ParamNames), so names built per call ($"x{i}") are
not kept once per trial. Never shared: categoricals (equal choices may be different objects, 1 and 1.0 under
Python equality, and a trial must report its own), a -0.0 bound (equal to 0.0, printed differently), a deprecated
subclass against its base. Compatibility is still checked first, with Optuna's messages. Distributions are never
mutated after construction (only the constructor sets FloatDistribution.Step), so sharing is unobservable.
Docs: TrialsPrefixView remarks (liveness, the in-memory hand-out), THREADING.md section 6.1 (the snapshot
guarantee as it is now kept) and the lock inventory (the view takes the in-memory storage monitor per unfinished
record; it is still the last lock a sampler takes, so the order of section 5.2 holds).
Tests:
- InMemoryStorageInPlaceWritesTests (13 cases): every read path (GetTrial, GetAllTrials with no filter, RUNNING,
WAITING, the view's indexer and enumerator, the three attribute reads, GetBestTrial of another trial) keeps its
snapshot through two writes of each kind, and every write lands; deep copies leave the record writable in place;
writes nobody reads between go in place (WriteCopyCount 0 over 20 parameters and 3 attributes) and each hand-out,
GetNTrials included, costs exactly one copy; a state change makes the next write copy once and a templated trial
never writes its template; the view returns the current record on every read and finished records as stored; a
finished trial refuses writes; an ask/suggest/tell loop copies at most once per trial. S7: identical float,
stepped float and int distributions shared, compatible-but-different ones keep the caller's instance; -0.0 keeps
its instance and repr; categoricals never shared (a [1.0, 2.0] trial after a [1, 2] one reports 1.0); names stored
once per study; IsBitwiseIdentical case by case; incompatible distributions still refused with Optuna's messages.
- StorageConcurrencyCoverageTests: new ConcurrencyGun case, the view and GetTrial read while 7 shooters write their
own trials in place (30 rounds): no fingerprint changes under the reader, and the writers did copy held records.
The existing GetAllTrials(deepcopy: false) case's summary now describes how the in-memory storage keeps it.
- Mutation check of the new tests (each reverted): the view handing out without marking fails 4 tests (including the
view race case); a write copy that shares IntermediateValues fails 10; GetAllTrials(deepcopy: false) not marking
fails 5 (including the existing race case).
Results: the CI unit filter (OpenBugs/HighMemory/RequiresPython/Slow excluded) 2022/2022 on net8.0 and on net10.0;
live-Python interop on net10.0 with the engine required 682 passed, 0 failed (24 not run here for services and Python
packages this machine lacks: Redis, and the pycma bridge and OptunaHub constraint fixtures); documentation gate 0
issues. Seeded 40-parameter studies hash identically to de495f8 (every trial's parameters and values, by bits):
random 2,000, NSGA-II 2,000, CMA-ES 1,500, TPE 150 and MOTPE 120 trials.
… history reads; same results as before Plan 13 (docs/plans/13-rustuna-derived-performance.md), phase 3: items S6, S5 and S3 (TPE's constant-liar read, QMC, the group-decomposed search space). Every change keeps OptunaSharp's results to the bit: seeded 40-parameter studies hash identically to de495f8 (TPE 1,000 trials, MOTPE 1,000, QMC), and the live-Python TPE suite, now with 1,000- and 300-trial long runs, stays within its erf tolerance. Measured (i9-13900K, Release, power throttling opted out, one rep each unless noted; 40 float parameters, seed 0): - TPE, 1,000 trials: 10.08 s -> 4.76 s (first rep, JIT included), 3.93 s steady; allocations 4.4 GB -> 1.6 GB, gen-2 collections 254 -> 2. - MOTPE (two objectives), 1,000 trials: 41.07 s -> 15.77 s; allocations 28.7 GB -> 3.4 GB, gen-2 713 -> 1. - MOTPE, 300 trials: 3.75 s -> 2.53 s. S6, the TPE scoring kernel (Samplers/Tpe/ProbabilityDistributions.cs, ParzenEstimator.cs, Internal/TruncNorm.cs, Internal/Erf.cs): - AddContinuousLogPdf computed log(sigma), the standardized bounds and their log Gaussian mass once per candidate for every (kernel, dimension) pair: 936k Math.Log calls per trial at 1,000 trials and 40 dimensions, more than half the run. They are now computed once per pair, in grids laid out by dimension, and AccumulateDimension adds one dimension's term to every kernel's sum on Vector<double> lanes. Each kernel's sum still runs over the dimensions in order from +0.0, lanes hold different kernels and never mix, and .NET never contracts a multiply and an add into an FMA, so every lane computes what the scalar loop computed; the remainder takes the scalar path. LogGaussMass is elementwise and the vectorized-erf switch depends only on how many elements it gets, so laying the grid out by dimension changes no element. - LogPdf takes each kernel's log weight once instead of once per candidate, and keeps its (candidates x kernels) matrix in a rented flat buffer: at 1,000 kernels it was a 192 KB large-object allocation per call, and the sampler waited on background GCs for them. - TruncNorm.LogGaussMass and Ndtr, and Erf.ComputeArray, gained span overloads over rented scratch (the array overloads now wrap them). The central elements are gathered into spans of exactly the central count, so the erf path is chosen from the same length as before; a test pins that a span of 1,999 inside a 5,000-element buffer takes the exact path. - ParzenEstimator.StableArgsort replaces Enumerable.Range(0, n).OrderBy(i => mus[i]): Array.Sort on a copy of the keys, then an index sort inside each run of equal keys (by CompareTo: NaNs together, -0.0 with 0.0), which is the same permutation the stable LINQ sort returned. - TPESampler.GetInternalRepr fills pre-sized columns by position, without the per-trial LINQ closure, the dictionary lookup per value and the list regrowth; same values in the same order. S5, the split cache (Internal/TpeTrialSplit.cs: TpeSplitCache; TPESampler): independent TPE (the default for several objectives) split the history once per parameter, and every parameter of a trial sees the same records, so a 40-parameter trial computed the same non-domination ranks and hypervolume tie-break 40 times. The cache returns the last split again only for the very same records (compared by reference, in order) of the same study and storage with the same n_below: finished records never change, and a record changes reference on any write after a reader got it (state changes publish a new record; InMemoryStorage copies a handed-out record before writing it). Returned lists are copies. One cache per sampler instance, so a PerWorker worker has its own (TPESampler.SplitCache, internal, for tests). S3: - TPE with constant liar (Optuna 5's default) read the whole history including running trials for every parameter and then dropped the current trial; that read handed out the current trial's record, so every suggestion copied it again under item S4. On a storage with a zero-copy view it now reads through the view and skips the current trial's position without reading it (TPESampler.ReadOtherTrials): the same records in the same order, minus the current one. - QMCSampler copied the COMPLETE/PRUNED history and sorted it by number on every ask to take the first one; FirstSuggestedTrial scans the history in number order through Study.GetTrialsForSampler and stops at the first match (O(1) on an in-memory storage, Optuna's read elsewhere). - GroupDecomposedSearchSpace filters the states while scanning Study.GetTrialsForSampler instead of a copied list. Tests: - TpeExactKernelTests (unit): the continuous log-pdf against a verbatim scalar copy of the replaced code, bit for bit, for every kernel count from 1 to three vectors and more (every remainder), 1-40 dimensions, log dimensions, candidates outside the bounds; 1,000-kernel grids that take the vectorized erf; an empty interval (NaN, as before); LogGaussMass over rented scratch against the replaced allocation-per-call code for 1 to 6,000 intervals around the 2,000 switch, NaN bounds and all three cases; the span erf choosing its path by the span's length; StableArgsort equal to the LINQ permutation over 300 arrays with ties, NaNs, both zeros and infinities. - TpeSplitCacheTests (unit): the cached split equals SplitTrials for every n_below over a history with completed, pruned (with and without reports), infeasible and running trials; reuse only for the same records (one fewer, another order, a rewritten running record, another study all miss); returned lists are copies; null arguments; an independent multi-objective study splits once per trial, not per parameter, with constant liar on; constant liar still sees another worker's running trial and leaves the current one out; a PerWorker instance has its own cache. - ZeroCopyHistoryReadTests (unit): QMC's search space comes from the lowest-numbered COMPLETE or PRUNED trial with trials finished out of order and running/failed trials before it; the running-trial union before anything finished; the group-decomposed space equals Optuna's decomposition of the copied, filtered history (same groups, order and distribution objects) with and without pruned trials and the cache. - Interop, live Python, [Slow]: TPE multivariate 40-D 1,000 trials and MOTPE 40-D 300 trials stay in lock-step with Optuna within the erf tolerance; NSGA-II 40 parameters 2,000 trials is bit-identical in full state (plan 13 phase 0 gates for S4-S6). - The 8 [OpenBugs] TPE/PED-ANOVA QA cases fail with byte-identical messages before and after (NaN acquisition values and NaN prior weights included), so the edge behavior they pin is unchanged. Also: OptunaHubBridgeParityTests.BenchmarkBridge_ToyProblem_ExposesSpaceAndOptimizes now seeds its TPE sampler. It ran an unseeded study (OS-entropy seed) and asserted that the best of 40 trials is below 1.0, which fails in a few percent of runs: it failed once in this phase's interop run (best 1.018) and passed eight reruns. Seeded studies are bit-identical before and after this phase, so every OS seed maps to the same outcome in both trees and the failure rate was unchanged; the test is about the bridge, so the seed is fixed (seed 0 passes). Results: the CI unit filter (OpenBugs/HighMemory/RequiresPython/Slow excluded) 2044/2044 on net8.0 and on net10.0 with S5/S6 and TPE's constant-liar read; the QMC and group-decomposition change and its 7 tests afterwards, with the QMC, group and TPE suites: 51/51. Live-Python interop on net10.0 with the engine required, the three new long runs included: 684 passed, 24 not run here (Redis, the pycma bridge and OptunaHub constraint fixtures), 1 failed: the stochastic OptunaHub toy test above. Long runs: TPE 1,000 trials 1 min 26 s, MOTPE 300 trials 58 s, NSGA-II 2,000 trials 16.5 s, all passing. Documentation gate 0 issues (5,609 declarations).
…s it as a view; database loads parse each distribution once Plan 13 (docs/plans/13-rustuna-derived-performance.md), phase 4: item S10, and item S9 as its measurement turned out. Same records, same order, same values as before. S10, CachedStorage (the wrapper Optuna puts around every RDB storage): - GetAllTrials sorted the cached dictionary by trial number on every read, and GetTrialsPrefixView fell back to that copy, so the genetic samplers, which take the view on every ask, paid O(n log n) per ask on a database storage. - StudyInfo.ByNumber mirrors the cache by trial number (AddTrialsToCache keeps it in step). A number not cached yet holds a null placeholder counted in MissingNumbers: a local create can cache trial n + 1 before a sync brings in a trial n another worker created. - With no placeholder, GetAllTrials walks the mirror (the same records in the same order as the sort) and GetTrialsPrefixView wraps it in a TrialsPrefixView after the same synchronization GetAllTrials does. Cached records are never changed in place (writes go to the backend; a sync swaps in fresh records), so the view needs no lock. With a placeholder, both use the sorted copy, as before. - HasZeroCopyTrialsView stays false here: samplers that share the per-trial cached list (GetTrialsForSampler) keep it, since a view read means a synchronization, which is a database round trip. S9, measured first as the plan asked (10,000-trial, 40-parameter CMA-ES study on SQLite, cold load with a new RDBStorage, i9-13900K, Release): - GetAllTrials took 2.1 s and allocated 1.22 GB. Decoding every attribute value (26k values, 30 MB of JSON, the CMA-ES pickles included) took 0.06-0.08 s of it: lazy attribute decoding would save under 4 % and move malformed-JSON errors from load to access. Not built. - The profile showed where the JSON time really goes: Distribution.FromJson for every parameter row, 400,000 parses of 40 distinct texts, about 40 % of the load. - Both RDB storages (Storage.Sqlite and Storage.EFCore) now parse each distinct distribution text once per load through the journal storage's JournalDistributionCache (plan 10, item 2.7): the shared object equals a fresh parse, only the three current distribution types are shared (the deprecated ones are built per use, so their warnings fire as before), and the cache is local to the call. Cold load: 2.1 s -> 1.4-1.5 s, 1.22 GB -> 479 MB. Tests: - CachedStorageOrderedReadsTests (unit, over a fake database: an in-memory store returning fresh records per read that can hide another worker's trial from a sync): every state filter with and without deep copies returns the backend's trials in number order over a history finished out of order; the view is a TrialsPrefixView holding the very records of the uncopied read and sees an unfinished trial a later sync refreshed; a gap falls back to the sorted copy (the view to a List) until a sync fills it; seeded NSGA-II (which reads the view every ask) and CMA-ES studies on the cached fake database equal the in-memory runs trial by trial. - SqliteRdbStorageTests.Reload_ParsesEachDistributionTextOnce_SharingOnlyTheCurrentTypes: a reloaded study shares one object per distinct text, each equal to a fresh parse of its stored text; a -0.0 bound is its own object and prints -0.0; a deprecated UniformDistribution is never shared; parameter values unchanged. Results: the CI unit filter 2056/2056 on net8.0 and on net10.0; the storage suite on net10.0 452 passed, 144 not run here (Redis, PostgreSQL and MySQL services), 0 failed; live-Python interop for the SQLite, EF Core, storage-URL and storage-recipe cases 33/33 (Python writes and reads databases this loader parses); documentation gate 0 issues.
…ce and Compatibility guide; README rerun Plan 13 (docs/plans/13-rustuna-derived-performance.md), phase 8: the documentation, and gate C's measurements. Before/after proof (gate C): one program (session 5417a82e, scratchpad/proof) built in Release against the de495f8 snapshot and against 656ab94; 40 float parameters, seed 0, sampler defaults; each workload before then after, back to back, on a machine other sessions loaded to 30-90 % CPU. Every pair hashed identically (every trial's parameters and values by bits): - CMA-ES 100,000 trials: 987.7 s -> 27.5 s (35.9x), allocations 519 GB -> 17.4 GB, peak working set 2.68 -> 1.88 GB; - CMA-ES 10,000: 25.4 -> 4.5 s (rep 0), 23.0 -> 2.6 s (rep 1); - NSGA-II 100,000: 73.2 -> 22.2 s, peak working set 1.53 -> 0.75 GB; NSGA-II 10,000: 3.8 -> 2.9 s / 2.7 -> 1.8 s; - TPE 1,000: 10.5 -> 5.0 s / 9.4 -> 4.1 s, gen-2 collections 254 -> 2; MOTPE 1,000: 47.4 -> 18.1 s; - QMC 10,000: 4.8 -> 2.5 s / 3.4 -> 1.05 s; Random 10,000: 5.8 -> 4.7 s / 4.9 -> 4.0 s. - The opt-in reuseToldOptimizer (D1): CMA-ES 10,000 2.0 s steady, 100,000 18.2 s; results differ, by design. docs/plans/13-rustuna-derived-performance.md: status BUILT except S8/S12; section 10 records each item, its commit, how it was gated and what was measured, the findings (the pickle's upper triangle behind D1, GetNTrials' hand-outs, dotnet-trace's suspensions showing up as lock and PollGC time), and the proof table. S9 is recorded as measured (attribute JSON 3-4 % of a cold load; the distribution parse cache built instead), S11 as measured and unchanged (the OpenBLAS thread count changes 100-D CMA-ES results, as it does NumPy's), S8 and S12 as left for the owner with their numbers, D2 as asked with both of its numbers (scoring is 41 % of TPE 1k and 49 % of MOTPE 1k after S6). docs/plans/README.md: plan 13 marked built; the workstream paragraph updated. docs/website-src/docs/performance.md (new, in the Guides TOC): what is fast and why results stay the same, the before/after table, the switches that change results (CmaEsSampler reuseToldOptimizer; TrialExecutorOptions.BlasThreads = 1 at nJobs 1) with whether seeded runs still match Python and whether Python workers can share the storage, and the thread-safety guarantee. docs/THREADING.md section 6.3: the plan 13 caches (CMA-ES restored optimizer and folded history, TPE split cache) are sampler state under the sampler monitor, one per PerWorker instance, keyed by what they were computed from. README.md, Performance: benchmark 14's C# probes rebuilt against 656ab94, three fresh processes each. Rows that moved beyond the busy machine's noise are updated (random over 1,000 parameters 13.5 -> 3.2 s, now faster than Python's 8.4 s, so the "one case where OptunaSharp is slower" sentence is gone; CMA-ES 200-D 2.8 -> 1.2 s; CMA-ES 6-D 1.75 -> 0.91 s; QMC 0.64 -> 0.39 s; random 6-D 1.34 -> 0.98 s; TPE 100-D 2.4 -> 2.1 s; the 50k blocks 0.6-0.8 s); the rest keep their numbers, and GP was checked interleaved against a de495f8 build (same within noise). A new "Large studies" table puts OptunaSharp next to Python Optuna 5.0 and Rustuna 0.2.1 on the announcement's workloads, with the measurement conditions stated (OptunaSharp's column ran on the busier machine). DocFX 2.78 builds the site with 0 errors; the new page adds no warning (the 1,160 warnings are the existing duplicate-<param> notes and one old link in our-methods/nsga-iii-cpt.md).
…5885 made ambiguous dce5885 (plan 13, item S6: the fused TPE kernel) added span overloads TruncNorm.LogGaussMass(ReadOnlySpan<double>, ReadOnlySpan<double>, Span<double>) and Erf.ComputeArray(ReadOnlySpan<double>, Span<double>) next to the array ones. Three doc comments still named the methods without a signature: - src/OptunaSharp.Core/Internal/TruncNorm.cs:21 (class remarks) -> Erf.ComputeArray(double[]), the array overload the sentence is about (Optuna's >= 2000-element vectorized erf); - src/OptunaSharp.Core/Samplers/Tpe/ProbabilityDistributions.cs:491-492 (the grid precompute's remarks) -> the span overloads, which is what that code calls. A plain build does not show this: the projects keep NoWarn for the doc warnings, and only the documentation workflow (.github/workflows/documentation.yml) rebuilds with -p:NoWarn= and CS0419 among WarningsAsErrors. Found by running that gate with --no-incremental; an incremental rebuild skips CoreCompile and reports 0 errors, so the gate has to run on a fresh compile. No code change.
…), with Optuna-named overloads Rustuna's storages take a batch of attributes (set_trial_user_attrs and friends) and write it under one lock / in one transaction. Optuna has no such call: a caller loops over set_trial_user_attr, and every call takes the storage lock, checks the trial, and (in a database) commits on its own. The owner chose to build it (2026-11-09: "Build it", "Also have optuna's api overload"), so the batch exists under Rustuna's plural names and under Optuna's singular names as overloads. The result never differs from the loop: every override leaves exactly what calling the single-key method once per entry, in the dictionary's enumeration order, would leave. What changes is the cost, and that a rejected batch writes nothing. API (all documented with params, exceptions and remarks): - BaseStorage: virtual SetStudyUserAttrs / SetStudySystemAttrs / SetTrialUserAttrs / SetTrialSystemAttrs(id, IReadOnlyDictionary<string, object?>). The default body is the loop, so every third-party storage supports the call unchanged (atomic only as far as each single call). Non-virtual Optuna-name overloads SetStudyUserAttr(id, dict), SetStudySystemAttr(id, dict), SetTrialUserAttr(id, dict), SetTrialSystemAttr(id, dict) forward to them. - BaseTrial.SetUserAttrs(dict) (virtual, default loop) + SetUserAttr(dict) overload; Trial.SetUserAttrs overrides it: one storage call, then the trial's own frozen view and the thread-cache mirror take every entry, exactly as SetUserAttr(key, value) updates them per key. - Study.SetUserAttrs(dict) + SetUserAttr(dict) overload -> storage.SetStudyUserAttrs. - Empty dictionary: writes nothing and checks nothing (no KeyError for an unknown id), in every override; null throws ArgumentNullException. Per storage: - InMemoryStorage: one lock, one existence/updatable check, and at most one record copy for the whole batch (WritableTrialUnlocked from S4: in place while no reader holds the record). Study attributes: one ShallowClone of the dictionary, every entry written into it, published once. - JournalStorage: the batch is N ordinary SINGLE-KEY records appended with one AppendLogs. Not one record with N keys: Python's replay asserts `len(log["user_attr"]) == 1` (optuna/storages/journal/_storage.py, _apply_set_trial_user_attr and siblings), so a multi-key record would make the file unreadable to Optuna. Under the thread lock: sync, check the trial (new JournalStorageReplayResult.ThrowIfTrialNotUpdatable: KeyError NotFoundMsg / UpdateFinishedTrialError UnupdatableMsgFormat, the replay's own messages) or the study (ThrowIfStudyMissing), stamp the worker id, append, then SyncAfterBatch. SyncAfterBatch exists because the replay throws for a rejected record OF THIS WORKER after advancing past it, leaving the rest of the batch unread: it syncs up to batch-size times, keeps the first exception and rethrows it after the batch is drained, so one rejected record (another process finished the trial between our check and our append) does not poison the next call. Test hooks: AppendBatchAndSyncForTests, WorkerIdForTests (internal). - CachedStorage: forwards each batch whole to the backend (its own cache is read-through for attributes, as for the single calls). - SQLite RDBStorage: one transaction per batch: the study or trial (FindTrialOrRaise + CheckTrialIsUpdatableOrRaise) is checked once, every entry upserted with the same SQL as the single call, one commit. A failure rolls back the whole batch. - EF Core RDBStorage: the same through Mutate(ctx => transaction; check; add/update each entry; SaveChanges; Commit). Docs: three crefs that named BaseTrial.SetUserAttr / Study.SetUserAttr without a signature became ambiguous with the new overload (ComponentLosses.cs, FrozenStudy.cs, StudySummary.cs, and one in JanusMooCmixSamplerTests.cs); they now name (string, object?). The documentation gate (check_documentation.cs over src: 0 issues; Core rebuilt --no-incremental with the workflow's WarningsAsErrors: 0 errors) passes. Tests: - test/OptunaSharp.Tests/Storages/BulkAttributeWritesTests.cs (new, 14 cases): batch == loop for in-memory, journal and cached storage (trial user/system, study user/system, overwrite order); the Optuna-name overloads; empty and null dictionaries; finished/unknown trial and unknown study write nothing and throw the single call's error; Trial.SetUserAttrs updates the trial's own view and a later frozen copy; a storage that does not override gets the loop (DelegatingStorage test double); the journal writes N single-key records with one worker id; and the drain test (a rejected own record followed by valid ones: the call throws the first error and the next call works). The drain test was checked against a mutation (SyncAfterBatch reduced to one sync): it fails there. - test/OptunaSharp.Tests.Storage/SqliteRdbStorageTests.cs (+3): batch leaves what single writes leave (read back through a fresh storage), all-or-nothing on a finished trial, and the batch through CachedStorage reaches the database. - test/OptunaSharp.Tests.Interop/JournalInteropTests.cs (+1, live Python): CSharp_WritesAttributeBatches_Python_ReadsEveryOne: C# writes a trial user batch (int, float, string, bool, None, list), a trial system batch, a study user batch and a study system batch to a journal file; live Optuna 5.0 loads the study, reads every attribute with its value (json.dumps compared byte-for-byte), continues the study with two trials, and C# reloads all three. Runs: full CI unit filter (TestCategory!=OpenBugs&!=HighMemory&!=RequiresPython&!=Slow) on both TFMs: 2070 passed, 1 skipped, 0 failed (net8.0 and net10.0). Storage suite cases and the interop case pass.
…n 13, item S12); same trials, a fifth of the memory
Owner, 2026-11-09: S12 "Build it", "If it changes optuna's/rustuna's behavior then make the feature optin". It changes
no result (seeded studies give the same trials, bit for bit, and live-Optuna parity holds on it), but finished trials
stop being dictionaries: their read-only accessors return views, numbers and long strings are rebuilt per read, and a
finished record no longer shares its dictionaries with the running record it was copied from. So it is a switch, off
by default:
new InMemoryStorage(compactFinishedTrials: true) // also: storage.CompactFinishedTrials
What a finished trial becomes (src/OptunaSharp.Core/Internal/CompactTrialParts.cs, new):
- Parameters: CompactParams = one internal-representation double per name against a ParamSchema (names +
distribution instances) shared by the study's trials through a per-study ParamSchemaCache (at most 256 shapes;
further shapes keep their dictionaries). A trial is accepted only when every stored value comes back
indistinguishable from d.ToExternalRepr(d.ToInternalRepr(stored)) (DistributionIdentity.IsSameValue: same type and
bits for numbers, -0.0 stays -0.0; same characters for strings; the same object otherwise), so reads return what
was stored. Fallbacks that keep the dictionary, each pinned by a test: an int under a float distribution (Optuna
keeps it an int), an integer beyond 2^53, a categorical choice Python-equal to an earlier one (1 then 1.0), a value
its distribution cannot convert, names listed in another order than the distributions.
- Distributions: the schema's one read-only DistributionsView, shared by every trial of the schema. Schemas match
distributions by DistributionIdentity.AreInterchangeable: bitwise for float/int (S7's rule), identical choices for
a plain CategoricalDistribution (every SuggestCategorical call builds a new distribution, so identity matching
would give every categorical trial its own schema), the same instance for anything else.
- Attributes: CompactAttrs = key and value arrays (one shared Empty), long ASCII strings as CompactString bytes:
ASCII one byte a char, lowercase hex two digits a byte with an odd trailing digit kept aside (CMA-ES splits its
pickle hex into 2,045-char chunks), uppercase hex and non-ASCII stay strings. Nested containers stay the same objects.
- Intermediate values: an empty dictionary becomes a shared sentinel never handed out; a non-empty one a copy sized
to its entries.
FrozenTrial (src/OptunaSharp.Core/FrozenTrial.cs):
- The four dictionary fields are typed IReadOnlyDictionary and hold either the OrderedDict or a compact form;
Params/Distributions/UserAttrs/SystemAttrs return the field as is (unchanged objects on the default path).
- ParamsDict/DistributionsDict/UserAttrsDict/SystemAttrsDict and IntermediateValues return real dictionaries: on a
compact record the first read materializes one and installs it with Interlocked.CompareExchange (Decompact), so
however many threads ask at once they all get the one dictionary the record keeps, and a write through it stays.
- New internal: CompactFinished(schemas), IsCompact, IntermediateValuesForRead (the non-pinning read).
- Equals, ToString, DeepCopy, LastStep, Constraints, Validate read through the views (comparing and printing never
decompact); SetUserAttr, Suggest's distribution write and SetConstraint write through the mutable accessors;
ShallowCopy gives a compact record its own dictionaries first and shares them, as Python's copy.copy shares dicts.
InMemoryStorage: the switch; compaction in SetTrialStateValues on the new record before it is published (under the
lock, only reading the dictionaries the old running record may still share) and in CreateNewTrial for a template
that is already finished (Study.AddTrial). GetTrialParam returns a compact trial's stored internal value directly.
Read paths kept compact: about 70 internal reads of finished trials moved from the mutable accessors to the
read-only ones (no change without the switch: both return the same object): CMA-ES/DE/ES/LM-CMA/VkD-CMA/PSO/JANUS
Transform(t.Params); GP, SAASBO, TuRBO, the terminators; TPE's GetParams (now IReadOnlyDictionary); NSGA-II/MOEA-D/
NSGA-III-CPT crossovers and conditional params; QMC, intersection and group-decomposed search spaces
(SearchSpaceGroup.AddDistributions takes IReadOnlyDictionary); BaseGASampler generations; SuccessiveHalving,
Percentile and Wilcoxon pruners; importance evaluators; dataframe; retry callbacks; Study.ShouldSkipEnqueue;
visualization; the CLI records; the BoTorch and pymoo bridges.
TPE fast path: GetInternalRepr copies a compact trial's stored internal value instead of boxing and converting the
external one, for parameters whose search-space distribution is interchangeable with the trial's
(ParamSchema.PositionsFor). Exact by construction: compaction stored internal = d.ToInternalRepr(stored) only when
d.ToExternalRepr(internal) is indistinguishable from stored, and an interchangeable distribution converts as d does.
Tests:
- test/OptunaSharp.Tests/Storages/CompactFinishedTrialsTests.cs (new, 35 cases): every kind of value reads back
identical (types, bits, order, reprs, constraints, LastStep, GetTrialParam); Equals/ToString match Optuna's
representation both ways; views read like dictionaries (generic and non-generic, CopyTo, KeyNotFound message) and
refuse writes; mutable accessors keep writes and stay single under 8-thread contention; the five fallbacks; schema
sharing per shape incl. per-trial categorical distributions and the 256 cap; identity rules; compact strings;
15 sampler families run seeded on and off to identical trials with every finished record still compact; CMA-ES
chunks stored as hex bytes and restored by a second sampler; 4 workers + 3 readers; retained memory below a
quarter. Mutation checks: TPE reading through ParamsDict fails exactly the 5 TPE rows; a dropped trailing hex digit
fails 19 cases; a plain assignment instead of the compare-exchange fails the contention test 3 runs of 3.
- test/OptunaSharp.Tests.Interop/CompactStorageParityTests.cs (new, 9 cases, live Optuna 5.0): Random with long ASCII,
odd hex, non-ASCII and nested attributes, NSGA-II mixed space, QMC and CMA-ES bit-identical full state (CMA-ES chunk
strings equal to an uncompacted C# run's), TPE multivariate within the erf tolerance; records stay compact.
Proof (scratchpad proof-s12, the plan 13 workloads: 40 floats, seed 0, plain then compact back to back, every pair
hash-identical; retained = heap the finished study keeps after a full collection):
workload retained plain -> compact peak WS time (steady) allocated
random 10k 64.2 -> 6.7 MB (6732->704 B) 200 -> 91 MB 3.52 -> 3.53 s 2.31 -> 2.32 GB
qmc 10k 64.2 -> 6.7 MB 198 -> 93 MB 1.02 -> 0.85 s 1.58 -> 1.60 GB
nsga2 10k 67.7 -> 9.7 MB (7099->1012 B) 205 -> 103 MB 1.57 -> 1.44 s 1.23 -> 1.29 GB
tpe 1k 13.4 -> 6.9 MB 109 -> 92 MB 3.84 -> 3.37 s 1.57 -> 1.60 GB
motpe 1k 7.2 -> 1.4 MB 76 -> 77 MB 12.7 -> 11.6 s 3.42 -> 3.45 GB
cmaes 10k 129.5 -> 26.5 MB 372 -> 137 MB 2.15 -> 2.46 s * 1.74 -> 1.85 GB
nsga2 100k 670.9 -> 87.5 MB (7035->917 B) 751 -> 167 MB 15.7 -> 14.6 s 12.4 -> 13.0 GB
cmaes 100k 1287.3 -> 256.8 MB 1846 -> 458 MB 23.2 -> 22.1 s 17.4 -> 18.4 GB
* repeated alternately 3x3: steady reps average 2.28 s plain vs 2.31 s compact, i.e. noise.
Before the two allocation fixes found by gc-verbose profiling (TPE boxing its history read; the constraints scan
rebuilding every attribute string), compact TPE allocated 3.06 GB vs 1.57 and MOTPE 3.90 vs 3.42; now equal.
Also fixed on the way (same results, both modes): GetConstraintsFromSystemAttrs scans keys and reads only matching
values, so a compact trial rebuilds no attribute string it does not need.
Gates on the final code: unit CI filter 2105 passed / 1 skipped / 0 failed on net8.0 and net10.0 (2070 before + 35);
live-Python interop (net10.0, engine required, minus Slow) 692 passed / 24 skipped / 0 failed; Storage 455 passed /
144 skipped (no Redis/PG/MySQL services) / 0 failed; CLI 91 passed + its 4 known [OpenBugs] QA reproductions;
check_documentation.cs 0 issues; Core and the touched satellites rebuilt --no-incremental with the doc gate's
WarningsAsErrors: 0 errors.
Docs: performance.md (Memory: compact finished trials; Writing several attributes at once), storage.md, README,
study-and-trials.md (SetUserAttrs), THREADING.md §6.1, plan 13 status, §10 rows (S8, S12, D2) and new §10.2.
…§10 row 17716c7 wrote "this commit" in the plan's §10 S12 row because a commit cannot know its own hash. This fills in the hash there and in the status line, and the row's peak working set now quotes the final proof run that §10.2's table reports (1,846 → 458 MB; the earlier figure, 1,845 → 460 MB, came from the second of three runs). No code change.
…parameter paths (plan 14, step A); random search 5x, NSGA-II 1.9x, same trials The second Rustuna comparison (catalog note optunasharp/notes/rustuna-comparison-2026-11-08.md, "Second pass") found the biggest remaining cost was not a Rustuna technique at all: every NumSharp NDArray has a finalizer, and the hot sampler paths created arrays per suggestion and dropped them. Each one survived its first collection on the finalization queue and was promoted, so almost every gen-0 collection became a gen-1 (random 10k: gen-0/1 154/153; NSGA-II 10k: 87/86). GC pauses were 47 % of a 40-parameter random search and 41 % of NSGA-II. What changed (no result changes; every draw is the same call on the same generator, in the same order): - New Internal/NumPyDraws: Rand(), Rand(n), RandomSample(n) and Uniform(low, high) read the generator's bit generator directly (BitGenerator.NextDouble under BitGenerator.@lock, exactly what NumSharp's scalar and fill paths do: one next_double per value, MT19937's 27 + 26 bits of two consecutive words), so they create no array at all. Uniform keeps NumPy's order: range = high - low, the finiteness check before any draw (OverflowError text "Range exceeds valid bounds"), then low + range * U. RandInt (masked rejection) and Normal (cached polar Gaussian) still call NumSharp and dispose the 0-d/vector result. - SearchSpaceTransform: TransformValues (the managed array Transform wrapped) and UntransformValues(ReadOnlySpan<double>) (the untransform without a NumSharp copy). Untransform(NDArray) now disposes its astype(float64) copy and delegates. The untransform result dictionary is sized up front. - RandomSampler.SampleIndependent draws the transform's bound rows one at a time through NumPyDraws.Uniform instead of rng.uniform(bounds[:, 0], bounds[:, 1]). Bit-identical: NumPy computes the same low + range * U per element from the same uniforms; the array form's only other difference (raising before any draw when a range is infinite) cannot arise, since a numerical parameter has one row and every categorical row is [0, 1]. It used to build five arrays per suggestion (bounds, two column views, the draws, the untransform copy). - NSGA-II family: NSGAIIChildGenerationStrategy (crossover decision, copy-parent pick, one decision per gene), NsgaiiCrossover (tournament randints, categorical uniform crossover, parents transformed straight from their params with TransformValues, child untransformed with UntransformValues, no population copy before the first pick), every crossover (Uniform, BLX-alpha, SBX/VSBX and their shared tail, SPX, UNDX) and PolynomialMutation go through NumPyDraws; result dictionaries are sized up front; NSGAIISampler.InferRelativeSearchSpace sizes its result. - CMA-ES: the per-ask seed draw, the told solutions' transform and the candidate's untransform use the helpers; the dimension comes from RawBounds instead of building the bounds array for its shape. QMC untransforms its point without a NumSharp copy. TPE's candidate sampling (mixture choice, categorical quantiles, truncated-normal quantiles) uses the helpers. Measured (proof harness of plan 13 built against the b3baef9 snapshot and this tree, 40 floats in [-10, 10], seed 0, warm rep, interleaved runs, machine at 25-48 % load from other sessions): random 10,000: 3.93-3.95 s -> 0.79-0.80 s (5.0x); allocated 2,305 -> 889 MB; gen-0/1/2 154/153/27 -> 56/16/7; GC pauses 1.67-1.69 -> 0.17-0.18 s; hash 5DC310069B28C5D7 both NSGA-II 10,000: 1.80-1.81 s -> 0.95-0.96 s (1.9x; Rustuna 0.2.1: 0.82 s); allocated 1,234 -> 962 MB; gen-0/1/2 87/86/20 -> 59/16/7; pauses 0.67 -> 0.20 s; hash 0B1FA7FBED2E9676 both CMA-ES 10,000 2.33 -> 2.20 s, TPE 1,000 3.80 -> 3.78 s, QMC 10,000 1.15 -> 1.21 s: unchanged within noise, hashes equal (their time is elsewhere; later plan 14 steps). Parity coverage (each affected area, byte for byte): - New unit NumPyDrawsTests: every helper against the NumSharp call it replaces, value bits and next stream state, on generators seeded alike (scalar/vector uniforms, TPE's sizes, every bound shape RandomSampler meets, row-by-row vs NumSharp's array uniform over a bounds column, non-finite ranges, randint seed ranges, Gaussians with the cached polar value, and NSGA-II's mixed draw sequence). - SearchSpaceTransformTests: TransformValues equals Transform and UntransformValues equals Untransform bit for bit (mixed space, with and without the unit cube, clamped and tied coordinates); the NDArray overload still reads float32/int64 input through a float64 copy and leaves the input usable. - RandomSamplerTests: 60 trials x 3 seeds over every distribution kind replay the pre-plan-14 array draw and match. - New live parity RandomSamplerParityTests.LongRun_AllDistributionKinds_IsBitIdentical_ToLiveOptuna: 300 trials x 2 seeds against live Optuna 5.0, one-hot categoricals with two and five mixed-type choices, log int to 2^40, stepped negative int, a float spanning +-1e300, every value compared through Python's repr. - Live parity suites rerun, all green against Optuna 5.0 (142/142, Slow long runs included): RandomSampler, NSGA-II (all crossovers, polynomial mutation, constraints, 2,000-trial 40-parameter run), NSGA-III, CMA-ES (+ mixed fleet), QMC, TPE (+ 1,000-trial and MOTPE 300-trial runs), compact storage, battle-tested usage, edge cases, partial fixed, brute force. - Hash probe (scratchpad p14/hashprobe, 35 seeded studies over every sampler family the change reaches directly or through RandomSampler and SearchSpaceTransform: random, TPE independent/multivariate/group/pruned/constrained, MOTPE, NSGA-II with every crossover, polynomial mutation and constraints, NSGA-III, SMS-EMOA, AGE-MOEA-2, MOEA/D, RVEA, CMA-ES plain/with margin/separable/IPOP, QMC Sobol/Halton, DE, PSO, SNES, xNES, OpenAI-ES, LM-CMA, VkD-CMA, JANUS, brute force): every hash equal before and after. - Unit suite, CI filter: net8.0 2136/2136; net10.0 2135/2136 with JournalConcurrencyTests.Lock_WithoutAGracePeriod_ WaitsForALeftBehindLockUntilItIsRemoved failing once under load and passing on 3 reruns (timing-sensitive journal lock test, untouched code). The OpenBugs subset of the touched areas fails the same 8 tests with the same messages before and after. - Documentation gate: check_documentation.cs 0 issues; Core builds with the doc warnings as errors (--no-incremental).
… exp call where it is exactly zero (plan 14, step B); TPE 1.5x, MOTPE 1.2x, same trials Stopwatch timing of TPE's stages on 1,000 trials over 40 floats (the profiler's sampled thread time had hidden it: EventPipe suspensions inflate some frames) showed where a run went before this step: scoring the candidates 1.74 s (40 %), building the Parzen estimators 1.34 s (31 %; the bandwidth argsort alone 1.05 s, 24 %), converting the history to internal values 0.60 s (14 %). Three changes, each keeping every bit: 1. Kernel widths in O(n) when that is provably exact (ParzenEstimator.BinnedRawSigmas; Rustuna's binned_sigmas idea, rustuna_core/src/parzen_estimator/model.rs). A width is the larger distance to the two sorted neighbours, clipped up to minsigma, so a neighbour nearer than minsigma cannot change it. Binned at that width, a centre that is neither the smallest nor the largest of its bin clips to minsigma, and only each bin's extremes can have a farther neighbour (the adjacent non-empty bin's extreme, or the domain end). Rustuna trusts the binning and can differ in the last bits where rounding puts a centre in the neighbouring bin; this version proves its result instead: bin assignment floor((v - low) / minsigma) is monotone, so bins are contiguous runs of the sorted order and equal centres share a bin; a bin's smallest is taken at its first index and its largest at its last (the stable sort's positions); distances are the sort's subtractions of the same operands; and every bin is checked to span at most minsigma (computed), which by the monotonicity of IEEE subtraction bounds every in-bin neighbour distance, so dropping those changes no clipped width. Anything else (fewer than two centres, a centre outside the domain or NaN, a degenerate domain, magic clip off, a wider bin) falls back to the sort, now SortedRawSigmas. ComputeSigmas applies the same clip to either. 2. TpeObservationCache (Rustuna ddd9c56 / e7a22d7, "reuse completed observations across trials"): each finished trial's internal value per (parameter name, search-space distribution), by trial number, or the fact that it lacks the name. GetInternalRepr copies a known row from dense columns instead of chasing each trial's parameter dictionary; an unknown row is converted exactly as before (the subset test first, conversions in search-space order, so the same exceptions) and remembered. Unfinished trials (constant liar's running ones) are never cached. Columns are keyed by distribution value equality (equal distributions convert every value alike); a call for another study or storage starts over; past 256 columns (dynamic ranges) it starts over. Each instance (per-worker copies too) owns its cache under its own monitor. The non-cached path also reads each value with one TryGetValue instead of ContainsKey plus the indexer. 3. The log-sum-exp stores 0.0 for every term below MixtureOfProductDistribution.ExpIsZeroBelow (-746) instead of calling Math.Exp: exp there is exactly +0.0 (exp(-746) is 0.42 of half the smallest subnormal; NumPy's, ucrt's, glibc's and Apple's exp all return zero below the underflow point, about -745.1332), and the zero still enters the pairwise sum, so the summation tree is unchanged. Measured on this machine: Math.Exp costs 5 ns for normal results, 16 ns on its underflow-to-zero path and 63 ns where the result is subnormal (that band stays computed). Measured (plan 13 proof harness built against the b3baef9 snapshot and this tree, 40 floats in [-10, 10], seed 0, interleaved runs; hashes equal before and after): TPE 1,000 (warm rep): 3.81-3.95 s -> 2.51-2.55 s (1.5x); allocated 1,571 -> 1,028 MB; hash 77CB9F115180830A MOTPE 1,000 (rep 0): 12.67 s -> 10.68 s (1.19x); allocated 3,418 -> 2,832 MB; hash A14E3A50093C4DF4 MOTPE 300 (rep 0): 2.31 s -> 1.83 s before change 3 Stage timing after changes 1-2 (rep 0, scratch instrumented copy): internal values 0.60 -> 0.15 s, estimator construction 1.34 -> 0.54 s; scoring 1.75 s unchanged then (change 3 came after). MOTPE's scoring is dominated by exp calls whose results are subnormal (63 ns each), which an exact port cannot skip. Parity coverage (byte for byte where OptunaSharp is compared with itself; TPE vs live Optuna is trajectory parity within the documented erf tolerance, as before): - New TpeBinnedSigmasTests: binned vs sorted widths bit for bit over uniform, tied (grids), clustered, on-edge, one-ulp-off-edge, domain-end and floor-spaced centres, 14 sizes from 2 to 1,500, ten domains (log, step-widened, tiny, huge, far from zero), both endpoint settings, three seeds; asserts the fast path answers on more than 90 % of uniform inputs so the equality checks are not vacuous; fallback cases; ComputeSigmas end to end with and without the magic clip. Scratch fuzz: 800,000 more generated cases, about 769,000 answered by the binned path, 0 mismatches. - New TpeObservationCacheTests: warm cache vs the cache emptied before every suggestion, identical trials: dynamic ranges + conditional parameter + pruned trials in both modes on plain and compact storage, MOTPE, constant liar with running trials (ask three, tell three) in both modes, an ever-changing range (bounded at MaxColumns), and a sampler reused for a second study. - New TpeExpUnderflowTests: Math.Exp returns +0.0 bits for the 100,000 doubles below the bound and 200,000 inputs down to -1e6 (and -inf, double.MinValue) on every platform CI runs; the bound sits below the underflow point. - New live parity in TpeSamplerParityTests: DynamicRangesConditionalAndPrunedTrials (both modes, 4 cases, 80 trials) and ConstantLiarWithRunningTrials_AskTell (both modes, 3 cases, 15 rounds of 3 running trials). The whole TPE live suite passes, 103/103 with the importance, compact-storage, docs and recipe parity cases, Slow long runs included (TPE 1,000 trials x 40 parameters, MOTPE 300); importance parity (PED-ANOVA shares the estimator) 40/40. - Hash probe (35 seeded studies across every sampler family): 0 differences before/after. - Unit suite, CI filter: 2154/2154 on net8.0 and net10.0. Documentation gate: 0 issues; Core builds with the doc warnings as errors.
…step C); QMC 2.4x, same trials QMCSampler finds its sample id in a study system attribute named "qmc:" + sha256(json.dumps(qmc_vars)), where qmc_vars holds the engine kind, the search space as str(distribution) by sorted name, and the scramble flag and seed (Optuna's _find_sample_id). OptunaSharp rebuilt that key on every trial: 40 distribution reprs (each an AsDict, a sort and a StringBuilder), a JSON document and a SHA-256. An allocation trace of 5,000 40-parameter trials put most of QMC's 1.58 GB per 10,000 trials there. The key is a pure function of the search space and the sampler's fixed settings (qmc type, scramble, seed are all readonly). QmcIdKey now keeps it for the last search space and reuses it while the next call passes the same names in the same order with the very same distribution objects, which a study's relative search space does from its first trial on (QMC builds it from the first trial's distributions). Reference identity, not value equality, decides: two categoricals can be equal by Python's rules (True == 1) and still print differently, so only the same objects are known to print the same; anything else recomputes. The sample-id read and write still happen on every call. The memo runs under the sampler's lock with the rest of sampling, and per-worker instances start without one. Also sized up front: the relative search space and the untransformed result dictionaries. Measured (plan 13 proof harness against the b3baef9 snapshot, 40 floats, seed 0, warm rep, interleaved): QMC 10,000: 1.24-1.30 s -> 0.52-0.53 s (2.4x); allocated 1,584 -> 472 MB; gen-0/1/2 98/22/10 -> 30/12/4; hash 1ADA76D940EDB34F both. Parity coverage: - New live gate QmcSamplerParityTests.LongRun_40Parameters_300Trials_FullState_IsBitIdentical: Sobol and Halton, scrambled and not, 37 floats + a log float + an int + a categorical over 300 trials; the whole study state, the qmc:<sha256> counter in the study system attrs included, bit-identical to live Optuna 5.0. The QMC live suite passes, 16/16; with battle-tested usage and the recipes 47/47 before the new gate. - New unit test QmcSamplerTests.QmcIdKey_Memoized_EqualsAFreshComputation: a memoized key equals a fresh sampler's for the same objects, for equal-by-value categoricals that print differently (True/False vs 1/0, which must and do give a different key), for a reordered space, and back again; scrambled and not. - QMC unit tests 31/31.
… step D; Rustuna 82b13d8); 160 MB -> 0.1 MB on a 4.3M-line journal, same records JournalFileBackend kept the byte offset of every log line it had read (Python's _log_number_offset dict), so a sync can seek to the next unread line. A 100,000-trial, 40-parameter study writes about 4.3 million lines: the dictionary alone held 160 MB for the life of the storage. Now it keeps what a read actually needs (Rustuna's scheme): the furthest line start any read has established (a sync asks for exactly the next unread line, so it still seeks straight there) and one checkpoint every OffsetCheckpointInterval = 4,096 lines. A read for an earlier line (another storage over a shared backend that is behind, a replay from the start) seeks to the nearest checkpoint at or before it and skips forward at most 4,095 lines, exactly as Python skips the lines before log_number_from when it reads from byte 0. The records each read yields, their numbering and the size snapshot are unchanged: - A line's successor start is recorded only after the line decodes (Python records it first and deletes it on a decode error), so an undecodable record never becomes a place a later read starts after; lines before the requested range are skipped undecoded and recorded, as Python records them. - Readers sharing a backend record the same offsets (the file only grows): the furthest start only moves forward, a checkpoint is appended by whichever reader passes it first, and a reader can only start past a checkpoint some read passed, so checkpoints stay in order. All bookkeeping is under the existing _offsetLock. Measured (journalmem probe, scratchpad p14/journalmem, built against the b3baef9 snapshot and this tree; a synthetic 4.3M-line, 1.09 GB journal of set_trial_param-shaped records): memory the backend retains after a full read: 160.3 MB -> 0.1 MB full read: 5.48-7.36 s -> 5.22-5.26 s; next sync (seek to the end): 0.28-0.35 ms -> 0.34-0.36 ms Parity coverage: - New unit JournalOffsetCheckpointTests: reads from 13 positions (both sides of each checkpoint, the end and past it) equal the matching slice of a full read on a cold and on a warm backend, and the warm backend kept 3 checkpoints for 8,692 lines; a storage-like reader that stopped half way resumes exactly after the file grows and another reader reaches the end; 12 concurrent readers from scattered positions on one backend all read exactly; an undecodable record followed by another fails every read that reaches it (from before it, from it) instead of being skipped; a partial last line is left for the next read. - New live gate JournalStateParityTests.LargeJournal_PythonWrites_CSharpReplaysAndContinues_SharedBackendCatchesUp_ FullState: Python writes 250 trials x 40 parameters (more than two checkpoint intervals); a C# storage replays Python's full state; two C# storages share one backend while one writes 30 more trials, so the other's next sync starts behind the furthest offset; the lagging storage, a cold C# storage and Python's fresh replay all give the identical full state (floats and datetimes as instants). - Journal suites: unit 95/95; live 30/30 (5 Redis cases skip without a Redis server; the Redis backend is untouched).
…an 14, step E; Rustuna d46ace5); 8,000 trials with a stale running trial 13.0 -> 9.6 s, same rows CachedStorage refreshes a study by asking the RDB storage for trial_id > watermark (the last finished trial) plus the unfinished trials it tracks at or below it: Optuna's query is `... WHERE study_id = ? AND (trial_id > ? OR trial_id IN (...)) ORDER BY trial_id`. That second part is non-empty only while a trial below the watermark is still RUNNING: a parallel worker's, or one a crashed worker left behind (then on every refresh for the rest of the study). SQLite answers the OR by walking every trial of the study in ix_trials_study_id: 2.6 ms per refresh at 20,000 trials (plan 14 research, scratch p14/sqlplan), so such a study's per-trial cost grew with its whole history. The SQLite RDBStorage now sends the two halves joined by UNION ALL, each indexable through the primary key (64 us there): `<select> AND trial_id > ? UNION ALL <same select> AND trial_id IN (...) ORDER BY trial_id`. The halves are disjoint (every included id is at or below the watermark, filtered as before), the final ORDER BY sorts the union, and both halves repeat the same named parameters, which SQLite binds to the same values: the rows and their order are the OR's. Rustuna also adds a (study_id, number) index; that changes Optuna's schema, so it is not taken. The EF Core storage (PostgreSQL, MySQL, whose planners combine ORed index ranges) keeps Optuna's form. Measured (sqlgrowth probe, scratchpad p14/sqlgrowth2, built against the b3baef9 snapshot and this tree: a seeded 10-parameter random search on a SQLite file in WAL mode with synchronous off, through CachedStorage, with one trial asked and left RUNNING before the run): 8,000 trials: 13.0 s -> 9.6 s; per trial by 2,000-trial chunk 1.77/1.38/1.55/1.78 ms -> 1.30/1.01/1.18/1.30 ms Parity coverage: - New live gate SqliteRdbInteropParityTests.RunningTrialBelowTheWatermark_CachedRefresh_MatchesPythonsRead: Python creates the study on a SQLite file, leaves its first trial running and completes 15; C# loads it through CachedStorage and optimizes 25 (every refresh takes the new query with the running trial included); Python then suggests a parameter, reports an intermediate value and tells the running trial; C#'s cached view equals Python's fresh read, field for field (41 trials, the formerly running one complete). - New unit SqliteRdbStorageTests.CachedRefresh_WithRunningTrialsBelowTheWatermark_EqualsAFreshRead: two storages on one file; running trials below the watermark, completions around them, a running trial changed, then finished out of order (one completed, one failed); after every step the cached view equals a fresh storage's read (numbers, states, params, values, intermediate values, in order). - SQLite suites: unit 52/52; live 21/21 (SQLite RDB, storage recipes, storage URLs). The schema-upgrade live cases (SqliteRdbUpgradeInteropParityTests) locate the repository root from the test binaries and so cannot run from the out-of-tree build used here; they exercise the migration chain, which this change does not touch. - Documentation gate: check_documentation.cs 0 issues; Storage.Sqlite builds with the doc warnings as errors.
…ice (plan 14, step F; Rustuna 198e484); categorical-heavy TPE 2.5-2.7 -> 1.8 s, same trials MixtureOfProductDistribution.LogPdf added, for every categorical dimension, log(weights[k, x]) for each candidate and kernel: Optuna's `np.log(np.take_along_axis(weights[None], xi, axis=-1))[..., 0]`, one Math.Log per (candidate, kernel). TPE scores 24 candidates against every kernel of both estimators, so a three-choice parameter over a thousand kernels took 24,000 logs per estimator, of which only 3,000 are distinct. Rustuna's 198e484 also cuts these log calls (with its own numerics, which OptunaSharp does not take). The categorical branch now takes the log of a (kernel, choice) column the first time some candidate draws that choice and reuses it for every other candidate with the same choice: one log per kernel and drawn choice. Math.Log is deterministic, so every added term has the bits it had; the terms are added to the same buffer elements in the same order (dimensions in order, then the log mixture weights, then the log-sum-exp), so every sum keeps its bits too. Out of range or negative choices still throw IndexOutOfRangeException (from the column array instead of the weight matrix). Measured (hash probe, scratchpad p14/hashprobe, built against the b3baef9 snapshot and this tree, interleaved): tpe-categorical-heavy (500 trials, 20 five-choice categoricals + 20 floats): 2.5-2.7 s -> 1.8 s, hash E015C07493B6830A on both sides. Parity coverage: - New unit TpeExactKernelTests.CategoricalLogPdf_OneLogPerKernelAndChoice_MatchesThePerCandidateLogsToTheBit (seeds 1 and 2): mixtures of 1, 2 and 4 categorical dimensions with 2-7 choices, weights drawn with some exact zeros (so some terms are -inf), 1 to 40 kernels, 24 candidates each; LogPdf equals the per-candidate reference (the code this replaces, written out in the test) to the bit for every candidate. - Hash probe: all 36 seeded scenarios (TPE independent / multivariate / group / pruned / constrained / MOTPE / categorical-heavy, NSGA-II with every crossover, NSGA-III, SMS-EMOA, AGE-MOEA-2, MOEA/D, RVEA, CMA-ES variants, QMC, DE, PSO, NES, LM-CMA, VkD-CMA, JANUS, BruteForce) hash the same before and after. - Unit: TPE + importance filter 103/103 (net10.0); TpeExactKernelTests 15/15. - Live Optuna (pythonnet, Python 3.12 + vendored optuna 5.0.0): TpeSamplerParityTests (non-Slow) + ImportanceParityTests 70/70. - Documentation gate: check_documentation.cs 0 issues; Core builds with the doc warnings as errors.
…d final proof; README and performance page numbers rerun after it
New docs/plans/14-rustuna-second-pass.md, the record of the six plan 14 steps on main (A cfe78ec NumSharp
temporaries, B 3050d81 TPE widths / observation cache / exp skip, C fe68085 QMC sample-id key, D 7537079 journal
sparse offsets, E 39fdd4b SQLite UNION ALL refresh, F 82b8a0b TPE categorical logs): the owner's request ("1. Each
affected area must have byte-for-byte parity test coverage 2. Improve performance - Implement."), the inventory and
verdicts (what was built, what was not taken because it changes bits or Optuna's schema, what stays open), what each
step does, the byte-for-byte gates per affected area (unit pins against the replaced code, the 36-study hash probe,
live Optuna 5.0), the final before/after proof, findings and open items (TPE ordered-array scoring and estimator
copies, NSGA-II per-trial precompute, CMA-ES in-memory restore, scrambled Sobol's O(n^2) point generation, NumSharp
temporaries in samplers that were not traced).
Final proof (plan 13 proof program built against the b3baef9 snapshot and 82b8a0b, 40 floats in [-10, 10], seed 0,
before/after back to back, 2026-11-09 14:23-14:27, ~30 % load from other sessions; every pair hash-identical):
random 10k rep 0 4.31-4.46 -> 0.81-0.82 s, rep 1 3.52-3.61 -> 0.50-0.52 s (6.8-7.2x); gen-0/1/2 130/129/2 -> 37/9/1
QMC 10k rep 1 0.99-1.01 -> 0.43-0.45 s (2.2-2.3x); allocated 1,584 -> 472 MB
NSGA-II 10k rep 1 1.51-1.57 -> 0.75 s (2.0-2.1x); 100k rep 0 17.19 -> 8.60 s (Rustuna 0.2.1: 19.4 s)
TPE 1k rep 1 3.86-3.91 -> 2.47 s (1.6x); compact storage 3.39 -> 2.43 s
MOTPE 1k 12.70-12.82 -> 10.44-10.52 s (1.2x)
CMA-ES 10k/100k unchanged within noise
TPE, 20 categoricals + 20 floats, 500 trials: 2.5-2.7 -> 1.8 s
full hash probe, 36 seeded studies over every sampler family: identical before and after
README: the sampler table's OptunaSharp column rerun (benchmark 14's C# probes built against b3baef9 and 82b8a0b,
three fresh processes each, interleaved; medians): TPE 6-D 1.12 -> 0.61 s, TPE multivariate 1.14 -> 0.66 s, TPE 100-D
2.1 -> 1.6 s, CMA-ES 6-D 0.91 -> 0.74 s, CMA-ES 200-D 1.2 -> 1.0 s, NSGA-II 0.60 -> 0.31 s, QMC 0.39 -> 0.13 s, random
6-D 0.98 -> 0.15 s, random 1000-D 3.2 -> 0.6 s, random 50k blocks 0.6-0.8 -> 0.08-0.25 s; GP moved less than the
noise and keeps 3.47 s. (The README's old numbers predate plan 14 and some were measured under heavier load; the plan
document has the interleaved before/after medians.) The large-studies table takes the final proof's rep 0 times: TPE
3.2 s, MOTPE 10.5 s, CMA-ES 10k 3.6 s and 100k 22.1 s, NSGA-II 10k 1.4 s and 100k 8.6 s.
docs/website-src/docs/performance.md: a "Second comparison" section with the plan 14 changes, their effect and the
before/after table. docs/THREADING.md: the two new sampler caches (TpeObservationCache, QMC's sample-id key) are sampler
state covered by the sampler monitor, one per PerWorker instance, like plan 13's. docs/plans/README.md: plan 14 row.
Final gates on 82b8a0b (out-of-tree Release build):
- Unit suite, CI filter: net8.0 2163 passed, 1 skipped; net10.0 2163 passed, 1 skipped (the skip is
StopSignalsTests.ARealSigterm_CancelsTheToken_AndTheProcessLivesOn, which self-skips on Windows).
- Live Optuna 5.0 (engine required), every area plan 14 touched, non-Slow: RandomSampler, NSGA-II, NSGA-III, CMA-ES
and the CMA-ES mixed fleet, QMC, TPE, importance, compact storage, battle-tested usage, edge cases, partial fixed,
brute force, journal state / interop / mixed fleet / codec, SQLite RDB: 228 passed, 1 skipped
(JournalMixedFleetInteropTests.PythonAndCSharpWorkers_OnOneRedisJournal_ProduceIdenticalPerfectStudies needs a Redis
server; the Redis backends are untouched).
- Storage suite (SQLite RDB storage and options, journal backend contract): 95/95.
…ecreated under a reused id is not read from the deleted one (plan 14 concurrency review); new concurrency tests for plan 14's state The owner asked whether plan 14 broke concurrency. The review (docs/plans/14-rustuna-second-pass.md section 8) went through everything plan 14 added that more than one thread can reach: - TpeObservationCache: reached only from GetInternalRepr <- BuildParzenEstimator <- Sample <- SampleRelative / SampleIndependent, all under the sampler monitor (BaseSampler.SyncRoot), which every study-driven call holds, also when several studies share the sampler. A PerWorker instance gets its own cache (the field initializer runs for the private copy constructor, which does not copy it). - QMC's sample-id key memo: reached only from SampleRelative, under the monitor; QMC has no worker instances. - JournalFileBackend's offsets: every access under _offsetLock, each read carries its own position, line starts are recorded after decode only, so the furthest start advances one line at a time and checkpoints are appended once, in order. - NumPyDraws: no state; each draw or fill holds BitGenerator.@lock, the lock NumSharp's own primitives take (BoundedIntegers, Generator) and ask outside callers to hold. - The rest (exact widths, exp skip, categorical logs, transform values, the SQLite query text) has no shared state. No threading defect was found. New tests pin it (below), and every concurrency-focused suite was run three more times. What the review did find is a correctness regression that is not about threads. TpeObservationCache started over only when the storage object or the study id changed. SQLite reuses the highest deleted study and trial ids, and a gRPC proxy or a custom storage passes them through as one storage object (PerStudyState's documentation describes exactly this, pre-release review item H3), so a reused TPESampler on a study deleted and recreated under the same id read the deleted study's internal values for every trial number it had cached. Optuna keeps no such cache and TPE before plan 14 re-read every trial, so results diverged from Optuna there. Reproduced end to end before the fix: the new storage-suite test ran the same scenario twice over a gRPC proxy to SQLite, once with the first study deleted (its id reused) and once without (a new id), and the reused sampler suggested different parameters from the first TPE trial on. The fix applies PerStudyState's TrialAnchor rule per row: TpeObservationCache keeps each cached row's trial identity (storage id and start time, by trial number) and ColumnsFor now takes the trials the caller is about to read and checks every one with a cached row before anything is read; one mismatch (another storage id, or the same id started later, since SQLite reuses trial ids too) and the cache starts over (IdentityRestarts counts it). An unknown start time on either side is not a difference. GetInternalRepr passes its trials and records the identity wherever a finished row is stored (present or absent). The cache also holds its storage through a WeakReference now, as PerStudyState does, so a long-lived sampler no longer keeps the last storage it served alive. Measured (plan 13 proof program, the 82b8a0b build against this tree, alternating): TPE 1k warm 2.68-2.90 s -> 2.58-3.04 s, MOTPE 1k 10.80-11.01 s -> 11.01-11.20 s, compact TPE 2.61 -> 2.56 s, allocations equal: unchanged within noise. Hash probe, 36 seeded studies over every sampler family: identical to 82b8a0b (and so to b3baef9). New tests: - Storage suite: GrpcStorageProxyTests.Tpe_ReusedOnARecreatedStudyUnderTheSameId_SamplesAsUnderANewId (failed before the fix, passes after; independent TPE, because Optuna's multivariate TPE refuses, as OptunaSharp's does, a sampler reused for a study under another id, which the reference run needs). - TpeObservationCacheTests: LiveStudy_NeverStartsOverOnATrialsIdentity (both modes, plain and compact, constant liar with running trials: no false alarm, the cache stays in use); TrialNumbersCarryingOtherTrials_StartTheCacheOver (later start under the same ids, other ids, an unknown start); Cache_DoesNotKeepTheStorageAlive; WorkerInstances_OwnTheirCaches_AndThePrototypesStaysUnused (PerWorker, nJobs 8: worker caches distinct, the prototype's unused); SharedByEightWorkers_EveryCachedRowIsItsTrialsOwnValue (shared sampler, nJobs 8, constant liar, pruned and conditional parameters: every cached row equals its trial's conversion to the bit, no identity restart). - JournalOffsetCheckpointTests.ReadersSyncingWhileOneWriterAppends_AcrossCheckpoints_AllReadExactly: one backend, one writer appending past three checkpoint intervals in batches, six readers syncing like storages from their own positions; each ends with exactly its slice of a full read, and the backend holds one checkpoint per interval. - Mutation checks: with RowsMatch forced true, TrialNumbersCarryingOtherTrials_StartTheCacheOver fails; with the storage held strongly, Cache_DoesNotKeepTheStorageAlive fails. Docs: plan 14 section 8 (the review and this finding); THREADING.md's cache paragraph now says how the TPE cache is keyed (study and storage plus each row's trial) and that it holds the storage weakly. Gates on this tree (out-of-tree Release build): - Documentation gate: check_documentation.cs 0 issues (5,788 declarations); Core builds with the doc warnings as errors. - Unit suite, CI filter: net8.0 2172 passed, 1 skipped; net10.0 2172 passed, 1 skipped (the Windows self-skip of StopSignalsTests.ARealSigterm_CancelsTheToken_AndTheProcessLivesOn). - Concurrency-focused classes (OptunaSharp.Tests.Concurrency, OptimizeConcurrencyTests, SamplerNJobsSweepTests, JournalConcurrencyTests, JournalOffsetCheckpointTests, TpeObservationCacheTests, QmcSamplerTests), three more runs in a row on net10.0: 342/342 each time. - Live Optuna 5.0 (engine required): TPE, compact storage, importance and battle-tested parity, 83/83. - Storage suite: GrpcStorageProxyTests, SqliteRdbStorageTests, SqliteRdbOptionsTests, JournalBackendContractTests, 114/114.
…d-ask child redraw, held running records (QA-56..58) Second 8-hour parity QA run against Optuna 5.0.0 (journal docs/qa/PARITY_QA_2026-11-10.md, started 21:38 JST). This commit holds the journal so far and the first three [OpenBugs] reproductions; nothing in src/ changes. Method so far - A/B oracle for the performance commits of master..main (plans 13 and 14): one probe, a JSON random-program interpreter (every parameter kind, conditional parameters, dynamic ranges, enqueued trials, constraints through set_constraint, pruners with reports, planned fail/prune/NaN/inf outcomes, batches told FIFO or LIFO, an ask-the-whole-batch-first mode), built against main and against de495f8 (the commit before the first plan 13 change, already on NumSharp 0.80.0). 6,400 programs over Random, TPE, CMA-ES, NSGA-II/III and QMC on in-memory, compact, journal and SQLite storage: every output line identical. The "same trials" claims hold on these paths. - A Python twin of the interpreter compares main with Optuna 5.0.0 on the same programs. Divergences are shrunk (greedy program shrinker), then traced: MT19937 states compared at every ask, Optuna instrumented by monkeypatching, the baseline copy of the port instrumented where needed. Findings - QA-56 (LOW) NSGA-III niche preservation: Optuna sorts the borderline front's (distance, index) pairs with list.sort(reverse=True); pairs holding a NaN distance are incomparable, so timsort keeps them in input order and pop() takes the last one first. The port's comparer orders NaN as equal and then by index, which reverses them. NaN distances arise whenever the front plus the elites has no finite value in one objective (_filter_inf). Population 2, a first generation of (inf, inf) trials, seed 0: Optuna's third trial has y=0.7151893663724195, the port's 0.5448831829968969. Found in 4 of ~120 NSGA-III fuzz programs, single- and multi-objective. - QA-57 (LOW, owner question Q4) the plan 12 running-trial redraw (HandedOutParams.Claim, d0558f5) also fires in a single-threaded seeded batched ask/tell, Optuna's documented batch pattern, and compares partial children: a child whose mutated parameter was dropped is called a duplicate of another running trial's partial child although each trial samples that parameter on its own. NSGA-II and NSGA-III, population 6, seed 311031104, batches of 5: trial 18's v5 is 0.00021556396515864647 (trial 1's value, from a fourth generated child) where Optuna samples 0.02621137972316322. Every MT19937 state up to that trial's mutation decisions is identical. - QA-58 (LOW) a FrozenTrial read with get_trials(deepcopy=False) while its trial runs keeps receiving that trial's later user attributes, intermediate values and system attributes in Optuna (in-memory and journal storage): Trial mutates the dictionaries of the record it was created from in place, and the storages' shallow copy-on-write shares every dictionary a write has not replaced. The port's record never changes. Tests (all fail today for the reason stated, net10.0): - test/OptunaSharp.Tests/QA/Run2SamplerParityBugTests.cs: Nsga3_AllInfiniteBorderlineFront_KeepsPythonsNichingOrder, Nsga_BatchedAskTell_PartialChildIsNotRedrawn - test/OptunaSharp.Tests/QA/Run2StorageParityBugTests.cs: HeldRunningRecord_SeesTheTrialsLaterAttributeWrites_LikeOptuna
…, QA-60); Wilcoxon, GP, Grid, Hyperband sweeps clean Parity QA run 2 (journal docs/qa/PARITY_QA_2026-11-10.md), second batch. Nothing in src/ changes. Findings - QA-59 (LOW) SQLite RDB storage: the cached refresh sends every unfinished trial below the last finished one as `trial_id IN (...)`, one bound variable each. Past SQLite's variable limit (32,766 in Microsoft.Data.Sqlite's e_sqlite3) the read throws SqliteException 'too many SQL variables' and the study can no longer be read. Optuna's RDBStorage._get_trials catches the OperationalError, logs "Caught an error from sqlalchemy: ... Falling back to a slower alternative." and reads every trial; its Anaconda SQLite 3.51.0 even allows 250,000 variables. Reproduced with 33,000 RUNNING trials (crashed workers) below a COMPLETE one: Optuna reads 33,003 trials twice, the port's second read throws. Found by a new technique: every raise/warn/log message literal of Optuna 5.0 (367) searched by its longest fragments in the port's sources; this fallback's warning was one of the two real misses. - QA-60 (LOW) CmaEsSampler(with_margin=True) on a search space that loses a dimension before the first generation is told: Optuna's CMAwM.tell refuses the ragged x_for_tell list (ValueError: setting an array element with a sequence ... inhomogeneous shape) and the trial fails; the port's Cma.Tell reads the first n_dim entries of each vector and carries on with coordinates that stand for other parameters. Seed 0, x int [0, 10], y [0, 1] ([0, 1.5] on trial 2): Optuna raises at trial 3/4/5 for population 2/3/4, the port finishes 12 trials. Found by the new dynamic-range program fuzz (11 of 68 CMA-ES programs, all with margin). Clean this batch (recorded in the journal's progress log) - WilcoxonPruner after the QA-14 fix: 3,000 random cases (1-150 steps, ties, zero differences, out-of-order and missing steps, NaN/inf, both directions, p thresholds, startup steps), decisions and warnings identical. - GP checked by acquisition value: the port's points are as good under Optuna's own LogEI (the 1e-4 L-BFGS-B tolerance class). - Near-miss message templates (difflib over 349 messages): no wording drift that C# can reach. - NSGA-II/III under HyperbandPruner (200 programs), Grid with batches (119), BruteForce sequential: identical. Tests (fail today for the reason stated, net10.0) - test/OptunaSharp.Tests/QA/Run2SamplerParityBugTests.cs: CmaWithMargin_SearchSpaceShrinksInFirstGeneration_RaisesLikeOptuna - test/OptunaSharp.Tests.Storage/Run2SqliteParityBugTests.cs: CachedRefresh_TooManyRunningTrialsBelowTheWatermark_FallsBackLikeOptuna
…y steps (QA-61, QA-62); storage and fuzz-mode sweeps clean Parity QA run 2 (journal docs/qa/PARITY_QA_2026-11-10.md), third batch. Nothing in src/ changes. Findings - QA-61 (LOW) CLI `-f yaml`: PyYAML's safe_dump writes every float with a decimal point and quotes the formatted datetimes; the port's YamlDotNet output writes 2.0 as `2`, 1e-07 as `1E-07`, 1e16 as `10000000000000000` and the datetimes bare. Read back with yaml.safe_load (YAML 1.1) the port's floats become the int 2, the string '1E-07' and the int 10000000000000000, and its datetimes become datetime objects where Optuna's output gives strings. OutputFormat.cs calls YAML byte parity best-effort; these change types, not bytes. - QA-62 (LOW, same root cause as QA-3, owner question Q1) Python parses str(x) into an exact Decimal and computes with 28 significant digits; .NET's decimal keeps at most 28 digits after the point, so a value whose repr needs more places (a 17-digit double below ~1e-11) is rounded on the way in. After the QA-2/QA-11 fix, 1,439 of 100,000 random stepped ranges still adjust `high` one ULP off and 184 of 5,000 BruteForce grids differ, every one among the inputs needing more than 28 places. FloatDistribution(4.1271208719049127e-14, 4.085121318968367e-13, step=1e-15).high: Optuna 4.0827120871904913e-13, the port one ULP below; _enumerate_candidates(2.942974008882182e-14, 5.614297400888218e-13, 2e-15): 266 values in Optuna, 267 in the port. Clean this batch - E13 (the previous run's unfinished SQLite half): 270 random programs written to SQLite and journal storages by both languages (Random, CMA-ES, QMC, NSGA-II/III, categorical TPE with interleaved asks): identical. - New fuzz modes: add_trial warm starts mid-run, Percentile/Threshold/Patient/Wilcoxon pruners, three objectives: 600 programs identical. - The journal's directions table now records the status of E1-E18 (the plan 13/14 commit review is complete: no regression in 6,400+ A/B programs). Tests (fail today for the reason stated, net10.0) - test/OptunaSharp.Tests.Cli/Run2CliParityBugTests.cs: TrialsYaml_FloatsAndDatetimes_KeepTheirTypesLikePyYaml - test/OptunaSharp.Tests/QA/Run2DistributionParityBugTests.cs: SteppedHigh_ValuesBeyond28DecimalPlaces_FollowPythonDecimal
…, MEDIUM); parallel-coordinate fix re-check clean Parity QA run 2 (journal docs/qa/PARITY_QA_2026-11-10.md), fourth batch. Nothing in src/ changes. Finding - QA-63 (MEDIUM) CLI `tell`: `optunasharp tell --study-name s --trial-number 0 --state fail` (or `--state pruned`) without `--values` fails. The `--values` option (Option<double[]?>, one-or-more) yields an empty array, not null, when it is omitted, so Core's tell sees values next to a PRUNED/FAIL state and raises "ValueError: Values were told. Values cannot be specified when state is TrialState.PRUNED or TrialState.FAIL."; the command prints an unhandled-exception stack, exits 1 and leaves the trial RUNNING. Optuna (argparse nargs="+", default None) logs "Told trial 0 with values None and state TrialState.FAIL." and exits 0. A bare `tell --trial-number n` also differs (warning "The number of the values 0 did not match..." and "with values []" instead of Optuna's "The value None could not be cast to float." and "with values None"). `--directions` already maps an empty array to null. CliParityTests' fixture runs these exact commands but ignores their exit codes, so it kept two RUNNING trials. Found by the new CLI differential (random ask/tell sequences with seeded samplers through both CLIs). Clean - The QA-26 fix: 600 random studies (None / bool / int / float / str choices including the string "None", missing parameters, pruned/failed trials, single and multi-objective) through _get_parallel_coordinate_info in both languages: identical. Test (fails today for the reason stated, net10.0) - test/OptunaSharp.Tests.Cli/Run2CliParityBugTests.cs: Tell_StateWithoutValues_FinishesTheTrialLikeOptuna
… scenario sweep
Parity QA run 2 (journal docs/qa/PARITY_QA_2026-11-10.md), fifth batch. Nothing in src/ changes.
Finding
- QA-64 (LOW) CLI `ask --sampler-kwargs`: Optuna builds the sampler with getattr(optuna.samplers, name)(**kwargs),
so JSON objects and arrays reach it; the port's SamplerFactory.TryCoerce binds only scalars. GridSampler's
search_space, CmaEsSampler's x0 and NSGAIIISampler's reference_points therefore fail with "Could not construct
sampler 'GridSampler' from the given --sampler-kwargs ({search_space})." where Optuna asks normally.
Also recorded
- 14 fixed CLI scenarios (60 commands) through both CLIs: exit codes and stdout match except QA-63/QA-64 and the
known QA-37; `tell --state complete` without values is QA-63's root cause too (added to its row).
Test (fails today for the reason stated, net10.0)
- test/OptunaSharp.Tests.Cli/Run2CliParityBugTests.cs: Ask_StructuredSamplerKwargs_ConstructTheSamplerLikeOptuna
…Optuna's error types and messages: KeyError, StorageInternalError, the full version, Python repr in LightGBM, FileExistsError on a directory, study.direction in MO TPE, n_jobs=0
Group 4 of the owner's "Fix now" list from the 2026-11-10 parity QA run (docs/qa/PARITY_QA_2026-11-10.md):
errors the port raised with another type or another message than Optuna 5.0.0 / optuna-integration 5.0.0.
Each finding's [OpenBugs] repro now passes and carries a "Fixed 2026-11-12" note.
QA-66 - storage-level errors (in-memory, journal, base implementation)
* BaseStorage.GetTrialParam and InMemoryStorage.GetTrialParam raise KeyError(paramName) for a parameter the
trial does not have (Python's `trial.distributions[param_name]`), not .NET's KeyNotFoundException; the base
implementation serves the journal, cached and gRPC-proxy storages, and the compact-record path raises the
same KeyError.
* The in-memory and journal storages raise DuplicatedStudyError("") - Optuna's bare `raise
DuplicatedStudyError` has an empty str(e) - instead of .NET's default text.
* GridSampler.AfterTrial: an enqueued trial without a grid_id raises KeyError("grid_id"), as Optuna's
`get_trial_system_attrs(...)["grid_id"]` does.
* Kept on purpose: the gRPC proxy client still forwards the server's detail where Optuna raises bare
KeyError() / ValueError() / ... `from` the RpcError. A .NET KeyError cannot carry the cause, and an empty
message alone would drop the detail Python keeps in __cause__. Recorded in the journal row.
QA-74 - database errors become StorageInternalError (Optuna's _create_scoped_session)
* EF Core: Mutate now catches DbUpdateException / DbException that is not an integrity error and raises
StorageInternalError("An exception is raised during the commit. This typically happens due to invalid data
in the commit, e.g. exceeding max length. ") wrapping the provider's error. Integrity errors propagate
unchanged (Optuna re-raises IntegrityError, which callers turn into DuplicatedStudyError and friends):
PostgreSQL SQLSTATE class 23 (psycopg2's mapping), MySQL exactly the seven numbers PyMySQL maps to
IntegrityError (1048, 1062, 1215, 1216, 1217, 1451, 1452; every other number, a check-constraint failure
included, is OperationalError there, which Optuna wraps). The translation sits outside the execution
strategy, so EnableRetryOnFailure still sees and retries transient errors.
* SQLite: under skip_table_creation the version-info step only reads version_info (Python creates no table
in that mode), and a database error there becomes StorageInternalError in the constructor - Optuna's
measured behavior on a database without its tables, with or without skip_compatibility_check. A constraint
error (another process inserted the row) is ignored, as ignore_integrity_error=True does. Every upgrade-test
database has version_info, so `storage upgrade` is unaffected (Storage and CLI suites green).
* StorageInternalError gained a (message, innerException) constructor - the `raise ... from e` analogue.
QA-75 - OptunaSharpVersion.OptunaCompatibilityTarget is "5.0.0", Optuna's __version__, so the RDB
compatibility error names the full version like Optuna's.
QA-76 - optuna-integration LightGBM messages in Python's forms, independent of the current culture
* LightGBMPruningCallback's not-found error prints the evaluation list with PythonRepr.Repr per element:
100.0, 1e-05 and 0.5 under de-DE (was 100, 1E-05, 0,5), quotes switched for a name holding a quote.
* The tuner's duplicated-aliases warning prints the dict in repr and the chosen value in str
({'sub_row': 0.5, 'subsample': 1.0}, bagging_fraction=1.0); the metric type error names the Python type
(<class 'int'>, <class 'float'>, <class 'NoneType'>, ...).
* Kept: a cross-validation entry prints four elements; Python's fifth is the standard deviation, which the
public LightGbmEvaluationResult record does not hold (adding it would change the record's API).
QA-78 - DownloadArtifact's existence check covers directories too (os.path.exists), so a directory at the
destination raises the IOException (Python FileExistsError) before anything is downloaded.
QA-79 - TpeTrialSplit.GetPrunedTrialScore reads study.Direction, which raises Optuna's RuntimeError ("A single
direction cannot be retrieved from a multi-objective study. ...") on a multi-objective study whose PRUNED
trials carry numeric reports, as Optuna's _get_pruned_trial_score does; a NaN report still scores
(-step, inf) before the direction is read, as in Python.
* TpeSplitCacheTests asserted the old behavior (its two-objective history had pruned trials with numeric
reports). The history builder now takes the number of objectives: one-objective histories keep numeric
reports, two-objective ones report NaN (the only report a multi-objective split accepts), and
TheCachedSplit_IsSplitTrials_AndTheSameRecordsReuseIt runs on both (6 rows). A new test,
AMultiObjectivePrunedReport_RaisesLikeOptuna_AndIsNotCached, checks the RuntimeError propagates through
the cache and that a throwing split is not stored.
QA-81 - OptunaSearchCV maps nJobs: 0 to 1 (optuna-integration's `n_jobs if n_jobs else 1`).
Also updated: CompactFinishedTrialsTests.EveryKindOfValue_ReadsBackIdentical expected KeyNotFoundException for
a missing parameter on both the plain and the compact record; it now expects KeyError with the name as message.
Gates (net10.0, out-of-tree artifacts, CI filter
TestCategory!=OpenBugs&TestCategory!=HighMemory&TestCategory!=RequiresPython&TestCategory!=Slow):
* OptunaSharp.Tests: 2,296 passed, 0 failed (first run: 6 failures, all the two tests above; fixed).
* OptunaSharp.Tests.Storage: 463 passed, 0 failed; the QA-74 EF Core rows ran for real on MySQL 8.0 and
PostgreSQL 16 through Testcontainers (OPTUNASHARP_EFCORE_REQUIRED=1); the 144 skips are the Redis suites
(no Redis server; group 4 does not touch them).
* OptunaSharp.Tests.Integration.LightGbm: 57 passed. OptunaSharp.Tests.Cli: 90 passed.
* check_documentation.cs over src: clean; Core, Storage.Sqlite, Storage.EFCore and Integration.LightGbm build
with the doc warnings as errors.
Journal: QA-66, 74, 75, 76, 78, 79 and 81 marked fixed 2026-11-12, with the two kept sub-items spelled out.
…ders (list.sort with NaN, set iteration), insertion-ordered study maps, SQLite's too-many-variables fallback, attributes in index key order; QA-77 kept on the logger, QA-58 deferred
Group 5 ("Orders + core edge cases") of the owner's "Fix now" list from the 2026-11-10 parity QA run
(docs/qa/PARITY_QA_2026-11-10.md). The owner answered the two open design questions on 2026-11-12: QA-58 "Defer",
QA-77 "Keep the logger".
QA-56 - NSGA-III niche preservation sorts NaN-bearing tuples as CPython does
* New src/OptunaSharp.Core/Internal/PyListSort.cs: CPython 3.12's list.sort ported step for step - reverse as
CPython does it (reverse, stable sort, reverse), count_run (3.12: ascending or strictly descending), binary
insertion up to minrun, the powersort merge policy (powerloop / found_new_run), merge_at with its gallop trims,
merge_lo / merge_hi with the adaptive min_gallop, merge_force_collapse. With a total order it is a plain stable
sort; with an inconsistent one (NaN) it reproduces CPython's exact comparison sequence, which is the only way to
land where CPython lands.
* NSGAIIIElitePopulationSelectionStrategy.PreserveNicheIndividuals sorts the borderline front with it and the new
DistanceIndexLessThan (Python's tuple <: equal distances by index, a NaN distance never less, never greater).
The old comparer treated NaN as equal and reversed an all-NaN front, so pop() took the other pair.
* CPython 3.13 rewrote count_run, so a 3.13 interpreter may order NaN-bearing lists differently; the port and the
corpus pin 3.12, the interop suite's interpreter (local 3.12.12, CI 3.12).
QA-68 - GridSampler lists unvisited grids in CPython's set-iteration order
* New src/OptunaSharp.Core/Internal/PyIntSet.cs: CPython 3.12's set table for non-negative ints (hash(k) == k):
8-slot minimum, home slot + 9 linear probes then i = (i*5 + 1 + perturb) & mask with perturb >>= 5, growth
when fill*5 >= mask*3 (to > used*4, used*2 past 50,000), dummies left by discards, the last passed dummy reused
by an insert (measured on 3.12: set(range(20)) minus {3, 5} plus 131 lands in slot 5), the dummy purge after a
difference update, set_merge's slot-for-slot copy, and set_difference's two strategies (copy-then-discard when
len(self) >> 2 > len(other), otherwise a fresh set filled in slot order).
* GridSampler.GetUnvisitedGridIds builds set(range(n)) - set(visited) - set(running) (and the unfinished
fallback) with it, so rng.choice indexes the same list as Optuna: list(set(range(16)) - {0, 1, 2, 3, 5, 6, 7, 9,
10, 12, 13, 14, 15}) is [8, 11, 4]. The old [Misaligned] note assumed only concurrency reaches that branch; a
single-threaded optimize reaches it whenever the numbering is shifted (add_trial, enqueue_trial, resumed study).
Oracle for both: test/OptunaSharp.Tests/oracle/cpython_order.json (546 KB), recorded from CPython 3.12.12 by
oracle/gen_cpython_order_oracle.py (refuses another minor version): 271 set-difference histories (both strategies,
the purge, sizes to 70,000 incl. both sides of 50,000; bitmaps in, full lists or count/head/tail/SHA-256 out) and
353 (distance, index) sorts up to 1,000 elements (NaN rates 0 to 100 %, ties, infinities, signed zeros, runs and
plateaus; distances as two-character pool indices). Internal/CPythonOrderOracleTests replays it with no Python:
every case matches; plus a total-order stability check against LINQ and argument checks.
QA-67 - InMemoryStorage._studies and JournalStorageReplayResult._studies are insertion-ordered OrderedDict<int, ...>
(Python dicts), so studies list in creation order across deletions; Dictionary reused a deleted study's slot
(create a, b, c; delete b; create d; delete a; create e, f: Optuna [c, d, e, f], the port [e, d, c, f]). The
journal snapshot codecs write and read studies in that order. Both maps are only touched under their locks.
QA-59 - SQLite cached refresh falls back like Optuna's _get_trials
* The multi-trial read catches the SqliteExceptions Python's sqlite3 raises as OperationalError (primary codes
ERROR - which "too many SQL variables" is - PERM, ABORT, BUSY, LOCKED, READONLY, INTERRUPT, IOERR, FULL,
CANTOPEN, PROTOCOL, EMPTY, SCHEMA), logs Optuna's "Caught an error from sqlalchemy: {e}. Falling back to a
slower alternative." and reads every trial of the base query (study and states), keeping the included ids and
those above the watermark, in trial-id order. 33,000 RUNNING trials below a COMPLETE one now read back.
* EF Core needs no fallback, measured with ToQueryString() on EF Core 8 and 9 (throwaway probe, not committed):
Pomelo inlines the ids (`trial_id IN (3, 5, 8, 13, 21)`), Npgsql passes one array (`= ANY (@__included_2)`).
QA-71 - attributes come back in the order Optuna's index scans return them
* SQLite: a study's attributes ORDER BY "key" and a trial's ORDER BY trial_id, "key" (the unique (..., key)
autoindex's BINARY order Optuna's unordered where_study_id / selectinload queries come back in); get_all_studies
keeps insertion order, as Optuna's table scan does.
* EF Core: the same on MySQL (ORDER BY key: the column collation's order, InnoDB's unique-index order; new
AttributesInKeyOrder, passed into the static GetTrialsInternal); PostgreSQL keeps insertion order, where
Optuna's own order is unstable.
* docs: the gRPC cross-language page no longer says every OptunaSharp storage keeps insertion order.
QA-77 - kept, [Misaligned] ("Keep the logger"): warnings stay on the optuna logger. Run2WarningParityBugTests'
two repros became [Misaligned] pins (Warnings_FollowTheLoggersVerbosity_SoErrorHidesThem,
RepeatedWarning_IsLoggedOnEveryOccurrence: 5 records for 5 trials) and OptunaWarnings documents the difference.
QA-58 - deferred by the owner: snapshots stay (copying Optuna's dictionary aliasing would undo the copy-before-write
that keeps n_jobs readers safe); the repro stays [OpenBugs] with a "Deferred" note.
Repros un-OpenBugged with "Fixed 2026-11-12" notes: Nsga3_AllInfiniteBorderlineFront_KeepsPythonsNichingOrder,
GridSampler_ShiftedNumbering_PicksUnvisitedGridsInCPythonSetOrder, StudyListing_AfterDeletions_
KeepsCreationOrderLikePythonDicts, CachedRefresh_TooManyRunningTrialsBelowTheWatermark_FallsBackLikeOptuna,
AttributeReads_ComeBackInKeyOrderLikeOptunasIndexScan, MySqlAttributeReads_ComeBackInKeyOrderLikeOptuna.
Gates (out-of-tree artifacts; CI filter
TestCategory!=OpenBugs&TestCategory!=HighMemory&TestCategory!=RequiresPython&TestCategory!=Slow):
* OptunaSharp.Tests: 2,306 passed on net10.0 and on net8.0 (the vendored OrderedDictionary path), 0 failed.
* OptunaSharp.Tests.Storage (net10.0, OPTUNASHARP_EFCORE_REQUIRED=1): 466 passed, 0 failed (the 25 upgrade
tests first failed only for the missing repository-root shim in the fresh artifacts folder and passed, 27/27,
once it was copied); the EF Core rows ran on MySQL 8.0 / PostgreSQL 16 through Testcontainers; 144 Redis skips.
* OptunaSharp.Tests.Interop GridSamplerParityTests + NsgaiiiSamplerParityTests against live Optuna: 21 passed.
* check_documentation.cs over src: 5,958 declarations, 0 issues; Core, Storage.Sqlite, Storage.EFCore build with
the doc warnings as errors.
Journal: QA-56, 59, 67, 68, 71 fixed, QA-77 kept [Misaligned], QA-58 deferred.
… shadow study in lockstep with the C# study; 12 registry samplers and 6 pruners now match native Optuna trial for trial
Group 6 ("OptunaHub bridges") of the owner's "Fix now" list from the 2026-11-10 parity QA run
(docs/qa/PARITY_QA_2026-11-10.md). The sampler bridge rebuilt a Python study per trial from
create_trial(state, values, params, distributions) and never forwarded before_trial / after_trial; the pruner bridge
rebuilt one per prune call. So every system attribute a hub object wrote was lost: PSO never left its random phase
(it builds its swarm in after_trial), MO-CMA failed with KeyError: 'mocma:g', the NSGA-II variants lost their
generations, AutoSampler differed from trial 1, successive halving and Hyperband never pruned (36 and 31 of 40
natively, 0 through the bridge).
New src/OptunaSharp.Interop.pythonnet/OptunaHubShadow.cs - one persistent in-memory Optuna study per OptunaSharp
study (keyed by storage instance + study id), shared design for both bridges:
* Sync (before every call into Python): every OptunaSharp trial, in number order and in any state, is mirrored with
its values, params, distributions, user and system attributes and intermediate values; an unfinished mirror
receives the params, reports and attributes written since, then its new state (WAITING -> RUNNING, or finished, so
a trial finished without after_trial - failed as stale - finishes too); the study's user and system attributes
follow. Finished mirrors are skipped. Reads go to the storage afresh, and the values last synchronized are kept
per key, so what the bridge wrote back is not echoed.
* The trial objects Python receives are native Optuna's: each running trial keeps its storage record from the
trial's start (Trial._cached_frozen_trial), updated in place by mirrored params, reports and user attributes as
Trial._suggest / report / set_user_attr do; LatestTrial is Trial._get_latest_trial() (a shallow copy with lazily
read system attributes) for sample_independent and prune; StoredTrial is storage.get_trial for after_trial.
* BeginTrial clears the shadow study's per-thread trial cache (study._thread_local.cached_all_trials) as Study.ask
does "once every trial". Without it the shadow kept its first read for good: restart CMA-ES diverged from trial
1, TPE from its first model-based trial, the NSGA-II variants from their first generation boundary (found by the
new native comparison; only samplers read that cache).
* The shadow carries the OptunaSharp study's name: HyperbandPruner assigns brackets by
crc32(f"{study.study_name}_{trial.number}").
* Call<T> snapshots the study's and the trial's user and system attributes around a Python call and writes every
key it set to the OptunaSharp storage (MO-CMA's mocma:g, PSO's swarm, successive halving's completed_rung_N), so
the C# study holds what a native run stores. Values with no JSON shape (NumPy arrays, arbitrary objects) stay in
the shadow only. Deep JSON-shaped conversion both ways (dict / list / tuple / scalars, NumPy scalars) and a
structural equality (numbers by value, NaN matching itself) live here; PyConvert's public surface is unchanged.
OptunaHubSampler: BeforeTrial (BeginTrial, cached record refreshed, before_trial), InferRelativeSearchSpace /
SampleRelative (cached record), SampleIndependent (latest trial), AfterTrial (stored record; the shadow trial is
finished in a finally, as _tell_with_warning writes the state even when after_trial raises), all through Call.
OptunaHubPruner: Prune syncs its own shadow and calls prune(study, latest trial) as Trial.should_prune does.
Both dispose every shadow in Dispose. Calls stay serialized on the Python thread under the bridge's gate.
Pythonic.cs: Python.optuna.samplers.BaseSampler.before_trial / after_trial; Python.optuna.create_study takes the
study name; new Python.optunasharp_shadow - a small helper module defined once in the embedded interpreter
(add_trial via create_trial + create_new_trial(template_trial=), begin_trial, get_trial, latest, set_param, report,
set_user_attr, set_system_attr, set_study_user_attr, set_study_system_attr, set_state, attrs, changed), Python source
so each helper reads as the Optuna code it mirrors.
Tests:
* Run2OptunaHubBridgeParityBugTests: the three repros pass (PSO trial 11 is the native swarm candidate bit for bit;
MO-CMA completes 30 trials; successive halving prunes 36 of 40, trial 3 after step 1); OpenBugs categories and
the CI=true -> Inconclusive guard removed, "Fixed 2026-11-12" notes added.
* New OptunaHubNativeParityTests: each registry object run natively and through the bridge in the same
interpreter (study name hub-parity, seeded), every trial's state, params (float.hex) and reported steps compared.
Samplers: pso, mocma, nsgaii_with_initial_trials, speaii, implicit_natural_gradient, differential_evolution,
restart_cmaes, levy_flight_sampler, nelder_mead, tpe_union_multivariate, random_search, and AutoSampler (GP-based,
~20 s a side, [Slow]). Pruners: successive_halving, hyperband, median, percentile, wilcoxon, threshold. All 18
match. A package whose imports or lazily imported optional modules are missing reports Inconclusive.
* BridgeSamplerParallelReadTests.OptunaHub_BothVolleys: the hub DE's pin moves from 49 collisions / 0 history
repeats to 0 / 4, in both volleys. The old numbers were artifacts of the rebuilt shadow (every read of a round
got the same shadow trial number, and the history cache never refreshed); a throwaway probe ran the same lockstep
pattern natively in Python: all 64 points identical, bit for bit. The threaded volley gives 0 / 4 as well (three
runs; the bridge serializes every call on the Python thread). The test gained a historyRepeats DataRow column.
Docs: docs/website-src/integrations/python/optunahub.md ("Compatibility and state" rewritten for the persistent
shadow, with the measured list and the remaining limits: Python-side state not persisted, non-JSON values stay in
the shadow, calls serialized with one shared history cache, a Hyperband-filtered C# study mirrored whole; the pruner
paragraph; the validation section points at the native-parity tests), integrations/python/index.md, the package
README.
Journal: QA-72 and QA-73 fixed 2026-11-12; a "Fixes (2026-11-12)" section maps the six groups to their commits and
tallies the run: 23 of 28 fixed, 3 kept [Misaligned] (QA-62, 69, 77), 2 deferred (QA-58, QA-82).
Gates (net10.0, live Python 3.12 + Optuna 5.0.0 + OptunaHub + pinned refs/optunahub-registry):
* OptunaSharp.Tests.Interop, everything but Slow: 727 passed, 0 failed, 30 skipped (pycma and Redis not
installed here). The 13 SqliteRdbUpgradeInteropParityTests failed first only for the repository-root shim
missing in the fresh artifacts folder, and passed (13/13) once it was copied.
* The OptunaHub classes on net8.0 as well: 29 passed. AutoSampler ([Slow]) passed on net10.0 (23 s).
* OptunaSharp.Tests.Hosting: green (it builds every package's DI host, the bridge included).
* check_documentation.cs over src: clean; OptunaSharp.Interop.pythonnet builds with the doc warnings as errors.
…blies built against 1.0.0 still bind ApiCompat (Microsoft.DotNet.ApiCompat.Tool 11.0.100, `apicompat package` of each 1.1.0 candidate package against the nuget.org 1.0.0 package, 2026-11-12) found one binary break across the 20 packages: CmaEsSampler's 14-parameter public constructor of 1.0.0 was gone. 99e9e73 (plan 13, item D1) appended the optional `reuseToldOptimizer` as a 15th parameter. Every source call site compiles unchanged, but the constructor got a new metadata signature, so a call compiled against 1.0.0 (a library built on OptunaSharp 1.0.0, used by an application on 1.1.0) throws MissingMethodException. The other 19 packages showed no breaking change; a positive control (Storage.Sqlite checked against the Storage.Redis 1.0.0 baseline) confirmed the tool reports breaks. The net8.0-vs-net10.0 OrderedDict findings inside Core are identical in 1.0.0 (the in-box OrderedDictionary base on net9+), so they are not new. Owner's decision, 2026-11-12: "Restore 1.0.0 overload". Change (src/OptunaSharp.Core/Samplers/CmaEsSampler.cs): - a public constructor with 1.0.0's exact 14 parameter types that chains to the public one with reuseToldOptimizer: false (Optuna's path), so a 1.0.0 caller gets exactly the sampler it got before; - no default values on it: with defaults, every call that leaves out an argument (new CmaEsSampler(), new CmaEsSampler(seed: 42)) would match both constructors equally well and fail to compile as ambiguous (CS0121); without them those calls bind to the public constructor, and only a call passing all fourteen arguments binds here, with the same result; - [EditorBrowsable(EditorBrowsableState.Never)], so IntelliSense offers only the public constructor. The CLI's SamplerFactory tries public constructors widest first and binds the 15-parameter one for any --sampler-kwargs subset, so it never reaches this one. - Documented in full: every parameter, the ValueError it can raise (the public constructor's checks), and remarks on why it exists and why it has no defaults. Test (test/OptunaSharp.Tests/Samplers/CmaEsSamplerCacheTests.cs): TheVersion1_0_0Constructor_StillBinds_AndIsOptunasPath looks the constructor up by 1.0.0's parameter types (null before this commit, as ApiCompat reported) and invokes it the way a 1.0.0-compiled call does, every argument explicit with 1.0.0's defaults; its 120 trials equal those of new CmaEsSampler(seed: 21, popsize: 6, reuseToldOptimizer: false). Gates: Core builds with the documentation warnings as errors (0 errors, only the known CS8632 nullable-context warnings); the 53 CmaEs* tests pass on net10.0 and net8.0. ApiCompat and the full unit filter run again on the final 1.1.0 candidate.
…to SciSharp main, so 155 citations in seven docs named commits that no longer exist there Owner's decision, 2026-11-12: rebase the local journey1 line onto origin/main (SciSharp main = f36573c, the owner's README edit made on GitHub on 2026-11-09) and open a PR from it, instead of merging. The rebase replayed the 125 commits since v1.0.0 (journey1 e9c3c81) with no conflict, and the rebased tip's tree is byte-identical to the merge of the two (c3f1376). Every commit got a new hash, so documents written before the rebase cited hashes the public history never has. This commit rewrites those citations; nothing else changes. How each citation was mapped (scratchpad script find_old_refs.py): - the 125 old/new pairs come from rev-list v1.0.0..e9c3c81 and f36573c..(rebased tip), in order, and were checked pair by pair: same author, author date and full message, and the same added/removed lines (0 mismatches); - a citation is a 7-40 character hex token, bounded by non-word characters, that is a prefix of exactly one old hash AND that git resolves to that commit alone (git rev-parse --disambiguate, commits only), so a token that only looks like a journey1 hash, or is ambiguous with another commit, is left alone; - the replacement keeps the token's length (a 7-character citation becomes the new commit's first 7 characters), and every new token resolves to exactly one commit. A re-scan finds no old citation left. Rewritten: docs/PORTING_PLAN.md (6), docs/plans/13-rustuna-derived-performance.md (24), 14-rustuna-second-pass.md (27), 15-performance-pass-3.md (43), plans/README.md (4), docs/qa/PARITY_QA_2026-11-02.md (11), PARITY_QA_2026-11-10.md (40): 155 citations of 49 commits, on 125 lines. Line endings are untouched (the files were rewritten as bytes). Not rewritten: the messages of the rebased commits themselves. 45 of them cite an earlier journey1 hash (126 citations); changing them would mean rewriting those commits again. A cited old hash maps to the commit with the same subject and author date among the 125 commits after f36573c.
…ASE_<version>.md, NumSharp's notes file, notes first and the install table last
Owner, 2026-11-12, asked where the 1.1.0 release notes should live: "See other release notes and produce same format".
The other release notes are NumSharp's (OptunaSharp's infrastructure template) and OptunaSharp's own 1.0.0:
- NumSharp commits its notes as docs/releases/RELEASE_<version>.md, written to its changelog skill's house style
(refs/NumSharp/.claude/skills/changelog/references/CHANGELOG_STYLE.md). Its create-release job appends that file to
the GitHub Release body after the install lines and the package table, and turns GitHub's generated notes off when
the file exists (refs/NumSharp/.github/workflows/build-and-release.yml, "Compose release notes").
- The published releases put the notes FIRST and the install table last, under a titled name: NumSharp 0.80.0
("NumSharp 0.80.0 - Strings Stable Release": intro, Overview, Detailed Breakdown, then Install via NuGet and
Packages) and OptunaSharp 1.0.0 ("OptunaSharp 1.0.0 - Stable": Features, then the install lines and the package
table). Both were rearranged by hand after their release runs.
OptunaSharp's create-release composed only the install lines, the package table and GitHub's generated notes, which
here are a bare compare link, because nothing merges through pull requests.
Change (.github/workflows/build-and-release.yml, create-release):
- A sparse checkout of docs/releases. The job had no checkout; the notes are read from the tagged commit.
- Compose release notes (now id: notes): when docs/releases/RELEASE_<version>.md exists, its H1
("# OptunaSharp <version> - <title>") becomes the release's name and the rest of the file opens the body, followed
by "---", the install lines and the package table, as before. The H1 must name the tagged version (exactly, or
followed by a space and a title); otherwise the job fails before anything is created, which catches notes copied
from the previous release and not renamed. Without the file the body is today's (an "## OptunaSharp v<version>"
header and the install table) and the name "OptunaSharp v<version>".
The name leaves the step through a delimited GITHUB_OUTPUT block, so a title with characters a key=value line would
misread passes through whole; HAS_NOTES is a plain key.
- Create Release takes name: from the step, and generate_release_notes only when there is no notes file.
Checked:
- The step's script, extracted from the YAML and run three ways: a stand-in notes file for 1.1.0 (the name taken
from the H1, the notes first, the install table after "---"), a version without a file (today's body and the
"OptunaSharp v<version>" name), and a file whose H1 names another version (exit 1 with the error, nothing written
to GITHUB_OUTPUT).
- actionlint 1.7.12 on the workflow (-shellcheck= -pyflakes=, neither tool is installed here): no findings.
…8 commit bodies since 1.0.0 read and bucketed, performance measured against the published 1.0.0 docs/releases/RELEASE_1.1.0.md, the file the create-release job now turns into the GitHub Release (previous commit). Owner, 2026-11-12: "See other release notes and produce same format". Format and process are NumSharp's changelog skill (refs/NumSharp/.claude/skills/changelog: CHANGELOG_STYLE.md for the shape, triage-taxonomy.md and verify-accuracy.md for the process), laid out like NumSharp 0.80.0's published release: an H1 that names the release, an intro, "Overview", then "Detailed Breakdown" in a collapsed <details> with Dashboards & Docs, New APIs & Modules, Performance, Parity & Fixes, Testing & Tooling and Breaking Changes, last. NumSharp-only sections (New NuGet Packages, ndarray surface) are omitted: 1.1.0 adds no package. Hashes are 7 characters, OptunaSharp's convention. .gitignore: the Visual Studio build-output rule [Rr]eleases/ hid docs/releases/, so the file could not be added; a !docs/releases/ exception follows the rule, as in NumSharp's .gitignore. Checked with git check-ignore: the notes file is tracked, while bin/ outputs and any other releases/ folder stay ignored. Title proposed: "OptunaSharp 1.1.0 - Performance & Parity Release". The two-sentence intro is a draft for the owner to rewrite in their own voice (NumSharp's notes open with the owner's paragraph). Process: every commit of v1.0.0..(this branch) was read, subject and body, oldest first, in batches of 16 (scratchpad batches/b01..b08), and put in one of the five buckets: - Cited, 45 commits: every perf commit (27), every fix but two, both feat(storage) commits, the expm1 refactor, the JIT-posture chore, and the QA run's closing journal commit (Testing & Tooling). - Folded, 2: dfc026f (the TPE observation cache's reused-id regression) and d7aa9d9 (step F's split regression) fix changes made inside this range, so their substance is in the lines of the features they repaired (style 10.4). - Excluded, 81: 46 docs commits (plans 13-15, the QA journal's progress entries, cref fixes, the README edit), 29 test(qa) commits (the [OpenBugs] repros; their fixes are cited instead), and the 6 example commits, which surface without hashes in Dashboards & Docs (style 7B). Two claims found in bodies were not carried over: - QA-41 / QA-42 "fixed": c5d79f9 credits plan 14 step A, 8a7e19e credits NumSharp 0.80.0's legacy RandomState rewrite, and 0.80.0 already shipped with OptunaSharp 1.0.0. Which release fixed them is not established, so no line. - 2189a12's "continuous sums kept in registers": 465f91c (QA-84) replaced that code with NumSharp's np.sum, so only the step's stepped-dimension work is in the TPE line. Numbers: the Overview and the first nine Performance lines were measured for these notes, against the PUBLISHED 1.0.0, because the passes measured against three different baselines and QA-84 then took part of TPE's gain back. A file-based consumer outside the repository (its own nuget.config: OptunaSharp* from a local folder holding the nuget.org OptunaSharp.1.0.0.nupkg and the candidate packed as 1.1.0-local.1 from e9c3c81, the pre-rebase tip, whose library code is this branch's except the 1.0.0 CmaEsSampler constructor overload; NUGET_PACKAGES isolated), Release, .NET 10, EcoQoS opted out, one process per (version, workload), rep 0 (JIT included) and rep 1, two alternating rounds on an i9-13900K shared with other sessions; each cell is the better round. Seed 0, shifted sphere over 40 floats unless stated; two-objective spheres for NSGA-II and MOTPE. workload 1.0.0 rep0 / rep1 1.1.0 rep0 / rep1 ratio allocated same trials random, 10,000 6.91 / 5.50 s 1.11 / 0.77 s x6.2->x7.1 3,608 -> 666 MB yes QMC (default), 10,000 5.16 / 3.72 s 1.29 / 0.75 s x4.0->x5.0 3,740 -> 485 MB yes QMC scrambled Sobol, 4.12 / 3.39 s 0.36 / 0.17 s x11.6->x20.4 1,255 -> 60 MB yes 4,000 over 10 floats NSGA-II, 10,000 4.72 / 3.29 s 1.96 / 0.98 s x2.4->x3.4 2,912 -> 723 MB yes TPE, 1,000 10.75 / 9.72 s 4.24 / 3.25 s x2.5->x3.0 4,441 -> 840 MB no (moves toward Optuna) TPE, 1,000 over 10 3.69 / 2.62 s 2.35 / 1.11 s x1.6->x2.4 1,296 -> 259 MB no MOTPE, 300 4.46 / 3.25 s 2.59 / 1.29 s x1.7->x2.5 2,386 -> 340 MB no CMA-ES, 10,000 27.60 / 25.45 s 4.46 / 3.66 s x6.2->x7.0 10,896 -> 1,691 MB yes GP, 150 over 10 17.86 / 16.40 s 11.49 / 9.12 s x1.6->x1.8 5,893 -> 1,513 MB yes (round 1 ran 1.0.0 first, round 2 the candidate first; round 1's TPE ran under heavy load on both sides, e.g. 1.0.0 TPE 25.9 s against round 2's 10.75 s. "Same trials" = an FNV hash of every trial's number, parameters and values by their bits, equal across versions and rounds.) The equal hashes are the evidence behind the Breaking Changes line "Random, QMC, NSGA-II, plain CMA-ES and single-objective GPSampler over floats give 1.0.0's trials bit for bit"; the TPE and MOTPE hashes differ, as the expm1 change (0e18eba) and QA-84 (465f91c) predict. Every other number is quoted from a commit body, using the final value where one moved: CMA-ES 100,000 trials 987.7 -> 27.5 s and the compact-storage retention (ca9b9fe, f0d62a0, measured against de495f8, whose library code is 1.0.0's); plan 15's final survey for BruteForce, SMS-EMOA / MOEA/D, the ES family and DE / PSO / LM-CMA / VkD-CMA (283876b); TuRBO (9533366); SQLite (2dc35cf, 9043b43); the journal offsets (45fadd1); the CLI cycle (27269b1); the parity counts from the fix commits and docs/qa/PARITY_QA_2026-11-10.md. Checks (verify-accuracy.md): all 45 cited hashes resolve to commits of the release range; the Overview's counts are the file's (43 parity fixes: 36 QA findings plus expm1, hypervolume / HSSP / UNDX, multi-objective GP standardization, the percentile pruner's zero signs, AGE-MOEA-2's row norms, JANUS's component mean and the journal lock; 18 measured speedups; 11 behavior changes; 130 commits since 1.0.0 including this one); no em or en dashes; the "fixed" verbs match bodies that fixed (repro tests that lost [OpenBugs]), and the deferred / kept findings are named as such.
…se note in Getting Started, the FAQ and the package README; the README's parallelism line no longer promises async-await
The two documentation follow-ups left open since the 1.0.0 release (catalog State, 2026-11-08). Owner, 2026-11-12:
"Fix them and after release commit them back in".
1. NativeAOT. The 1.0.0 smoke test found that a consumer written as a .NET 10 file-based app fails in trial 0: file-based
apps default to PublishAot=true, which turns RuntimeFeature.IsDynamicCodeSupported off, and NumSharp compiles its
kernels at run time (DirectILKernelGenerator). Only the root README's contributor section ("Scripting with dotnet
run") showed #:property PublishAot=false. Now:
- docs/website-src/docs/getting-started.md, Install: OptunaSharp needs dynamic code, does not run under NativeAOT, and
a file-based app needs the property next to its package directive; the exact error it fails with otherwise.
- docs/website-src/docs/faq.md: "Can I publish with NativeAOT, or run OptunaSharp from a file-based app?" with a
complete file-based example; a regular project needs nothing extra.
- src/OptunaSharp.Core/README.md (the README nuget.org shows): the same note in two sentences.
Checked 2026-11-12 with nuget.org's OptunaSharp (isolated NUGET_PACKAGES, nuget.org as the only source): the
Getting Started directives with a 5-trial TPE study print "ok 1.0.0 ..."; the same file without the property fails
trial 0 with "PlatformNotSupportedException: Dynamic code generation is not supported on this platform." (the text
quoted in all three places); the FAQ's code block, extracted from the page as written, compiles and runs.
#:package OptunaSharp@* floats to the newest release, so the pages do not go stale with each version.
2. README.md, line 27 (the owner's line of 2026-11-08): "reused dedicated threads and ThreadPool async-await" becomes
"reused dedicated threads or the .NET ThreadPool". OptunaSharp has no async API (the 2026-11-08 async evaluation
built none); the link already points at the three executors the page documents (nJobs: 1, TrialWorkerScheduler,
TrialExecutorKind.ThreadPool).
…n commit (6fa30e7), now part of the release Owner, 2026-11-12, in the PR round: "Include in this PR" for the two documentation fixes. 6fa30e7 was written for after the release (its message quotes the owner's earlier "Fix them and after release commit them back in"); it is now on the release branch, so the notes account for it. docs/releases/RELEASE_1.1.0.md: - Dashboards & Docs gains one line: Getting Started, the FAQ and the package README say that OptunaSharp needs dynamic code, does not run under NativeAOT, and that a .NET 10 file-based app needs #:property PublishAot=false, with the error it fails with otherwise. Like the examples line, it carries no hash (CHANGELOG_STYLE 7B); 6fa30e7 is a docs commit that documents a user-facing contract, the one kind of docs commit the style lets into the notes. - The Overview's count: "of the release's 132" (v1.0.0..this commit). The cited set is unchanged at 45 commits; the README parallelism line in the same commit is cosmetic and gets no line.
…passes moved - UNDXCrossover, AGE-MOEA-2 at 8+ objectives, the hypervolume Found while reading the development sessions' transcripts (Agentmaster, --minimal) for the numbers the 1.1.0 PR description now carries, at the owner's request in the PR round: "Read the transcripts (minimal) via agentmaster cli to find the numbers of each development". The Breaking Changes bullet "Seeded results move toward Optuna's" listed TPE, CMA-ES with margin, NSGA-III, GridSampler, multi-objective / stepped GP and one-thread NSGA batches, then said "Random, QMC, NSGA-II, plain CMA-ES and single-objective GPSampler over floats give 1.0.0's trials bit for bit". Two NumSharp commits move results it did not name: - c600736 (Optuna's 1-D np.dot / np.linalg.norm through NumSharp's OpenBLAS ddot / dgemv): UNDXCrossover's products and norms. Its body: NsgaiiSamplerParityTests.UndxCrossover_ManyParameters_FullState_IsBitIdentical (24 float parameters, 48 trials, seeds 0 and 7) "failed on both seeds before, bit-identical full state after"; the 3-parameter UNDX test passed before and after. So a seeded NSGA-II / NSGA-III study with UNDXCrossover over many parameters changes. The same commit moved 37 of 240 hypervolumes by an ulp or two (HypervolumeParityTests, live against Optuna 5.0.0), so a hypervolume can differ from 1.0.0's by an ulp. - 156ce8a (reductions in NumPy's order): AgeMoeaSurvival's per-row norms now follow np.linalg.norm(front, p, axis=1), "equal below eight objectives; NumPy's pairwise rows from eight on". AGE-MOEA-2 is matched to pymoo, so it moves toward pymoo's, not Optuna's. docs/releases/RELEASE_1.1.0.md: - The seeded-results bullet adds "`UNDXCrossover` over many parameters" to the moved list, then "AGE-MOEA-2 at 8 or more objectives moves toward pymoo's, and a hypervolume can differ from 1.0.0's by an ulp"; the unchanged list says "NSGA-II with its default crossover"; the bullet cites c600736 as well. c600736 was already cited by the Parity and Testing lines, so the notes still cite 45 distinct commits. - The Overview's count: "of the release's 133" (v1.0.0..this commit).
|
On hold: waiting for NumSharp 0.81.0 for its fixes and performance boosts. The What 1.1.0 waits for:
|
…get; QA-85 fixed upstream Owner, 2026-11-14: "NumSharp 0.81.0 was released, using am cli find all the minimal transcripts about numsharp needing to do some fixing or changes, well it did all of them and now we need to align, fix, migrate to NumSharp. Commit and push and prepare cicd to release". PR #6 had been on hold for it ("waiting for NumSharp 0.81.0 for fixes and performance boosts"). The bump - Directory.Build.props: $(NumSharpVersion) 0.80.0 -> 0.81.0 (NumSharp and NumSharp.Interop.OpenBLAS, both on nuget.org). The whole solution compiled unchanged against 0.81.0's 20 breaking changes (NumPy's signatures, Python exception classes, read-only scalars); refs/NumSharp moves to the v0.81.0 tag (65ef305). What 0.81.0 changes for OptunaSharp, measured - NumSharp.Interop.OpenBLAS 0.81.0 bundles numpy 2.5.4's scipy-openblas 0.3.34.106.0 (0.80.0 bundled numpy 2.4.2's). The 36-study seeded hash probe (plan 14/15 harness), base 38735c2 vs the bump: 31 studies identical, 5 moved in their last bits, all through BLAS / LAPACK: cmaes-mixed and cmaes-ipop (eigh), xnes (eigh), nsga2-undx (ddot, norm) and janus. The 8-study GP probe is identical. So the byte-exact target moves to NumPy 2.5.4: - CI pins numpy==2.5.4 where it pinned 2.4.2 (build-and-release.yml: the satellite job's vendored Optuna, the interop job's install, its re-pin after torch/numba and the assert), and so does the package-parity Dockerfile; numba 0.68 (numpy < 2.6) and scipy 1.18 (numpy < 2.8) accept it, and the sdk:10.0 image is Ubuntu 24.04 with Python 3.12, which numpy 2.5.4 (>= 3.12) needs. - The CMA-ES pickle oracle is regenerated under numpy 2.5.4: CmaPickleOracleTests' 30-dimensional case compared a told state whose eigendecomposition came from numpy 2.4.2's OpenBLAS; 27/27 pass on the regenerated file. Its generator could not run: 73dd305 had written its newline="\n" argument as a literal line break (a syntax error), repaired byte-exactly; its docstring now names numpy 2.5.4 and says why two runs never write the same file (cmaes' np.empty _funhist_values, gzip timestamps) while the tests still compare. - NumPy's exception classes: 0.81.0's np.random uniform raises NumSharp.OverflowError (Python's class, under ArithmeticException) where 0.80.0 raised .NET's OverflowException. NumPyDraws.Uniform mirrors that call, so it raises OverflowError("Range exceeds valid bounds") too, as NumPy does; NumPyDrawsTests and the QA-41 test (RandomSampler over an infinite range) expect the class exactly. A trial failed this way now logs Optuna's own line, OverflowError('Range exceeds valid bounds'). - QA-85 is fixed upstream: 0.81.0's several-kth partition runs NumPy's introselect_ statement for statement (its QB43 fix), so PercentilePruner.NanPercentile's median of [0.0, -0.0, -0.0, -1.0] is -0.0 as in NumPy 2.4.2 and 2.5.4 (both checked live). PrunerTests.NanPercentile_TiedSignedZeros_KeepNumpysZeroSign passes and loses [OpenBugs]; the pruner's remarks, the test docs and PARITY_QA_2026-11-10.md say so. - Doc alignment: both GP search spaces document NumSharp's np.arange ZeroDivisionError for a zero step (0.81.0's class; it was DivideByZeroException); TPE's np.sum remark records that NumSharp's two pinned costs behind QA-84 (SumOverShortRows_CostsAFewLongRowSums, SumIntoOut_CostsAboutAFreshSum) are still [OpenBugs] in 0.81.0, so QA-84's 27-66 % TPE cost stays; CLAUDE.md names 0.81.0, the bundled OpenBLAS and the pin that follows it. Not in 0.81.0 (checked in its sources and its OptunaSharp consumer pins), so OptunaSharp keeps its code - The two np.sum(axis=-1) performance pins: still [OpenBugs] in OpenBugs.Consumers.OptunaSharp.cs. - np.einsum at optimize=False: NumSharp's owner kept the pairwise BLAS contraction ("Keep the BLAS route"), so NSGA-III's 4-operand einsum stays a hand port. - The default argsort tie order (QB-52): still [OpenBugs] in NumSharp. - A public triangular solve: OpenBlasEngine's Trsm is still behind internal interfaces. The premise, live (interop, net10.0, the vendored Optuna 5.0.0): the 57 BLAS-backed parity cases (CmaEsSamplerParityTests, LinAlgParityTests, NsgaiiSamplerParityTests, HypervolumeParityTests) fail 14 times against numpy 2.4.2 with NumSharp 0.81.0 (CMA-ES long runs, lr_adapt, warm start, UNDX) and pass 57/57 against numpy 2.5.4. LinAlg's remark records it, with the footgun that the pin must follow the bundled OpenBLAS. Verified on the tree with this commit and the four after it (none of them moves a seeded result: the 36- and 8-study probes are identical to this commit's alone): whole solution builds; OptunaSharp.Tests on net10.0 and net8.0 2,310 passed, 1 skipped, 41 failed, all 41 [OpenBugs] (none unexpected); CmaPickleOracleTests 27/27.
…or and FileNotFoundError from NumSharp 0.81.0 where the port raised .NET stand-ins
NumSharp 0.81.0 ships every exception class of CPython 3.12 and NumPy 2.5.4, each under its .NET twin's base
(OverflowError / ZeroDivisionError under ArithmeticException, OSError and its subclasses under IOException). Until
now the port raised .NET types where Optuna, CPython or NumPy raise these, because NumSharp had no such class; the
code comments said so ("DivideByZeroException is this port's ZeroDivisionError"). The owner asked to "align, fix,
migrate to NumSharp" once 0.81.0 shipped, so every such stand-in now raises the Python class with the same text.
Raised as Python's class now (the message is unchanged in every case)
- OverflowError: PyStrFormat's "%c arg not in range(0x110000)" and "int too large to convert to float"
(str.format / the SQLAlchemy and storage URL templates); SqlAlchemyUrl's int() of an infinity ("cannot convert
float infinity to integer"); the EF Core MySQL URL's C int arguments ("Python int too large to convert to C long",
"signed integer is greater than maximum" / "... less than minimum"); the Redis URL's TLS version and timeouts
("Python int too large to convert to C long", "timestamp out of range for platform time_t"); the SQLite URL's C int
arguments ("Python int too large to convert to C int").
- ZeroDivisionError: separable CMA-ES with popsize 1 ("division by zero", cmaes' int division; QA-80's fix raised
DivideByZeroException).
- FileExistsError: DownloadArtifact onto an existing file or directory ("File already exists: <path>", QA-78's
fix raised IOException). FileExistsError IS an IOException, so a catch (IOException) still sees it.
- FileNotFoundError(2, "No such file or directory"): the MySQL URL's ssl_ca / ssl_cert files and the Redis URL's
ssl_ca_certs (they raised FileNotFoundException with the same text). The Redis client certificate and key now
check their files first too: Python's load_cert_chain raises that FileNotFoundError for a missing one, where
.NET's PEM loader threw FileNotFoundException with .NET's own text.
Why it matters beyond the type
- A trial that fails on one of these logs Optuna's exact line, "... because of the following error:
ZeroDivisionError('division by zero')", because the warning prints the exception's class name.
- The QA-5 repro tests (IntDistribution.ToExternalRepr(inf), suggest_int of an enqueued inf; still [OpenBugs])
now expect Python's OverflowError("cannot convert float infinity to integer") exactly, the fix they pin.
Kept as .NET types (not Python errors): RDBStorage's 32-bit step column limit, journal and pickle size limits,
Int32/Int64 conversions of stored values, and every other OverflowException that has no Python counterpart.
Breaking for C# callers that caught the old types: OverflowException, DivideByZeroException and
FileNotFoundException are not the new classes' bases (catch OverflowError / ZeroDivisionError / FileNotFoundError,
or ArithmeticException / IOException). Listed under the 1.1.0 notes' Breaking Changes.
Tests: EFCoreDriverParityTests, RedisUrlTests and RedisTlsTests expect the classes by exact type; the separable
CMA-ES and artifact QA tests use ThrowExactly; the interop StorageUrlStandardParityTests maps OverflowError to
Python's "OverflowError|..." outcome.
…replaces the vendored concurrent collections The owner's TODO of 2026-10-27 (834a368), above both vendored types: "once NumSharp 0.80.0 is released, use its NumSharp.Collections.ConcurrentOrderedDictionary<TKey,TValue> instead of this type ... then delete this file. ConcurrentOrderedDictTests pins this copy's contract: point the tests that still apply at the NumSharp type and drop the rest." 0.80.0 shipped neither type publicly; 0.81.0 ships ConcurrentOrderedDictionary (public sealed, NumSharp.Collections) and keeps its ConcurrentDictionary internal. - Deleted src/OptunaSharp.Core/Collections/Concurrent/ConcurrentOrderedDict.cs (internal), ConcurrentDictionary.cs and ConcurrentDictionaryInternals.cs (the vendored dotnet ConcurrentDictionary, internal, whose only consumer was ConcurrentOrderedDict): about 3,550 lines no library code used (THREADING.md §5.1 said so). Nothing public changes, so ApiCompat against 1.0.0 is unaffected. - ConcurrentOrderedDictTests -> NumSharpConcurrentOrderedDictionaryTests: all 22 contract tests still apply (NumSharp's type is a superset of the vendored API: the same constructors, key and index paths, GetOrAdd / AddOrUpdate / TryUpdate, RemoveAt, snapshot enumeration) and now construct NumSharp's type; the two smoke tests of the vendored ConcurrentDictionary clone are dropped with it. - docs/THREADING.md: §5.1 and the test table name NumSharp's type.
…0.81.0's np.prod, bit for bit NumSharp's fix brief (session ab93c03a, 2026-11-12) named the consumer follow-up: "After items 2 and 3 land, OptunaSharp can drop its hand-ported sequential products (Hypervolume, HypervolumeHistory, FanovaTree)". Item 2 landed: 0.81.0's float np.prod multiplies in NumPy's sequential multiply.reduce order on every route (0.80.0's SIMD lanes did not, which is why the port wrote loops). The loops stay, by the rule the NumSharp migration of 2026-11-11 set (owner-approved): where a hand-written helper already gives NumPy's exact result, it stays if it runs per point or per trial, because a NumSharp call costs about 2 µs there (MathUtil's sums, TPE's argmax). ProdRefMinus runs once per point inside the hypervolume / HSSP / box-decomposition loops, HypervolumeHistory.Prod once per trial, FanovaTree.GetCardinality once per tree node. What changes instead - SequentialProductTests (new): the three private helpers, reached by reflection as NumPyReductionOrderTests reaches its own, equal NumSharp's np.prod(a - b) bit for bit: 400 random rows of 2-16 objectives (magnitudes 1e-3..1e3, both signs) for the two hypervolume helpers, 400 random bound pairs of 1-16 features for the cardinality, plus rows with an infinite distance, zero times infinity and a signed-zero distance. If a NumSharp release reorders np.prod again, these fail and name the helper. - The three helpers' comments and Hypervolume's class remark say np.prod is now NumPy's order in NumSharp too, why the loop stays, and which test pins the equivalence. Item 3 (np.einsum at optimize=False computing like c_einsum) did not land: NumSharp's owner kept the pairwise BLAS contraction, so NSGA-III's 4-operand einsum stays a hand port, unchanged.
… 2's QA-83 and QA-80, still marked open Found by the NumSharp 0.81.0 bump's full unit run, which includes the [OpenBugs] category: three known-failing tests passed. One was QA-85, fixed by NumSharp. The other two passed on the 0.80.0 base as well (re-run on 38735c2): - ErrorContractParityBugTests.CmaEs_X0MissingParameter_FailsWithPythonsMessage (QA-17, run 1): an x0 missing a parameter raises Python's "Parameter configuration must contain all distributions." since 1e8aba6 made SearchSpaceTransform.TransformValues check it (run 2's QA-83). - SamplerParityBugTests.CmaEs_X0OutsideBounds_FailsTrialsLikeCmaes (QA-34, run 1): Cma / SepCma assert cmaes' _is_valid_bounds since 1e8aba6 (run 2's QA-80). 1e8aba6 marked QA-80 / QA-83 fixed in the run 2 journal but left run 1's repro tests marked [OpenBugs], so CI (which excludes the category) never ran them. Both lose the attribute and gain a "fixed by 1e8aba6" note; PARITY_QA_2026-11-02.md marks QA-17 and QA-34 fixed. Also, a follow-up to 9f77939: the QA-80 (separable CMA-ES, popsize 1) and QA-78 (DownloadArtifact) tests' docs name the classes they now expect, NumSharp.ZeroDivisionError and NumSharp.FileExistsError (an IOException).
…r NumPy 2.5.4 The NumSharp 0.81.0 bump (9d3285f) moved the live oracle to numpy 2.5.4, the NumPy whose OpenBLAS 0.81.0 bundles. NumPy 2.5 removed np.row_stack, and pymoo 0.6.1.3 calls it in NSGA-II's crowding distance (rank_and_crowding/metrics.py: np.row_stack([F, np.full(n_obj, np.inf)])). The full interop suite on net10.0 against numpy 2.5.4 (2026-11-14): 675 passed, 30 failed, 52 skipped; 17 of the failures were every pymoo-bridge case (PymooSamplerParityTests, BridgeSamplerParallelReadTests' two pymoo cases, PythonThreadTests' PymooSampler case), all "module 'numpy' has no attribute 'row_stack'". The other 13 are SqliteRdbUpgradeInteropParityTests, which cannot find the repository root from an out-of-tree build folder (the known out-of-tree limitation; CI runs them in-tree). Why not pymoo 0.6.2 (the latest, which no longer calls row_stack): it changed AGE-MOEA-2's arithmetic. Diffing its age2.py against 0.6.1.3 without docstrings, the Newton-Raphson curvature now evaluates point**x as exp(x * log(point + 1e-10)) with overflow guards, and the distances sum with np.sum where 0.6.1.3 used Python's sum; the native AgeMoea2Optimizer reproduces 0.6.1.3 generation by generation, so 0.6.1.3 stays the oracle. What changes: PythonSession, after setting NPY_DISABLE_CPU_FEATURES and before Optuna's import, sets numpy.row_stack = numpy.vstack when NumPy lacks it. NumPy 2.0-2.4 defined row_stack as exactly that alias, so no result changes. Every other NumPy name pymoo 0.6.1.3 uses that 2.5.4 lacks (np.Inf, np.NaN, np.math, np.product) was already gone in NumPy 2.0, so no test reaches it. The product's pythonnet runtime leaves the user's NumPy untouched: with NumPy 2.5, a pymoo bridge user needs pymoo 0.6.2. Also checked: NumPy 2.5.4 keeps the CPU dispatch names PythonSession disables on AVX-512 Linux hosts (X86_V4, AVX512_ICL, AVX512_SPR), so that guard still applies. After this change: the pymoo, AGE-MOEA-2, bridge-parallel-read and Python-thread suites 66 passed, 5 skipped, 0 failed against numpy 2.5.4.
…e target, Python's exception classes, QA-85 / QA-17 / QA-34, the benchmark re-run The NumSharp 0.81.0 migration (9d3285f..8e7418e) changed what the 1.1.0 notes promise, and the benchmark re-run on it changed their numbers. docs/releases/RELEASE_1.1.0.md now says so; the PR description follows. Intro and Overview - A sentence and an Overview bullet for NumSharp 0.81.0: its OpenBLAS is NumPy 2.5.4's (scipy-openblas 0.3.34.106.0), so the bit-for-bit target is Optuna running on NumPy 2.5.4; QA-85 is fixed upstream; Python's OverflowError, ZeroDivisionError, FileExistsError and FileNotFoundError are raised where Optuna raises them. - The binary-compatibility bullet names the new behavior changes (Python's exception classes, BLAS-backed results following NumPy 2.5.4). - The count line, recounted from the notes by a script, by the rules that gave the earlier figures: 48 parity fixes (39 QA ids, 9 items without one; was 43 = 36 + 7), 18 measured speedups (unchanged), 14 behavior changes (was 11), 50 cited commits (was 45) of the release's 140 (v1.0.0..this commit; was 133), 2,310 unit tests on each framework (was 2,307), 751 live-Optuna interop tests against NumPy 2.5.4 (was 727). The 751 are the complete non-Slow interop suite on net10.0, run 2026-11-14 with every backend installed (numpy 2.5.4, cma 4.4.4, the vendored optunahub and its registry, JANUS_REF_PATH): 751 passed, 0 failed, 6 skipped - five Redis cases without a local server (CI's Linux leg starts one) and PreV050_AppliesNoConstraints_MatchBaseProblemCall, which needs optunahub < 0.5.0. Performance - re-measured 2026-11-14 on NumSharp 0.81.0 by the same method as e3a25cb: the published 1.0.0 package against the 1.1.0-local.4 candidate, one process per version and workload, two reps (rep 0 cold, rep 1 warm), best of two alternating rounds, an isolated NUGET_PACKAGES, on the shared i9-13900K. - random x6.2->x7.1 -> x5.3->x8.1; CMA-ES x6.2->x7.0 -> x5.3->x10.3; QMC x4.0->x5.0 -> x4.1->x6.0; scrambled Sobol x11.6->x20.4 -> x12.1->x16.8; NSGA-II x2.4->x3.4 -> x2.3->x3.2; TPE (40 floats) x2.5->x3.0 -> x2.7->x3.4, 4.4 GB -> 0.81 GB (was 0.84); TPE over 10 floats x1.6->x2.4 -> x1.5->x2.3; multi-objective TPE x1.7->x2.5 -> x1.7->x2.3; GP x1.6->x1.8 -> x1.5->x1.6, 5.9 GB -> 1.3 GB (was 1.5), gen-2 collections 7x to 9x fewer (was about 4.5x). The Overview's span of reductions becomes 4.0x to 21x (was 3.9x). - CMA-ES's 10,000-trial workload no longer hashes to 1.0.0's trials: its eigendecomposition follows NumPy 2.5.4's OpenBLAS now. The Overview and the CMA-ES line say so and cite 9d3285f. Random, QMC, scrambled Sobol, NSGA-II and GP still give 1.0.0's trials, hash for hash; the TPE workloads differed already (QA-84, expm1). Parity & Fixes - The CMA-ES line adds run 1's QA-17 and QA-34 (5b0e4fd: fixed by 1e8aba6, their repro tests run in CI again). - New lines: NumSharp 0.81.0's OpenBLAS (the 57 BLAS-backed live cases pass, 14 of them fail against NumPy 2.4.2; 9d3285f), QA-85 (the median of tied signed zeros is NumPy's -0.0, NumSharp's several-kth partition being NumPy's introselect_; 9d3285f), and Python's exception classes (14 throw sites in 8 files; 9f77939, 9d3285f). Testing & Tooling - NumPy 2.5.4 as the live oracle's NumPy, the regenerated CMA-ES pickle oracle and the np.row_stack alias for the pinned pymoo 0.6.1.3 (9d3285f, 8e7418e); SequentialProductTests, 800 random cases against np.prod (784f495). Breaking Changes - DownloadArtifact raises FileExistsError (an IOException), separable CMA-ES with popsize=1 ZeroDivisionError (9f77939 joins 3009c4a and 1e8aba6). - "plain CMA-ES" leaves the list of samplers that give 1.0.0's trials bit for bit. - New: BLAS- and LAPACK-backed seeded results follow NumPy 2.5.4's OpenBLAS (5 of 36 probe studies moved: CMA-ES over mixed parameters, IPOP CMA-ES, NSGA-II with UNDX, xNES and JANUS; CMA-ES with margin, separable CMA-ES and the 8 GP studies did not); Python's exception classes replace .NET stand-ins, and OverflowException, DivideByZeroException and FileNotFoundException are not their bases; the NumSharp 0.81.0 dependency, whose own breaking changes reach code that calls NumSharp directly. Not cited: 1592d05, which deletes the vendored concurrent collections (internal, unused) for NumSharp's public ConcurrentOrderedDictionary, changes nothing a user sees.
|
Hold lifted: NumSharp 0.81.0 is in. Six commits move the branch to it and a seventh updates the notes; the description above is updated. What 0.81.0 fixed for 1.1.0:
What it did not fix:
Found on the way: NumPy 2.5 removed Verified locally: unit tests 2,310 on net8.0 and net10.0; live-Optuna interop 751 passed, 0 failed, 6 skipped against NumPy 2.5.4 (5 Redis cases without a local server, which CI runs; 1 by design); 20 packages publishable; ApiCompat clean against 1.0.0. Next: this PR's checks and a full manual run of the release matrix (3 OSes, both frameworks, Slow) on |
…tes written since 1.0.0 pinned to x64 for macOS, crash dumps for Windows test hosts The release-matrix rehearsal of 8c31fb6 (workflow_dispatch run 43545001631, 2026-11-14: 3 OSes, both TFMs, the Slow category) passed everything but its two macOS jobs; PR #6's own run of the same commit (43544913859) passed everything but one Windows test host, which crashed. This commit answers both. 1. macOS The rehearsal ran macOS for the first time since the v1.0.0 tag run. Four bit-exact gates written after that run failed there; none of them had run on macOS before, and all four compare last bits that differ on arm64 for reasons the suite already documents and pins to x64 elsewhere (PythonSession.BitExactHost; the unit suite's own x64 guards in AgeMoea2OperatorParityTests, CmaPickleOracleTests, ShapleyImportanceEvaluatorTests, SklearnOracleParityTests and the ES digest tests). Interop (macos-26, net8.0: 248 passed, 3 failed, 506 skipped): - HypervolumeParityTests.ComputeHypervolume_MatchesOptunaBitForBit_TwoToFiveObjectives (c600736, 2026-11-11): 54 of 240 volumes a last bit off, e.g. trial 9, m=3 n=37: 3ff02f8a9b00e63c against Optuna's 3ff02f8a9b00e63b. - HypervolumeParityTests.SolveHssp_SelectsOptunasIndices_ThreeAndFourObjectives (c600736): 5 of 240 selections differ (trial 66, m=3 n=17 k=12: [...,13,12,16,14] against Optuna's [...,14,10,15,11]), the zero-contribution ties. Both compare NumSharp's OpenBLAS ddot/dgemv with NumPy's @ / np.dot. That is bit-exact only where NumPy's wheel bundles the same scipy-openblas, i.e. x64; the macOS runner installs numpy-2.5.4-cp312-cp312-macosx_14_0_arm64.whl, NumPy's macOS 14+ wheel, which is built against Apple's Accelerate. - CmaEsSamplerParityTests.SeparableCma_LongRun_FullState_IsBitIdentical (99e9e73, 2026-11-08): trial 11's pickled optimizer state differs. Every other full-state CMA-ES test here was already EnsureBitExactHostOrInconclusive; the short SeparableCma_FullState_IsBitIdentical still runs everywhere (it passed on macOS), the 30-generation run meets the documented arm64 last-bit difference. All three now call PythonSession.EnsureBitExactHostOrInconclusive() instead of EnsureOrInconclusive(). Unit (macos-26, net8.0: 2,284 passed, 1 failed, 20 skipped): - SamplerParityBugTests.GpSampler_SteppedFloat_SuggestsNumpyGridValues (QA-49; written 2026-11-03 as [OpenBugs], unmarked by the 2026-11-11 fix, so it never ran on macOS before): the fourth trial is 0.9999999999999998 where Optuna on x64 (and the port on Windows and Linux) gives 1.0; every other trial matches and picks the same grid point. The grid itself is NumPy's on macOS too (GpSearchSpace_SteppedFloatGrid_UsesNumpyArangeStep passed in the same run), so the GP's last-bit arithmetic on arm64 reached that point by another route: the value equals the rounding path's low + k*step (0.9999999999999999) after the normalize / unnormalize round trip. The test now goes Inconclusive off x64, with the reason. Each test's docs say why it is pinned and what macOS showed. No product code changes: these gates hold on the hosts the parity claims are made for (x64, where NumPy and NumSharp share one OpenBLAS and one libm family), and macOS keeps running the tolerance-based parity cases. On x64 all four still run: the three interop gates pass locally against numpy 2.5.4 (3/3, net10.0). 2. The Windows test-host crash PR #6's test (windows-2025) job, net10.0, aborted after 1 min 8 s: "The active test run was aborted. Reason: Test host process crashed", with nothing after "crashed" (no managed exception, no stack overflow message). Its TRX (artifact test-results-windows-2025) holds 915 results; the last ones, GridSamplerTests at 02:14:48.47, come right before the JANUS suites in discovery order, and the host died about 0.46 s later, so one of the next few tests was running (VSTest batches results, so the last few can be lost). No earlier run in SciSharp's last 60 Build and Release runs crashed a test host. It did not come back: - the rehearsal's test (windows-2025) job ran both TFMs on the same commit, green; - locally (i9-13900K, Windows 11, OPENBLAS_CORETYPE=Haswell as in CI, DOTNET_GCgen0size=0x40000 so collections and NumSharp finalizers run constantly, --blame-crash): the JANUS / GridSampler / MOEA/D / GP-primitives window 10 of 10 rounds green (71 tests each), the whole CI-filtered net10.0 suite 3 of 3 rounds green (2,310 passed, 1 skipped). So the Windows leg of the unit test step now runs with --blame-crash --blame-crash-dump-type mini, and the upload keeps TestResults' *.dmp and Sequence_*.xml beside the TRX files: a recurrence leaves the running test's name and a dump. Linux and macOS keep the plain run (the flag was only tried on Windows). The bash array form expands to nothing off Windows, checked under CI's `bash -e -o pipefail`; the SDK's blame collector names its file Sequence_<id>.xml. .claude/CLAUDE.md's test-job bullet says where the dump lands. Also: the notes' "of the release's 140" becomes 141 (v1.0.0..this commit); the PR body's commit count follows.
|
Release matrix rehearsed - a manual run of Build and Release on
A second rehearsal runs on the new head. |
… fault there was PR #6's Windows test-host crash PR #6's test (windows-2025) job crashed its test host on 8c31fb6 and on 7d73f5a, both times in JanusDeSamplerTests.Sampler_ScalarMode_Runs (VSTest blame, added by 7d73f5a), with nothing logged and no dump, while both release-matrix rehearsals and every local run passed. Three diagnostic rounds on a throwaway branch (ci/win-crash-diag, each leg a fresh windows-2025 VM) found it: - Round 1, run 43548926143 (the unit suite as the PR runs it, four legs): crash on AMD EPYC 9V74 (Zen 4) and 9V45 (Zen 5), also with the JANUS-DE suite run alone; pass on AMD EPYC 7763 (Zen 3) and an Intel Xeon 8370C. No WER dump, no Application Error event; the host's own log ends at "Executing test Sampler_ScalarMode_Runs". - Round 2, run 43549856491 (the scenario as a console program, six legs, seven variants each, full exit codes): on the three Zen 4 legs, NumSharp 0.81.0's OpenBLAS 0.3.34.106.0 with OPENBLAS_CORETYPE=Haswell exits 0xC0000005 (access violation) between trials 150 and 200, on net10.0 and net8.0, with OPENBLAS_NUM_THREADS=1, and with .NET's AVX-512 / wide vectors off. The same build PASSES with OpenBLAS's own kernel choice, and NumSharp 0.80.0's OpenBLAS 0.3.31.22.0 (NUMSHARP_OPENBLAS_VERSION override) passes WITH the pin. Zen 3 x2 and an Intel Xeon 8573C pass every variant. So: OpenBLAS 0.3.34's Haswell kernels, forced on an AMD CPU with AVX-512, fault; CI forces them. - Round 3, run 43550350376 (the whole unit suite, both TFMs, OPENBLAS_CORETYPE unset, six legs): no crash anywhere. Zen 3 x4 green. Zen 4 and Zen 5 fail the same six Haswell-captured oracles on both TFMs: CmaPickleOracleTests (cma, cma_lr_adapt, cma_30d), EvolutionStrategyExtraCandidateTests' xNES digest, AgeMoea2OperatorParityTests' Normalize_IsBitExact and Do_Age2_IsBitExact. (One Zen 3 leg also failed the 16-worker REMBO no-duplicate-evaluation test once on net8.0; it passed on every other leg and run, unrelated to the pin.) The product is not affected in its default configuration: a user's OpenBLAS picks its own kernels, the variant that passed on the same machines. Only a process that forces OPENBLAS_CORETYPE=Haswell on Zen 4/5 meets the fault. What changes: - .github/scripts/openblas_coretype.sh (new), sourced by all five x64 test steps that pinned Haswell inline (unit, slow unit, satellites, interop main, interop slow): it exports OPENBLAS_CORETYPE=Haswell as before, except on an AMD CPU with AVX-512 (Windows: PowerShell 7 asks .NET for the vendor and Avx512F; Linux: /proc/cpuinfo; Git Bash's emulated /proc/cpuinfo is deliberately not trusted for the flag), where it leaves OpenBLAS to choose, and prints which and why. Live NumPy parity still holds there: the interop suite's NumPy and NumSharp load the same OpenBLAS build, which makes the same choice. Checked locally under CI's `bash -e -o pipefail` and `-uo pipefail`, on x64 Intel (pins Haswell) and with RUNNER_ARCH=ARM64 (does nothing). - The three Haswell-captured oracle guards also require OpenBlasEngine.CoreName == "Haswell" (openblas_get_corename), and report Inconclusive with the kernel name otherwise: AgeMoea2OperatorParityTests.RequireBitExactLinalgHost, CmaPickleOracleTests.RequireOracleHost (every variant), EvolutionStrategyExtraCandidateTests' xNES row (OpenAI-ES and SNES never reach OpenBLAS and held). Locally: OPENBLAS_CORETYPE=Haswell 52/52 of those classes pass; OPENBLAS_CORETYPE=Sandybridge 37 pass and 15 report Inconclusive, none fail. - The Windows --blame-crash comment and .claude/CLAUDE.md say what the blame data led to; CLAUDE.md gains the pin rule (a new Haswell-captured oracle needs the same guard). Upstream: worth an OpenBLAS issue (0.3.34, DYNAMIC_ARCH, OPENBLAS_CORETYPE=Haswell on Zen 4/5 -> access violation), with the console repro of the diagnostic branch; NumSharp 0.81.0 ships that build.
…d PR #6's Windows test-host crash (4cbbef1) Testing & Tooling gains one line for 4cbbef1: CI no longer forces OpenBLAS's Haswell kernels on AMD CPUs with AVX-512, because NumSharp 0.81.0's OpenBLAS 0.3.34 running them there faults (an access violation in JANUS-DE's quadratic chart, found on Zen 4 and Zen 5 runners by the diagnostic runs 43548926143, 43549856491 and 43550350376); OpenBLAS's own kernel choice and 0.3.31 pass, and the Haswell-captured oracles report Inconclusive on other kernels. The line also tells a reader who sets OPENBLAS_CORETYPE=Haswell on such a CPU that they meet the same fault; a default-configured process picks its own kernels and does not. The Overview's count line, recounted by count_notes.py: 51 cited commits (was 50) of the release's 143 (v1.0.0..this commit; was 141, +4cbbef1 +this one). Parity fixes (48), measured speedups (18) and behavior changes (14) are unchanged: Testing & Tooling is not counted in them. 7d73f5a (the macOS x64 pins, test-only) stays uncited, as test-only commits do.
|
The Windows test-host crash: traced and fixed ( It is OpenBLAS 0.3.34 (NumSharp 0.81.0's bundle) running the Haswell kernels CI forces with
Runs: 43548926143, 43549856491, 43550350376. A third rehearsal of the release matrix runs on |
…uns forced Haswell fine, and unpinned it broke a live CMA-ES gate 4cbbef1 left OPENBLAS_CORETYPE unset on every AMD CPU with AVX-512. The third release-matrix rehearsal (dispatch 43551908135 on 07f8232) then put its ubuntu Slow interop leg on such a runner and LongRun_40D_FullState_IsBitIdentical failed: trial 31's stored 40-D CMA-ES state differed from live Optuna's in the same process. Meanwhile the fault behind 4cbbef1 had only ever been seen on Windows. A check on throwaway branch ci/linux-haswell-check (run 43553335934, six fresh ubuntu-24.04 VMs, the console repro of the fault, Haswell forced and unpinned, net10.0 and net8.0) settled it: three AMD EPYC 9V74 (Zen 4) legs ran the forced Haswell kernels to the end, exit 0 on both TFMs (OpenBLAS's own choice there is SkylakeX); so did EPYC 7763 x2 and a Xeon 8370C. The fault is in OpenBLAS 0.3.34's Windows build only. So .github/scripts/openblas_coretype.sh applies the exception only when RUNNER_OS is Windows; Linux keeps the Haswell pin on every CPU, as before 4cbbef1, which keeps the committed oracles and the live byte-for-byte gates (the 40-D CMA-ES state among them) bit-exact on Linux Zen 4 too. The script's header gives both measurements; the workflow comments, the three oracle guards' remarks, .claude/CLAUDE.md and the notes line say "Windows on AMD CPUs with AVX-512". Checked locally under `bash -e -o pipefail`: RUNNER_OS=Windows on this Intel host and RUNNER_OS=Linux pin Haswell, RUNNER_ARCH=ARM64 does nothing. The notes' count line becomes "of the release's 144" (v1.0.0..this commit); the cited-commit count (51) is unchanged: the notes line keeps citing 4cbbef1, which this commit narrows.
…s optimizer repeats evaluated points one trial at a time too, a model bug, not a lost read
The `rembo` row of SamplerParallelReadTests.Optimize_SixteenWorkers_SamplerThatNeverRepeats_EvaluatesNoPointTwice
failed 2 of about 45 CI runs of the 1.1.0 release branch (net8.0), for example:
rembo evaluated these points more than once:
x=9.999999999999998|y=-3.6688215204910746|z=0.9715361410124286 x4
The owner's decision (2026-11-14): investigate first, then open an OpenBug and proceed with the release.
Investigation (2026-11-14, scripts kept in the session scratchpad, not in the repo)
-------------------------------------------------------------------------------------
1. Reproduction. The test's exact scenario (RandomEmbeddingSampler(2, seed: 1), 60 trials, nJobs: 16, objective
(x-2)^2 + (y+1)^2 + z over x, y in [-10, 10], z in [0, 1]) repeated a point in 3 of 80 runs on 32 CPUs and
0 of 80 pinned to 4 CPUs (.NET 11, the scratchpad lies outside the repo's global.json). Each repeat had one
latent stamp ("rembo:latent") on every copy, e.g. [-1.414205579096142, 1.4142055790961423]: the center of
DIRECT's corner cell of the latent box [-sqrt2, sqrt2]^2 after 11 trisections (sqrt2 - (2*sqrt2/3^11)/2).
2. Not a lost read. A storage decorator (an InMemoryStorage subclass) numbered every read and write under one
lock: ALL = the per-trial history snapshot, RUN = the running-trial read (the pending set), STAMP = a latent
stamp, TELL = a final state. For every stamp, the proposing thread's last ALL and RUN reads give exactly the
history and the pending set its proposal used. Over 220 runs and 13,200 proposals (.NET 10): no proposal
missed an earlier proposal's latent stamp. GetOrComputeLatent reads the running trials first (fresh,
useCache: false) and the history right after (the first useCache read of the trial), all inside `_gate`, and
every stamp is written before `_gate` is released. Every repeated point was visible to the proposal that
repeated it, as a finished observation (most) or as a running trial's pessimistic lie (one), and the
acquisition's optimizers still returned the same DIRECT cell. The repeats seen under 16 workers were exact
latent repeats at corners of the latent box at poor values (about 72 for the CI point; the minimum is 0).
One concurrency imprecision showed on the way: in about 1.4 % of proposals (186 of 13,200) a trial told
between the RUN and the ALL read is in both, so FitWithPendingLies holds its real value and its lie at one
input. No repeat coincided with it. Left as is; noted in the OpenBugs test.
3. Not concurrency at all. The same objective and configuration with ONE worker (nJobs: 1), seeds 1..120:
82 runs evaluated a parameter point twice; counting exact latent repeats only, 67 runs repeated 149 latent
points (138 at values <= 2 near the best point found, 11 from 2.05 up, among them three corners of the
latent box in seed 48 at 67 and 122). Seed 1 repeats nothing alone, which is why the 16-worker test passed
about 95 % of the time: an interleaving acts as another seed (as the test's xNES remark already says).
4. Two routes, both with one worker (seed 95, 60 trials):
- Clipping. x = p_X(A*y) clips onto the parameter box, so a region of the latent box lands on one corner:
(10, 10, 0) at #7, #18, #34 and (-10, -10, 1) at #19, #45, each from a different latent point. The default
low-dimensional kernel measures distances between latent points and sees them as far apart - the waste
Wang et al. (2016, sec. 3.2) name as that kernel's weakness ("two points far apart in this region may be
projected via p_X to nearby points on the boundary of X"), which their high-dimensional kernel avoids.
- The incumbent. The GP interpolates (the Cholesky never needs more than its 1e-6 nugget; the posterior std
at an evaluated point is 2e-4..5e-4 of the profiled amplitude from trial 29 on), so expected improvement at
the incumbent is that std times phi(0): small but not zero. Once the GP is confident across the box,
DIRECT and CMA-ES find nothing larger and return the incumbent's own coordinates: #35's latent point again
at #36-#37, #40's at #41-#44, #46's at #47.
(Ruled out on the way: jitter escalation - it stays at 1e-6; log-EI underflow - Acqf.StandardLogEi has the
erfcx branch below z = -25; a spurious per-study state reset - TrialAnchor compares by id and start time.)
What changed
------------
- SamplerParallelReadTests: the `rembo` DataRow leaves the shared 16-worker test (the other 14 rows stay), and
the new Optimize_SixteenWorkers_Rembo_EvaluatesNoPointTwice, marked [OpenBugs], runs that same scenario
through the shared method (whose 60-trial condition for "rembo" stays live). Its remarks carry the CI
failures, the local rates, the trace and the corner cells. The class remarks say REMBO is an open
exception, kept out of RepeatsFinishedPoints because it is a bug, not a design choice; its volley rows
(ParallelReads_SharedSampler_HandsNoNewPointToTwoTrials), which have never repeated, stay asserted.
- RandomEmbeddingSamplerTests: new [OpenBugs] Optimize_OneWorker_EvaluatesNoPointTwice, the deterministic
reproduction (seed 95, one worker), documenting both routes and a fix direction (keep the acquisition's
maximizer off evaluated and running points, compared in the projected parameters). It fails today on
net8.0 and net10.0 with "x=9.999999999999998|y=9.999999999999998|z=0 at #7,#18,#34" first.
- docs/THREADING.md: the whole-run and history-repeat passages of sec. 6.3 and the test-map row name REMBO as the
open exception, with the tests that pin it.
No source change: RandomEmbeddingSampler and RemboGpProposer are untouched, so every seeded REMBO result is
the same.
Verification
------------
- dotnet test test/OptunaSharp.Tests -c Release, filter RandomEmbeddingSampler|SamplerParallelReadTests:
net8.0 92 of 93 pass, net10.0 91 of 93. Optimize_OneWorker_EvaluatesNoPointTwice fails on both (intended);
Optimize_SixteenWorkers_Rembo_EvaluatesNoPointTwice failed on net10.0 in that run and passed on net8.0
(flaky, intended - now outside CI). Every other test of the two classes passes on both.
- The CI filter (TestCategory!=OpenBugs&!=HighMemory&!=RequiresPython&!=Slow) lists neither new test.
…r no-repeat check is [OpenBugs] beside a one-worker reproduction (6bed3d7) The owner's call (2026-11-14) on the REMBO flake that had failed 2 of about 45 CI runs: investigate first, then open an OpenBug and proceed with the release. 6bed3d7 did both; this commit tells the release's readers. - Testing & Tooling gains one line: REMBO's 16-worker no-repeat check is now [OpenBugs], beside a one-worker reproduction; RandomEmbeddingSampler evaluates a point twice one trial at a time too (82 of 120 seeds of the check's objective), from latent points the projection clips onto one box corner and by proposing a confident GP's incumbent again; a trace of 13,200 concurrent proposals found no lost read - 6bed3d7. - The Overview's count line, recounted: 52 cited commits (was 51; 6bed3d7 is new) of the release's 146 (v1.0.0..this commit, which counts itself; was 144), 2,309 unit tests on each framework (was 2,310: the rembo row left the CI set and both new tests are [OpenBugs]; measured with the CI filter on both TFMs, 2,309 passed and 1 skipped, ARealSigterm_..., a POSIX-signal test that self-skips on Windows, as in the earlier count). The other figures are untouched by this change: 18 measured speedups and 14 behavior changes recount the same with count_notes.py; the 48 parity fixes keep the item rule 8c31fb6 used (39 QA ids + 9 items without one, where the script's bullet rule gives 5), since neither commit touches the Parity & Fixes section.
OptunaSharp 1.1.0: the 145 commits on top of
main. Release notes:docs/releases/RELEASE_1.1.0.md. Each change carries the numbers its development session measured.x5.3->x8.1, CMA-ESx5.3->x10.3, QMCx4.1->x6.0, NSGA-IIx2.3->x3.2, TPEx1.5->x3.4, GPx1.5->x1.6UNION ALL(8,000 trials with one left running: 13.0 -> 9.6 s); SQLite and EF Core parse each distribution once per load (a 10,000-trial study: 2.1 -> 1.4-1.5 s, 1.22 GB -> 479 MB); journal files keep sparse line offsets (4.3 million lines: 160 -> 0.1 MB)ask+tellpair 1,794 -> 1,578 mslist.sortand set order (624 cases recorded from Python)KeyError,StorageInternalError, LightGBM's repr, and moretellwithout--values, structured--sampler-kwargs, empty--log-fileisclose,clipexpm1(truncated-normal quantiles bit-identical to Optuna's: 44.2 % -> 58.6 %) and the pairwise continuous sum (QA-84; 27-66 % slower than the left-to-right sum, TPE still faster than 1.0.0)-0.0); run 1's QA-17 and QA-34, already fixed with QA-80 and QA-83, lose their open marksOverflowError,ZeroDivisionError,FileExistsErrorandFileNotFoundErrorat 14 sites where the port threw .NET stand-insConcurrentOrderedDictionaryreplaces the vendored concurrent collections (3,550 lines deleted)np.prodis sequential now: the hand-written hypervolume and fANOVA products stay (no array per call) and are pinned to it over 800 random casesnp.row_stackalias (NumPy 2.5 removed it)np.sum(axis=-1)short-row andout=costs, so QA-84's 27-66 % stays; NSGA-III keeps its term-by-termeinsumport and the GP its own triangular solvesInMemoryStorage: 100,000 CMA-ES trials retain 257 MB instead of 1,287 MB, 100,000 NSGA-II trials 88 MB instead of 671 MB; same trials, same run timeSetTrialUserAttrsand siblings)CmaEsSampler(reuseToldOptimizer: true): 100,000 trials 27.5 -> 18.2 s; seeded results leave Optuna's, by designOverflowException,DivideByZeroExceptionandFileNotFoundExceptionare not their bases)CmaEsSampler's 1.0.0 constructor restored: binary-compatible with 1.0.0 (ApiCompat)[OpenBugs]beside a one-worker reproduction: REMBO's optimizer evaluates a point twice one trial at a time too (82 of 120 seeds of that objective), through the projection's clipping and its GP's incumbent; a trace of 13,200 concurrent proposals found no lost readdocs/releases/RELEASE_<version>.mdas the GitHub Release; the 1.1.0 notes in NumSharp's format, benchmarked again on NumSharp 0.81.0Verified locally: 20 packages packed, signed and verified as publishable; ApiCompat finds no break against 1.0.0; doc gate clean; examples 59/59; unit tests 2,309 on net10.0 and net8.0; live-Optuna interop 751 passed, 0 failed against NumPy 2.5.4 (6 skipped: 5 need a Redis server, which CI starts; 1 needs optunahub < 0.5.0); DocFX 0 errors.
Merge: fast-forward
mainto this branch's head, so the cited hashes stay valid; not GitHub's "Rebase and merge" or "Squash and merge".