Extensive improvement over native optimization & exposed strategy controls - #239
Open
pavelmk wants to merge 1 commit into
Open
Extensive improvement over native optimization & exposed strategy controls#239pavelmk wants to merge 1 commit into
pavelmk wants to merge 1 commit into
Conversation
Ten-second native comparison (one thread). Each entry is the largest confirmed sampled size / observed failed upper size; two repetitions and all family variants must pass at the reported lower size. | Problem (input unit) | Original | Auto + race | Configured | |---|---:|---:|---:| | Knapsack (items) | 33 / 34 | 2304 / 2336 | 2304 / 2336 | | Assignment (tasks) | 17 / 18 | 64 / 66 | 31 / 32 | | Facility location (sites) | 8 / 9 | 44 / 45 | 54 / 56 | | Bin packing (items) | 17 / 18 | 47 / 48 | 30 / 31 | | Production planning (periods) | 11 / 12 | 27 / 29 | 28 / 29 | | Routing (cities) | 21 / 22 | 21 / 22 | 21 / 22 | | Weighted queens | 23 / 24 | 23 / 24 | 23 / 24 | Original is the preserved pre-algorithm optimization facade, not pristine upstream Gecode. Automatic exploration and restarting share the ten-second limit. These are historical sampled brackets, not speed ratios or general capacity guarantees. Production auto size 28 had mixed confirmations. Witnesses were independently checked; optimality is backend-reported. The combined-policy study used development families and does not isolate racing's contribution. It predates MiniZinc controls and this source cleanup. Full measurements, report, methods and evidence-verification commands: https://github.com/pavelmk/gecode-optimization-benchmarks Reproduce the source regression cases with CMake 3.21+, C++17, HiGHS 1.15.1 (commit 04024d701f79feb8e2f18bc3df0dffc04ef05088), and MiniZinc 2.10.1. Replace the two dependency paths with their local installations/checkouts: cmake -S . -B build/optimize -DCMAKE_BUILD_TYPE=Release \ -DGECODE_ENABLE_OPTIMIZE=ON \ -DGECODE_OPTIMIZE_HIGHS_SOURCE=/path/to/HiGHS \ -DGECODE_OPTIMIZE_MINIZINC_REGISTRATION=ON \ -DGECODE_OPTIMIZE_MINIZINC_EXECUTABLE=/path/to/minizinc \ -DGECODE_ENABLE_QT=OFF -DGECODE_ENABLE_GIST=OFF cmake --build build/optimize --config Release --parallel 4 ctest --test-dir build/optimize -C Release --output-on-failure Focused native and frontend cases: ctest --test-dir build/optimize -C Release --output-on-failure \ -R '^optimize-(native|flatzinc|minizinc)'
pavelmk
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi there! [Meta-note: this is human-written, and everything below the dotted line is AI-written.] My name is Pavel, and I've used Gecode extensively in my consulting work (abseil.ai) in operations scheduling across robotics and factories. I work for an AI lab now and had about $400 of GPT6 credits for personal use, so I was curious with all the recent advancements in frontier math how well the AI (specifically Codex with /goal mode) could optimize Gecode performance, and ran it overnight on that use case. It was quite successful! Here are the results (a combination of several strategic and algorithmic improvements):
Best of auto/configured, selected separately for size and speed:
(here's the HTML version: https://pavelmk.github.io/gecode-optimization-benchmarks/ , and here is the benchmark suite: https://github.com/pavelmk/gecode-optimization-benchmarks/ )
Because of the substantial performance improvements, I wanted to make this available to the community, however I'm also aware that this is an enormous change to drop into an open-source project, and candidly I've overwhelmingly trusted GPT6 to perform code review, though I was sure to include here an extensive test suite. I'm confident in its correctness, but not that it successfully followed all stylistic cues and project philosophy.
Aesthetically, I think there are substantial questions arising now regarding the meaning and value of AI-vs-human-written code, the erosion of codebase understanding (e.g. https://www.seangoedecke.com/in-defense-of-not-understanding-your-codebase/), purpose of code as a pedagogical tool vs. a performance-maximizing project, etc. So, I will welcome guidance from this maintainer community whether any of these changes warrant incorporation, or what that interaction should look lie with this project.
If you'd like to incorporate this in a modified form, I'm happy to take any feedback which I will forward to my agent :). And thank you for all of your work over the years!
Best,
Pavel
(PS, I published a corresponding MiniZinc IDE revision with the configuration options built-in: https://github.com/pavelmk/MiniZincIDE )
[Machine-written below this point]
Add optional optimization APIs and adaptive native solving
Introduce
Gecode::Optimizewith explicit result guarantees, reusable optimization workflows and automatic selection of native algorithms.Racing may increase CPU work or solve time through exploration and restarting, but can identify a substantially better strategy for the remaining solve.
Performance measured on seeded instances with one thread and a ten-second solve budget against the project’s pre-algorithm runtime. Sizes are confirmed sampled results. Speedups use recorded median solve time, excluding startup; most improved timings have only one run per instance. These are observed combined-policy ratios, not guaranteed speedups or isolated evidence for racing.