feat: mix full rewrites into diff-based evolution, plus trace fixes (v0.4.1)

configs/default_config.yaml

@@ -11,6 +11,8 @@ random_seed: 42                       # Random seed for reproducibility (null =
 
 # Evolution settings
 diff_based_evolution: true            # Use diff-based evolution (true) or full rewrites (false)
+full_rewrite_probability: 0.0         # With diffs on, chance (0-1) an iteration asks for a full rewrite instead
+                                      # (the mode used is saved as metadata["evolution_mode"] on each program)
 max_code_length: 10000                # Maximum allowed code length in characters
 enforce_evolve_blocks: false          # Revert LLM edits outside EVOLVE-BLOCK-START/END markers
 
@@ -181,6 +183,9 @@ evolution_trace:
   format: 'jsonl'                     # Output format: 'jsonl', 'json', or 'hdf5'
   include_code: false                 # Include full program code in traces
   include_prompts: true               # Include prompts and LLM responses
+  include_failures: false             # Also log failed iterations (error, mode, and with
+                                      # include_prompts the prompt and response) to
+                                      # <trace name>_failures.jsonl (.jsonl.gz with compress)
   output_path: null                   # Path for trace output (defaults to output_dir/evolution_trace.{format})
   buffer_size: 10                     # Number of traces to buffer before writing
   compress: false                     # Compress output file (jsonl only)

docs/configuration.md

@@ -0,0 +1,257 @@
+# Configuration
+
+Every option, with its default, is in [`configs/default_config.yaml`](../configs/default_config.yaml). This page covers the ones people most often need.
+
+```yaml
+# Advanced Configuration Example
+max_iterations: 1000
+random_seed: 42  # Full reproducibility
+
+# Optional prefix for all logger names, e.g. "run1" logs as "run1.openevolve.*";
+# useful to tell apart log output from multiple concurrent runs
+logger_prefix: "run1"
+
+llm:
+  # Ensemble configuration
+  models:
+    - name: "gemini-2.5-pro"
+      weight: 0.6
+    - name: "gemini-2.5-flash"
+      weight: 0.4
+  temperature: 0.7
+
+database:
+  # MAP-Elites quality-diversity
+  population_size: 500
+  num_islands: 5  # Parallel evolution
+  # island_initial_programs: [null, "seeds/alt.py"]  # Optional per-island starting programs
+  migration_interval: 20
+  feature_dimensions: ["complexity", "diversity", "performance"]
+
+  # Optional novelty filtering with Gemini embeddings
+  embedding_model: "gemini-embedding-001"
+  similarity_threshold: 0.99
+  # Or any OpenAI-compatible embedding endpoint (OpenRouter, local servers):
+  # embedding_api_base: "https://openrouter.ai/api/v1"  # or OPENAI_EMBEDDING_BASE_URL
+
+evaluator:
+  enable_artifacts: true      # Error feedback to LLM
+  cascade_evaluation: true    # Multi-stage testing
+  use_llm_feedback: true      # AI code quality assessment
+  evaluator_args:             # Keyword arguments forwarded to evaluate()
+    weight: 2.0
+
+prompt:
+  # Sophisticated inspiration system
+  num_top_programs: 3         # Best performers
+  num_diverse_programs: 2     # Creative exploration
+  include_artifacts: true     # Execution feedback
+
+  # Custom templates
+  template_dir: "custom_prompts/"
+  use_template_stochasticity: true  # Randomized prompts
+```
+
+For Gemini embeddings, set `GEMINI_API_KEY`. `GOOGLE_API_KEY` is also supported
+as a fallback. OpenEvolve uses Google's OpenAI-compatible endpoint automatically.
+
+## Command-Line Options
+
+```bash
+openevolve-run initial_program.py evaluator.py \
+  --config config.yaml \
+  --iterations 100 \
+  --target-score 0.95 \
+  --checkpoint path/to/checkpoint_directory \
+  --logger-prefix run1
+```
+
+`--target-score` stops the run once `combined_score` reaches the value. `--checkpoint` resumes from a saved run. `--logger-prefix` keeps log output distinguishable across parallel runs (it logs as `run1.openevolve.*`).
+
+## Evolution Modes
+
+```yaml
+diff_based_evolution: true      # SEARCH/REPLACE diffs; false asks for a full rewrite every iteration
+full_rewrite_probability: 0.2    # With diffs on, chance an iteration asks for a full rewrite instead
+enforce_evolve_blocks: true      # Revert edits outside the EVOLVE-BLOCK-START/END markers
+
+evolution_trace:
+  enabled: true
+  include_failures: true         # Also log failed iterations to <trace name>_failures.jsonl
+```
+
+## Feature Engineering
+
+**Control how programs are organized in the quality-diversity grid:**
+
+```yaml
+database:
+  feature_dimensions: 
+    - "complexity"      # Built-in: code length
+    - "diversity"       # Built-in: structural diversity
+    - "performance"     # Custom: from your evaluator
+    - "memory_usage"    # Custom: from your evaluator
+
+  feature_bins:
+    complexity: 10      # 10 complexity levels
+    performance: 20     # 20 performance buckets
+    memory_usage: 15    # 15 memory usage categories
+```
+
+**Important**: Return raw values from evaluator, OpenEvolve handles binning automatically.
+
+## Passing Arguments to Your Evaluator
+
+If your `evaluate()` function takes extra keyword arguments beyond the program
+path, pass them with the `--evaluator-args` CLI flag (a JSON object):
+
+```bash
+openevolve-run initial_program.py evaluator.py \
+  --evaluator-args '{"weight": 2.0, "label": "experiment-1"}'
+```
+
+or configure them in YAML:
+
+```yaml
+evaluator:
+  evaluator_args:
+    weight: 2.0
+    label: "experiment-1"
+```
+
+OpenEvolve inspects the signature of your `evaluate()` function: arguments are
+forwarded only when the function accepts them (named parameters or `**kwargs`),
+and a startup error lists the offending keys and the actual signature if it
+does not — so typos fail fast instead of silently doing nothing. Evaluators
+with a single `program_path` argument are unaffected.
+
+This replaces the earlier workaround of smuggling per-run settings to the
+evaluator through environment variables. (Note: custom top-level keys in
+`config.yaml` are dropped silently by the config parser — use
+`evaluator.evaluator_args` or the CLI flag instead.)
+
+## Custom Prompt Templates
+
+**Advanced prompt engineering** with custom templates:
+
+```yaml
+prompt:
+  template_dir: "custom_templates/"
+  use_template_stochasticity: true
+  template_variations:
+    greeting:
+      - "Let's enhance this code:"
+      - "Time to optimize:"
+      - "Improving the algorithm:"
+    improvement_suggestion:
+      - "Here's how we could improve this code:"
+      - "I suggest the following improvements:"
+      - "We can enhance this code by:"
+```
+
+**How it works:** Place `{greeting}` or `{improvement_suggestion}` placeholders in your templates, and OpenEvolve will randomly choose from the variations for each generation, adding diversity to prompts.
+
+See [prompt examples](../examples/llm_prompt_optimization/templates/) for complete template customization.
+
+## Island Scheduling
+
+Pass an optional selector to `run_evolution`:
+
+```python
+def select_island(context):
+    # Schedule the next iteration on the island with the fewest pending tasks.
+    return min(range(len(context.islands)), key=lambda i: context.pending_counts[i])
+
+result = run_evolution(
+    initial_program="program.py",
+    evaluator="evaluator.py",
+    island_selector=select_island,
+)
+```
+
+The selector receives a read-only `IslandSelectionContext` with the iteration number,
+pending task counts, and each island's population size, best and average score, diversity, and
+generation. It must return an island ID from `0` to `num_islands - 1`. It runs in the
+main process before each iteration is submitted. Without a selector, OpenEvolve uses
+its existing balanced island scheduling.
+
+## Population Strategies
+
+Population management can use decision hooks. For example, this policy lets a
+candidate replace an island's MAP-Elites cell occupant when their scores tie:
+
+```python
+from openevolve import run_evolution
+from openevolve.population import PopulationStrategy
+
+strategy = PopulationStrategy(
+    replace_cell=lambda state, candidate, incumbent, island: (
+        candidate.metrics["combined_score"] >= incumbent.metrics["combined_score"]
+    ),
+)
+result = run_evolution("program.py", "evaluator.py", population_strategy=strategy)
+```
+
+Other optional hooks are `admit(state, candidate, island) -> bool`,
+`archive(state, candidate) -> ArchiveDecision`,
+`evict(state, count, protected_ids) -> sequence of program IDs`,
+`migration_due(state) -> bool`, and `migrate(state) -> sequence of MigrationMove`.
+`state` is a detached `PopulationSnapshot` of programs, island memberships, cell owners,
+the archive, population limits, and migration generations. Unspecified hooks retain
+the existing rules. `ProgramDatabase` validates decisions and applies all changes;
+the strategy never receives the mutable database. The initial seed cannot be rejected.
+
+## Artifacts
+
+**Artifacts side-channel** provides rich feedback to accelerate evolution:
+
+```python
+# Evaluator can return execution context
+from openevolve.evaluation_result import EvaluationResult
+
+return EvaluationResult(
+    metrics={"performance": 0.85, "correctness": 1.0},
+    artifacts={
+        "stderr": "Warning: suboptimal memory access pattern",
+        "profiling_data": {...},
+        "llm_feedback": "Code is correct but could use better variable names",
+        "build_warnings": ["unused variable x"]
+    }
+)
+```
+
+**Next generation prompt automatically includes:**
+
+```markdown
+## Previous Execution Feedback
+Warning: suboptimal memory access pattern
+LLM Feedback: Code is correct but could use better variable names
+Build Warnings: unused variable x
+```
+
+This creates a **feedback loop** where each generation learns from previous mistakes!
+
+## Visualization
+
+**Real-time evolution tracking** with interactive web interface:
+
+```bash
+# Install visualization dependencies
+pip install -r scripts/requirements.txt
+
+# Launch interactive visualizer
+python scripts/visualizer.py
+
+# Or visualize specific checkpoint
+python scripts/visualizer.py --path examples/function_minimization/openevolve_output/checkpoints/checkpoint_100/
+```
+
+**Features:**
+
+- **Evolution tree** with parent-child relationships
+- **Performance tracking** across generations
+- **Code diff viewer** showing mutations
+- **MAP-Elites grid** visualization
+- **Multi-metric analysis** with custom dimensions
+
+![OpenEvolve Visualizer](../openevolve-visualizer.png)

docs/faq.md

@@ -0,0 +1,55 @@
+# FAQ
+
+### How much does it cost to run?
+
+See [Cost Estimation](llm-providers.md#cost-estimation) for detailed pricing information and cost-saving tips.
+
+### 🆚 How does this compare to manual optimization?
+
+| Aspect | Manual | OpenEvolve |
+|--------|--------|------------|
+| **Initial Learning** | Weeks to understand domain | Minutes to start |
+| **Solution Quality** | Depends on expertise | Consistently explores novel approaches |
+| **Time Investment** | Days-weeks per optimization | Hours for complete evolution |
+| **Reproducibility** | Hard to replicate exact process | Perfect reproduction with seeds |
+| **Scaling** | Doesn't scale beyond human capacity | Parallel evolution across islands |
+
+**OpenEvolve shines** when you need to explore large solution spaces or optimize for multiple objectives simultaneously.
+
+### Can I use my own LLM?
+
+**Yes!** OpenEvolve supports any OpenAI-compatible API:
+
+- **Commercial**: OpenAI, Google, Cohere
+- **Local**: Ollama, vLLM, LM Studio, text-generation-webui
+- **Advanced**: OptiLLM for routing and test-time compute
+
+Just set the `api_base` in your config to point to your endpoint.
+
+### What if evolution gets stuck?
+
+**Built-in mechanisms prevent stagnation:**
+
+- **Island migration**: Fresh genes from other populations
+- **Temperature control**: Exploration vs exploitation balance
+- **Diversity maintenance**: MAP-Elites prevents convergence
+- **Artifact feedback**: Error messages guide improvements
+- **Template stochasticity**: Randomized prompts break patterns
+
+**Manual interventions:**
+- Increase `num_diverse_programs` for more exploration
+- Add custom feature dimensions to diversify search
+- Use template variations to randomize prompts
+- Adjust migration intervals for more cross-pollination
+
+### How do I measure success?
+
+**Multiple success metrics:**
+
+1. **Primary Metric**: Your evaluator's `combined_score` or metric average
+2. **Convergence**: Best score improvement over time
+3. **Diversity**: MAP-Elites grid coverage
+4. **Efficiency**: Iterations to reach target performance
+5. **Robustness**: Performance across different test cases
+
+**Use the visualizer** to track all metrics in real-time and identify when evolution has converged.

docs/how-it-works.md

@@ -0,0 +1,35 @@
+# How OpenEvolve Works
+
+OpenEvolve implements a sophisticated **evolutionary coding pipeline** that goes far beyond simple optimization:
+
+![OpenEvolve Architecture](../openevolve-architecture.png)
+
+### **Core Innovation**: MAP-Elites + LLMs
+
+- **Quality-Diversity Evolution**: Maintains diverse populations across feature dimensions
+- **Island-Based Architecture**: Multiple populations prevent premature convergence
+- **LLM Ensemble**: Multiple models with intelligent fallback strategies
+- **Artifact Side-Channel**: Error feedback improves subsequent generations
+
+## Features
+
+### Scientific Reproducibility
+
+- **Comprehensive Seeding**: Every component (LLM, database, evaluation) is seeded
+- **Default Seed=42**: Immediate reproducible results out of the box
+- **Deterministic Evolution**: Exact reproduction of runs across machines
+- **Component Isolation**: Hash-based isolation prevents cross-contamination
+
+### Advanced LLM Integration
+
+- **Universal API**: Works with OpenAI, Google, Claude Code CLI, local models, and proxies
+- **Intelligent Ensembles**: Weighted combinations with sophisticated fallback
+- **Test-Time Compute**: Enhanced reasoning through proxy systems (see [OptiLLM setup](llm-providers.md))
+- **Plugin Ecosystem**: Support for advanced reasoning plugins
+
+### Evolution Algorithm Innovations
+
+- **Double Selection**: Different programs for performance vs inspiration
+- **Adaptive Feature Dimensions**: Custom quality-diversity metrics
+- **Migration Patterns**: Ring topology with controlled gene flow
+- **Multi-Strategy Sampling**: Elite, diverse, and exploratory selection