Develop

.github/workflows/ci.yml

@@ -0,0 +1,78 @@
+name: CI
+
+on:
+  push:
+    branches: ["main", "develop", "workspace-esparso"]
+  pull_request:
+    branches: ["main", "develop"]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          # Núcleo puro: sem dissmodel nem nenhum extra geoespacial.
+          # Garante que engine.py, disk/workspace.py, disk/convergence.py
+          # continuam com zero dependências em runtime.
+          - name: "core (sem extras)"
+            extras: "dev"
+          # dissmodel + todos os extras geoespaciais opcionais que não
+          # dependem de repositórios externos ainda em construção
+          # (brmangue-dissmodel fica de fora de propósito).
+          - name: "full (dissmodel + geo extras)"
+            extras: "dev,dissmodel,geotiff,zarr,zarr-test,geomosaic,dissmodel-ca"
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install package (${{ matrix.name }})
+        run: pip install -e ".[${{ matrix.extras }}]"
+
+      - name: Run test suite
+        run: pytest -v
+
+      - name: Extract and smoke-test Python blocks from docs/
+        if: matrix.name == 'full (dissmodel + geo extras)'
+        run: |
+          python - <<'EOF'
+          import re, subprocess, sys, pathlib
+
+          docs = pathlib.Path("docs").glob("*.md")
+          failures = []
+          # Palavras que indicam um bloco ilustrativo (referencia dado
+          # externo que não existe neste repo, ou escala grande demais
+          # pra rodar em CI) -- pulamos, não são candidatos a bug de
+          # import/herança, que é o que este smoke-test mira.
+          SKIP_MARKERS = ("50_000", "study_area.tif", "tile_001.zarr", "/tmp/my_large_workspace")
+
+          for doc in docs:
+              text = doc.read_text(encoding="utf-8")
+              blocks = re.findall(r"```python\n(.*?)```", text, re.DOTALL)
+              for i, block in enumerate(blocks):
+                  if any(marker in block for marker in SKIP_MARKERS):
+                      continue
+                  if "import" not in block:
+                      continue
+                  print(f"--- {doc.name} bloco {i} ---")
+                  result = subprocess.run(
+                      [sys.executable, "-c", block],
+                      capture_output=True, text=True, timeout=120,
+                  )
+                  if result.returncode != 0:
+                      failures.append((doc.name, i, result.stderr))
+                      print(result.stderr)
+
+          if failures:
+              print(f"\n{len(failures)} bloco(s) de código quebrado(s) em docs/:")
+              for name, i, err in failures:
+                  print(f"  - {name} bloco {i}")
+              sys.exit(1)
+          print("Todos os blocos de código testáveis em docs/ rodaram sem erro.")
+          EOF

.gitignore

@@ -2,4 +2,5 @@ __pycache__/
 *.pyc
 .pytest_cache/
 *.egg-info/
-.venv
+.venv
+raster_map_frames/

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Sergio Costa, LambdaGeo / UFMA
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -6,6 +6,15 @@ Domínio** com **zonas de Halo** (Ghost Cell Pattern), integrado ao
 pip — sem viver dentro da pasta do repo `dissmodel` core, que está
 atualmente sob revisão JOSS (issue #10827).
 
+> [!NOTE]
+> **Complete English Documentation Available:**  
+> For the complete documentation in English covering API specifications, theory, architecture, and recipes, see the [`docs/`](docs/) directory:
+> - [**Documentation Index**](docs/README.md)
+> - [**Theory and Core Concepts**](docs/theory_and_concepts.md)
+> - [**Architecture & Design Patterns**](docs/architecture.md)
+> - [**API Reference**](docs/api_reference.md)
+> - [**Tutorials & Practical Recipes**](docs/tutorials_and_recipes.md)
+
 ## Motivação
 
 O padrão bloco+halo usado no `chunked_engine.py` (BR-MANGUE,
@@ -93,6 +102,17 @@ pytest
 
 Depende de `dissmodel>=0.6.3` (PyPI) como dependência real de runtime.
 
+> **Nota:** `examples/brmangue_validation/` (scripts, testes de
+> equivalência e fixtures que validavam contra o dataset real da Ilha
+> do Maranhão e o golden do TerraME) foi removido desta branch — o
+> modelo BR-MANGUE (`brmangue-dissmodel`) ainda está em construção
+> ativa, e o exemplo dependia de uma versão específica dele. Os
+> "Achados" abaixo permanecem documentados porque continuam válidos
+> tecnicamente (o bug e a causa raiz não mudam), mas os caminhos de
+> arquivo que citam (`examples/brmangue_validation/...`) não existem
+> mais neste repositório — servem como registro histórico da
+> investigação, não como referência executável.
+
 ## Achado: `boundary_value=0` não é seguro para todo domínio
 
 Ao validar contra `MangroveModel` (não só `FloodModel`), um teste

docs/README.md

@@ -0,0 +1,154 @@
+# `haloexec` Documentation
+
+Welcome to the comprehensive documentation for **`haloexec`**, a high-performance execution engine for large-scale geospatial Cellular Automata (CA) and spatial models using **Domain Decomposition** with **Halo Zones** (Ghost Cell Pattern).
+
+---
+
+## Documentation Navigation
+
+This documentation is organized into modular guides covering foundational theory, software architecture, complete API specifications, and end-to-end practical tutorials:
+
+1. [**Theory and Core Concepts**](./theory_and_concepts.md)  
+   *Detailed mathematical, algorithmic, and computational foundations.*  
+   Domain decomposition, ghost cell exchange, multi-hop spatial dependency reach, external boundary sentinel pitfalls, virtual memory and POSIX sparse disk memmaps, double-buffering race conditions, iterative Gauss-Seidel convergence, and Zarr axis-ordering anomalies.
+
+2. [**Architecture and Design Patterns**](./architecture.md)  
+   *Software design and structural mechanics.*  
+   Layered decoupling (zero-dependency core engine vs. ecosystem adapters), cooperative multiple inheritance via Python's Method Resolution Order (MRO), runtime backend-swapping pattern, and memory-mapped double-buffer slot lifecycle.
+
+3. [**API Reference**](./api_reference.md)  
+   *Exhaustive interface specification for all modules, classes, and functions.*  
+   Signatures, parameter descriptions, invariants, return types, exceptions, and side effects across RAM, disk, I/O, convergence, and visualization subsystems.
+
+4. [**Tutorials and Practical Recipes**](./tutorials_and_recipes.md)  
+   *Step-by-step implementation walkthroughs.*  
+   Building in-memory chunked CAs, executing out-of-core billion-cell models, windowed GeoTIFF/VRT ingestion, multi-tile Zarr assimilation, unbounded connectivity propagation, and memory profiling (`RssAnon` vs. `RssFile`).
+
+---
+
+## Overview & Mission
+
+In geospatial modeling and complex systems simulation (e.g., land-use and land-cover change [LUCC], tidal hydrology, mangrove migration, wildland fire propagation), spatial grids frequently exceed available physical RAM. For example, high-resolution national or regional grids can encompass tens or hundreds of millions—even billions—of cells.
+
+Traditional execution engines process such grids **monolithically**: the entire spatial array is held in RAM, and neighborhood operations (e.g., focal sums, directional shifts) are computed globally. When the domain cannot fit into memory, the simulation fails with an out-of-memory (OOM) error or suffers thrashing due to unmanaged swap paging.
+
+`haloexec` solves this challenge by providing:
+- **Zero-Cognitive-Overhead Integration**: Maintains 100% contract parity with base models (such as `dissmodel`'s `RasterCellularAutomaton` and `SyncRasterModel`). Scientists do not rewrite their transition rules or equations.
+- **Strict Domain Decomposition**: Partitions arbitrary 2D rasters into manageable computational blocks.
+- **Ghost Cell Pattern (Halo Padding)**: Seamlessly stitches local block boundaries so that neighborhood operations across block partitions evaluate identically to monolithic runs.
+- **Out-of-Core Disk Execution**: A zero-dependency disk workspace built on memory-mapped files (`numpy.memmap`) and double-buffering, allowing billion-cell simulations on standard hardware.
+- **Iterative Convergence Orchestration**: A block-wise Gauss-Seidel engine for unbounded spatial dependencies (e.g., hydrological connectivity and flow routing) that cannot be resolved with fixed-depth halos.
+
+---
+
+## Architectural Map
+
+```mermaid
+graph TD
+    subgraph Storage & Ingestion Tier
+        TIFF[GeoTIFF / VRT] -->|load_geotiff_into_workspace| WS[(MemmapRasterWorkspace)]
+        ZARR[Zarr Store / Tiles] -->|load_zarr_into_workspace| WS
+    end
+
+    subgraph Core Partitioning Engine
+        WS -->|make_blocks| BLK[Block Partitioning]
+        BLK -->|halo_window| WIN[Halo Window Computation]
+    end
+
+    subgraph Execution Tiers
+        subgraph In-Memory RAM Tier
+            RAM_CA[HaloChunkedRasterCellularAutomaton]
+            RAM_SYNC[HaloChunkedSyncRasterModel]
+        end
+
+        subgraph Out-of-Core Disk Tier
+            DISK_CA[DiskChunkedRasterCellularAutomaton]
+            DISK_SYNC[DiskChunkedSyncRasterModel]
+            CONV[sweep_until_convergence]
+        end
+    end
+
+    subgraph Visualization Tier
+        WS -->|as_backend| BACKEND[WorkspaceRasterBackend]
+        BACKEND --> CRMAP[CheckpointRasterMap]
+    end
+
+    WIN --> RAM_CA
+    WIN --> RAM_SYNC
+    WIN --> DISK_CA
+    WIN --> DISK_SYNC
+    WIN --> CONV
+```
+
+---
+
+## Quickstart
+
+### 1. In-Memory Chunked Simulation
+
+When the domain fits into RAM, but you want to enforce block-wise processing or prepare for scaling:
+
+```python
+import numpy as np
+from dissmodel.core import Environment
+from dissmodel.geo.raster.backend import RasterBackend
+from haloexec import HaloChunkedRasterCellularAutomaton
+
+class GameOfLife(HaloChunkedRasterCellularAutomaton):
+    def rule(self, arrays: dict[str, np.ndarray]) -> dict[str, np.ndarray]:
+        state = arrays["state"]
+        neighbors = self.backend.focal_sum_mask(state == 1)
+        born = (state == 0) & (neighbors == 3)
+        survive = (state == 1) & np.isin(neighbors, [2, 3])
+        return {"state": np.where(born | survive, 1, 0).astype(np.uint8)}
+
+# Allocate global domain
+backend = RasterBackend(shape=(2000, 2000))
+backend.set("state", np.random.randint(0, 2, (2000, 2000), dtype=np.uint8))
+
+env = Environment(start_time=1, end_time=20)
+GameOfLife(backend=backend, block_h=500, block_w=500, halo=1)
+env.run()
+```
+
+### 2. Out-of-Core Disk-Backed Simulation
+
+When the domain exceeds physical memory, initialize a [`MemmapRasterWorkspace`](../src/haloexec/disk/workspace.py#L81) and run out-of-core:
+
+```python
+from pathlib import Path
+import numpy as np
+from dissmodel.core import Environment
+from dissmodel.geo.raster.cellular_automaton import RasterCellularAutomaton
+from haloexec import (
+    MemmapRasterWorkspace,
+    DiskChunkedRasterCellularAutomaton,
+)
+
+# DiskChunkedRasterCellularAutomaton é um mixin cooperativo: precisa vir
+# primeiro no MRO, com uma classe real do dissmodel (aqui,
+# RasterCellularAutomaton) como segunda base — do contrário setup()
+# não tem para onde delegar via super() e a classe falha ao instanciar.
+class LargeScaleGameOfLife(DiskChunkedRasterCellularAutomaton, RasterCellularAutomaton):
+    def rule(self, arrays: dict[str, np.ndarray]) -> dict[str, np.ndarray]:
+        state = arrays["state"]
+        neighbors = self.backend.focal_sum_mask(state == 1)
+        born = (state == 0) & (neighbors == 3)
+        survive = (state == 1) & np.isin(neighbors, [2, 3])
+        return {"state": np.where(born | survive, 1, 0).astype(np.uint8)}
+
+# Allocate out-of-core workspace (50,000 x 50,000 = 2.5 Billion cells)
+ws = MemmapRasterWorkspace.create(
+    root=Path("/tmp/sim_workspace"),
+    shape=(50_000, 50_000),
+    arrays={"state": np.uint8},
+    block_h=1024,
+    block_w=1024,
+    halo=1,
+)
+
+env = Environment(start_time=1, end_time=5)
+LargeScaleGameOfLife(workspace=ws)
+env.run()
+ws.flush()
+```