broad_phase: the broad phase, and an overflow-free key hash

README.md

@@ -40,6 +40,7 @@ so a test written against the reference reads the same here.
 | `aephysics.compound` | the baked compound: capsules, hulls, meshes and spheres in one block under a static tree, hulls and meshes shared by content, materials by value; bounds, overlap, ray and shape casts, the box query, the mover's planes | done, `test_compound.ae` (159 checks); [same results as the reference, build 0.7x, queries 1.3-1.6x](bench/RESULTS.md#compound) |
 | `aephysics.shape` | the shape of any kind (sphere, capsule, hull, mesh, height field, compound) with the dispatch over every kind under a transform: bounds, swept and fat bounds, centroid, areas, mass, extent, ray and shape casts, overlap, the mover's planes, the proxy; the collision filters | done, `test_shape.ae` (440 checks); [same results as the reference, rays and masses at parity, radius casts 2.5x](bench/RESULTS.md#shape) |
 | `aephysics.mover` | the character mover's plane solver: pushes accumulated and clamped over twenty sweeps, the velocity clip | done, `test_mover.ae` (56 checks); [same results as the reference, 0.9x its time](bench/RESULTS.md#mover) |
+| `aephysics.broad_phase` | the broad phase: a tree per body type, proxies keyed by type, the pair update through the moved siblings and cross-tree seeds with the filter and compound lookups as visitors, the pair set, the keys sorted | done, `test_broad_phase.ae` (36 checks against a brute force); [10,000 moving boxes at 4.6 ms a step](bench/RESULTS.md#broad_phase) |
 | `aephysics.dynamics` | bodies, contacts, the constraint graph, islands, the Soft Step solver, joints (spherical, revolute, prismatic, distance, motor, weld, wheel), sensors, the character mover, the world | |
 | `aephysics` | the public API | |
 

aephysics/core/module.ae

@@ -565,14 +565,24 @@ const MAX_CHILD_SHAPES = 1048576
 // integers and xor alone collides. The murmur3 finaliser, with logical
 // shifts written out. The low 32 bits are the hash; zero is the empty
 // slot's mark, so a zero hash is nudged to one.
+// A 32-bit mix, overflow-free: the 27-bit constant 0x45d9f3b keeps every
+// product under 2^59 (a signed long overflow is undefined in the C
+// underneath, and gcc at -O2 made two inlined copies of a 64-bit mixer
+// disagree, so a key stored by one was not found by the other).
+mix32(value: long) -> long {
+    x = value & (4294967295 as long)
+    x = ((x ^ lsr(x, 16)) * (73244475 as long)) & (4294967295 as long)
+    x = ((x ^ lsr(x, 16)) * (73244475 as long)) & (4294967295 as long)
+    return x ^ lsr(x, 16)
+}
+
+// A 32-bit hash of a long, never zero (zero marks an empty slot): the
+// low half mixed, the high half folded in and mixed again, so a pair of
+// small ints packed in the two halves hashes by both and by their order.
 key_hash(key: long) -> int {
-    h = key
-    h = h ^ lsr(h, 33)
-    h = h * (0 - 48043033244054079 as long)     // 0xff51afd7ed558ccd
-    h = h ^ lsr(h, 33)
-    h = h * (0 - 4265267296055464877 as long)   // 0xc4ceb9fe1a85ec53
-    h = h ^ lsr(h, 33)
-    hash = (h & 4294967295) as int
+    x = mix32(key)
+    x = mix32(x ^ (lsr(key, 32) & (4294967295 as long)))
+    hash = x as int
     if hash == 0 { hash = 1 }
     return hash
 }

bench/RESULTS.md

@@ -201,8 +201,8 @@ binned SAH, edges identified each time; 100,000 rays cast down onto it;
 
 | phase | aephysics | Box3D |
 |---|---|---|
-| 10 builds, median split (80,000 triangles) | 199 ms | **129** |
-| 10 builds, binned SAH | 316 | **256** |
+| 10 builds, median split (80,000 triangles) | 221 ms | **129** |
+| 10 builds, binned SAH | 335 | **256** |
 | 100,000 ray casts | 20.8 | **10.2** |
 | 100,000 box queries | 70.4 | **39.5** |
 | 10,000 shape casts | 24.7 | **14.3** |
@@ -322,3 +322,32 @@ velocities, 1,662,623 iterations here against 1,662,624 there (one
 convergence test on the float's side of the slop). The solver runs
 at 0.9x: the reference reads its planes through a pointer per pass
 where the loop here indexes the array.
+
+## broad_phase
+
+`bench/broad_phase.ae`: 10,000 dynamic unit boxes on a jittered 25 x 16
+x 25 grid at 1.4 spacing over a static ground, the first update finding
+every pair, then 20 steps in which every box drifts a little and only
+the new pairs come out, each update's keys adopted into the pair set.
+The reference finds its pairs inside its world (broad_phase.c filters
+through the shapes and creates the contacts itself), so it has no
+free-standing counterpart; its pair update is measured against ours
+through the world benchmarks once the world steps.
+
+| phase | aephysics |
+|---|---|
+| 10,000 proxies created | 3.5 ms |
+| first update (2,900 pairs) | 2.2 |
+| 20 steps of 10,000 moves and an update (757 new pairs) | 91 |
+
+A step is 4.6 ms, most of it the 10,000 proxy moves (a leaf removed and
+re-inserted each); the update itself walks only the sibling pairs a
+moved node touched. The test checks every update against a brute force
+over the boxes.
+
+The hash under the pair set and every map (core's `key_hash`) was
+rewritten during this layer: the previous 64-bit mixer multiplied
+signed longs, which is undefined in the C underneath, and gcc at -O2
+made two inlined copies of it disagree, so a key stored by one copy was
+not found by the other. The new mixer stays in 32-bit products; the
+mesh builds above moved from 199 to 221 ms and 316 to 335 ms with it.

bench/broad_phase.ae

@@ -0,0 +1,96 @@
+// The broad phase on its own: 10,000 dynamic unit boxes on a jittered
+// 25 x 16 x 25 grid at 1.4 spacing over a static ground (so a quarter
+// of the neighbours touch), the first update finding every pair, then
+// 20 steps in which every box drifts a little and only the new pairs
+// come out, each update's keys adopted into the pair set. The reference
+// finds its pairs inside its world (broad_phase.c calls into shapes and
+// contacts), so it has no free-standing counterpart here; its pair
+// update is measured against ours through the world benchmarks. Single
+// thread, wall time per phase, with the pair counts as the checksum.
+import std.string
+import std.os
+import aephysics.math
+import aephysics.core
+import aephysics.dynamic_tree
+import aephysics.broad_phase
+
+extern calloc(count: int, size: int) -> ptr
+extern free(p: ptr)
+extern sin(x: float) -> float
+extern cos(x: float) -> float
+
+clock() -> long { return os.now_monotonic_ns() }
+ms(ns: long) -> float { return (ns as float) / 1000000.0 }
+
+const COUNT = 10000
+const STEPS = 20
+
+var g_bodies: ptr = null
+
+should_pair(a: int, b: int, context: ptr) -> bool { return a != b }
+no_compound(shape: int, context: ptr) -> ptr { return null }
+identity(shape: int, context: ptr) -> Transform { return math.transform_identity() }
+
+all_bits() -> long { return (0 as long) - (1 as long) }
+
+adopt(bp: *BroadPhase) {
+    n = broad_phase.pair_key_count(bp)
+    i = 0
+    while i < n {
+        broad_phase.add_pair(bp, broad_phase.pair_key(bp, i))
+        i = i + 1
+    }
+}
+
+main() {
+    bp_block = calloc(1, sizeof(BroadPhase))
+    bp = bp_block as *BroadPhase
+    broad_phase.create_broad_phase(bp, Capacity { static_shape_count: 16, dynamic_shape_count: COUNT, contact_count: 4 * COUNT })
+    boxes_block = calloc(COUNT + 1, sizeof(AABB))
+    boxes = boxes_block as AABB[]
+    keys_block = calloc(COUNT + 1, 4)
+    keys = keys_block as int[]
+    visitors = PairVisitors { should_pair: should_pair, compound_tree_of: no_compound, compound_transform: identity, context: null }
+
+    t0 = clock()
+    i = 0
+    while i < COUNT {
+        x = (i % 25) as float
+        y = ((i / 25) % 16) as float
+        z = (i / 400) as float
+        jitter = 0.3 * sin(7.0 * (i as float))
+        c = math.vec3(1.4 * x + jitter, 1.0 + 1.4 * y, 1.4 * z + 0.3 * cos(11.0 * (i as float)))
+        boxes[i] = AABB { lower: math.sub(c, math.vec3(0.5, 0.5, 0.5)), upper: math.add(c, math.vec3(0.5, 0.5, 0.5)) }
+        keys[i] = broad_phase.broad_phase_create_proxy(bp, broad_phase.BODY_DYNAMIC, boxes[i], all_bits(), i, false)
+        i = i + 1
+    }
+    boxes[COUNT] = AABB { lower: math.vec3(0.0 - 5.0, 0.0 - 1.0, 0.0 - 5.0), upper: math.vec3(40.0, 0.6, 40.0) }
+    keys[COUNT] = broad_phase.broad_phase_create_proxy(bp, broad_phase.BODY_STATIC, boxes[COUNT], all_bits(), COUNT, false)
+    t1 = clock()
+    first = broad_phase.update_broad_phase_pairs(bp, visitors, false)
+    adopt(bp)
+    t2 = clock()
+
+    found = 0
+    step = 0
+    while step < STEPS {
+        i = 0
+        while i < COUNT {
+            d = math.vec3(0.05 * sin((i + step) as float), 0.05 * cos((3 * i + step) as float), 0.05 * sin((5 * i + 2 * step) as float))
+            boxes[i] = AABB { lower: math.add(boxes[i].lower, d), upper: math.add(boxes[i].upper, d) }
+            broad_phase.broad_phase_move_proxy(bp, keys[i], boxes[i])
+            i = i + 1
+        }
+        n = broad_phase.update_broad_phase_pairs(bp, visitors, false)
+        found = found + n
+        adopt(bp)
+        step = step + 1
+    }
+    t3 = clock()
+
+    println("aephysics broad_phase: ${COUNT} proxies created ${ms(t1 - t0)} ms, first update ${ms(t2 - t1)} ms (${first} pairs), ${STEPS} steps of moves and updates ${ms(t3 - t2)} ms (${found} new pairs, ${broad_phase.pair_count(bp)} in the set)")
+    broad_phase.destroy_broad_phase(bp)
+    free(keys_block)
+    free(boxes_block)
+    free(bp_block)
+}

design.md

@@ -140,13 +140,16 @@ started until its tests pass.
      (the point culling and the per-cluster reduction are pure; the
      triangle cache it refreshes is the contact's, so the entry point
      takes the cache as a struct).
-   - `aephysics.broad_phase`: broad_phase.c's trees per body type,
-     proxies keyed by type in the low bits, the moved-sibling gathering,
-     the self and cross pair walks and the pair set; the pair filter and
-     the pair emission are visitors, since the reference does its shape
-     filtering and contact creation inside. Own test: pairs found and
-     not found across moves, a compound's children, the pair set's
-     persistence.
+   - `aephysics.broad_phase` (done): broad_phase.c's trees per body
+     type, proxies keyed by type in the low bits, the moved-sibling
+     gathering, the self and cross pair walks and the pair set; the pair
+     filter and the compound lookups are visitors, and the update leaves
+     sorted keys for the client to turn into contacts. 36 checks against
+     a brute force: pairs found and not found across moves, the filter,
+     a forced static proxy, a destroyed proxy, a compound's children.
+     Found and fixed on the way: core's key_hash multiplied signed
+     longs (undefined in C; gcc at -O2 made two inlined copies disagree)
+     -- it now mixes in 32-bit products.
    - `aephysics.dynamics`: one module for the world's state and its
      bookkeeping -- the World with its arrays (bodies, shapes, contacts,
      joints, islands, solver sets), the ids with generations, the

scripts/bench.sh

@@ -10,10 +10,13 @@ cd "$root"
 mkdir -p target
 layers="${1:-tree}"
 for layer in $layers; do
-    gcc -O3 -Ireference/box3d/include "bench/${layer}_box3d.c" -o "target/${layer}_box3d" -Lreference/box3d/out/src -lbox3d -lm || exit 1
+    # A layer the reference only has inside its world (the broad phase) runs ours alone.
+    if [ -f "bench/${layer}_box3d.c" ]; then
+        gcc -O3 -Ireference/box3d/include "bench/${layer}_box3d.c" -o "target/${layer}_box3d" -Lreference/box3d/out/src -lbox3d -lm || exit 1
+        ref="target/${layer}_box3d"; [ -x "$ref" ] || ref="$ref.exe"
+        "$ref"
+    fi
     AETHER_LIB_DIR="$root" ae build "bench/$layer.ae" -o "target/$layer" >"target/$layer.log" 2>&1 || { cat "target/$layer.log"; exit 1; }
-    ref="target/${layer}_box3d"; [ -x "$ref" ] || ref="$ref.exe"
     ours="target/$layer"; [ -x "$ours" ] || ours="$ours.exe"
-    "$ref"
     "$ours"
 done