Skip to content

The aligner is dead. Long live the aligner. - #739

Open
candytaco wants to merge 20 commits into
mainfrom
claude/awesome-wright-5bz9m3
Open

candytaco wants to merge 20 commits into
mainfrom
claude/awesome-wright-5bz9m3

Conversation

@candytaco

@candytaco candytaco commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

I had claude re-implement the mayavi aligner in webgl.

cortex.align.webgl_maual(<surface>, <transform>) gives a webgl viewer that can edit transforms.

Updated to show the data also on the surface in real-time as you move the transform.

Controls:
WASD or RMB to translate, QE to rotate, hold shift to move in 1/10th size increments. I've specifically removed the scaling/stretching options.

Aligner_.TZfs.aligner-test.-.Vivaldi.2026-09-19.13-28-07.webm

Edit histories are tracked and can be undone to any edit
Aligner_ TZfs aligner-test - Vivaldi 2026-09-19 21-51-39.webm

(I am very proud of the PR title, which, unlike the code, came solely from my brain)

Port of the mayavi aligner to the WebGL viewer: the functional reference
volume stays on its own voxel grid, the pial and white matter surfaces
are moved into its space by rotations and translations only, and they
are cut off at the displayed slices so that their outline shows on each
slice.

- cortex/webgl/aligner.py: tornado server (reference mosaic, CTM pack,
  page, save endpoint), the world frame (voxel grid in mm permuted to
  RAS) and the JSAligner handle with tagged calls and frame waits
- resources/js/aligner.js, aligner.html, resources/css/aligner.css:
  coronal, axial, sagittal and 3D views, cursor and slice navigation,
  mesh translation and rotation by mouse and keyboard with undo,
  colormap with range, brightness, contrast, gamma and flip, mesh color
  and opacity, a mode painting the volume onto the surface, saving
- shaderlib.js: aligner_volume and aligner_mesh shader builders
- menu.js: color picker controls
- cortex.align.webgl_manual, [webgl_aligner] config defaults, docs and
  tests

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
The browser aligner refused to open a transform that had cached masks.
It now opens, and deletes those masks when it saves: they were cut out
of the reference volume through the alignment being replaced, nothing
else invalidates them, and db.save_xfm refuses to write over a transform
that still has them.

- cortex/webgl/aligner.py: cached_masks and clear_masks, called from the
  save handler before db.save_xfm; the save response names what it
  deleted, and the page config lists the masks
- the page warns, on opening a transform that has masks, that saving
  deletes them and that data masked with them has to be masked again
- view_only stays an explicit choice rather than something masks force

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
The browser aligner replaces it, and it had not worked since mayavi
stopped running on anything newer than Ubuntu 14.04.

- drop cortex/mayavi_aligner.py, cortex.align.mayavi_manual and the
  [mayavi_aligner] config section
- drop the mayavi sections of docs/align.rst and their screenshots, and
  point docs/transforms.rst at the browser aligner
- the mayavi imports elsewhere (segmentation, surface display in
  freesurfer and database) are untouched, as are the mypy overrides they
  need

Also make the aligner tests redirect the transform's mask paths into a
directory of their own, so they neither depend on nor delete the masks
another test file leaves in the bundled filestore.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
- WASD moves the mesh in the view under the mouse, like the arrow keys,
  in both cases with shift for the fine step
- the colormap dropdown draws a strip of each colormap beside its name,
  through select2, the library the viewer's colormap picker uses; a css
  block in the shared template lets the aligner's stylesheet load last,
  since the viewer hides the select2 control it now needs
- a transform field above the save button names what the alignment is
  saved under, so it can be saved as a new transform, leaving the one it
  was loaded from and its masks alone; the name is checked before it
  reaches the filesystem
- an asterisk on the save button and in the title marks an alignment
  that differs from the one last saved
- cortex.align.webgl_manual spells out every option rather than taking
  **kwargs

menu.js keeps the controller of a button action, so its label can change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
A `layout` control chooses where the surface is shown. "4 panels" is the
layout as it was. "panels + surface" keeps the three slices showing their
planes and outlines and turns the corner the 3D view occupies into the
viewer, so the mesh can be nudged in the slices while the data on the
surface follows. "surface" gives that viewer the whole window, framed on
the surface the way the viewer opens on one.

Both carry the reference data through the alignment as it stands, saved
or not, and redraw as the mesh moves, so an alignment can be judged from
the pattern the data makes on the cortex before it is committed.

Also corrects the documented mouse buttons of the 3D view: its controls
are the viewer's, which pan on the middle button and zoom on the right.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
- replace the `view` and `layout` dropdowns with one `display` control:
  3 ortho + 3D slices, 3 ortho + 3D brain, or the data on the surface
- draw the data view's surfaces in the anatomy's own frame with the morph
  targets of the CTM pack, so they inflate and flatten there while the
  alignment only moves where the volume is sampled
- add `unfold` and `pivot`, and cull the medial wall while the flatmap shows
- take the keyboard back from the panel when the mouse enters a view, and
  start with a view under the keys
- keep the open colormap list within the window, which was scrolling the
  page sideways when it reached past the right edge

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
@candytaco
candytaco marked this pull request as ready for review September 19, 2026 20:59
Copilot AI lite review requested due to automatic review settings September 19, 2026 20:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The save path has unresolved authorization, data-loss, completion, and rendering edge cases.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity · 3 Medium severity

Open (4)

Comment thread cortex/webgl/aligner.py
Comment thread cortex/webgl/aligner.py Outdated
Comment thread cortex/webgl/aligner.py Outdated
Comment thread cortex/webgl/resources/js/shaderlib.js Outdated
@mvdoc

mvdoc commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

@claude review

@claude

claude Bot commented Sep 19, 2026

Copy link
Copy Markdown

Claude finished @mvdoc's task in 1m 52s —— View job


Review of #739 — the WebGL aligner

  • Gather context (PR diff, files, prior Copilot review comments)
  • Review cortex/webgl/aligner.py (new tornado server + save endpoint)
  • Review cortex/align.py changes (webgl_manual entry point)
  • Review cortex/webgl/resources/js/aligner.js and shaderlib.js changes
  • Review tests in cortex/tests/test_webgl_aligner.py
  • Post final review

This is a substantial, well-documented replacement of the mayavi aligner. The docstrings are thorough and the test suite (cortex/tests/test_webgl_aligner.py, 748 lines) is unusually complete, including headless-browser Playwright coverage. I independently verified the four issues Copilot's automated review already flagged — all four are real, confirmed against the current code, not artifacts of a stale diff:

1. /save has no auth/CSRF protection (cortex/webgl/aligner.py:511-539)
SaveHandler.post performs a destructive filesystem write (db.save_xfm) and deletion (clear_masks) with no token or Tornado XSRF check. WebApp (cortex/webgl/serve.py:334) binds via bind_sockets(port) with no address restriction, so this is reachable from any host that can route to the port, not just loopback. This is a shared risk with cortex.webgl.show, but that server is read-only — this is the first place the webgl server exposes a destructive write, so it's worth tightening (bind to 127.0.0.1 and/or require the per-session token that's already generated for RPC calls).

2. save() returns before the save actually completes (aligner.py:288-290, resources/js/aligner.js:998-1024)
JSAligner.save()'s docstring says it saves "like the Save button," but under the hood Aligner.prototype.save in JS fires an async $.ajax POST and returns the literal string "saving" immediately — success/failure is only reported later via showStatus(), which the Python side never sees. Confirmed by the test itself: test_aligner_in_headless_browser (test_webgl_aligner.py:506-511) has to separately poll recorder.wait(1) after calling handle.save(), rather than trusting the return value — i.e., the test works around the gap instead of exercising a contract that holds. A caller who calls .save() and then tears down the server immediately can lose the write silently.

3. get_client() / get_local_client() don't attach server (aligner.py:544-550)
The open_browser=True path manually does object.__setattr__(client, "server", server) (aligner.py:569) after calling get_client(), but get_client()/get_local_client() themselves don't do this. So the documented open_browser=False flow (server = webgl_manual(...); client = server.get_client()) produces a JSAligner whose _call sees server is None and silently falls back to WebApp.send's ~2s timeout instead of the intended 120s tagged-reply loop (aligner.py:200-232). This is masked in every current test by manually patching it in (e.g. test_webgl_aligner.py:455: object.__setattr__(handle, "server", server)), which means the "normal" documented path is untested. Worth setting self as server inside get_client/get_local_client directly.

4. Unguarded vmin == vmax division in aligner_volume shader (resources/js/shaderlib.js:1015)
float norm = (value - vmin) / (vmax - vmin); has no zero-denominator guard. show() in aligner.py:422-425 guards the initial value, but the colormap min/max sliders in the UI can still be moved to equal values interactively, producing NaN/Inf sampling. Note this mirrors an existing pattern elsewhere in shaderlib.js (e.g. lines 20-21), so it's not a new class of bug introduced by this PR, but it's worth a max(vmax - vmin, epsilon) guard in the new shader since it's easy to trigger from the aligner's own UI.

Everything else — the coordinate-frame math (reference_frame, world/coord conventions), mask invalidation on save, the transform-name validation (XFM_NAME regex blocking path traversal), and the CTM/mesh handling — looked correct and carefully commented. No functional issues found there.

No code changes were made — this was a review-only request, per the instructions ("review" → provide feedback, don't implement).
(branch: claude/awesome-wright-5bz9m3)

- WebApp binds the loopback interface unless a caller asks otherwise; the
  viewer opens up to every interface only when domain_name names one
- the aligner's save endpoint takes only posts carrying the token of the
  page it served, so nothing else reaching the port can write a transform
- JSAligner.save waits for the save to land and returns the server's
  message, raising when it was refused
- get_client hands the handle its server, which the documented
  open_browser=False flow needs for the tagged-reply loop
- guard a zero vmin/vmax span in the aligner shader
- drop get_local_client, which called an attribute WebApp does not have
- add an edit history to the panel: a row per edit since load, with how far
  it moved and turned the surfaces; clicking one puts that alignment back
- confirm closing the page while the alignment differs from the saved one

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
bind_local_sockets binds one socket per address the names in serve.LOCAL
resolve to, all on one port: localhost, 127.0.0.1 and whatever this
computer's hostname stands for. A name that does not resolve is passed
over, and one that resolves to an address another name already covers is
bound once, so a machine whose hostname is its loopback address still gets
a server.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
- the link the servers print names this computer, which is what a port
  forward from another machine is set up under; a machine whose name leads
  nowhere falls back to localhost
- the history reads newest first
- a translation says where it took the surfaces along the anatomical axes
  ("2.00 mm left, 3.00 mm anterior"), a rotation how far and which way
  round it turned them in the plane it was made in ("0.20 CW in coronal")

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
Every handler, the websocket and the static files included, is wrapped in
serve.TokenGuard: the address the server prints carries the token, the page
it opens is handed it back as a per-port cookie, and a request with neither
gets a 403. Build addresses with WebApp.url, which carries it; pass
token="" to answer anything that reaches the port.

The aligner's save token is now posted as save_token, so that it and the
session token cannot stand in for each other.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
@candytaco

Copy link
Copy Markdown
Contributor Author

This now adds a history panel so you can undo to any edit in the edit history

claude and others added 9 commits September 20, 2026 05:06
The picker sizes its parts by their content and lays them out to add up to
its own width. The panel sits inside a w2ui one, whose reset counts border
and padding within every width instead, which left the hue bar too wide to
fit beside the saturation square and dropped it below the picker. The
picker shows only while the pointer is over it, so the bar could not be
reached: the picker closed on the way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
The corner of the four panels and the data view now draw the same surface,
in the anatomy's own frame, so unfold, pivot and depth carry from one to
the other and act on both. r, i and f jump to the folded surface, the
inflated one and the flatmap, as they do in the viewer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
sliceplanes > ortho_views (or the v key) splits the canvas between the
three slice planes, each drawn straight down its own axis by a camera of
its own, and the 3D view, which keeps the last quarter, the camera the
controls move and the surface as it has always been drawn. The keys that
step through the slices act on the views they are showing. Data with no
volume to slice, such as data on the vertices, leaves the surface in view
instead, seen down each of the three axes.

A view in jsplot.Axes3D can now carry a camera, a prepare callback and a
surface index of its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
A pick in the 3D view takes the three slice views to the slices that point
is on and draws a crosshair there, so the four views agree on where it is.
The crosshair lies in the plane of the view that draws it and goes on top
of the slice in a pass of its own, since the slice is transparent and is
drawn over everything opaque. It is not the picker's own marker, which
rides with the surface as it unfolds; the 3D view keeps that one.

The picker draws the 3D view over the whole canvas, so a pick in the
corner the split gives it is mapped there first, and a pick in a slice
view picks nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
- turn the scissor test off once the split has been drawn, so that the
  picker's own passes cover the canvas instead of the last view of the loop,
  which is what kept a click on the brain in the corner from picking it
- leave the crosshair where it is when a click picks nothing, rather than
  taking it away
- read a click in a slice view against the frame that view is drawn in, and
  take the other two views to the point it lands on

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
- draw the crosshair from a scene of its own, laid over each of the four
  views once the rest of it is in, so that a point picked in a slice view is
  marked in the 3D one as well
- give the key its own entry: menu.js calls a descriptor's action with no
  arguments, which threw on the pair that names a method
- name the control "ortho views", and draw it as a switch
- tell dat.GUI what a control was set to from outside it, so that the click
  after a key press is not spent asking for the value it already has

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
The picker marks the vertex on the surface, in the 3D view alone; the
crosshair marks the point in the volume, in all four views. Both were up at
once in the split, so the picker's marker stands down there and comes back
when the 3D view has the canvas to itself. Its own passes now put back
whatever visibility the markers had rather than turning them on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ
@candytaco

Copy link
Copy Markdown
Contributor Author

@claude review

@claude

claude Bot commented Sep 20, 2026

Copy link
Copy Markdown

Claude finished @candytaco's task in 1m 58s —— View job


Review of #739 — the WebGL aligner (update)

  • Gather context (PR diff vs origin/main, prior Copilot/Claude review threads)
  • Verify the four previously-flagged issues are actually fixed as claimed
  • Review cortex/webgl/serve.py changes (loopback binding, token guard)
  • Review cortex/align.py changes (webgl_manual entry point, mayavi removal)
  • Review the newer, not-yet-reviewed commits (slice views / crosshair / picking)
  • Review cortex/tests/test_webgl_aligner.py and test_webgl_headless.py
  • Post final review

I re-checked the four issues Copilot and my earlier review flagged (/save auth, save() returning before completion, get_client() not attaching server, the shader's vmin == vmax division), and they're all genuinely fixed by b38ae14 — confirmed against the current code, not just the comment claiming it:

  • serve.py now binds via bind_local_sockets/LOOPBACK by default (cortex/webgl/serve.py:52-113), and every handler is wrapped in TokenGuard (serve.py:344-379).
  • SaveHandler.post (aligner.py:558) checks a per-page save_token with hmac-based same_token before touching the filesystem.
  • Aligner.prototype.save (aligner.js:1179-1220) sets this._saveState = {status: "saving", ...} synchronously, before the async $.ajax fires, so JSAligner.save()'s poll loop (aligner.py:288-317) can't read a stale prior result.
  • WebApp.get_client (aligner.py:593-604) now attaches server to the returned handle itself, so the documented open_browser=False path gets the real 120s tagged-reply loop.

Test coverage for all of this in test_webgl_aligner.py (1123 lines) is genuinely thorough, not superficial — e.g. test_save_needs_the_token_of_the_page and test_every_page_needs_the_session_token directly exercise the auth fix, and test_aligner_in_headless_browser asserts db.save_xfm was actually called by the time save() returns rather than polling afterwards.

New finding, in code not covered by the earlier reviews (the slice-view/crosshair/picking commits added since):

Crosshair can desync from the slice-plane images if a pick happens before "ortho views" is ever turned oncortex/webgl/resources/js/mriview.js:1611-1645 (setCursor) and :1764-1779 (_showAllPlanes).

setCursor() only moves the slice planes to the picked voxel when this._sliceviews is already true:

if (this._sliceviews) {
    var slices = {x: voxel.x, y: voxel.y, z: voxel.z};
    for (var name in this.sliceplanes) { ... .update(slices[name]); }
}

But pick() (mriview.js:976-996) calls setCursor(coords) unconditionally, including while still in the plain 3D view (_sliceviews === false, the default state). In that case _cursorAt is set true and the crosshair's world position is recorded, but the three slice images are left wherever they were. Toggling into ortho view afterward (v key / menu) calls setSliceViews(true)_showAllPlanes(), which makes the crosshair visible in all four views (_sliceviews===true && _cursorAt===true) — but the slice planes were never moved to the picked point's slice. The crosshair ends up shown against unrelated slice images, which directly contradicts the comment documenting the intent at mriview.js:1608-1609: "so that the four views agree on where it is."

This isn't exercised by the new tests: test_ortho_views_split_the_canvas always splits the view before picking, so the "pick first, then split" ordering is untested. A fix would be to drop the if (this._sliceviews) guard in setCursor (updating the plane's slice index is presumably harmless while the plane is hidden), or to re-apply the last picked voxel's slice indices when setSliceViews(true) turns ortho view on.

Fix this →

Minor / non-blocking:

  • cortex/align.py:51-53 (webgl_manual docstring) describes only two display modes ("surface outlines... or the volume painted on the surface"), but there are actually three (slices/brain/surface, per aligner.py:48-50 and aligner.js:41-43), and it omits the history panel and depth/unfold controls that docs/align.rst documents accurately. Worth syncing the docstring with docs/align.rst at some point, but not a functional bug.
  • cortex/tests/test_webgl_headless.py:914 — a docstring still says `ortho_views` (the old snake_case control name), while the control was renamed to "Show ortho views" in cc9461d/55a0925. Stale comment only.

Everything else — cortex/align.py's handling of reference/xfmname, the [webgl_aligner] config section replacing [mayavi_aligner], doc/image cleanup (no dangling references to deleted docs/aligner/*.png or mayavi_aligner anywhere in the repo), axes3d.js's per-view scissor/camera handling, and the CSS for the new ortho-view layout — checked out fine.
·

A point picked with the 3D view to itself set the crosshair and left the
planes alone, so the views opened with the crosshair over slices it is not
on. The slices are now taken to it as they open, which is the only moment
they can disagree: once open, a pick moves them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115CeZYMxr1wvRSL2V8meKZ

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants