Agent examples: an LLM driving the MCP tools (real runs, replayed in CI) - #16
Merged
Merged
Conversation
…al runs docs/AGENT_EXAMPLES.md — two worked examples, each run for real by Claude (Opus 5.5) in Claude Code against the `materia` server: 1. Look at a structure before any data (Mn₃Ga): space group, Wyckoff sites, first Cu Kα reflections, shortest contact — four calls on refs. 2. (Example 4 in the plan) GaTa₄Se₈ X-ray PDF fitted block by block — scale + cell 35.3 %, ADPs 19.3 %, Qdamp 12.2 %, δ1 8.61 %, positions 8.33 % (demo: 8.12 %) — with the judgement calls stated (Qdamp freed without a standard, Qbroad at its bound dropped, δ1 not δ2); then a boxcar scan: the 1.5–7.5 Å box fits worst and U(Se2) doubles with r while the cell stays flat; the reverse scan agrees, so the drift is in the data. examples/mcp/ holds the app's bundled demo data as plain files (plus MnO and a POWGEN .irf for the Rietveld example to come). src/mcp/examples.test.ts keeps both honest: the files must equal their TS sources (UPDATE_MCP_EXAMPLES=1 rewrites them), and each example's tool sequence is replayed through an MCP client with the page's claims asserted (~33 s; MCP_EXAMPLES_FULL=1 adds each block and the reverse scan). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first worked examples from the LLM-usage plan. Both were run for real by Claude (Opus 5.5) in Claude Code against the
materiaMCP server, then written up indocs/AGENT_EXAMPLES.md.Examples
1. Look at a structure before any data (Mn₃Ga). The prompt asks for the space group, the Wyckoff sites, the first Cu Kα reflections, and any unphysical distance. The model answered in 4 calls, passing refs between them:
2. A PDF fit block by block, then local vs average (GaTa₄Se₈, X-ray, 1.5–28 Å).
The run states three judgement calls:
It then ran a boxcar scan. The 1.5–7.5 Å box fits worst (11.0% against 3–6%), and U(Se2) nearly doubles with r while the cell stays flat. A reverse scan agrees within the esds, so the drift is in the data. Conclusion: the local structure differs from the cubic average, and naming the distortion needs a lower-symmetry model.
Keeping the page honest
examples/mcp/holds the app's bundled demo data as plain files, so a model can load them bypath. It also includes the MnO CIF and a POWGEN.irffor the Rietveld example that comes next.src/mcp/examples.test.tsdoes two jobs:UPDATE_MCP_EXAMPLES=1rewrites them.MCP_EXAMPLES_FULL=1adds each block's Rw and the reverse scan (about 80 s), which I ran and which passes.Next
check_cell_symmetryandexport_bundle. This session'smateriaserver was started before those merged, so it has to be reconnected before that run can be recorded.Both are listed under "Other planned work" in AGENT_TOOLS.
Tests: 196 files, 1,490 tests pass.
tsc -bandeslint .are clean.🤖 Generated with Claude Code