Atelier

Atelier turns a published paper into a FindSim experiment definition. Given a PDF and a figure or panel to target, the pipeline reads the paper, builds the experiment description, digitizes the plotted data, and writes a FindSim record.

Stages

Stage	Module	Output
1. Paper to JSON	core/Paper2Json.py	paper.json
2. Experiment builder	experimentBuilder.py	experiment.json, findsim_preview.json
3. Data builder	core/data_builder.py	data.json, extracted_data.csv
4. Model builder (optional)	core/model_builder.py	experimentMap.json
5. Fold	fold.py	final findsim_preview.json

pipeline.py is the single orchestrator that runs all of them.

Install

Python 3.11 or newer (developed on 3.14).

git clone https://github.com/<your-org>/FSW-AI.git
cd FSW-AI
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Choose a backend

Every stage uses one LLM backend, set with --backend:

--backend	Requirement
cc (default)	The Claude Code CLI on your PATH, logged in
api	export ANTHROPIC_API_KEY=...
openai	export OPENAI_API_KEY=...

Use --model to override the model for the chosen backend.

Inputs

The pipeline needs two things at minimum:

1. A PDF of the paper, passed with --pdf, or a directory of PDFs passed with --pdf-dir.
2. A target, meaning which figure or panel to build an experiment from. Pass it inline with --target, or list many targets in a manifest file.

Two optional inputs enable the model builder stage. Supply both or neither:

3. --sim-model, a simulation model file, either kkit .g or HillTau .json.
4. --sim-map, the matching model map JSON, for example NOS_ma_map.json.

Run one paper

cd FSW-AI/pipeline
python pipeline.py \
    --pdf /path/to/Abrams1991.pdf \
    --target "Figure 2:A" \
    --output-dir out/ \
    --backend cc

--target is repeatable. It accepts "Figure 2" for a whole figure or "Figure 2:A" for one panel. Omit it entirely to process every figure the paper contains.

Run many papers

A manifest is a JSON list of targets. PDFs are resolved as <pdf-dir>/<paper>.pdf.

[
  {"paper": "Abrams1991",   "figure": "Figure 2", "panel": null},
  {"paper": "Alano2010",    "figure": "Figure 3", "panel": "B"},
  {"paper": "Fujiwara2014", "figure": "Figure 3", "panel": "C"}
]

python pipeline.py \
    --manifest manifests/list1.json \
    --pdf-dir /path/to/pdfs \
    --output-dir out/ \
    --backend api \
    --parallel 4

Example manifests are in FSW-AI/pipeline/manifests/. To build one from a directory of ground truth filenames such as Abrams1991_Fig2.json:

python manifest_from_groundtruth.py --groundtruth-dir /path/to/Expts --output targets.json

Size --parallel to your API rate limit, not your CPU count. Every worker draws from the same account wide budget.

Outputs

One directory per paper, one subdirectory per figure or panel:

out/
└── Abrams1991/
    ├── paper.json                  # stage 1
    ├── figures/                    # extracted figure images
    └── figure2A/
        ├── experiment.json         # stage 2
        ├── data.json               # stage 3
        ├── extracted_data.csv      # digitized points
        ├── extracted_plot.png      # digitizer overlay
        ├── comparison_plot.png     # digitized vs original
        ├── experimentMap.json      # stage 4, only with --sim-model/--sim-map
        └── findsim_preview.json    # final FindSim record

Useful flags

Flag	Effect
--reuse-paper-json	Skip stage 1 and reuse an existing paper.json
--reuse-experiment-json	Also reuse experiment.json and rerun from stage 3
--force-redigitize	Ignore a cached extracted_data.json and call the digitizer again
--limit N	Process only the first N papers of a manifest
-v	DEBUG level logging

Run python pipeline.py --help for the full list.

Optional: validate against the FindSim schema

If a FindSim checkout sits beside this repository, its FindSim-Schema.json is found automatically. Otherwise point at it directly:

export FINDSIM_SCHEMA=/path/to/FindSim/FindSim-Schema.json

Without it, folding falls back to the Pydantic mirror in core/FindSim_schema.py.

Other entry points

Script	Purpose
data_extractor.py	Digitize a single figure image or PDF, no paper context needed
pdf_annotator.py	Flask UI for reviewing and correcting paper.json
inspect_experiment.py	Print a readable summary of an experiment.json
plot_result.py	Plot digitized data against the source figure

Tests

cd FSW-AI/pipeline
pytest -m unit

Unit tests mock every LLM call. Tests marked integration need a live backend, and those marked docling run real PDF layout extraction. Tests that compare against FindSim itself skip unless FINDSIM_DIR points at a checkout.

Repository layout

separate_panels_by_whitespace.py    # standalone panel splitter, imported by the pipeline
WIP docs/                           # flowchart and curation notes
FSW-AI/pipeline/
├── pipeline.py                     # main orchestrator
├── experimentBuilder.py            # stage 2 CLI
├── dataBuilder.py                  # stage 3 CLI
├── modelBuilder.py                 # stage 4 CLI
├── fold.py                         # stage 5, merges data into the FindSim record
├── core/                           # library modules for every stage
├── agents/                         # LLM prompts, one CLAUDE.md per agent
├── manifests/                      # example target lists
└── templates/                      # pdf_annotator UI template

Disclaimer

The algorithms here are built by the authors, and the code is generated by the authors with the help of LLMs, specifically Claude's Sonnet 4.5 and Sonnet 5.0.