Skip to content

Add SmartBench - Evidence-constrained code diagnosis - #1871

Open
xianyu-sheng wants to merge 1 commit into
analysis-tools-dev:masterfrom
xianyu-sheng:add-smartbench-clean
Open

Add SmartBench - Evidence-constrained code diagnosis#1871
xianyu-sheng wants to merge 1 commit into
analysis-tools-dev:masterfrom
xianyu-sheng:add-smartbench-clean

Conversation

@xianyu-sheng

Copy link
Copy Markdown

SmartBench is a code diagnosis workbench that combines deterministic static analysis with evidence-constrained LLM review.

Key features:

  • Language-neutral semantic analyzers (CFG/ICFG, resource lifecycle, state machines)
  • Evidence-gated architecture: static analyzers own facts, LLM proposes hypotheses, validators decide
  • Real upstream bugs found and verified: stunner #89, qscan #22, sniproxy #203, Robyn #1432
  • Reproducible benchmark corpus with 20 before/after cases
  • Python 3.10+, MIT license

Why add it:

  • Proven results: found real bugs in production open-source projects
  • Unique architecture: evidence-constrained to reduce false positives
  • Complete verification chain: static audit → runtime reproduction → local fix → upstream submission

GitHub: https://github.com/xianyu-sheng/SmartBench

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant