Skip to content

tools: generate the per-CPU SIMD inventory from LLVM's TableGen source - #330

Merged
AdaWorldAPI merged 1 commit into
masterfrom
claude/llvm-inventory-generator
Sep 24, 2026
Merged

AdaWorldAPI merged 1 commit into
masterfrom
claude/llvm-inventory-generator

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

What this adds

tools/gen_llvm_inventory.py builds the per-CPU SIMD feature inventory from LLVM's public TableGen source instead of maintaining it by hand.

  • Input: llvm-project's X86.td, AArch64Processors.td and AArch64Features.td, plus the instruction and predicate .td files. They're read at the llvmorg-* tag that matches rustc -vV, which is llvmorg-22.1.8 under the pinned 1.98.1, so the inventory is the one cfg(target_feature) actually sees.
  • Resolution: it resolves every processor's feature set: !listconcat / !listremove chains, foreach P = [...] alias blocks, ProcessorAlias, implied-feature closure, aarch64 architecture default extensions, and tune records that imply ISA features.
  • Output:
    • tools/llvm-inventory/inventory.json: full per-CPU feature sets for 92 x86 and 99 aarch64 CPUs;
    • .claude/knowledge/llvm-cpu-inventory.md: the readable table. It includes the backend that the simd.rs cfg ladder selects, and which instruction implements each crypto-relevant primitive on that CPU: mul_lo32, variable u32 shift and permute, IFMA, carry-less multiply, and the AES round.
  • Primitive rows: they name the LLVM instruction def and its gating predicate, and the generator asserts both still exist in the .td files. An LLVM rename fails the run instead of leaving a stale row.

It follows the gen_ternlog_bodies.py pattern: the output is committed and the script is its provenance.

python3 tools/gen_llvm_inventory.py --write          # regenerate
python3 tools/gen_llvm_inventory.py                  # exit 1 if the committed output is stale
python3 tools/gen_llvm_inventory.py --verify-rustc   # compare every CPU with rustc

Verification

--verify-rustc compares every CPU with rustc's own --print cfg -C target-cpu=.... It covers all features stable rustc exposes: 56 on x86 and 37 on aarch64. It uses rustc's own Rust↔LLVM feature-name table, parsed from rustc_codegen_llvm/src/llvm_util.rs at the pinned release. 191/191 CPUs agree.

The comparator failed first, and each failure drove a real fix:

  • evaluating !listremove;
  • reading Architecture64's implied list from the right argument;
  • including aarch64 architecture default extensions (apple-m4 ssbs);
  • the x86_64 ABI baseline (sse/sse2);
  • tune records implying ISA features (TuneOryon implies FullFP16, FP16FML and SPE).

I also checked that the staleness mode fails: editing one row makes it exit 1, and regenerating clears it.

What the table shows

  • x86-64-v3, the portable baseline, has no aes and no pclmul. Compile-time dispatch on the v3 build cannot reach AES-NI or PCLMULQDQ.
  • haswell has pclmul but not aes in LLVM's table.
  • arrowlake has avxifma, the VEX-encoded IFMA, without AVX-512.
  • aarch64 neoverse-v2 (Graviton4, Grace) and cortex-x925 have no aes, so no hardware AES and no 64-bit PMULL without an explicit +aes.

Not in this PR

  • The backend column mirrors the simd.rs cfg ladder by hand; it isn't parsed from simd.rs.
  • No wasm rows: wasm has no processor table.
  • No CI job runs the staleness check yet.
  • LLVM fork: rustc builds on rust-lang's fork of LLVM (rustc/22.1-2026-05-19). Its AArch64 .td files matched the upstream tag byte for byte; --verify-rustc is the guard if that changes.
  • No src/ changes. This is the generator's first stage; the missing primitives themselves come in later PRs.

🤖 Generated with Claude Code

https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm


Generated by Claude Code

tools/gen_llvm_inventory.py reads llvm-project's X86.td, AArch64Processors.td
and AArch64Features.td (plus the instruction/predicate .td files) at the
llvmorg-* tag matching `rustc -vV` (22.1.8 under the pinned 1.98.1), resolves
every processor's feature set, and writes:

  tools/llvm-inventory/inventory.json       full per-CPU feature sets
  .claude/knowledge/llvm-cpu-inventory.md   the human-readable table

Each row also names the backend the simd.rs cfg ladder selects, and which
instruction implements each crypto-relevant primitive (mul_lo32, variable
u32 shift/permute, IFMA, carry-less multiply, AES round) on that CPU. The
instruction defs and gating predicates are asserted to exist in the .td
files, so an LLVM rename fails the generator instead of leaving a stale row.

Like gen_ternlog_bodies.py, the output is committed and the script is its
provenance; the default mode exits 1 if the committed output is stale.

--verify-rustc compares every CPU (92 x86, 99 aarch64) with rustc's own
`--print cfg -C target-cpu=...` over all features stable rustc exposes,
using rustc's Rust<->LLVM name table parsed from rustc_codegen_llvm at the
pinned release. 191/191 agree. The comparator failed first and drove five
fixes: !listremove, the Architecture64 implied-list position, aarch64
architecture default extensions, the x86 ABI baseline, and tune records
that imply ISA features (TuneOryon -> FullFP16/FP16FML/SPE).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HnekoM1EidTwQLS3oFVFm
@coderabbitai

coderabbitai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Essentials

Run ID: 1ee21f5c-cd34-4b61-93cf-d71d17feff05

📥 Commits

Reviewing files that changed from the base of the PR and between 07c7b4b and 11de642.

📒 Files selected for processing (4)
  • .claude/blackboard.md
  • .claude/knowledge/llvm-cpu-inventory.md
  • tools/gen_llvm_inventory.py
  • tools/llvm-inventory/inventory.json
 ________________________________________________________
< A spoonful of AI helps the bitter code review go down. >
 --------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@cursor

cursor Bot commented Sep 24, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_5884f8e0-4523-4c1d-9bf7-f996cf49d41f)

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review September 24, 2026 22:59
@AdaWorldAPI
AdaWorldAPI merged commit a1ad0bc into master Sep 24, 2026
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants