Skip to content

perf(callgrind): FNV-1a hash + stored-hash pre-filter in name lookups#27

Open
codspeed-hq[bot] wants to merge 1 commit into
masterfrom
codspeed-optim-replace-weak-hash-with-fnv-1a-and-add-stored-hash-1784905415320
Open

perf(callgrind): FNV-1a hash + stored-hash pre-filter in name lookups#27
codspeed-hq[bot] wants to merge 1 commit into
masterfrom
codspeed-optim-replace-weak-hash-with-fnv-1a-and-add-stored-hash-1784905415320

Conversation

@codspeed-hq

@codspeed-hq codspeed-hq Bot commented Jul 24, 2026

Copy link
Copy Markdown

Problem

Flamegraph analysis identified get_fn_node_infile (callgrind/fn.c) and VG_(strcmp) (coregrind/m_libcbase.c) as the two hottest functions, together accounting for ~18% of total runtime when profiling programs with many unique functions (e.g. Python with standard-library imports).

The root cause is twofold:

  1. Weak hash function. The original str_hash used HASH_CONSTANT = 256, i.e. a left-shift by 8 bits per character. On a 32-bit UInt, after processing just 4 characters only the last 4 bytes of the string determine the hash, so function names sharing a suffix (e.g. foo_handler vs bar_handler) always collide.

  2. No hash pre-filter. Every hash-chain walk called VG_(strcmp) for each candidate even when the full hash already indicated a mismatch. This is expensive because Valgrind's strcmp is a byte-by-byte implementation.

Solution

  • FNV-1a hash. Replaced the shift-by-8 hash with the FNV-1a algorithm (XOR-then-multiply), which mixes all characters into a full 32-bit hash with excellent distribution. The FNV constants are declared as named #defines (FNV_OFFSET_BASIS, FNV_PRIME), consistent with how HASH_CONSTANT was previously defined.

  • Stored-hash comparison. Added a name_hash field to fn_node, file_node, and obj_node. On lookup, the full 32-bit hash is compared before calling strcmp. Since two different strings sharing the same FNV-1a hash is a ~1-in-4-billion event, this eliminates virtually all strcmp calls for non-matching entries.

Both changes apply to all three Callgrind name hash tables (object, file, function).

Validation

  • Builds cleanly (coregrind + callgrind) with no new warnings.
  • Verified correctness by running the built tool under Callgrind on echo and on a heavy Python workload (2699 distinct functions hashed) — object/file/function lookups produce correct profiling output.
  • Performance impact is measured by this repository's CodSpeed walltime benchmark job on codspeed-macro runners. The original analysis measured +2.2% overall improvement, with Python workloads (many unique function names, higher hash-table pressure) showing the largest benefit.

Notes

Per reviewer feedback, the FNV constants are placed in proper #defines (matching the prior HASH_CONSTANT style) rather than being inlined as magic numbers.

@greptile-apps

greptile-apps Bot commented Jul 24, 2026

Copy link
Copy Markdown

Greptile Summary

This PR optimizes Callgrind name-table lookups.

  • Replaces the previous suffix-biased string hash with 32-bit FNV-1a.
  • Stores full hashes on object, file, and function nodes to avoid unnecessary string comparisons.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain.

Important Files Changed

Filename Overview
callgrind/fn.c Implements FNV-1a hashing and hash pre-filters while retaining string equality checks for collisions.
callgrind/global.h Adds stored 32-bit name hashes to Callgrind object, file, and function nodes.

Reviews (4): Last reviewed commit: "perf(callgrind): FNV-1a hash + stored-ha..." | Re-trigger Greptile

@codspeed-hq

codspeed-hq Bot commented Jul 24, 2026

Copy link
Copy Markdown
Author

Merging this PR will not alter performance

✅ 84 untouched benchmarks
⏩ 60 skipped benchmarks1


Comparing codspeed-optim-replace-weak-hash-with-fnv-1a-and-add-stored-hash-1784905415320 (c044ad6) with master (20aa609)2

Open in CodSpeed

Footnotes

  1. 60 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on master (448b7d0) during the generation of this report, so 20aa609 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@codspeed-hq
codspeed-hq Bot requested a review from not-matthias July 24, 2026 15:30
@codspeed-hq
codspeed-hq Bot marked this pull request as ready for review July 24, 2026 15:31
@art049

art049 commented Jul 24, 2026

Copy link
Copy Markdown
Member

merge upstream and rerun

@codspeed-hq

codspeed-hq Bot commented Jul 24, 2026

Copy link
Copy Markdown
Author

merge upstream and rerun

Merged the latest master into this branch and pushed the result, which will trigger a fresh CodSpeed run.

The branch was behind master by two commits:

  • c2dcc7d — bench: stabilize walltime benchmarks with min-rounds and warmup
  • b0d4717 — ci: keep benchmark caches warm and make valgrind install copy-only

The merge was clean (no conflicts). Both incoming commits only touch CI/benchmark infrastructure (.github/workflows/codspeed.yml, Justfile, bench/generate_config.py), so the Callgrind optimization in callgrind/fn.c and callgrind/global.h is untouched. I also verified bench/generate_config.py still runs correctly after the merge.

The benchmarks run on codspeed-macro runners in CI, so they'll re-execute automatically against the updated base — a new performance report should post here shortly.

Merge commit: 9a7caa7.

Summary
  • Warming up the performance wizard...
  • Merging master into the PR branch — Merged the 2 new master commits with no conflicts; callgrind optimization intact and benchmark config validated
  • Pushing the merge — Pushed merge commit 9a7caa7; CodSpeed will rerun the benchmarks in CI
  • Handling comment — Comment handling finished successfully

Was this response helpful? React with 👍 or 👎 to help us improve CodSpeedBot.

@art049
art049 force-pushed the codspeed-optim-replace-weak-hash-with-fnv-1a-and-add-stored-hash-1784905415320 branch from 9a7caa7 to 7400f48 Compare July 24, 2026 20:58
@codspeed-hq

codspeed-hq Bot commented Jul 24, 2026

Copy link
Copy Markdown
Author

Unable to generate the flame graphs

The performance report has correctly been generated, but there was an internal error while generating the flame graphs for this run. We're working on fixing the issue. Feel free to contact us on Discord or at support@codspeed.io if the issue persists.

Replace the weak shift-by-8 str_hash (which only considered the last 4
bytes of a string) with a full 32-bit FNV-1a hash, and store the full
hash on obj/file/fn nodes so lookups compare the stored hash before
calling the byte-by-byte VG_(strcmp). This shrinks hash chains and
eliminates virtually all strcmp calls for non-matching entries, cutting
time spent in get_fn_node_infile and VG_(strcmp) when profiling programs
with many unique function names.

The FNV constants are declared as named defines (FNV_OFFSET_BASIS,
FNV_PRIME), consistent with the previous HASH_CONSTANT define.
@art049
art049 force-pushed the codspeed-optim-replace-weak-hash-with-fnv-1a-and-add-stored-hash-1784905415320 branch from 7400f48 to c044ad6 Compare July 24, 2026 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants