Skip to content

chore: Add unit tests for arm64 disassembler - #3245

Open
filzrev wants to merge 18 commits into
dotnet:masterfrom
filzrev:chore-add-arm64-unittests
Open

chore: Add unit tests for arm64 disassembler#3245
filzrev wants to merge 18 commits into
dotnet:masterfrom
filzrev:chore-add-arm64-unittests

Conversation

@filzrev

@filzrev filzrev commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

This PR contains following changes.

1. Cleanup arm64 disassembler related code to preparing to add unit tests

See following PR comment for details.

2. Add AsmArm64 package reference

AsmArm64 package to unit test project.
Currently it's used for test purpose.
It's expected existing arm64 disassembler is replaced to AsmArm64 based implementation. (#3246)

3. Add arm64 disassembler related unit tests.

To ensure existing arm64 disassembler behavior.
Unit test codes are added for major code paths. (It can confirm code coverage results with Analyze Code Coverage on VS)

Note:
Almost of unit tests on .NET Framework are excluded by #if NET directive.

  • private methods are tested with UnsafeAccessor (It requires .NET 8 or later)
  • Capstone's native dependencies seems not copied to bin directory when using xUnit v2 (Because it's Library project)

@timcassell

Copy link
Copy Markdown
Collaborator
AI review

Accumulator bugs

Arm64RegisterValueAccumulator.cs:64ExpectingAdd never resets. Unlike ExpectingMovk, this case has no fall-through to LookingForPossibleLdr, so any instruction that isn't the expected ADD
leaves the state machine parked with the stale ADRP value. adrp x0,#0x1000 ; movz x0,#0x100 ; add x0,x0,#0x100 gives HasValue == true, Value == 0x1100 after x0 was clobbered, so a following BR/BLR x0
resolves to a bogus address and the disassembly prints the wrong symbol. The PR's own skipped AdrpThenOther_ThenAdd_ShouldResetValue fails on this today.

Arm64RegisterValueAccumulator.cs:68ADD ignores lsl #12. The match checks only Operands[2].Type == Immediate, then does _value | Immediate. adrp x0,#0x1000000 ; add x0,x0,#0xfff, lsl #12
yields page | 0xFFF instead of page + 0xFFF000, with HasValue still true — a silently wrong address rather than a bail-out. At minimum, reject a shifted immediate. (Skipped
AdrpThenAdd_WithShiftedImm_ShouldCalculateAddress covers it.)

Arm64RegisterValueAccumulator.cs:41MOVZ discards the shift. Same class: _value = details.Operands[1].Immediate drops the lsl amount, so movz x0,#0x1234, lsl #16 seeds 0x1234 instead of
0x12340000.

Arm64RegisterValueAccumulatorTests.Movz.cs:31 — skipped test asserts MOVK semantics. Movz_WithShiftedImmediateValue_ShouldStartNewValue expects 0x3333_2222_1111 from three MOVZs. A real MOVZ
zeroes the rest of the register, so the architectural result is 0x3333_0000_0000. Whoever un-skips this and "fixes" the accumulator to satisfy it will encode a decoding bug.

Tests that can't fail

Arm64DisassemblerTests.TryFollowJumpTrampoline.cs:73 — slot displacement untested. In all four stub tests the getPointer callback returns ExpectedResultAddress for any address, unlike the
TryResolvePrecode tests which assert the requested slot. Changing parseBase + (ulong)(long)off0 to parseBase + 4 + (ulong)(long)off0 in the StubPrecode branch leaves the whole suite green. Assert the
address inside getPointer, as TryResolvePrecode_* does.

Arm64DisassemblerTests.TryFollowJumpTrampoline.cs:178NonStubHead bails on length, not shape. The test supplies one instruction, so the reader returns 4 bytes and TryReadStubHead bails at read < 12; the stub-shape rejection it names is never reached, and the test passes with all stub matching deleted. Pad to >= 4 non-stub instructions.

Arm64InstructionFormatterTests.cs:19 — padding disabled. The comment says "Use DisassemblyDiagnoserConfig default config value" but FirstOperandCharIndex = 10 is commented out, so the theory runs at
Iced's default of 0 and asserts strings BDN never emits ("b #8" vs. production "b #8"DisassemblyDiagnoserConfig.cs:86). The shipped column width is only incidentally covered by the one
FirstOperandCharIndex = 6 case.

Arm64InstructionFormatterTests.cs:61 — empty symbols map. FormatInstruction_B_WithReferencedAddress passes no symbols, so TryGetValue always misses and the one thing gated on ReferencedAddress
— the Operand.Replace($"#0x{addr:x}", name) substitution — is never run. Add an entry mapping 0x10000 to a name.

Arm64DisassemblerTests.TryGetReferencedAddress.cs:11 — helper skips Init(runtime). _runtime stays null; it works only because no test here feeds an LDR. The first one that does will NRE inside
Feed. The sibling helper (Arm64RegisterValueAccumulatorTests.cs:15) does call Init.

Arm64DisassemblerTests.TryGetReferencedAddress.cs:23_With_BL builds BR X0. Duplicate of _With_BLR; BL is never covered.

Minor

Arm64Disassembler.cs:145 — constant is ISHLD, not ISH. 0xD50339BF has CRm = 0b1001 (DMB ISHLD); DMB ISH is 0xD5033BBF. The new tests construct Arm64BarrierOperationLimitKind.ISHLD,
confirming it. As written a stub prefixed with a plain DMB ISH isn't recognised and precode resolution silently fails — fix the comment/name or widen the match.

ClrMdDisassembler.cs:114IClrRuntime switch introduced unchecked downcasts. foreach (ClrModule module in state.Runtime.EnumerateModules()) and the nested foreach (ClrType type in ...) went from
statically-typed iteration to runtime casts, since both interface members are explicit implementations returning IClrModule/IClrType. Nothing breaks against ClrMD 4.0.732401 (the concrete instances
still come back), but FilterAndEnqueue now throws InvalidCastException for any other IClrRuntime — including the MockClrRuntime this PR adds, which makes that path untestable by the harness being
introduced.

Helpers/Arm64TestInstructions.cs:180ValidateMultipleOf8 throws "...must be multiple of 4".

It looks like it found some possible bugs in the accumulator (I did not verify myself). Fine if you want to fix them here, or defer for out-of-scope.

@filzrev
filzrev force-pushed the chore-add-arm64-unittests branch from 7aa8d2b to 52dc08c Compare September 6, 2026 23:23
@filzrev

filzrev commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

It looks like it found some possible bugs in the accumulator (I did not verify myself). Fine if you want to fix them here, or defer for out-of-scope.

This PR intended to add tests to verify existing arm64 disassembler behaviors.
So following tasks are handled on another PRs.

  • Refactor Arm64ValueAccumulator/Arm64Disassebler codes. And fix some existing issues. (and reviewed content)
  • Migrate code to use AsmArm64.

@timcassell

Copy link
Copy Markdown
Collaborator

Findings are all in the new test code; the production changes look good (the formatter padding is a real fix — a mnemonic reaching FirstOperandCharIndex previously ran straight into its operand with no separator).

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Helpers/Arm64DisassemblerTestBase.cs:71 — the read delegate ignores its address parameter and always serves rawInstructions from index 0, so the address the disassembler reads from is never asserted for TryReadStubHead, TryFollowJumpTrampoline, or the multi-hop loop. Mutating the trampoline read to dataReader.Read(address + 0x1000, head) leaves all 79 tests green. Suggest keying the delegate off address: serve bytes at a registered base, return 0 elsewhere.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Arm64DisassemblerTests.TryTranslateAddressToName.cs:97NonAlignedAddress = 0x10004 is below MinValidAddress on macOS-arm64 (ClrMdDisassembler.GetMinValidAddress returns 0x100000000 there), so the test returns on the first line and asserts three empty collections without reaching the GetMethodByHandle/GetTypeByMethodTable path it documents. macos-latest is arm64 in the CI matrix. Forcing GetMinValidAddress to 0x100000000 locally keeps all 79 tests passing. Address1 + 4 would fix it — same class of issue as 0ca5d3f/019acb64e, just missed for this one address.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Arm64InstructionFormatterTests.cs:97BeEquivalentTo on a List<string> is order-insensitive, so the instruction ordering Decode is responsible for isn't asserted (swapping two expected lines still passes). Should().Equal(...) instead.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Arm64InstructionFormatterTests.cs:142{ MOVZ(X0, 0x100), "movz x0, #0x100" } duplicates line 140 verbatim; xUnit silently drops it (Skipping test case with duplicate ID in the run output). Presumably one row was meant to be a different value.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Helpers/Arm64DisassemblerTestBase.cs:13DummyMethodNotUsed = default! is a null MockClrMethod. Safe only because the current tests return before TryTranslateAddressToName reaches method.NativeCode == currentMethod.NativeCode; a future test that lets GetMethodByInstructionPointer resolve will NRE inside production code instead of failing usefully.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Arm64DisassemblerTests.Decode.cs:28,39,63,74 (and the same copy-paste in the TryFollowJumpTrampoline/TryResolvePrecode files) — trailing comments name x11/x0 where the code uses x10/x1. The register number is exactly what distinguishes StubPrecode (x10/x12) from FixupPrecode (x11/x12), so someone "fixing" code to match a comment here would break stub recognition.

src/BenchmarkDotNet/Disassemblers/Arm64Disassembler.cs:146 — nit: now that the comment correctly reads DMB ISHLD, the constant name DmbIshInstr is the remaining misnomer (DmbIshLdInstr). Separately, only ISHLD is matched — a build emitting dmb ish (0xD5033BBF) would silently fail stub-head detection. Out of scope for this PR, just noting it.

Reviewed with Claude Code.

@filzrev
filzrev force-pushed the chore-add-arm64-unittests branch from 84d7a22 to a01d20d Compare September 8, 2026 12:06
@filzrev
filzrev force-pushed the chore-add-arm64-unittests branch from a01d20d to 2fdfa41 Compare September 8, 2026 12:07
@filzrev
filzrev force-pushed the chore-add-arm64-unittests branch from 2fdfa41 to 025c00a Compare September 8, 2026 17:58

@timcassell timcassell left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up on the updated head. The fixes from the last round all look right — the macOS-arm64 address, WithStrictOrdering, the duplicate MOVZ row, DmbIshLdInstr, and the Decode.cs comments. A few items are still open, plus one new one and one follow-up note.

Reviewed with Claude Code.

@timcassell timcassell left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another pass on 7124b6c3a. AddFalsePointer and the absolute-address assertion in TryGetReferencedAddress_With_BranchRelative are both good additions. Remaining items below — the UnreachableException visibility one is the only one that affects shipped code.

Reviewed with Claude Code.

/// <summary>
/// Exception thrown when the program executes an instruction that was thought to be unreachable.
/// </summary>
public sealed class UnreachableException : Exception

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This polyfill is public, so BenchmarkDotNet's netstandard2.0 and net6.0 assets now export a public System.Diagnostics.UnreachableException. A consumer on those TFMs who references another library exporting the same polyfill gets CS0433 ("exists in both"), and one who declares their own gets CS0436 — purely from referencing BenchmarkDotNet.

Its only consumer is MockClrMethod in the test assembly, and Properties/AssemblyInfo.cs:9 already grants InternalsVisibleTo("BenchmarkDotNet.Tests"), so internal works. The sibling polyfills that declare types (AsyncEnumerable, FileExtensions) are both internal.

@@ -0,0 +1,41 @@
#if !NET8_0_OR_GREATER

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System.Diagnostics.UnreachableException shipped in .NET 7, not .NET 8. This guard is correct today only because the TFM list (netstandard2.0;net6.0;net8.0;net9.0;net10.0) happens to skip net7.0 — add net7.0 and the polyfill duplicates the BCL type (CS0436), which TreatWarningsAsErrors in build/common.props makes fatal.

Should be #if !NET7_0_OR_GREATER.

{
private Arm64RegisterValueAccumulator CreateValueAccumulator(Arm64RegisterX register, ushort initialValue)
{
using var clrRuntime = CreateMockClrRuntime();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using disposes the runtime when this helper returns, but the accumulator it returns keeps it in _runtime (Init(clrRuntime) on the next line). Every caller therefore gets an accumulator holding a disposed IClrRuntime.

Harmless only while the BR/BLR paths never dereference it — the first MOVZ->LDR case added here reads through the disposed object, since Arm64RegisterValueAccumulator.Feed calls _runtime.DataTarget.DataReader.ReadPointer. Drop the using, or return the runtime alongside the accumulator so the caller owns its lifetime.

{
if (JitHelperFunctionNames.TryGetValue(address, out var value))
return value;
throw new ArgumentException($"Specified address(0x{address:X}) is not registered.");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These four lookups (also :41, :48, :55) throw for an unregistered address, where ClrMD returns null and the production code treats null as an ordinary outcome. So the null branches in TryTranslateAddressToName can't be reached through this mock, and a fixture that forgets one registration fails with an opaque exception from inside the mock instead of a readable assertion.

Arm64InstructionFormatterTests.FormatInstruction_IntroDisassembly_SumLocal only survives because every branch target happens to be pre-seeded into AddressToNameMapping; adding one more branch instruction to that fixture without registering its symbol would throw.

AddFalsePointer handled the equivalent problem on the MockMemory side nicely — same idea would fit here. (One gap remains there too: ClrMdDisassembler.FlushCachedDataIfNeeded's Read(...) <= 0 branch is still unreachable, since a 1-byte region returns 1 rather than 0.)

@@ -0,0 +1,33 @@
using AwesomeAssertions;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFM guards are inconsistent across the new files: #if NET in seven, #if NET8_0_OR_GREATER in five, and none in eight — including this one, which calls the Capstone-backed Arm64TestInstructions.Movz. It survives on net472 only because the single test here is Skipped.

The two guards are equivalent for today's TFMs (net10.0 and net472), but DmbIshldOffset is defined under #if NET and consumed under #if NET8_0_OR_GREATER, so adding a net6.0/net7.0 test TFM — or un-skipping this test — breaks the build. Worth settling on one guard and applying it consistently.


file static class ExtensionMethods
{
public static ReadBytesDelegate ToGetReadBytesDelegate(this uint[] rawInstructions)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ToGetReadBytesDelegate and ToTryReadPointerDelegate (:71) are no longer referenced anywhere — MockMemory replaced them, and the single-delegate MockDataReader constructors they fed were removed in 2304b834.

Worth deleting: they're the old address-ignoring helpers, so leaving them around invites reintroducing exactly the gap MockMemory closed.


/// <summary>
/// Create MockClrRuntime with MockDataReader that returns following data.
/// Read: Throw InvalidOperationException.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale since 2304b834: MockDataReader(ulong) now sets _read = (_, _) => 0, so Read returns 0 rather than throwing. (Returning 0 is the right call — it matches the IDataReader.Read contract; it's just the doc that needs updating.)


private readonly ulong BaseAddress;

public MockMemory(ulong baseAddress = 0)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The baseAddress parameter is never passed — every call site is new MockMemory() — and it's applied only in AddBytes, not in AddJitHelperFunctionName/AddMethodByHandle/AddMethodByInstructionPointer/AddTypeByMethodTable. If anyone does pass a base, instruction addresses would rebase while the lookup mappings silently would not. Either drop the parameter or apply it uniformly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants