Skip to content

Fix DispatchProxy pointer parameters and returns - #136452

Open
caraioniurie47 wants to merge 6 commits into
dotnet:mainfrom
caraioniurie47:issue-136451
Open

caraioniurie47 wants to merge 6 commits into
dotnet:mainfrom
caraioniurie47:issue-136451

Conversation

@caraioniurie47

@caraioniurie47 caraioniurie47 commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

A DispatchProxy method generated for an interface method with a pointer parameter converted the pointer to object with castclass object on the raw pointer, so on CoreCLR a call with a non-null pointer usually threw NullReferenceException before Invoke ran, and Mono's JIT rejected the method with InvalidProgramException on every call. Pointer returns and ref/out pointer write-backs used castclass to the pointer type, so on CoreCLR a pointer returned by Invoke threw InvalidCastException.

Pointers now cross DispatchProxy.Invoke as follows:

  • A non-null pointer argument reaches Invoke in args as a System.Reflection.Pointer of the parameter's type (Pointer.Box); a null pointer reaches it as null, as on CoreCLR before this change. ref and in pointer arguments pass their current value the same way.
  • A pointer returned by Invoke, or written back to args for a ref/out pointer parameter, may be a Pointer, an IntPtr, or null for a null pointer. Anything else is unboxed as an IntPtr, so an int or a string throws InvalidCastException.
  • The pointer type stored in a returned Pointer is not checked, since Pointer exposes it only internally: a Pointer of long* returned for an int* gives its address. MethodInfo.Invoke rejects such a Pointer for an int* argument.

Apart from such a Pointer, MethodInfo.Invoke takes these forms for pointer arguments, and it returns a Pointer for a pointer return, so an Invoke that forwards with method.Invoke(target, args) works for pointer parameters passed by value and for pointer returns. For ref/out pointer parameters, the write-back works when Invoke sets args[i] itself. Forwarding a method with a ref, out or in pointer parameter through MethodInfo.Invoke still fails. This change leaves that as it is.

The conversions are C# methods in DispatchProxyGenerator that the generated proxies call directly; the proxy assembly gets an IgnoresAccessChecksTo attribute for System.Reflection.DispatchProxy once a proxy converts a pointer, as for internal proxied types. The IL generated for methods without pointers is unchanged, and so is the IL for function pointers.

The new tests in DispatchProxyTests cover pointer parameters, returns, ref/out/in pointers, void* and int**, forwarding through MethodInfo.Invoke, and pointer proxies in new AssemblyLoadContexts; the trimming test calls a pointer method.

Mono reads a proxy assembly's IgnoresAccessChecksTo attributes only once, so it misses one that a pointer proxy or a proxy of an internal type adds later; the new DispatchProxyTests tests are skipped on Mono, as asked in r4407294021.

Resolves #136451

Note

AI-generated, written at my direction and reviewed by me before posting.
Verified with this change on f0c7a08: DispatchProxy tests on Windows x64 CoreCLR, new ones failing without it; on Linux x64, CoreCLR with the trimming test and Mono JIT and interpreter, on an earlier version of the change.

The generated proxy method stored each argument in the object[] it
passes to DispatchProxy.Invoke with Convert(paramType -> object), which
for a pointer or function pointer fell through to `castclass object` on
the raw pointer. A non-null pointer argument made the call throw
NullReferenceException on CoreCLR before Invoke ran (or pass the pointee
to Invoke as an object when its first word was a MethodTable pointer),
and Mono rejected the method with InvalidProgramException on every call.
Pointer returns and by-ref pointer write-backs had the same problem in
the other direction (`castclass` to the pointer type).

Box and unbox pointers in the forms MethodInfo.Invoke uses for pointer
arguments and returns: a pointer argument reaches Invoke as a
System.Reflection.Pointer of the parameter's type and a function pointer
as an IntPtr; a pointer returned or written back by Invoke may be a
Pointer (pointers only), an IntPtr or null (a null pointer), and
anything else throws InvalidCastException.
By-ref pointers are read and written with ldind.i/stind.i. The IL for
every non-pointer parameter and return is unchanged.

Fix dotnet#136451

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Oct 25, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @steveisok, @dotnet/area-system-reflection
See info in area-owners.md if you want to be subscribed.

@AaronRobinsonMSFT

Copy link
Copy Markdown
Member

Thanks for reporting this. Fixing the invalid IL for ordinary pointer parameters, returns, and ref/out write-back looks like the right thing to do, but I’d prefer to keep this PR narrowly focused on that.

Please remove the function-pointer support. I believe proxy creation still fails for those signatures on CoreCLR, and addressing that broader capability brings additional signature-encoding and cross-runtime support considerations that aren’t necessary to fix the reported issue.

For the ordinary pointer fix, please make the intended conversion contract explicit, particularly the change from null to a boxed null pointer, and accepting a Pointer whose stored type differs from the declared pointer type. Those choices become observable behavior we need to maintain. Please also distinguish custom interceptor write-back from forwarding through MethodInfo.Invoke(), whose existing by-reference pointer limitations remain outside this fix.

The goal here should be a bounded correctness fix, not a commitment to supporting every CLR signature through DispatchProxy.

il.Emit(OpCodes.Conv_U);
il.Emit(OpCodes.Br_S, done);
il.MarkLabel(notNull);
il.Emit(OpCodes.Unbox_Any, typeof(IntPtr));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this match the conversion done by regular MethodInfo.Invoke for UIntPtr?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. On CoreCLR, MethodInfo.Invoke takes an IntPtr for any pointer parameter and a UIntPtr only for void*; on Mono it takes a UIntPtr for any pointer parameter. The proxy takes an IntPtr back for any pointer type and, on CoreCLR, throws InvalidCastException for a UIntPtr. Matching CoreCLR would take a second unbox helper that also accepts a UIntPtr, used only for void* returns and write-backs. Should I add it?

private static readonly MethodInfo s_dispatchProxyInvokeMethod = typeof(DispatchProxy).GetMethod("Invoke", BindingFlags.NonPublic | BindingFlags.Instance)!;
private static readonly MethodInfo s_getTypeFromHandleMethod = typeof(Type).GetMethod("GetTypeFromHandle", new Type[] { typeof(RuntimeTypeHandle) })!;
private static readonly MethodInfo s_makeGenericMethodMethod = GetGenericMethodMethodInfo();
private static readonly MethodInfo s_pointerBoxMethod = typeof(Pointer).GetMethod(nameof(Pointer.Box))!;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to be very rare path. It would be better to initialize these MethodInfos lazily.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 8322ba2: with the conversions moved to C# (the other thread), the generator no longer looks up Pointer.Box and Pointer.Unbox; the two Func<>.Invoke methods it emits calls to are looked up on first use.

}
}

// Unboxes a pointer from what MethodInfo.Invoke returns for one (a System.Reflection.Pointer for a pointer,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where possible, it is better to write more complex logic like this in C# and just call the C# method instead of manually inlining the IL.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 8322ba2: the conversions are now C# methods, BoxPointer and UnboxPointer in DispatchProxyGenerator. The generated code calls them through Func<> delegates held in two static fields of a PointerHelpers type, one per proxy assembly.

The proxies can't call the library's internal methods directly, and two other ways failed on Mono: IgnoresAccessChecksTo on the proxy assembly (Mono reads that attribute list once, so a pointer call before a proxy with an internal base type threw MethodAccessException), and [UnsafeAccessor] methods on an emitted type (Mono's TypeBuilder rejects a method without a body). With the delegates, the tests pass on CoreCLR, and passed on Mono, JIT and interpreter, before the last edits (comments, nameof, the load-context test). The trimming test now calls a pointer method.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-runtime pointer IL, conversion compatibility, and the Mono-only function-pointer path warrant final maintainer review.

No open issues 🎉

What changed in this PR

This PR fixes pointer argument and return handling in DispatchProxy by replacing invalid casts with explicit pointer conversions in generated IL.

Changes:

  • Adds Pointer and IntPtr conversions, including null handling and pointer-sized ref/out reads and writes.
  • Adds regression tests for ordinary, generic, and function pointers, plus reflection forwarding.
  • Enables unsafe test code; function-pointer tests exclude CoreCLR because of its existing signature-encoding limitation.
File Description
src/​libraries/​System.Reflection.DispatchProxy/​tests/​TestTypes.cs Adds pointer interfaces and forwarding implementations.
src/​libraries/​System.Reflection.DispatchProxy/​tests/​System.Reflection.DispatchProxy.Tests.csproj Enables unsafe blocks for pointer tests.
src/​libraries/​System.Reflection.DispatchProxy/​tests/​DispatchProxyTests.cs Tests pointer conversions, write-backs, and forwarding.
src/​libraries/​System.Reflection.DispatchProxy/​src/​System/​Reflection/​DispatchProxyGenerator.cs Emits pointer boxing, unboxing, and indirect-access instructions.

Review effort: Balanced

Leave function pointer parameters and returns as before:
DispatchProxy.Create still throws for them on CoreCLR, where
TypeBuilder can't encode function pointer types. A null pointer
argument now reaches Invoke as null, as it did on CoreCLR before,
instead of as a Pointer holding null.

Convert pointers in two C# methods instead of inline IL. Generated
proxies can't call the library's internal methods, so they call them
through delegates held in static fields of a PointerHelpers type,
defined once per proxy assembly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@caraioniurie47

Copy link
Copy Markdown
Contributor Author

@AaronRobinsonMSFT

Thanks for reporting this. Fixing the invalid IL for ordinary pointer parameters, returns, and ref/out write-back looks like the right thing to do, but I’d prefer to keep this PR narrowly focused on that....

Thanks, updated in 8322ba2, and the description now states the contract:

  • Function pointers are removed: their IL is as before, and the function pointer test is gone.
  • A null pointer argument now reaches Invoke as null, as it did on CoreCLR before this PR; a non-null pointer reaches it as a Pointer of the parameter's type.
  • Accepted back from Invoke (return or ref/out write-back): a Pointer, an IntPtr, or null as a null pointer; anything else is unboxed as an IntPtr, so an int or a string throws InvalidCastException. The pointer type stored in a Pointer isn't checked, since Pointer exposes it only internally.
  • ref/out write-back works when Invoke sets args[i] itself. Forwarding a method with a ref, out or in pointer parameter through MethodInfo.Invoke still fails; this PR doesn't change that.

private static MethodInfo? s_unboxPointerInvokeMethod;

private static MethodInfo BoxPointerInvokeMethod =>
s_boxPointerInvokeMethod ??= typeof(Func<IntPtr, Type, object?>).GetMethod(nameof(Func<IntPtr, Type, object?>.Invoke))!;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use field as the backing storage for lazily initialized properties

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next push moves the four lazily initialized properties this PR adds to field. One existing property in the file has the same pattern, DispatchProxyGenerator.cs:168-169:

internal ConstructorInfo IgnoresAccessChecksAttributeConstructor =>
    _ignoresAccessChecksToAttributeConstructor ??= IgnoreAccessChecksToAttributeBuilder.AddToModule(_mb);

Should it move to field in this PR too?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 747c7c8: the lazily initialized properties this PR adds use field. The two that return the delegate fields used to fill both fields from one helper type; each delegate now has a helper type of its own, so each property fills only its own backing field.

Type helpers = tb.CreateType();

const BindingFlags Flags = BindingFlags.NonPublic | BindingFlags.Static;
helpers.GetField(nameof(BoxPointer), Flags)!.SetValue(null, new Func<IntPtr, Type, object?>(BoxPointer));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So these delegates always point to the same method. Why do we need them? Can these helper methods be called directly?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the AI generated comments just rephrase what the code does in English. Such comments are not very useful.

Good comments explain why the code does what it does.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So these delegates always point to the same method. Why do we need them? Can these helper methods be called directly?

Not directly: the proxies live in a separate dynamic assembly, and BoxPointer and UnboxPointer aren't accessible outside System.Reflection.DispatchProxy, so a direct call needs IgnoresAccessChecksTo for it on the proxy assembly. Tested with the helpers internal, that passes on CoreCLR, but on Mono, JIT and interpreter, a pointer call followed by a proxy with an internal base type in the same load context throws MethodAccessException: Mono reads a dynamic assembly's IgnoresAccessChecksTo attributes only once, so it misses the one added later for the internal base type. A call through a delegate needs no IgnoresAccessChecksTo. If that Mono limitation is acceptable, I can switch to direct calls and drop the delegates.

Good comments explain why the code does what it does.

The next push also cuts or rewrites the comments that restate the code, and the delegates get a comment saying why. If the new comments look right, I'll do the same in #136450 and #136454. Thanks for the patience.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 747c7c8: the comments that restated the code are removed or rewritten to say why.

caraioniurie47 and others added 2 commits October 27, 2026 23:20
The lazily initialized properties this change adds use the field
keyword. The two that return the delegate fields used to fill both
fields from one helper type; each delegate now has a helper type of
its own, so each property fills only its own backing field.

Comments that restated the code are removed or rewritten to say why.

The load-context test is conditional on HasAssemblyFiles through
ConditionalTheory, as tests.instructions.md asks; the older tests
beside it return early instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CA1859 flagged the FieldInfo return type in the builds that run
analyzers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

// Proxies call BoxPointer and UnboxPointer through a delegate because they can't call them directly:
// the methods aren't accessible outside System.Reflection.DispatchProxy. IgnoresAccessChecksTo would
// allow the call on CoreCLR, but Mono reads a dynamic assembly's IgnoresAccessChecksTo attributes only

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mono is going to be deleted in near future. Workarounds for Mono bugs are not necessary - just disable the failing tests on Mono instead.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9cd377b: the proxies call BoxPointer and UnboxPointer directly; the delegates and their helper types are gone. A proxy that converts a pointer gets an IgnoresAccessChecksTo attribute for System.Reflection.DispatchProxy on its proxy assembly, through the existing EnsureTypeIsVisible. Mono reads a proxy assembly's IgnoresAccessChecksTo attributes only once, so it misses one that a pointer proxy or a proxy of an internal type adds later. The six pointer tests are skipped on Mono; without the skips they all failed there with MethodAccessException, on the JIT and the interpreter.

Proxies call BoxPointer and UnboxPointer directly instead of through
delegates in helper types. A proxy that converts a pointer gets its
proxy assembly an IgnoresAccessChecksTo attribute for
System.Reflection.DispatchProxy, through EnsureTypeIsVisible, as for
internal proxied types.

Mono reads a dynamic assembly's IgnoresAccessChecksTo attributes only
once, so it misses the one a pointer proxy or a proxy of an internal
type adds later. The pointer tests are skipped on Mono.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-System.Reflection community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DispatchProxy: calling a method with a non-null pointer argument throws NullReferenceException

4 participants