consteval: Rework caller location handling by theemathas · Pull Request #163746 · rust-lang/rust

theemathas · 2026-10-04T09:50:19Z

Instead of walking up the call stack each time we need the caller location, we emulate the run time behavior of passing an implicit caller location argument in #[track_caller] functions. We record a representation of that argument in each frame. For inlined functions though, we still do a "stack walk" within that one frame.

This PR is a prerequisite for #163396

Fixes rust-lang/miri#5351

theemathas · 2026-10-04T09:53:21Z

+                // Figure out if the callee receives a caller location argument.
+                // In the case where we are calling the fallback body of
+                // a `#[rustc_intrinsic] #[track_caller] fn`, we pretend that
+                // the caller doesn't pass a caller location argument (for checking ABI),
+                // but the caller receives a caller location argument from thin air anyway.
+                let callee_receives_caller_location = with_caller_location
+                    || (instance.def.requires_caller_location(*self.tcx)
+                        && matches!(instance.def, InstanceKind::Item(def_id) if self.tcx.intrinsic(def_id).is_some()));
+


This is extremely hacky, but I don't have a better idea how to handle this.

View changes since the review

Yeah this is very strange. What problem is this even solving?

For a #[rustc_intrinsic] #[track_caller] fn, there are two Instances involved.

An InstanceKind::Intrinsic. This is what the caller of the intrinsic "sees". requires_caller_location returns false.

An InstanceKind::Item. This is the fallback body, and is therefore what the callee "sees". requires_caller_location returns true.

In Machine::call_intrinsic, when such an intrinsic is called, "replaces" the former Instance with the latter Instance. However, it does not recheck to see if the new Instance needs a caller location.

callee_receives_caller_location is whether the callee receives a caller location argument, regardless of whether the caller passes one. These two things can mismatch for #[rustc_intrinsic] #[track_caller] fn.

Now that I think about it again, I probably should move this hack to next to the Machine::call_intrinsic call.

For a #[rustc_intrinsic] #[track_caller] fn

I am not sure that's a combination anyone ever thought about. But having two difference instances like that with different signature sounds like a bug. Cc @oli-obk

Some more context at #t-compiler/help > Help with `async_fn_track_caller` @ 💬

IIRC, codegen explicitly handles this case by replacing a call to the InstanceKind::Intrinsic with a call to the InstanceKind::Item, and the replacement happens before the decision on whether to pass an extra caller location argument.

Intrinsics in the stdlib that are currently #[track_caller]: unchecked_disjoint_bitor, unchecked_funnel_shl, unchecked_funnel_shr

Sigh, what a mess.

For us, that replacement happens around here:

rust/compiler/rustc_const_eval/src/interpret/call.rs

Lines 769 to 779 in 693bf50

assert!(!self.tcx.intrinsic(fallback.def_id()).unwrap().must_be_overridden);

assert_matches!(fallback.def, ty::InstanceKind::Item(_));

return self.init_fn_call(

FnVal::Instance(fallback),

(caller_abi, caller_fn_abi),

args,

with_caller_location,

destination,

target,

unwind,

);

So we could probably update with_caller_location there?

Yeah, that's what I meant by "I probably should move this hack to next to the Machine::call_intrinsic call."

theemathas · 2026-10-06T15:43:55Z

r? @RalfJung
(or reassign)

rustbot · 2026-10-06T15:44:01Z

RalfJung is not on the review rotation at the moment.
They may take a while to respond.

rustbot · 2026-10-06T15:44:02Z

Some changes occurred to the CTFE machinery

cc @RalfJung, @oli-obk, @lcnr

miri is developed in its own repository. If the Miri part of this change can be broken out, consider making this change to rust-lang/miri instead. However, if Miri needs adjusting for rustc changes, just ignore this message.

cc @rust-lang/miri

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri

RalfJung

Thanks a lot!

Overall I like it, except for what's happening during stack frame init. That part I haven't really understood yet but it's not like what I naively thought it should be.

View changes since this review

RalfJung · 2026-10-15T20:50:33Z

        let _trace = enter_trace_span!(M, step::init_stack_frame, %instance, tracing_separate_thread = Empty);
        let def_id = instance.def_id();

+        assert!(!with_caller_location || callee_receives_caller_location);


Why are these two separate flags and what is the point of this assertion?

RalfJung · 2026-10-15T20:52:12Z

                .collect::<Vec<_>>()
        );

+        self.frame_mut().track_caller_arg = caller_location;


Suggested change

self.frame_mut().track_caller_arg = caller_location;

// Initialize special caller_location "local", if this function has it.

self.frame_mut().track_caller_arg = caller_location;

RalfJung · 2026-10-15T20:52:46Z

        let def_id = instance.def_id();

+        assert!(!with_caller_location || callee_receives_caller_location);
+        let caller_location = callee_receives_caller_location.then(|| self.caller_location());


Doesn't the Instance tell us whether the callee will get a caller location?

RalfJung · 2026-10-15T20:53:39Z

+        // we will not pass normally; our `caller_location` intrinsic implementation handles
+        // that separately).
        assert_eq!(
            args.len() + if with_caller_location { 1 } else { 0 },
            caller_fn_abi.args.len(),
            "mismatch between caller ABI and caller arguments",
        );


So now we need caller location hacks in two places. If we could instead integrate this with the argument passing we'd reduce the number of hacks here rather than increasing them -- that would be a lot nicer I think.

RalfJung · 2026-10-15T20:54:25Z

+                // Figure out if the callee receives a caller location argument.
+                // In the case where we are calling the fallback body of
+                // a `#[rustc_intrinsic] #[track_caller] fn`, we pretend that
+                // the caller doesn't pass a caller location argument (for checking ABI),
+                // but the caller receives a caller location argument from thin air anyway.
+                let callee_receives_caller_location = with_caller_location
+                    || (instance.def.requires_caller_location(*self.tcx)
+                        && matches!(instance.def, InstanceKind::Item(def_id) if self.tcx.intrinsic(def_id).is_some()));
+


Yeah this is very strange. What problem is this even solving?

RalfJung · 2026-10-15T20:55:20Z

+        assert_eq!(
+            frame.track_caller_arg.is_some(),
+            frame.instance.def.requires_caller_location(*self.tcx)
+        );


Strange to check this here rather than during frame initialization.

By the way, I just found out that this assertion fails on both of the code snippets in rust-lang/miri#5351

Those snippets should cause UB before ever reaching the assertion, shouldn't they? So I think the assertion should be put back. The PR also still has a comment on the track_caller_arg field claiming this assertion as an invariant.

The updated PR now detects UB at the point the mismatched function call happens.

Yes. But asserting the invariant here would still be useful IMO. We have a lot of assertions in the interpreter because it's so easy to accidentally invalidate an invariant.

RalfJung · 2026-10-15T20:55:41Z

    /// frame is popped.
    pub(super) va_list: Vec<MPlaceTy<'tcx, Prov>>,

+    /// The implicit caller location argument that is passed by this function's caller, if any.


Suggested change

/// The implicit caller location argument that is passed by this function's caller, if any.

/// The implicit caller location argument that is passed by this function's caller, if any.

/// This is `Some` if and only if `instance.def.requires_caller_location` returns `true`.

RalfJung · 2026-10-15T20:59:53Z

        }

-        span_bug!(self.cur_span(), "no non-`#[track_caller]` frame found")
+        frame.body.caller_location_span(source_info, frame.track_caller_arg, *self.tcx, identity)


Suggested change

frame.body.caller_location_span(source_info, frame.track_caller_arg, *self.tcx, identity)

// Directly returning `frame.track_caller_arg` would be wrong if we are

// in code that got inlined from another function. `caller_location_span`

// handles that for us.

frame.body.caller_location_span(source_info, frame.track_caller_arg, *self.tcx, identity)

theemathas · 2026-10-17T06:32:21Z

@rustbot author

rustbot · 2026-10-17T06:32:26Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

theemathas · 2026-10-17T11:36:04Z

@rustbot ready

This PR now also fixes rust-lang/miri#5351

I'll squash the commits into one before merging.

theemathas · 2026-10-17T12:12:04Z

what on earth is going on with #131787 lol

theemathas · 2026-10-17T12:24:19Z

For some reason, 974786e caused #131787 to stop ICE-ing?

theemathas · 2026-10-17T12:33:07Z

Calling push_stack_frame_raw directly without going through init_stack_frame causes the caller location check to be bypassed in the current PR's state. This call is done at

rust/compiler/rustc_const_eval/src/const_eval/eval_queries.rs

Line 114 in 158f79d

ecx.push_stack_frame_raw(

. What should we do about this, if anything? Maybe I should keep the assertion as-is in caller_location, instead of removing it as in 974786e?

RalfJung · 2026-10-18T08:12:27Z

+        // they go to. (For the former, we expect the caller to do any special handling
+        // that might be required here (e.g. for untupling)).


Why is this a parenthetical now?
The new wording isn't worse than the old one but also not better IMO so I am surprised to see a diff here.

View changes since the review

This was moved from being part of a separate paragraph into being in the same paragraph. I think it's clearer that the text is about a new "topic" if it's in parentheses.

The old structure of the comment was as follows:

There are two iterators

"where they come from" part is easy, we expect the caller to do any special handling begins setting up the first iterator

Now we have to spread them out across the callee's locals, begins setting up the 2nd iterator

I can see how that was not very clear, but your changes make it even worse IMO. I would instead suggest making the two sections this is delineating more explicit. (Happy to draft a commit for that if you prefer.)

RalfJung · 2026-10-18T08:14:49Z

+        // the two iterators should again have the same length.
+        //
+        // We could have an assertion that checks the iterator lengths, but plumbing through
+        // the necessary information is annoying, so we don't bother.


I am very unhappy about losing this assertion. Checking the number of arguments is a pretty basic sanity check.

It is also very concerning that we'd be using the wrong caller ABI here. That's something we should fix, not sweep under the rug.

View changes since the review

Do you mean that something in the original code is incorrect, and we shouldn't need to add 1 for the caller location argument in the assertion?

Or maybe.... I should re-compute the function ABI when I do the intrinsic fallback?

I think I figured out a better way of doing this.

Do you mean that something in the original code is incorrect, and we shouldn't need to add 1 for the caller location argument in the assertion?

Yeah. Apparently the ABI can change when swapping an intrinsic for its fallback, and the caller will actually use the updated ABI, so we have to re-compute the caller ABI.

RalfJung · 2026-10-18T08:19:36Z

+        assert_eq!(
+            frame.track_caller_arg.is_some(),
+            frame.instance.def.requires_caller_location(*self.tcx)
+        );


Those snippets should cause UB before ever reaching the assertion, shouldn't they? So I think the assertion should be put back. The PR also still has a comment on the track_caller_arg field claiming this assertion as an invariant.

theemathas · 2026-10-18T14:32:48Z

@rustbot ready

Not 100% sure if shuffling around the tests like this makes sense. @RalfJung

RalfJung · 2026-10-18T14:52:39Z

Hm, this seems to have tested something that's not specific to static / const-eval at all, so the way it ICEs now is new. @JonathanBrouwer is there another way to test #105594 that doesn't rely on bogus MIR being passed to const-eval?

RalfJung · 2026-10-18T14:57:12Z

+    // Note: Promoteds don't require a caller location,
+    // even when inside an `Instance` with `#[track_caller]`
+    assert_eq!(body.arg_count, 0);
+    assert!(cid.promoted.is_some() || !cid.instance.def.requires_caller_location(tcx));


Maybe an instead of an assertion we can make this a delay_bug, with a FIXME pointing at #131787 saying that it should be an assertion? The delay_bug gives us a ErrorGuaranteed, we can then call something like throw_inval!(AlreadyReported(ReportedErrorInfo::non_const_eval_error)) to avoid executing the bogus MIR.

View changes since the review

Done. However, this seems to have gotten rid of the ICE in #131787 entirely. What should I now do with tests/crashes/131787.rs?

Both of these tests do almost the same thing, don't they? I'd suggest adding a panic to the body of the static in issue-105594-invalid-attr-validation.rs, then deleting the other test as a duplicate.

Done. This PR should be ready to merge now.

JonathanBrouwer · 2026-10-18T15:08:35Z

I can't immediately think of anything, ideally the code from the current test should just not pass bogus MIR to const eval at all ofcourse
I've been thinking a while that attributes that are rejected by target checking should be filtered out

Instead of walking up the call stack each time we need the caller location, we emulate the run time behavior of passing an implicit caller location argument in #[track_caller] functions. We record a representation of that argument in each frame. For inlined functions though, we still do a "stack walk" within that one frame.

RalfJung · 2026-10-18T16:22:46Z

Next try...
@bors r+

rust-bors · 2026-10-18T16:22:49Z

📌 Commit ba7e70d has been tentatively approved by RalfJung

It will be put into the queue for this repository once PR CI succeeds.

rust-bors · 2026-10-18T18:26:01Z

⌛ Testing commit ba7e70d with merge 5055b09...

Workflow: https://github.com/rust-lang/rust/actions/runs/39154127389

consteval: Rework caller location handling Instead of walking up the call stack each time we need the caller location, we emulate the run time behavior of passing an implicit caller location argument in `#[track_caller]` functions. We record a representation of that argument in each frame. For inlined functions though, we still do a "stack walk" within that one frame. This PR is a prerequisite for #163396 Fixes rust-lang/miri#5351

…on, r=RalfJung consteval: Rework caller location handling Instead of walking up the call stack each time we need the caller location, we emulate the run time behavior of passing an implicit caller location argument in `#[track_caller]` functions. We record a representation of that argument in each frame. For inlined functions though, we still do a "stack walk" within that one frame. This PR is a prerequisite for rust-lang#163396 Fixes rust-lang/miri#5351

JonathanBrouwer · 2026-10-18T18:29:51Z

@bors yield
Yielding to enclosing rollup

rust-bors · 2026-10-18T18:29:55Z

Auto build was cancelled. Cancelled workflows:

https://github.com/rust-lang/rust/actions/runs/39154127389

The next pull request likely to be tested is #164531.

…uwer Rollup of 2 pull requests Successful merges: - #163746 (consteval: Rework caller location handling) - #163611 (add core::process::resumable_breakpoint)

…uwer Rollup of 4 pull requests Successful merges: - #160912 (Rename `bstr` module/feature to `byte_str`) - #164463 (perf: inline `Clause::map_bound`) - #163746 (consteval: Rework caller location handling) - #163611 (add core::process::resumable_breakpoint)

Rollup merge of #163746 - theemathas:consteval-caller_location, r=RalfJung consteval: Rework caller location handling Instead of walking up the call stack each time we need the caller location, we emulate the run time behavior of passing an implicit caller location argument in `#[track_caller]` functions. We record a representation of that argument in each frame. For inlined functions though, we still do a "stack walk" within that one frame. This PR is a prerequisite for #163396 Fixes rust-lang/miri#5351

…uwer Rollup of 4 pull requests Successful merges: - rust-lang/rust#160912 (Rename `bstr` module/feature to `byte_str`) - rust-lang/rust#164463 (perf: inline `Clause::map_bound`) - rust-lang/rust#163746 (consteval: Rework caller location handling) - rust-lang/rust#163611 (add core::process::resumable_breakpoint)

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Oct 4, 2026

theemathas force-pushed the consteval-caller_location branch from a401e74 to 48523ba Compare October 4, 2026 09:52

theemathas commented Oct 4, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

theemathas mentioned this pull request Oct 5, 2026

Make #[track_caller] async fn track the caller, not the poller/awaiter #163396

Open

theemathas force-pushed the consteval-caller_location branch from 48523ba to e0647c4 Compare October 6, 2026 14:27

This comment has been minimized.

Sign in to view

theemathas force-pushed the consteval-caller_location branch from e0647c4 to ab6523e Compare October 6, 2026 14:37

rustbot assigned RalfJung Oct 6, 2026

theemathas marked this pull request as ready for review October 6, 2026 15:43

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 6, 2026

rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Oct 6, 2026

theemathas force-pushed the consteval-caller_location branch from ab6523e to e620c7f Compare October 6, 2026 15:45

RalfJung reviewed Oct 15, 2026

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 17, 2026

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 17, 2026

This comment has been minimized.

Sign in to view

RalfJung reviewed Oct 18, 2026

View reviewed changes

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 18, 2026

theemathas mentioned this pull request Oct 18, 2026

ICE: no non-'#[track_caller]' frame found #131787

Open

RalfJung reviewed Oct 18, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

theemathas force-pushed the consteval-caller_location branch 2 times, most recently from 45058f9 to 28f4050 Compare October 18, 2026 15:39

theemathas force-pushed the consteval-caller_location branch from 28f4050 to ba7e70d Compare October 18, 2026 15:56

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 18, 2026

JonathanBrouwer mentioned this pull request Oct 18, 2026

Rollup of 2 pull requests #164531

Closed

JonathanBrouwer mentioned this pull request Oct 18, 2026

Rollup of 4 pull requests #164535

Merged

rust-bors Bot merged commit 66d841c into rust-lang:main Oct 18, 2026
14 of 15 checks passed

rustbot added this to the 1.101.0 milestone Oct 18, 2026

theemathas deleted the consteval-caller_location branch October 19, 2026 02:11

	assert!(!self.tcx.intrinsic(fallback.def_id()).unwrap().must_be_overridden);
	assert_matches!(fallback.def, ty::InstanceKind::Item(_));
	return self.init_fn_call(
	FnVal::Instance(fallback),
	(caller_abi, caller_fn_abi),
	args,
	with_caller_location,
	destination,
	target,
	unwind,
	);

	self.frame_mut().track_caller_arg = caller_location;
	// Initialize special caller_location "local", if this function has it.
	self.frame_mut().track_caller_arg = caller_location;

	/// The implicit caller location argument that is passed by this function's caller, if any.
	/// The implicit caller location argument that is passed by this function's caller, if any.
	/// This is `Some` if and only if `instance.def.requires_caller_location` returns `true`.

-        frame.body.caller_location_span(source_info, frame.track_caller_arg, *self.tcx, identity)
+        // Directly returning `frame.track_caller_arg` would be wrong if we are
+        // in code that got inlined from another function. `caller_location_span`
+        // handles that for us.
+        frame.body.caller_location_span(source_info, frame.track_caller_arg, *self.tcx, identity)

		// they go to. (For the former, we expect the caller to do any special handling
		// that might be required here (e.g. for untupling)).

Conversation

theemathas commented Oct 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theemathas Oct 4, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

theemathas commented Oct 6, 2026

Uh oh!

rustbot commented Oct 6, 2026

Uh oh!

rustbot commented Oct 6, 2026

Uh oh!

RalfJung left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

theemathas commented Oct 17, 2026

Uh oh!

rustbot commented Oct 17, 2026

Uh oh!

theemathas commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

theemathas commented Oct 17, 2026

Uh oh!

theemathas commented Oct 17, 2026

Uh oh!

theemathas commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RalfJung Oct 18, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

theemathas commented Oct 4, 2026 •

edited

Loading

theemathas Oct 4, 2026 •

edited by rustbot

Loading

RalfJung left a comment •

edited by rustbot

Loading

theemathas commented Oct 17, 2026 •

edited

Loading

theemathas commented Oct 17, 2026 •

edited

Loading

RalfJung Oct 18, 2026 •

edited by rustbot

Loading

RalfJung Oct 18, 2026 •

edited

Loading

RalfJung Oct 18, 2026 •

edited by rustbot

Loading

theemathas commented Oct 18, 2026 •

edited

Loading

RalfJung Oct 18, 2026 •

edited by rustbot

Loading