Rollup of 7 pull requests - #163189
Merged
Merged
Rollup of 7 pull requests#163189
Conversation
…erparts on OnceLock public
Update deprecated rustc_hir imports Followup to rust-lang#160336 That's ~half of them, I'm not doing it all in one go to reduce conflicts.
alloc: stabilise `Allocator` # Allocator stabilisation report Reference PR: - rust-lang/reference#2364 This is the stabilisation report for a subset of the feature `allocator_api`, with tracking issue [rust-lang#32838](rust-lang#32838) under the purview of wg-allocators, initially proposeed by [RFC rust-lang#1398](https://rust-lang.github.io/rfcs/1398-kinds-of-allocators.html). The remainder of the feature will be renamed to `allocator_ext`. This was a collaborative effort of t-libs, wg-allocators, members of t-types, t-lang, and t-opsem, alongside interested parties in the ecosystem and contributors to the initial attempt at stabilisation on GitHub. See also the new [wg-allocators roadmap](rust-lang/wg-allocators#150) on the matter. ## Summary The following is a proposal following several conversations, [in-person](rust-lang/all-hands-2026#48) and [online](https://rust-lang.zulipchat.com/#narrow/channel/197181-t-libs.2Fwg-allocators), with libs team members and interested ecosystem participants and represents an attempt at stabilising an MVP for the `Allocator` trait and its implementation safety requirements, alongside minimal functionality to make its use possible in the standard library. While an effort was made to align with the stated positions of the team, the opinions and rationale stated are **the author's own**, and should **not** be seen as representative of the libs(-api) team as a whole except insofar as individual members therein choose to endorse the contents of report. Any mention of "we", "us", etc. should be understood to refer to the author alongside those who have explicitly expressed agreement. ## API & considerations The stabilised API surface consists of: ```rust unsafe trait Allocator { // Required methods fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError>; unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout); // Provided methods fn allocate_zeroed( &self, layout: Layout, ) -> Result<NonNull<[u8]>, AllocError> { ... } unsafe fn grow( &self, ptr: NonNull<u8>, old_layout: Layout, new_layout: Layout, ) -> Result<NonNull<[u8]>, AllocError> { ... } unsafe fn grow_zeroed( &self, ptr: NonNull<u8>, old_layout: Layout, new_layout: Layout, ) -> Result<NonNull<[u8]>, AllocError> { ... } unsafe fn shrink( &self, ptr: NonNull<u8>, old_layout: Layout, new_layout: Layout, ) -> Result<NonNull<[u8]>, AllocError> { ... } } // N.B.: This particular point is lang-relevant since `Box` // would be stabilised without being fundamental over `A`. struct Box<T, #[stable(...)] A: Allocator>(...) impl<T, A: Allocator> Box<T, A> { fn new_in(x: T, alloc: A) -> Box<T, A>; } impl<T: ?Sized, A: Allocator> Box<T, A> { unsafe fn from_raw_in(raw: *mut T, alloc: A) -> Self; unsafe fn from_non_null_in(raw: NonNull<T>, alloc: A) -> Self; fn into_raw_with_allocator(b: Self) -> (*mut T, A); fn into_non_null_with_allocator(b: Self) -> (NonNull<T>, A); fn allocator(b: &Self) -> &A; } struct Vec<T, #[stable(...)] A: Allocator> { ... } impl<T, A: Allocator> Vec<T, A> { fn new_in(alloc: A) -> Vec<T, A>; fn with_capacity_in(capacity: usize, alloc: A) -> Self; unsafe fn from_raw_parts_in( ptr: *mut T, length: usize, capacity: usize, alloc: A, ) -> Self; unsafe fn from_parts_in( ptr: NonNull<T>, length: usize, capacity: usize, alloc: A, ) -> Self; fn into_raw_parts_with_allocator(self) -> (*mut T, usize, usize, A); fn into_parts_with_allocator(self) -> (NonNull<T>, usize, usize, A); fn allocator(&self) -> &A; } struct Global; // implementor of Allocator struct System; // implementor of Allocator unsafe impl<A: Allocator + ?Sized> Allocator for &A { ... } unsafe impl<A: Allocator + ?Sized> Allocator for &mut A { ... } unsafe impl<A: Allocator + ?Sized> Allocator for Box<A, _> { ... } unsafe impl<A: Allocator + ?Sized> Allocator for Rc<A, _> { ... } unsafe impl<A: Allocator + ?Sized> Allocator for Arc<A, _> { ... } ``` The `by_ref` method on `Allocator` was removed, as it was only a postfix syntax convenience (equivalent to writing `(&alloc)`). The safety requirements on implementors of `Allocator` were tightened to the most restrictive sound form we expect to possibly want, in order to enable us to iterate on the design in the future and relax these bounds if it is deemed possible. Notable changes from the form assumed before the stabilisation effort started: - the implicit requirement that the standard library's `Clone for Arc<T, A>` implementation relied on but was improperly documented, for implementors not to invalidate allocated memory on drop or mutable access was made explicit; - implementors are now required not unwind from any of the methods on the trait or from drop; - the safety invariants implementors of `Allocator + Clone` must uphold were moved to their own unstable marker subtrait as the requirement was deemed unjustifiable. Additionally, [it was decided](rust-lang#156906) that `Allocator` will be `dyn`-compatible, as the resulting constraints on the design were deemed acceptable given the significant increased flexibility for users. ## Soundness developments The process of attempting stabilisation resulted in several soundness issues arising, especially with regard to the interaction between custom allocators and `Box`es. Thus, some points had to be adjusted: - there existed a requirement for trait implementors to obey certain semantics if an implementor of `Allocator` is also `Clone`, which constituted possible UB if broken. These have been dropped, as unsafe implementors [cannot guard against possible unsoundness](rust-lang#156920) from incorrect implementations in downstream safe code, but the equivalent functionality may be added backwards-compatibly with an unsafe marker trait or language mechanism; - `Box::into_pin` will not yet be possible with custom allocators. This is because of a [soundness bug](rust-lang#157089) relating to an interaction between the possibility of manually implementing `Clone for Box<T, A>` and `Box` being covariant over `A`, allowing for a pinned box to be cloned with a non-`'static` allocator from one with a correct `'static` allocator subtyped to a non-static one. Making `Box` invariant over the allocator was considered, but was deemed far too limiting and would technically be a [breaking change](rust-lang#153607) to reverse later. Thus, for now, an unstable and unsafe marker trait `StaticAllocator` will be introduced to mark an allocator as guaranteeing that its allocations live for `'static` (i.e. will never be lost unless explicitly de/reallocated). This will be implemented for the `Global` and `System` allocators; - due to `Pin`'s preexisting implementation of a safe `Pin::new` for any pointer type where `<Ptr as Deref>::Target: Unpin`, it *will* be stably possible to call `Pin::new()` on a box with a custom allocator as changing this would require significant special-casing in trait resolution. Experiments in this direction surfaced a [soundness bug](rust-lang#159445 (comment)), addressed by tightening the requirements of `impl PinSafePointer for Box` to necessitate a pin-safe `StaticAllocator`; - further discussion revealed that these same semantics are necessary for integrating custom allocators into LLVM's proposed semantics for allocator intrinsics, as below; - a preexisting hack whereby `Box` had `noalias` semantics for its pointer if and only if the allocator is `Global` - alongside a similar hack to make ["unleaking"](https://doc.rust-lang.org/std/boxed/struct.Box.html#method.leak) work - is to be moved to an unstable wrapper type `NativeAllocator<A: StaticAllocator>`, which enables us to make use of LLVM's new [allocator intrinsics](https://rust-lang.zulipchat.com/#narrow/channel/136281-t-opsem/topic/Enabling.20compiler.20magic.20on.20custom.20allocators). `Global` would then be equivalent to `NativeAllocator<A>` with the concrete allocator substituted in. Per a conversation with members of opsem, this appeared to be a reasonable way forward; - unwinding out of many allocation-related methods was found to be a pervasive source of unsoundness. Thus, language on allocator cloning and allocation methods was expanded so as to ensure unwinds never come out of methods on an `Allocator`, clones, or drops. ## Backwards-compatible changes Several designs were considered to extend or modify the trait's semantics. We have opted to defer full consideration of many of these for later, as we have determined they can be added backwards-compatibly to the existing API. A list of these is present below, alongside rationale for their postponement. ### `Store` API This is an alternative, more complex proposal for custom allocators (see the [draft RFC](rust-lang/rfcs#3446)). Per a conversation in-person with one of the authors of the `Store` proposal, we have established that it could be added backwards-compatibly (in `Store` terminology, the stabilised `Allocator` trait is effectively a storage with pointer handles). The details were thus deferred for potential post-stabilisation changes. ### Split `Deallocator` trait [Supertrait item shadowing](rust-lang#89151) alongside a blanket `impl<A: Allocator + ?Sized> Deallocator for A` will allow us to add a `Deallocator` supertrait backwards-compatibly, and to relax the requirements for collection types to insted hold a `Deallocator`. Conversations with those involved in the above issue suggest it is likely for a PR implementing this to be merged in the near future. ### `fn reallocate()` The current design uses dedicated `grow`, `grow_zeroed`, and `shrink` methods instead of a way to reallocate between arbitrary sizes. However, such a function could be added with a defaulted body in the future, forwarding to the extant `grow`/`shrink` implementations. ### Conditional reentrancy in `std` Not all allocators will be [reentrant in `std`](rust-lang/libs-team#743), and thus the standard library may want to be able to conditionally call the global allocator in areas it has otherwise promised not to. Thus, the proposed unstable `GlobalAllocator: Allocator` marker trait could be extended with a defaulted associated constant `REENTRANT_IN_STD: bool = true` wherein implementors could promise that a certain allocator never calls *any* part of `std`. ## Possible but less clean additions Several options appeared to signal compelling usecases, but were sufficiently niche that we did not consider them to be blocking for an MVP stabilisation so long as it was realistically possible to express their semantics. ### Associated constants Several usecases would be facilitated by having certain associated items on the `Allocator` trait; notably, `const MIN_ALIGN: usize` for the minimum alignment an allocator is always guaranteed to return. Adding the semantics of these backwards-compatibly would rely on maybe trait bounds being stabilised, which per conversations with the lang & types teams we believe is feasible in the near future. Alternatively, much of the same functionality could be added with defaulted `const` methods, which are also on the stabilisation path. ### `grow_in_place()` There is currently no obvious way to signal through the API whether a move of the data is acceptable when reallocating memory. Though messy, a way to express these semantics with the current design does exist, even if non-obvious: ```rust struct A; // `grow`/`grow_zeroed` have non-in-place semantics unsafe impl Allocator for A { /* ... */ } struct B(A); // in-place-grow semantics unsafe impl Allocator for B { /* ... */ } impl A { fn as_pinning(self) -> B { B(self) } } impl B { fn as_nonpinning(self) -> A { self.0 } } ``` We have decided that this is acceptable, given that it is "only" a point of design and not underlying functionality. A cleaner way to signal such semantics would be of interest for future extensions to the trait. Notably, in-place growing and/or shrinking without invalidating preexisting pointers (i.e. actually changing the size of the allocation in the abstract machine) needs proper support from LLVM which may not happen in the near future. ### Allocation flags A similar transmute-based mechanism as for the above can be used to reference a local inside of the allocator, though this could be UB-prone. Alternatively, and much more nicely, argument splatting could allow us to backwards-compatibly extend the trait (assuming implementors as well as callers may ignore optional fields). However, this would depend on the details of such a proposal. The main stakeholder who approached us with concerns on this topic - Rust for Linux - signalled willingness to maintain a downstream extension trait for such functionality for the time being. ## Rejected alternative proposals The following changes were explicitly not made to the API pre-stabilisation, despite it being unlikely that their semantics could be nicely expressed in the (near) future. In all cases, notable arguments existed to make the requested change, but we decided they were not sufficiently compelling. Should a way to express these semantics emerge in the future backwards-compatibly, we would be open to re-reviewing them. ### `NonZeroLayout` arguments An idea had been proposed to change the signature of the allocating/freeing methods to take a `Layout` that is guaranteed to have a nonzero size. We determined that API cleanliness and potential simplification of library code (once `const Trait`s are stable, collection types could drop special-case logic when using a `const Allocator` at zero capacity) outweigh the arguments for not allowing zero-sized allocations. As we see it, in the cases where it would genuinely be problematic, this will only move the branch on zero-sized allocation to the other side of the call. At worst, it would put marginally more pressure on a branch predictor. Though the possibility of zero-size allocations being probematic is often mentioned, we have not seen sufficiently convincing concrete cases where this is the case. One pointed-to example was that of highly performance-sensitive allocators (e.g. bump allocators); however, it appears most of these cases can trivially support zero-size allocations (e.g. bumping by zero). Consequently, we have decided to keep the nicer logic for downstream users of the trait. An argument had also been made around `jemalloc` being unable to correctly handle zero-sized allocations, but this appears to only apply to internal APIs. A similar idea wherein `allocate` was an unsafe method and support for zero-sized allocations was implementation-defined was rejected on similar usability grounds. Several members of the libs team expressed their opinion that the design of `GlobalAlloc` (featuring a similar unsafe allocating method wherein the caller must guarantee the size is nonzero) was not desirable in hindsight. ### `NonNull<u8>` return type Lacking a better way to signal returned vs. requested capacity, and not wishing to duplicate all of the allocating methods, we have decided that we would prefer to keep the wide-pointer return value and potentially use that logic to determine capacity in returned allocations. This will never be an issue with regard to performance, as no architecture allocates expects fewer than two registers to be clobbered by a function call, and so there is no cost to returning the wide pointer. Some callers will elect to ignore extra capacity; similarly, some implementors will elect not to offer it. The language around implementation safety ensures that these cases are supported, and recommends that implementors not signal extra capacity if it would be expensive to do so. That is to say, both the caller and implementor must cooperate for the excess to be meaningfully usable; otherwise there is no performance impact in a correct implementation. ### Associated types Having an associated type, especially for the returned error on allocation failure, had been mentioned as a possible addition; however, doing so would add significant complexity to the trait while also making `dyn`-compatibility impossible. Few concrete usecases came up where the allocator itself has meaningful error information that would be actionable to callers, and therefore it was elected to keep the current ZST `AllocError`. ## Future work A large part of the standard library will need review as we determine what the correct way is for various collection and pointer types to work with custom allocators. Notably, there are multiple outstanding proposals for integrating fallible allocation APIs into the standard library, and a stable mechanism needs to be decided on for exposing the `Allocator` + `Clone` interaction. ## Outlined potential extensions The following is a possible future outline of what the `Allocator` trait and related might look like under this proposal, assuming both of supertrait item shadowing and defaulted associated items being added: ```rust unsafe trait Allocator: Deallocator { fn allocate( &self, layout: Layout, ) -> Result<NonNull<[u8]>, AllocError>; unsafe fn deallocate( &self, ptr: NonNull<u8>, layout: Layout, ); // Provided methods unsafe fn reallocate( &self, ptr: NonNull<u8>, old_layout: Layout, new_layout: Layout, ) -> Result<NonNull<[u8]>, AllocError> { ... } /// Minimum alignment that will always be returned, regardless /// of what alignment is requested. const fn min_align(&self) -> usize { 1 } } unsafe trait Deallocator { unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout); } // Enabled by supertrait item shadowing. impl<A: Allocator> Deallocator for A { unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout) { <Self as Allocator>::deallocate(self, ptr, layout) } } /// The allocator is suitable for use as the global allocator. /// /// # Safety /// /// `reentrant_in_std` must never be `false` incorrectly, and /// if `true`, the allocator may only use those parts of `std` /// which explicitly allow themselves to be called from the global /// allocator (such as thread-locals). unsafe trait GlobalAllocator: Allocator + Sync + 'static { const REENTRANT_IN_STD: bool; } /// `Clone` will create an equivalent (de)allocator (i.e. both /// can deallocate the same memory), and `Copy` either is /// the same as clone (i.e. `Clone` is a memcpy) or impossible /// to implement. unsafe trait AllocatorClone: Deallocator + Clone {} /// Polls allocator equivalence. unsafe trait AllocatorEq<Other: AllocatorEq = Self>: Deallocator { /// If `other` can free something, so can `self`. /// Implementors must never incorrectly return `true`, /// and equality must be transitive and reflexive. fn is_equivalent(&self, other: &Other) -> bool; } /// The allocator in question will not break `Pin` guarantees /// even if subtyped with a shorter lifetime; that is, memory /// is never deallocated except via an explicit call to `deallocate` /// (and not via dropping the allocator, etc.). unsafe trait StaticAllocator: Allocator {} impl<T, A, D> Box<T, D> where A: Allocator + AllocatorEq<D>, D: AllocatorEq<A>, { // bikeshed better names fn new_in_with(x: T, alloc: A, dealloc: D) -> Self { if dealloc.is_equivalent(&alloc) { unsafe { Box::new_in_with_unchecked(...) } } } fn with_dealloc(boxed: Box<T, A>, dealloc: D) -> Self { ... } } impl<T, A: StaticAllocator> Box<T, A> { fn into_pin(boxed: Box<T, A>) -> Pin<Box<T, A>> { ... } } /// Calls to this allocator are not considered part of program /// behaviour, and thus may be elided or created by the optimiser. /// Additionally, such an allocator will never return an excess /// and makes no promises about alignment beyond what is requested. #[lang = "native_allocator"] struct NativeAllocator<A: StaticAllocator>(A); impl<A: StaticAllocator> Allocator for NativeAllocator<A> { ... } impl<A: StaticAllocator> StaticAllocator for NativeAllocator<A> {} ``` cc @rust-lang/libs @rust-lang/libs-api @rust-lang/opsem r? libs
…hlin Borrow the destination place with a raw pointer in MIR inlining Background: if the destination place in the caller involves a `Deref` or `Index` projection then the inliner will capture a mutable reference to the destination place before the call and write back the return value using that pointer at the end of the call. This is necessary because such projections are unstable and may cause the place to evaluate to a different location at the end of the call. However the use of `&mut` instead of `&raw mut` causes 2 problems: 1. Because call arguments are copied after the destination place is evaluated, if the same place is present in an argument and the destination then it would invalidate the destination reference, causing the later write through it to be UB. 2. Creating a mutable reference to an uninhabited type is immediate UB according to rust-lang/unsafe-code-guidelines#413, and this is enforced by Miri. While the first issue could be addressed by using two-phased borrows or by copying arguments beforehand, a raw pointer is the only way to handle the second. It's also not possible to special-case uninhabited types since it may not be possible to determine whether the destination is inhabited in generic code. r? wg-mir-opt
link the offload with in-tree lld if possible I have multiple LLVM builds on all my computers, and often enough ended up with an older lld version on the path, which often breaks Offload builds. I've also seen autodiff builds take forever since some of it's files are huge, and builds somehow end up using ld. This fixes both by just using our up-to-date in-tree lld (if available). I used an LLM to implement a couple of bootstrap fixes, including this one. I then split this one out and cleaned it up. r? kobzol closes: rust-lang#160584
…ked, r=LawnGnome implement (OnceLock,LazyLock)::(get_unchecked,get_unchecked_mut) Tracking issue: rust-lang#162716 Based on the original work of @HomelikeBrick42 from rust-lang#138914.
…cs, r=nia-e Make it clear what the alignment needs to be for `deallocate` As noticed on [zulip](https://rust-lang.zulipchat.com/#narrow/channel/197181-t-libs.2Fwg-allocators/topic/Question.3A.20layout.20alignment.20for.20Allocator.3A.3Adeallocate.28.29/with/626111397), this is insufficiently precise r? clarfonthey
…ram-type-union, r=chenyukang Test unused lifetime params for 'type' and 'union' I was reminded of rust-lang#82365 and noticed a few test cases are missing in `tests/ui/variance/variance-unused-region-param.rs`
rustbot
added
A-attributes
Member
Author
Contributor
rust-bors
Bot
added
S-waiting-on-bors
This comment has been minimized.
This comment has been minimized.
rust-bors Bot
pushed a commit
that referenced
this pull request
Sep 23, 2026
Rollup of 7 pull requests try-job: dist-various-1 try-job: test-various try-job: test-x86_64-gnu-aux try-job: test-x86_64-gnu-llvm-21-3 try-job: test-x86_64-msvc-1 try-job: test-aarch64-apple-1 try-job: test-aarch64-apple-2 try-job: test-x86_64-mingw-1 try-job: test-i686-msvc try-job: test-armhf-gnu
This comment has been minimized.
This comment has been minimized.
Contributor
rust-bors
Bot
added
merged-by-bors
Contributor
This was referenced Sep 23, 2026
Contributor
What is this?This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.Comparing 1a82b5d (parent) -> 4f37171 (this PR) Test differencesShow 1955 test diffsStage 1
Stage 2
Additionally, 1946 doctest diffs were found. These are ignored, as they are noisy. Job group index
Test dashboardRun cargo run --manifest-path src/ci/citool/Cargo.toml -- \
test-dashboard 4f371718739cc2a5374119a18ee20e74bd2096c9 --output-dir test-dashboardAnd then open Job duration changes
How to interpret the job duration changes?Job durations can vary a lot, based on the actual runner instance |
Collaborator
|
Finished benchmarking commit (4f37171): comparison URL. Overall result: no relevant changes - no action needed@rustbot label: -perf-regression Instruction countThis perf run didn't have relevant results for this metric. Max RSS (memory usage)Results (primary 2.6%, secondary 4.2%)A less reliable metric. May be of interest, but not used to determine the overall result above.
CyclesResults (primary 7.2%, secondary 5.1%)A less reliable metric. May be of interest, but not used to determine the overall result above.
Binary sizeThis perf run didn't have relevant results for this metric. Bootstrap: 490.182s -> 489.245s (-0.19%) |
Contributor
|
📌 Perf builds for each rolled up PR:
parent commit: 1a82b5d4f4 In the case of a perf regression, run the following command with the SHAs of each PR you suspect might be the cause: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Successful merges:
Allocator#156882 (alloc: stabiliseAllocator)deallocate#163171 (Make it clear what the alignment needs to be fordeallocate)r? @ghost
Create a similar rollup