Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

align_offset, align_to: no longer allow implementations to spuriously fail to align #121201

Merged
merged 1 commit into from
Mar 8, 2024

Conversation

RalfJung
Copy link
Member

For a long time, we have allowed align_offset to fail to compute a properly aligned offset, and align_to to return a smaller-than-maximal "middle slice". This was done to cover the implementation of align_offset in const-eval and Miri. See #62420 for more background. For about the same amount of time, this has caused confusion and surprise, where people didn't realize they have to write their code to be defensive against align_offset failures.

Another way to put this is: the specification is effectively non-deterministic, and non-determinism is hard to test for -- in particular if the implementation everyone uses to test always produces the same reliable result, and nobody expects it to be non-deterministic to begin with.

With #117840, Miri has stopped making use of this liberty in the spec; it now always behaves like rustc. That only leaves const-eval as potential motivation for this behavior. I do not think this is sufficient motivation. Currently, none of the relevant functions are stably const: align_offset is unstably const, align_to is not const at all. I propose that if we ever want to make these const-stable, we just accept the fact that they can behave differently at compile-time vs at run-time. This is not the end of the world, and it seems to be much less surprising to programmers than unexpected non-determinism. (Related: rust-lang/rfcs#3352.)

@thomcc has repeatedly made it clear that they strongly dislike the non-determinism in align_offset, so I expect they will support this. @oli-obk, what do you think? Also, whom else should we involve? The primary team responsible is clearly libs-api, so I will nominate this for them. However, allowing const-evaluated code to behave different from run-time code is t-lang territory. The thing is, this is not stabilizing anything t-lang-worthy immediately, but it still does make a decision we will be bound to: if we accept this change, then

  • either align_offset/align_to can never be called in const fn,
  • or we allow compile-time behavior to differ from run-time behavior.

So I will nominate for t-lang as well, with the question being: are you okay with accepting either of these outcomes (without committing to which one, just accepting that it has to be one of them)? This closes the door to "have align_offset and align_to at compile-time and also always have compile-time behavior match run-time behavior".

Closes #62420

@rustbot
Copy link
Collaborator

rustbot commented Feb 16, 2024

r? @cuviper

rustbot has assigned @cuviper.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Feb 16, 2024
@RalfJung RalfJung added T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. I-lang-nominated The issue / PR has been nominated for discussion during a lang team meeting. I-libs-api-nominated The issue / PR has been nominated for discussion during a libs-api team meeting. and removed T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Feb 16, 2024
@oli-obk
Copy link
Contributor

oli-obk commented Feb 16, 2024

Note that the opposite of this PR exists at #105296

@RalfJung
Copy link
Member Author

Not sure in which sense that is the "opposite". I think having these APIs you propose is a good idea no matter what we do with align_to. But I also think having those APIs does not stop us from removing a footgun in align_to if we don't have sufficient motivation to keep the footgun.

@oli-obk
Copy link
Contributor

oli-obk commented Feb 16, 2024

But I also think having those APIs does not stop us from removing a footgun in align_to if we don't have sufficient motivation to keep the footgun.

those were only ever added to support making str::from_utf8 const at some point. That was their original motivation. If we close the door for that, it'll just mean we have to add the same function, but fallible, again.

I guess public usage of the functions does show what people want out of these APIs, so 🤷 let's just do it and figure things out when we're ready to make more stuff const

@RalfJung
Copy link
Member Author

RalfJung commented Feb 16, 2024

those were only ever added to support making str::from_utf8 const at some point. That was their original motivation. If we close the door for that, it'll just mean we have to add the same function, but fallible, again.

We're not closing the door on that though. from_utf8 can use align_offset and it will be correct

  • at run-time, for obvious reasons
  • at compile-time, because from_utf8 is written in a way to be robust to align_offset failing due to compile-time constraints

@RalfJung RalfJung added S-waiting-on-team Status: Awaiting decision from the relevant subteam (see the T-<team> label). and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 22, 2024
@m-ou-se
Copy link
Member

m-ou-se commented Feb 27, 2024

We discussed this in the libs-api meeting, and those present were happy with this change.

@rfcbot merge

@rfcbot
Copy link

rfcbot commented Feb 27, 2024

Team member @m-ou-se has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@rfcbot rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Feb 27, 2024
@m-ou-se m-ou-se removed the I-libs-api-nominated The issue / PR has been nominated for discussion during a libs-api team meeting. label Feb 27, 2024
@rfcbot rfcbot added final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. and removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. labels Feb 27, 2024
@rfcbot
Copy link

rfcbot commented Feb 27, 2024

🔔 This is now entering its final comment period, as per the review above. 🔔

@joshtriplett
Copy link
Member

With my lang hat on:

If I'm understanding correctly, these can differ at compile time precisely because the concept of addresses and them having alignment isn't the same at compile-time and runtime. That seems fine. We will still have options at compile-time; for instance, if we want to track the concept of "sufficiently aligned" at compile time, we could decide to do so in an abstract way.

@RalfJung
Copy link
Member Author

If I'm understanding correctly, these can differ at compile time precisely because the concept of addresses and them having alignment isn't the same at compile-time and runtime.

Yes. The only time compile-time would give a "weird" answer is when the information required to give the expected answer is inherently unavailable at compile-time.

We will still have options at compile-time; for instance, if we want to track the concept of "sufficiently aligned" at compile time, we could decide to do so in an abstract way.

I think that might be tricky, e.g. if one crate declares a [u8; N] buffer stored somewhere then even tracking abstract alignment means downstream crates could be adding alignment constraints to that, e.g. deciding that it must be 4-aligned. I also don't think we actually want align_to to change the alignment of these static global objects.

@Amanieu Amanieu added T-lang Relevant to the language team, which will review and decide on the PR/issue. and removed T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Feb 28, 2024
@rfcbot rfcbot added finished-final-comment-period The final comment period is finished for this PR / Issue. to-announce Announce this issue on triage meeting and removed final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. labels Mar 8, 2024
@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 8, 2024
@cuviper
Copy link
Member

cuviper commented Mar 8, 2024

Wait, hold on -- should the docs for <*mut>::align_offset be updated too?

@bors r-

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 8, 2024
@RalfJung
Copy link
Member Author

RalfJung commented Mar 8, 2024

Ah, good point. Done that. I also added a sentence to explicitly state that unstable things are unstable.

@cuviper
Copy link
Member

cuviper commented Mar 8, 2024

@bors r

@bors
Copy link
Contributor

bors commented Mar 8, 2024

📌 Commit 507583a has been approved by cuviper

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 8, 2024
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 8, 2024
…iaskrgr

Rollup of 9 pull requests

Successful merges:

 - rust-lang#121201 (align_offset, align_to: no longer allow implementations to spuriously fail to align)
 - rust-lang#122076 (Tweak the way we protect in-place function arguments in interpreters)
 - rust-lang#122100 (Better comment for implicit captures in RPITIT)
 - rust-lang#122157 (Add the new description field to Target::to_json, and add descriptions for some MSVC targets)
 - rust-lang#122164 (Fix misaligned loads when loading UEFI arg pointers)
 - rust-lang#122171 (Add some new solver tests)
 - rust-lang#122172 (Don't ICE if we collect no RPITITs unless there are no unification errors)
 - rust-lang#122197 (inspect formatter: add braces)
 - rust-lang#122198 (Remove handling for previously dropped LLVM version)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 948d32d into rust-lang:master Mar 8, 2024
11 checks passed
@rustbot rustbot added this to the 1.78.0 milestone Mar 8, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Mar 8, 2024
Rollup merge of rust-lang#121201 - RalfJung:align_offset_contract, r=cuviper

align_offset, align_to: no longer allow implementations to spuriously fail to align

For a long time, we have allowed `align_offset` to fail to compute a properly aligned offset, and `align_to` to return a smaller-than-maximal "middle slice". This was done to cover the implementation of `align_offset` in const-eval and Miri. See rust-lang#62420 for more background. For about the same amount of time, this has caused confusion and surprise, where people didn't realize they have to write their code to be defensive against `align_offset` failures.

Another way to put this is: the specification is effectively non-deterministic, and non-determinism is hard to test for -- in particular if the implementation everyone uses to test always produces the same reliable result, and nobody expects it to be non-deterministic to begin with.

With rust-lang#117840, Miri has stopped making use of this liberty in the spec; it now always behaves like rustc. That only leaves const-eval as potential motivation for this behavior. I do not think this is sufficient motivation. Currently, none of the relevant functions are stably const: `align_offset` is unstably const, `align_to` is not const at all. I propose that if we ever want to make these const-stable, we just accept the fact that they can behave differently at compile-time vs at run-time. This is not the end of the world, and it seems to be much less surprising to programmers than unexpected non-determinism. (Related: rust-lang/rfcs#3352.)

`@thomcc` has repeatedly made it clear that they strongly dislike the non-determinism in align_offset, so I expect they will support this. `@oli-obk,` what do you think? Also, whom else should we involve? The primary team responsible is clearly libs-api, so I will nominate this for them. However, allowing const-evaluated code to behave different from run-time code is t-lang territory. The thing is, this is not stabilizing anything t-lang-worthy immediately, but it still does make a decision we will be bound to: if we accept this change, then
- either `align_offset`/`align_to` can never be called in const fn,
- or we allow compile-time behavior to differ from run-time behavior.

So I will nominate for t-lang as well, with the question being: are you okay with accepting either of these outcomes (without committing to which one, just accepting that it has to be one of them)? This closes the door to "have `align_offset` and `align_to` at compile-time and also always have compile-time behavior match run-time behavior".

Closes rust-lang#62420
@RalfJung RalfJung deleted the align_offset_contract branch March 9, 2024 08:16
@scottmcm
Copy link
Member

Ah, align_to isn't const fn. Saying it won't be, and thus we can make strict promises about that function makes good sense to me, given all the complaints about it.

We can add a different function for const fn later if really needed.

@apiraino apiraino removed the to-announce Announce this issue on triage meeting label Mar 14, 2024
@traviscross
Copy link
Contributor

@rustbot labels -I-lang-nominated

We discussed this in the lang call on 2024-03-13 and agreed that we were OK with this having been merged and the consequences of that noted in the original nomination.

@rustbot rustbot removed the I-lang-nominated The issue / PR has been nominated for discussion during a lang team meeting. label Mar 19, 2024
@RalfJung
Copy link
Member Author

Sorry, I hadn't realized the T-lang discussion was still open when this got approved.

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request May 4, 2024
Pkgsrc changes:
 * Adapt checksums and patches, some have beene intregrated upstream.

Upstream chnages:

Version 1.78.0 (2024-05-02)
===========================

Language
--------
- [Stabilize `#[cfg(target_abi = ...)]`]
  (rust-lang/rust#119590)
- [Stabilize the `#[diagnostic]` namespace and
  `#[diagnostic::on_unimplemented]` attribute]
  (rust-lang/rust#119888)
- [Make async-fn-in-trait implementable with concrete signatures]
  (rust-lang/rust#120103)
- [Make matching on NaN a hard error, and remove the rest of
  `illegal_floating_point_literal_pattern`]
  (rust-lang/rust#116284)
- [static mut: allow mutable reference to arbitrary types, not just
  slices and arrays]
  (rust-lang/rust#117614)
- [Extend `invalid_reference_casting` to include references casting
  to bigger memory layout]
  (rust-lang/rust#118983)
- [Add `non_contiguous_range_endpoints` lint for singleton gaps
  after exclusive ranges]
  (rust-lang/rust#118879)
- [Add `wasm_c_abi` lint for use of older wasm-bindgen versions]
  (rust-lang/rust#117918)
  This lint currently only works when using Cargo.
- [Update `indirect_structural_match` and `pointer_structural_match`
  lints to match RFC]
  (rust-lang/rust#120423)
- [Make non-`PartialEq`-typed consts as patterns a hard error]
  (rust-lang/rust#120805)
- [Split `refining_impl_trait` lint into `_reachable`, `_internal` variants]
  (rust-lang/rust#121720)
- [Remove unnecessary type inference when using associated types
  inside of higher ranked `where`-bounds]
  (rust-lang/rust#119849)
- [Weaken eager detection of cyclic types during type inference]
  (rust-lang/rust#119989)
- [`trait Trait: Auto {}`: allow upcasting from `dyn Trait` to `dyn Auto`]
  (rust-lang/rust#119338)

Compiler
--------

- [Made `INVALID_DOC_ATTRIBUTES` lint deny by default]
  (rust-lang/rust#111505)
- [Increase accuracy of redundant `use` checking]
  (rust-lang/rust#117772)
- [Suggest moving definition if non-found macro_rules! is defined later]
  (rust-lang/rust#121130)
- [Lower transmutes from int to pointer type as gep on null]
  (rust-lang/rust#121282)

Target changes:

- [Windows tier 1 targets now require at least Windows 10]
  (rust-lang/rust#115141)
 - [Enable CMPXCHG16B, SSE3, SAHF/LAHF and 128-bit Atomics in tier 1 Windows]
  (rust-lang/rust#120820)
- [Add `wasm32-wasip1` tier 2 (without host tools) target]
  (rust-lang/rust#120468)
- [Add `wasm32-wasip2` tier 3 target]
  (rust-lang/rust#119616)
- [Rename `wasm32-wasi-preview1-threads` to `wasm32-wasip1-threads`]
  (rust-lang/rust#122170)
- [Add `arm64ec-pc-windows-msvc` tier 3 target]
  (rust-lang/rust#119199)
- [Add `armv8r-none-eabihf` tier 3 target for the Cortex-R52]
  (rust-lang/rust#110482)
- [Add `loongarch64-unknown-linux-musl` tier 3 target]
  (rust-lang/rust#121832)

Refer to Rust's [platform support page][platform-support-doc]
for more information on Rust's tiered platform support.

Libraries
---------

- [Bump Unicode to version 15.1.0, regenerate tables]
  (rust-lang/rust#120777)
- [Make align_offset, align_to well-behaved in all cases]
  (rust-lang/rust#121201)
- [PartialEq, PartialOrd: document expectations for transitive chains]
  (rust-lang/rust#115386)
- [Optimize away poison guards when std is built with panic=abort]
  (rust-lang/rust#100603)
- [Replace pthread `RwLock` with custom implementation]
  (rust-lang/rust#110211)
- [Implement unwind safety for Condvar on all platforms]
  (rust-lang/rust#121768)
- [Add ASCII fast-path for `char::is_grapheme_extended`]
  (rust-lang/rust#121138)

Stabilized APIs
---------------

- [`impl Read for &Stdin`]
  (https://doc.rust-lang.org/stable/std/io/struct.Stdin.html#impl-Read-for-&Stdin)
- [Accept non `'static` lifetimes for several `std::error::Error`
  related implementations] (rust-lang/rust#113833)
- [Make `impl<Fd: AsFd>` impl take `?Sized`]
  (rust-lang/rust#114655)
- [`impl From<TryReserveError> for io::Error`]
  (https://doc.rust-lang.org/stable/std/io/struct.Error.html#impl-From-for-Error)

These APIs are now stable in const contexts:

- [`Barrier::new()`]
  (https://doc.rust-lang.org/stable/std/sync/struct.Barrier.html#method.new)

Cargo
-----

- [Stabilize lockfile v4](rust-lang/cargo#12852)
- [Respect `rust-version` when generating lockfile]
  (rust-lang/cargo#12861)
- [Control `--charset` via auto-detecting config value]
  (rust-lang/cargo#13337)
- [Support `target.<triple>.rustdocflags` officially]
  (rust-lang/cargo#13197)
- [Stabilize global cache data tracking]
  (rust-lang/cargo#13492)

Misc
----

- [rustdoc: add `--test-builder-wrapper` arg to support wrappers
  such as RUSTC_WRAPPER when building doctests]
  (rust-lang/rust#114651)

Compatibility Notes
-------------------

- [Many unsafe precondition checks now run for user code with debug
  assertions enabled] (rust-lang/rust#120863)
  This change helps users catch undefined behavior in their code,
  though the details of how much is checked are generally not
  stable.
- [riscv only supports split_debuginfo=off for now]
  (rust-lang/rust#120518)
- [Consistently check bounds on hidden types of `impl Trait`]
  (rust-lang/rust#121679)
- [Change equality of higher ranked types to not rely on subtyping]
  (rust-lang/rust#118247)
- [When called, additionally check bounds on normalized function return type]
  (rust-lang/rust#118882)
- [Expand coverage for `arithmetic_overflow` lint]
  (rust-lang/rust#119432)

Internal Changes
----------------

These changes do not affect any public interfaces of Rust, but they represent
significant improvements to the performance or internals of rustc and related
tools.

- [Update to LLVM 18](rust-lang/rust#120055)
- [Build `rustc` with 1CGU on `x86_64-pc-windows-msvc`]
  (rust-lang/rust#112267)
- [Build `rustc` with 1CGU on `x86_64-apple-darwin`]
  (rust-lang/rust#112268)
- [Introduce `run-make` V2 infrastructure, a `run_make_support`
  library and port over 2 tests as example]
  (rust-lang/rust#113026)
- [Windows: Implement condvar, mutex and rwlock using futex]
  (rust-lang/rust#121956)
greaka added a commit to greaka/rust that referenced this pull request Jul 6, 2024
greaka added a commit to greaka/rust that referenced this pull request Jul 6, 2024
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jul 12, 2024
as_simd: fix doc comment to be in line with align_to

In rust-lang#121201, the guarantees about `align_offset` and `align_to` were changed. This PR aims to correct the doc comment of `as_simd` to be in line with the new `align_to`.

Tagging rust-lang#86656 for good measure.
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jul 12, 2024
Rollup merge of rust-lang#127422 - greaka:master, r=workingjubilee

as_simd: fix doc comment to be in line with align_to

In rust-lang#121201, the guarantees about `align_offset` and `align_to` were changed. This PR aims to correct the doc comment of `as_simd` to be in line with the new `align_to`.

Tagging rust-lang#86656 for good measure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. finished-final-comment-period The final comment period is finished for this PR / Issue. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-lang Relevant to the language team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

align_offset guarantees