feat: stabilize lockfile v4 #12852

weihanglo · 2023-10-19T01:28:42Z

What does this PR try to resolve?

Stabilize lockfile v4, which effectively contains only one change: #12280.

How should we test and review this PR?

v4 cannot be constructed at this moment unless people deliberately change their lockfiles to version = 4. No user-perceivable change is involved.

The change of SourceId serialization is postponed to the time v4 becoming the default lockfile version. That seems more reasonable than doing it now.

Additional information

I failed to resolve any other issue listed in #12120. They are either too complicated or not worthy. Therefore I think we can just ship this without waiting.

rustbot · 2023-10-19T01:28:46Z

r? @ehuss

(rustbot has picked a reviewer for you, use r? to override)

epage · 2023-10-19T02:14:49Z

I failed to resolve any other issue listed in #12120. They are either too complicated or not worthy. Therefore I think we can just ship this without waiting.

Is there a reason we are wanting to go forward with this now rather than wait?

weihanglo · 2023-10-19T02:31:04Z

Is there a reason we are wanting to go forward with this now rather than wait?

I cannot predict how long we will wait for other issues getting addressed. It's more meaningful to ship a fix when we already have one.

I've waited for #10801 but #11938 failed. I gave a shot on #12464 but after chatting with Eh2406 today, I quit. Fixing #12464 entails a fundamental overhaul of the current resolver that I am not capable of. pubgrub is going to provide a different (and clearer?) way to address patches.

epage · 2023-10-19T13:57:29Z

src/cargo/core/resolver/resolve.rs

@@ -80,11 80,9 @@ pub enum ResolveVersion {
 /// V3 by default staring in 1.53.
 #[default]


What do we plan to do to track updating the default?

btw now that we have MSRV piped into the resolve logic (#12560), should we make the default based on MSRV with a fallback if no MSRV is set?

See #12120 (comment)

I'll post another PR for doc update.

For MSRV-aware lockfile generating, see #12861.

At the latest, we might want to default to the new format for 2024-edition crates, since those crates won't build with old cargo anyway so there won't be any "flapping" between formats for different Cargo versions.

Further more, it does sound like we might want to check if edition and rust-version are in sync and don't conflict with each other.

We should have checks that the rust-version isn't older than the edition allows.

Falling back to an edition check if the rust-version isn't set would be useful. The problem is we don't have a workspace edition field. We could do something like rust-version and check the oldest edition present.

src/cargo/ops/lockfile.rs

epage · 2023-10-19T14:02:20Z

src/cargo/core/resolver/encode.rs

- let unstable_lockfile_version_allowed = ws.config().cli_unstable().next_lockfile_bump;
 let path_deps = build_path_deps(ws)?;
 let mut checksums = HashMap::new();

 let mut version = match self.version {
- Some(4) if ws.config().nightly_features_allowed => {
- if unstable_lockfile_version_allowed {


Should we create V5 at the same time?

Nice suggestion. Done.

src/cargo/core/source_id.rs

epage · 2023-10-19T14:05:49Z

@rfcbot fcp merge

This will stabilize the V4 lockfile which includes #12280. The idea being to go ahead and get this into people's hands now rather than waiting for a certain number of fixes to make it in.

rfcbot · 2023-10-19T14:05:51Z

Team member @epage has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

bors · 2023-10-20T12:59:28Z

☔ The latest upstream changes (presumably #12860) made this pull request unmergeable. Please resolve the merge conflicts.

rfcbot · 2024-01-03T15:54:50Z

🔔 This is now entering its final comment period, as per the review above. 🔔

rfcbot · 2024-01-13T15:58:14Z

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

ehuss · 2024-01-30T19:47:17Z

Thanks, sorry for the delay!

@bors r

bors · 2024-01-30T19:47:20Z

📌 Commit f8a47fa has been approved by ehuss

It is now in the queue for this repository.

bors · 2024-01-30T19:48:28Z

⌛ Testing commit f8a47fa with merge f8c152d...

bors · 2024-01-30T20:31:32Z

☀️ Test successful - checks-actions
Approved by: ehuss
Pushing f8c152d to master...

bors · 2024-01-30T20:31:32Z

☀️ Test successful - checks-actions
Approved by: ehuss
Pushing f8c152d to master...

bors · 2024-01-30T20:32:31Z

👀 Test was successful, but fast-forwarding failed: 422 Changes must be made through a pull request.

Update cargo 24 commits in 7bb7b539558dc88bea44cee4168b6269bf8177b0..cdf84b69d0416c57ac9dc3459af80dfb4883d27a 2024-01-20 00:15:32 0000 to 2024-02-02 19:39:16 0000 - Apply `-Zpanic-abort-tests` to doctests too (rust-lang/cargo#13388) - Don't print rustdoc command lines on failure by default (rust-lang/cargo#13387) - Ensure `nonzero_exit_code` test isn't affected by developers `RUST_BACKTRACE` setting (rust-lang/cargo#13385) - Add `docsrs` cfg as a well known `--check-cfg` (rust-lang/cargo#13383) - fix(new): Print a note, rather than a comment, for more information (rust-lang/cargo#13371) - Change tests to support changes to suggestion (rust-lang/cargo#13382) - chore(ci): enable m1 runner (rust-lang/cargo#13377) - fix(toml): Improve map/sequence error message (rust-lang/cargo#13376) - fix(diagnostic): Don't panic on empty spans (rust-lang/cargo#13375) - doc: Hide `cargo-fetch` description in offline man page (rust-lang/cargo#13364) - feat: stabilize lockfile v4 (rust-lang/cargo#12852) - fix(new): Print a 'Creating', rather than 'Created' status (rust-lang/cargo#13367) - fix: use spec id instead of name to match package (rust-lang/cargo#13335) - refactor(shell): Use new fancy anstyle API (rust-lang/cargo#13368) - feat(cargo-update): `--precise` to allow yanked versions (rust-lang/cargo#13333) - refactor: remove unnecessary Option in `Freshness::Dirty` (rust-lang/cargo#13361) - doc: Replace version with `latest` for jobserver link (rust-lang/cargo#13366) - test: data layout fix for `x86_64-unknown-none-gnu` (rust-lang/cargo#13362) - docs(ref): Try to improve reg auth docs (rust-lang/cargo#13351) - fix typo of rustbuild, instead of rustuild (rust-lang/cargo#13357) - fix(config): Deprecate non-extension files (rust-lang/cargo#13349) - fix(cli): Improve errors related to cargo script (rust-lang/cargo#13346) - fix list option description starting with uppercase (rust-lang/cargo#13344) - Fix typo in test (rust-lang/cargo#13342)

Pkgsrc changes: * Adapt checksums and patches, some have beene intregrated upstream. Upstream chnages: Version 1.78.0 (2024-05-02) =========================== Language -------- - [Stabilize `#[cfg(target_abi = ...)]`] (rust-lang/rust#119590) - [Stabilize the `#[diagnostic]` namespace and `#[diagnostic::on_unimplemented]` attribute] (rust-lang/rust#119888) - [Make async-fn-in-trait implementable with concrete signatures] (rust-lang/rust#120103) - [Make matching on NaN a hard error, and remove the rest of `illegal_floating_point_literal_pattern`] (rust-lang/rust#116284) - [static mut: allow mutable reference to arbitrary types, not just slices and arrays] (rust-lang/rust#117614) - [Extend `invalid_reference_casting` to include references casting to bigger memory layout] (rust-lang/rust#118983) - [Add `non_contiguous_range_endpoints` lint for singleton gaps after exclusive ranges] (rust-lang/rust#118879) - [Add `wasm_c_abi` lint for use of older wasm-bindgen versions] (rust-lang/rust#117918) This lint currently only works when using Cargo. - [Update `indirect_structural_match` and `pointer_structural_match` lints to match RFC] (rust-lang/rust#120423) - [Make non-`PartialEq`-typed consts as patterns a hard error] (rust-lang/rust#120805) - [Split `refining_impl_trait` lint into `_reachable`, `_internal` variants] (rust-lang/rust#121720) - [Remove unnecessary type inference when using associated types inside of higher ranked `where`-bounds] (rust-lang/rust#119849) - [Weaken eager detection of cyclic types during type inference] (rust-lang/rust#119989) - [`trait Trait: Auto {}`: allow upcasting from `dyn Trait` to `dyn Auto`] (rust-lang/rust#119338) Compiler -------- - [Made `INVALID_DOC_ATTRIBUTES` lint deny by default] (rust-lang/rust#111505) - [Increase accuracy of redundant `use` checking] (rust-lang/rust#117772) - [Suggest moving definition if non-found macro_rules! is defined later] (rust-lang/rust#121130) - [Lower transmutes from int to pointer type as gep on null] (rust-lang/rust#121282) Target changes: - [Windows tier 1 targets now require at least Windows 10] (rust-lang/rust#115141) - [Enable CMPXCHG16B, SSE3, SAHF/LAHF and 128-bit Atomics in tier 1 Windows] (rust-lang/rust#120820) - [Add `wasm32-wasip1` tier 2 (without host tools) target] (rust-lang/rust#120468) - [Add `wasm32-wasip2` tier 3 target] (rust-lang/rust#119616) - [Rename `wasm32-wasi-preview1-threads` to `wasm32-wasip1-threads`] (rust-lang/rust#122170) - [Add `arm64ec-pc-windows-msvc` tier 3 target] (rust-lang/rust#119199) - [Add `armv8r-none-eabihf` tier 3 target for the Cortex-R52] (rust-lang/rust#110482) - [Add `loongarch64-unknown-linux-musl` tier 3 target] (rust-lang/rust#121832) Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Bump Unicode to version 15.1.0, regenerate tables] (rust-lang/rust#120777) - [Make align_offset, align_to well-behaved in all cases] (rust-lang/rust#121201) - [PartialEq, PartialOrd: document expectations for transitive chains] (rust-lang/rust#115386) - [Optimize away poison guards when std is built with panic=abort] (rust-lang/rust#100603) - [Replace pthread `RwLock` with custom implementation] (rust-lang/rust#110211) - [Implement unwind safety for Condvar on all platforms] (rust-lang/rust#121768) - [Add ASCII fast-path for `char::is_grapheme_extended`] (rust-lang/rust#121138) Stabilized APIs --------------- - [`impl Read for &Stdin`] (https://doc.rust-lang.org/stable/std/io/struct.Stdin.html#impl-Read-for-&Stdin) - [Accept non `'static` lifetimes for several `std::error::Error` related implementations] (rust-lang/rust#113833) - [Make `impl<Fd: AsFd>` impl take `?Sized`] (rust-lang/rust#114655) - [`impl From<TryReserveError> for io::Error`] (https://doc.rust-lang.org/stable/std/io/struct.Error.html#impl-From-for-Error) These APIs are now stable in const contexts: - [`Barrier::new()`] (https://doc.rust-lang.org/stable/std/sync/struct.Barrier.html#method.new) Cargo ----- - [Stabilize lockfile v4](rust-lang/cargo#12852) - [Respect `rust-version` when generating lockfile] (rust-lang/cargo#12861) - [Control `--charset` via auto-detecting config value] (rust-lang/cargo#13337) - [Support `target.<triple>.rustdocflags` officially] (rust-lang/cargo#13197) - [Stabilize global cache data tracking] (rust-lang/cargo#13492) Misc ---- - [rustdoc: add `--test-builder-wrapper` arg to support wrappers such as RUSTC_WRAPPER when building doctests] (rust-lang/rust#114651) Compatibility Notes ------------------- - [Many unsafe precondition checks now run for user code with debug assertions enabled] (rust-lang/rust#120863) This change helps users catch undefined behavior in their code, though the details of how much is checked are generally not stable. - [riscv only supports split_debuginfo=off for now] (rust-lang/rust#120518) - [Consistently check bounds on hidden types of `impl Trait`] (rust-lang/rust#121679) - [Change equality of higher ranked types to not rely on subtyping] (rust-lang/rust#118247) - [When called, additionally check bounds on normalized function return type] (rust-lang/rust#118882) - [Expand coverage for `arithmetic_overflow` lint] (rust-lang/rust#119432) Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Update to LLVM 18](rust-lang/rust#120055) - [Build `rustc` with 1CGU on `x86_64-pc-windows-msvc`] (rust-lang/rust#112267) - [Build `rustc` with 1CGU on `x86_64-apple-darwin`] (rust-lang/rust#112268) - [Introduce `run-make` V2 infrastructure, a `run_make_support` library and port over 2 tests as example] (rust-lang/rust#113026) - [Windows: Implement condvar, mutex and rwlock using futex] (rust-lang/rust#121956)

Cargo.lock was updated to version 4 manually. All Git refs in our Cargo.lock appear to be encoded correctly. See: - rust-lang/cargo#12280 - rust-lang/cargo#12852 The clippy::incompatible_msrv lint may be useful in the future. We currently do not specify rust-version in Cargo.toml. Our libraries are not designed to be used outside Grandine. rust-toolchain.toml makes rust-version redundant for builds inside this repository. See: - https://doc.rust-lang.org/1.78.0/cargo/reference/manifest.html#the-rust-version-field - https://rust-lang.github.io/rust-clippy/rust-1.78.0/#/incompatible_msrv The clippy::lint_groups_priority lint produces false positives: - rust-lang/cargo#12918 (comment) - rust-lang/rust-clippy#12270 Luckily, lints inherited from a workspace do not trigger it. The issue appears to be fixed in rust-lang/rust-clippy#12827. The reason public functions trigger clippy::single_call_fn is actually our setting of avoid-breaking-exported-api. We assumed it was a bug.

In 1.78.0 Cargo introduce v4 lockfile: rust-lang/cargo#12852 The v3-to-v4 change is minimal: encode URL params with URL encoding. However, roundtrip test failed because v4 we can only have one serialization implementation at a time. I don't know how to proceed. Maybe we could * Make v4 serialization the default when it becomes the default in Cargo * Provide API for people to generate different versions of lockfiles. * Work with upstream Cargo with an in-tree pacakge for lock serialization. (This is on me?) Opened this PR for discussion.

Cargo.lock was updated to version 4 manually. All Git refs in our Cargo.lock appear to be encoded correctly. See: - rust-lang/cargo#12280 - rust-lang/cargo#12852 The clippy::incompatible_msrv lint may be useful in the future. We currently do not specify rust-version in Cargo.toml. Our libraries are not designed to be used outside Grandine. rust-toolchain.toml makes rust-version redundant for builds inside this repository. See: - https://doc.rust-lang.org/1.78.0/cargo/reference/manifest.html#the-rust-version-field - https://rust-lang.github.io/rust-clippy/rust-1.78.0/#/incompatible_msrv The clippy::lint_groups_priority lint produces false positives: - rust-lang/cargo#12918 (comment) - rust-lang/rust-clippy#12270 Luckily, lints inherited from a workspace do not trigger it. The issue appears to be fixed in rust-lang/rust-clippy#12827. The reason public functions trigger clippy::single_call_fn is actually our setting of avoid-breaking-exported-api. We assumed it was a bug.

rustbot assigned ehuss Oct 19, 2023

epage reviewed Oct 19, 2023

View reviewed changes

epage approved these changes Oct 19, 2023

View reviewed changes

epage reviewed Oct 19, 2023

View reviewed changes

src/cargo/ops/lockfile.rs Outdated Show resolved Hide resolved

epage reviewed Oct 19, 2023

View reviewed changes

src/cargo/core/source_id.rs Show resolved Hide resolved

epage added the T-cargo Team: Cargo label Oct 19, 2023

rfcbot added proposed-final-comment-period An FCP proposal has started, but not yet signed off. disposition-merge FCP with intent to merge labels Oct 19, 2023

This was referenced Oct 20, 2023

refactor: centralize logic of getting max resolve version #12860

Merged

feat: respect rust-version when generating lockfile #12861

Merged

weihanglo force-pushed the lockfile-v4 branch from 75f409f to bec5a26 Compare October 21, 2023 01:36

rfcbot added final-comment-period FCP — a period for last comments before action is taken and removed proposed-final-comment-period An FCP proposal has started, but not yet signed off. labels Jan 3, 2024

weihanglo force-pushed the lockfile-v4 branch from bec5a26 to 5245a73 Compare January 3, 2024 20:45

weihanglo added 2 commits January 3, 2024 15:58

feat(lockfile): stabilize v4 and prepare for v5

6af53da

refactor: make clear SourceId serialization will change in v4

f8a47fa

weihanglo force-pushed the lockfile-v4 branch from 5245a73 to f8a47fa Compare January 3, 2024 20:58

rfcbot added finished-final-comment-period FCP complete to-announce and removed final-comment-period FCP — a period for last comments before action is taken labels Jan 13, 2024

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 30, 2024

bors merged commit f8c152d into rust-lang:master Jan 30, 2024
20 checks passed

weihanglo deleted the lockfile-v4 branch January 30, 2024 20:36

weihanglo mentioned this pull request Jan 31, 2024

Update cargo rust-lang/rust#120508

Merged

Shnatsel mentioned this pull request Feb 13, 2024

Support Cargo.lock v4 format rustsec/rustsec#1118

Open

This was referenced Feb 25, 2024

Record package hashes in the generated SBOM CycloneDX/cyclonedx-rust-cargo#620

Merged

Hashes for packages from registries with parameters in the URL may not be recorded CycloneDX/cyclonedx-rust-cargo#629

Open

ehuss added this to the 1.78.0 milestone Mar 2, 2024

hanabi1224 mentioned this pull request May 2, 2024

chore: bump rust toolchain version to 1.78.0 ChainSafe/forest#4284

Merged

4 tasks

weihanglo mentioned this pull request Jun 24, 2024

feat(cargo-lock): support lockfile v4 rustsec/rustsec#1206

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: stabilize lockfile v4 #12852

feat: stabilize lockfile v4 #12852

weihanglo commented Oct 19, 2023

rustbot commented Oct 19, 2023

epage commented Oct 19, 2023

weihanglo commented Oct 19, 2023 •

edited

Loading

epage Oct 19, 2023

weihanglo Oct 20, 2023

joshtriplett Oct 24, 2023

weihanglo Oct 24, 2023

epage Oct 24, 2023

epage Oct 19, 2023

weihanglo Oct 21, 2023

epage commented Oct 19, 2023

rfcbot commented Oct 19, 2023 •

edited by Muscraft

Loading

bors commented Oct 20, 2023

rfcbot commented Jan 3, 2024

rfcbot commented Jan 13, 2024

ehuss commented Jan 30, 2024

bors commented Jan 30, 2024

bors commented Jan 30, 2024

bors commented Jan 30, 2024

bors commented Jan 30, 2024

bors commented Jan 30, 2024

		@@ -80,11 80,9 @@ pub enum ResolveVersion {
		/// V3 by default staring in 1.53.
		#[default]

feat: stabilize lockfile v4 #12852

feat: stabilize lockfile v4 #12852

Conversation

weihanglo commented Oct 19, 2023

What does this PR try to resolve?

How should we test and review this PR?

Additional information

rustbot commented Oct 19, 2023

epage commented Oct 19, 2023

weihanglo commented Oct 19, 2023 • edited Loading

epage Oct 19, 2023

Choose a reason for hiding this comment

weihanglo Oct 20, 2023

Choose a reason for hiding this comment

joshtriplett Oct 24, 2023

Choose a reason for hiding this comment

weihanglo Oct 24, 2023

Choose a reason for hiding this comment

epage Oct 24, 2023

Choose a reason for hiding this comment

epage Oct 19, 2023

Choose a reason for hiding this comment

weihanglo Oct 21, 2023

Choose a reason for hiding this comment

epage commented Oct 19, 2023

rfcbot commented Oct 19, 2023 • edited by Muscraft Loading

bors commented Oct 20, 2023

rfcbot commented Jan 3, 2024

rfcbot commented Jan 13, 2024

ehuss commented Jan 30, 2024

bors commented Jan 30, 2024

bors commented Jan 30, 2024

bors commented Jan 30, 2024

bors commented Jan 30, 2024

bors commented Jan 30, 2024

weihanglo commented Oct 19, 2023 •

edited

Loading

rfcbot commented Oct 19, 2023 •

edited by Muscraft

Loading