Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resolve: Unload speculatively resolved crates before freezing cstore #119592

Merged
merged 1 commit into from
Feb 8, 2024

Conversation

petrochenkov
Copy link
Contributor

@petrochenkov petrochenkov commented Jan 4, 2024

Name resolution sometimes loads additional crates to improve diagnostics (e.g. suggest imports).
Not all of these diagnostics result in errors, sometimes they are just warnings, like in #117772.

If additional crates loaded speculatively stay and gets listed by things like query crates then they may produce further errors like duplicated lang items, because lang items from speculatively loaded crates are as good as from non-speculatively loaded crates.
They can probably do things like adding unintended impls from speculatively loaded crates to method resolution as well.
The extra crates will also get into the crate's metadata as legitimate dependencies.

In this PR I remove the speculative crates from cstore when name resolution is finished and cstore is frozen.
This is better than e.g. filtering away speculative crates in query crates because things like DefIds referring to these crates and leaking to later compilation stages can produce ICEs much easier, allowing to detect them.

The unloading could potentially be skipped if any errors were reported (to allow using DefIds from speculatively loaded crates for recovery), but I didn't do it in this PR because I haven't seen such cases of recovery. We can reconsider later if any relevant ICEs are reported.

Unblocks #117772.

@rustbot
Copy link
Collaborator

rustbot commented Jan 4, 2024

r? @TaKO8Ki

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jan 4, 2024
@petrochenkov
Copy link
Contributor Author

cc @surechen

@compiler-errors
Copy link
Member

compiler-errors commented Jan 4, 2024

They can probably do things like adding unintended impls from speculatively loaded crates to method resolution as well.

Yes, this happens and causes spurious knock-on errors in typeck.

@bors

This comment was marked as resolved.

@bors

This comment was marked as resolved.

Copy link
Member

@compiler-errors compiler-errors left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me if you'd like to land this, needs rebase. Speculative crate loading has been a headache for spurious diagnostics.

@compiler-errors compiler-errors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 5, 2024
@petrochenkov
Copy link
Contributor Author

@bors r=compiler-errors

@bors
Copy link
Contributor

bors commented Feb 6, 2024

📌 Commit a2ab48c has been approved by compiler-errors

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 6, 2024
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 7, 2024
…rors

resolve: Unload speculatively resolved crates before freezing cstore

Name resolution sometimes loads additional crates to improve diagnostics (e.g. suggest imports).
Not all of these diagnostics result in errors, sometimes they are just warnings, like in rust-lang#117772.

If additional crates loaded speculatively stay and gets listed by things like `query crates` then they may produce further errors like duplicated lang items, because lang items from speculatively loaded crates are as good as from non-speculatively loaded crates.
They can probably do things like adding unintended impls from speculatively loaded crates to method resolution as well.
The extra crates will also get into the crate's metadata as legitimate dependencies.

In this PR I remove the speculative crates from cstore when name resolution is finished and cstore is frozen.
This is better than e.g. filtering away speculative crates in `query crates` because things like `DefId`s referring to these crates and leaking to later compilation stages can produce ICEs much easier, allowing to detect them.

The unloading could potentially be skipped if any errors were reported (to allow using `DefId`s from speculatively loaded crates for recovery), but I didn't do it in this PR because I haven't seen such cases of recovery. We can reconsider later if any relevant ICEs are reported.

Unblocks rust-lang#117772.
bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 7, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#119162 (Add unstable `-Z direct-access-external-data` cmdline flag for `rustc`)
 - rust-lang#119592 (resolve: Unload speculatively resolved crates before freezing cstore)
 - rust-lang#120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
 - rust-lang#120455 ( Add FileCheck annotations to MIR-opt SROA tests)
 - rust-lang#120470 (Mark "unused binding" suggestion as maybe incorrect)
 - rust-lang#120619 (Assert that params with the same *index* have the same *name*)
 - rust-lang#120633 (pattern_analysis: gather up place-relevant info)
 - rust-lang#120726 (Don't use bashism in checktools.sh)

r? `@ghost`
`@rustbot` modify labels: rollup
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 7, 2024
…rors

resolve: Unload speculatively resolved crates before freezing cstore

Name resolution sometimes loads additional crates to improve diagnostics (e.g. suggest imports).
Not all of these diagnostics result in errors, sometimes they are just warnings, like in rust-lang#117772.

If additional crates loaded speculatively stay and gets listed by things like `query crates` then they may produce further errors like duplicated lang items, because lang items from speculatively loaded crates are as good as from non-speculatively loaded crates.
They can probably do things like adding unintended impls from speculatively loaded crates to method resolution as well.
The extra crates will also get into the crate's metadata as legitimate dependencies.

In this PR I remove the speculative crates from cstore when name resolution is finished and cstore is frozen.
This is better than e.g. filtering away speculative crates in `query crates` because things like `DefId`s referring to these crates and leaking to later compilation stages can produce ICEs much easier, allowing to detect them.

The unloading could potentially be skipped if any errors were reported (to allow using `DefId`s from speculatively loaded crates for recovery), but I didn't do it in this PR because I haven't seen such cases of recovery. We can reconsider later if any relevant ICEs are reported.

Unblocks rust-lang#117772.
bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 7, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#119592 (resolve: Unload speculatively resolved crates before freezing cstore)
 - rust-lang#120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
 - rust-lang#120206 (hir: Make sure all `HirId`s have corresponding HIR `Node`s)
 - rust-lang#120214 (match lowering: consistently lower bindings deepest-first)
 - rust-lang#120596 ([rustdoc] Correctly generate path for non-local items in source code pages)
 - rust-lang#120688 (GVN: also turn moves into copies with projections)
 - rust-lang#120702 (docs: also check the inline stmt during redundant link check)
 - rust-lang#120739 (improve pretty printing for associated items in trait objects)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 8, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#119592 (resolve: Unload speculatively resolved crates before freezing cstore)
 - rust-lang#120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
 - rust-lang#120206 (hir: Make sure all `HirId`s have corresponding HIR `Node`s)
 - rust-lang#120214 (match lowering: consistently lower bindings deepest-first)
 - rust-lang#120596 ([rustdoc] Correctly generate path for non-local items in source code pages)
 - rust-lang#120688 (GVN: also turn moves into copies with projections)
 - rust-lang#120702 (docs: also check the inline stmt during redundant link check)
 - rust-lang#120739 (improve pretty printing for associated items in trait objects)

r? `@ghost`
`@rustbot` modify labels: rollup
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 8, 2024
…rors

resolve: Unload speculatively resolved crates before freezing cstore

Name resolution sometimes loads additional crates to improve diagnostics (e.g. suggest imports).
Not all of these diagnostics result in errors, sometimes they are just warnings, like in rust-lang#117772.

If additional crates loaded speculatively stay and gets listed by things like `query crates` then they may produce further errors like duplicated lang items, because lang items from speculatively loaded crates are as good as from non-speculatively loaded crates.
They can probably do things like adding unintended impls from speculatively loaded crates to method resolution as well.
The extra crates will also get into the crate's metadata as legitimate dependencies.

In this PR I remove the speculative crates from cstore when name resolution is finished and cstore is frozen.
This is better than e.g. filtering away speculative crates in `query crates` because things like `DefId`s referring to these crates and leaking to later compilation stages can produce ICEs much easier, allowing to detect them.

The unloading could potentially be skipped if any errors were reported (to allow using `DefId`s from speculatively loaded crates for recovery), but I didn't do it in this PR because I haven't seen such cases of recovery. We can reconsider later if any relevant ICEs are reported.

Unblocks rust-lang#117772.
bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 8, 2024
…iaskrgr

Rollup of 9 pull requests

Successful merges:

 - rust-lang#119592 (resolve: Unload speculatively resolved crates before freezing cstore)
 - rust-lang#120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
 - rust-lang#120206 (hir: Make sure all `HirId`s have corresponding HIR `Node`s)
 - rust-lang#120214 (match lowering: consistently lower bindings deepest-first)
 - rust-lang#120596 ([rustdoc] Correctly generate path for non-local items in source code pages)
 - rust-lang#120688 (GVN: also turn moves into copies with projections)
 - rust-lang#120727 (exhaustiveness: Prefer "`0..MAX` not covered" to "`_` not covered")
 - rust-lang#120734 (Add `SubdiagnosticMessageOp` as a trait alias.)
 - rust-lang#120739 (improve pretty printing for associated items in trait objects)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 8, 2024
…iaskrgr

Rollup of 9 pull requests

Successful merges:

 - rust-lang#119592 (resolve: Unload speculatively resolved crates before freezing cstore)
 - rust-lang#120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
 - rust-lang#120206 (hir: Make sure all `HirId`s have corresponding HIR `Node`s)
 - rust-lang#120214 (match lowering: consistently lower bindings deepest-first)
 - rust-lang#120688 (GVN: also turn moves into copies with projections)
 - rust-lang#120702 (docs: also check the inline stmt during redundant link check)
 - rust-lang#120727 (exhaustiveness: Prefer "`0..MAX` not covered" to "`_` not covered")
 - rust-lang#120734 (Add `SubdiagnosticMessageOp` as a trait alias.)
 - rust-lang#120739 (improve pretty printing for associated items in trait objects)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 4e11d03 into rust-lang:master Feb 8, 2024
11 checks passed
@rustbot rustbot added this to the 1.78.0 milestone Feb 8, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Feb 8, 2024
Rollup merge of rust-lang#119592 - petrochenkov:unload, r=compiler-errors

resolve: Unload speculatively resolved crates before freezing cstore

Name resolution sometimes loads additional crates to improve diagnostics (e.g. suggest imports).
Not all of these diagnostics result in errors, sometimes they are just warnings, like in rust-lang#117772.

If additional crates loaded speculatively stay and gets listed by things like `query crates` then they may produce further errors like duplicated lang items, because lang items from speculatively loaded crates are as good as from non-speculatively loaded crates.
They can probably do things like adding unintended impls from speculatively loaded crates to method resolution as well.
The extra crates will also get into the crate's metadata as legitimate dependencies.

In this PR I remove the speculative crates from cstore when name resolution is finished and cstore is frozen.
This is better than e.g. filtering away speculative crates in `query crates` because things like `DefId`s referring to these crates and leaking to later compilation stages can produce ICEs much easier, allowing to detect them.

The unloading could potentially be skipped if any errors were reported (to allow using `DefId`s from speculatively loaded crates for recovery), but I didn't do it in this PR because I haven't seen such cases of recovery. We can reconsider later if any relevant ICEs are reported.

Unblocks rust-lang#117772.
flip1995 pushed a commit to flip1995/rust that referenced this pull request Feb 26, 2024
…iaskrgr

Rollup of 9 pull requests

Successful merges:

 - rust-lang#119592 (resolve: Unload speculatively resolved crates before freezing cstore)
 - rust-lang#120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
 - rust-lang#120206 (hir: Make sure all `HirId`s have corresponding HIR `Node`s)
 - rust-lang#120214 (match lowering: consistently lower bindings deepest-first)
 - rust-lang#120688 (GVN: also turn moves into copies with projections)
 - rust-lang#120702 (docs: also check the inline stmt during redundant link check)
 - rust-lang#120727 (exhaustiveness: Prefer "`0..MAX` not covered" to "`_` not covered")
 - rust-lang#120734 (Add `SubdiagnosticMessageOp` as a trait alias.)
 - rust-lang#120739 (improve pretty printing for associated items in trait objects)

r? `@ghost`
`@rustbot` modify labels: rollup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants