Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip in struct impl and other cases #4707

Open
wants to merge 1 commit into
base: rustfmt-2.0.0-rc.2
Choose a base branch
from

Conversation

davidBar-On
Copy link
Contributor

@davidBar-On davidBar-On commented Feb 18, 2021

Suggest fix for issue #4499 - skip attribute in struct impl and in other cases. The changes are basically to all calls to push_skipped_with_span() where the 2nd parameter was x.span. In all these cases, the 2nd parameter was changed to x.span(). This is because x.span() includes the attributes blobk, while item.span does not (found that based on the other calls to push_skipped_with_span - not sure why is that so).

Currently, per the check done by visit_attrs(), only the first line in an attributs block that contains a skip is indented, regardless of the where the skip attribute is in the block. It seems that it was assumed that a skip will always be the first in a block of attributes. If this functionality is not intentional, further enhancements may be required, e.g. not not format all the attributes block (see this short discussion). Note that per this skip.rs test case, at least the attributes following the skip should not be formatted.

The fix includes a change to a test case for issue #4398. It seems that the fix for that issue was not complete, and therefor a skip line was indented, although it is the second line in the attributes block (Documentation comment is regarded as an Attribute - AST attribute kind DocComment).

@calebcartwright
Copy link
Member

Note that currently, per the check done by visit_attrs(), only the first line in a block of attributes is formatted, regardless of the where the skip is in the block

This is a rather confusing statement, especially within the context of the PR (and still so even with the links). It will always be helpful, both for the initial PR review and for posterity, for pertinent topics to be expressed clearly and concisely. That"s because folks reviewing the PR at the time as well as those circling back to it months/years later won"t necessarily have the same mental context as the developer at the time as the latter was in the technical weeds.

I don"t think the intended message there is that only the first attribute in a block is formatted, as that"s clearly not true:

      #[foo] #[bar]
fn   lorem  () { Qux // asdf
}

I assume there"s specific context about the indentation of the first attribute in a block of attributes which contains a skip attr, but if so let"s be explicit and clear about that:

    #[foo]
 #[baz]
    #[rustfmt::skip]
     #[bar]
    fn       foo() {
             Bar
        //
    }

@davidBar-On
Copy link
Contributor Author

I assume there"s specific context about the indentation of the first attribute in a block of attributes which contains a skip attr, but if so let"s be explicit and clear about that

Yes, this was the intention. Sorry for the confusion. Somehow I missed mentioning the skip attribute in the block. Fixed the original description to make it clear for reviewers.

Copy link
Member

@calebcartwright calebcartwright left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, and good spot on the span usage! This is largely in good order and should be good to go after addressing the inline items. Additionally, I have two other asks:

  • Could you rename the parameters in push_skipped_with_span to make it more explicit that the first should include the attributes while the second should not? The current names are not sufficiently clear
  • Could you take your current tests/source/issue-4499.rs file and run it through the latest 1.x rustfmt (use rustup-installed rustfmt from stable or nightly, in a separate project) and then run your updated version of rustfmt with these changes against that to ensure it"s idempotent? (note there could be some minor unrelated changes between 2.x and 1.x, but still want to run this regardless)

src/formatting/visitor.rs Outdated Show resolved Hide resolved
src/formatting/visitor.rs Outdated Show resolved Hide resolved
src/formatting/visitor.rs Outdated Show resolved Hide resolved
src/formatting/visitor.rs Outdated Show resolved Hide resolved
src/formatting/visitor.rs Outdated Show resolved Hide resolved
@@ -0,0 +1,180 @@
// Different "rustfmt::skip" (not)formatting tests

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the thoroughness of the test suite! I do have a couple requested changes though:

  • please try to keep issue-specific test files specific to those reported issues. it"s always great to add more tests, but those don"t need to be stuffed in to the same file
  • this file is a little jarring given the size, and the excessive misformatting. i believe it will be much more readable and maintainable to break this into smaller chunks, so lets split these up in multiple files (feel free to create a new directory, e.g. skip to place them if nothing along those lines exists yet)
  • lets refrain from having using comment text in these test cases which describes behavior that is fixed. it"s okay for a small snippet from the original issue (though still free to change the words of the comment, as the actual words don"t matter), but we don"t want to repeat wording that will be very confusing to someone looking at this down the road
  • lets avoid having duplicative attributes everywhere, again doing so makes it more difficult to review and eyeball since one has to try to manually match up the 5th instance of the same attribute between two files. The actual attribute doesn"t really matter for our formatting purposes (doesn"t have to be real/valid) so always prefer distinctness
  • i"d advise against having so much indentation, unless you feel it"s really necessary to visually tell that the content is indeed skipped. I can certainly understand one or two levels, but some of these look a bit excessive for a test that"s basically targeting idempotency between source and target (albeit with a couple exceptions)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please try to keep issue-specific test files specific to those reported issues. it"s always great to add more tests, but those don"t need to be stuffed in to the same file .... this file is a little jarring given the size, and the excessive misformatting. i believe it will be much more readable and maintainable to break this into smaller chunks .....

Split the test file into several files in the skip folder (that already exited in target). Did keep the "issue-4499" prefix for the file names.

lets refrain from having using comment text in these test cases which describes behavior that is fixed .... lets avoid having duplicative attributes everywhere ....

Change. Hopefully it is o.k. now. It seems that I prepared a draft of the test cases and somehow forgot that it is a draft ...

i"d advise against having so much indentation ...

Changed all source indentations to 3 tabs. The reason is that 1 or 2 tabs may be the output of formatted code. I hope this is o.k.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did keep the "issue-4499" prefix for the file names.

Please do not do this. The point of having issue specific file names is to be able to quickly and succinctly tie back a test case to a reported issue. It is always fine to add additional test cases not directly related to a reported issue, but don"t conflate the additional test cases with the reported issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did keep the "issue-4499" prefix for the file names.

Please do not do this.

O.k. Removed the "issue-4499-" prefix from the test files names.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won"t block the PR as this is getting into bikeshedding territory, but you didn"t need to go from one extreme to the other 😄

Having a specific file for a reported issue to be used for system+idempotence tests is not mutually exclusive with adding other test files.

The way we handle this is pretty straightforward in my mind, but please let me know if this is unclear and/or if you have any questions. When creating specific test case content to cover a reported issue, there"s a few options depending on the report/preference:

  • Create a single source/target file pair that contains the MCVE (repro case, minimal complete and verifiable example) provided in the original issue, and name the file to match with a dash or underscore (e.g.tests/{source,target}/issue-5004.rs)
  • If the originally provided MCVE requires multiple files to reproduce (e.g. module loading scenarios) then create a directory named after the issue and add the files within (issue name does not need to be tacked on to the file names, e.g. tests/{source,target}/issue-5000/{foo.rs,bar.rs})
  • Add the MCVE snippet to an existing test file, and use inline comments and/or function names to add the tie back to the reported issue, e.g.
    fn issue184(source: &str) {
    for c in source.chars() {
    if index < "a" {
    continue;
    }
    }
    }

If during the process of resolving a reported issue you discover that there"s other scenarios we should have test cases for then that"s great! However, you do not need to, and should not, tie those additional scenarios to the reported issue just because they were conjured while thinking about or working on a reported issue. The above naming conventions and test file locations only apply for the reported MCVEs, and other tests can be added elsewhere

@davidBar-On davidBar-On force-pushed the issue-4499-skip-in-struct-impl branch 2 times, most recently from 7443503 to 0a16936 Compare March 7, 2021 09:11
@calebcartwright
Copy link
Member

Also, have you had a chance to run through the exercise I asked about?

Could you take your current tests/source/issue-4499.rs file and run it through the latest 1.x rustfmt (use rustup-installed rustfmt from stable or nightly, in a separate project) and then run your updated version of rustfmt with these changes against that to ensure it"s idempotent? (note there could be some minor unrelated changes between 2.x and 1.x, but still want to run this regardless)

@davidBar-On
Copy link
Contributor Author

davidBar-On commented Mar 8, 2021

Could you take your current tests/source/issue-4499.rs file and run it through the latest 1.x rustfmt (use rustup-installed rustfmt from stable or nightly, in a separate project) and then run your updated version of rustfmt with these changes against that to ensure it"s idempotent?

Sorry, overlooked that request. There is one difference between 1.x (1.4.36-nightly) and the updated 2.x version, which is because of the issue fixed by this PR.

1.x output is:

impl Foo {
    // Comment 1
                        #[rustfmt::skip]
                        #[allow(non_snake_case)]
    // Comment 2
                        #[allow(non_snake_case)]
    pub fn foo(&self) {}
}

while the updated version output is:

impl Foo {
    // Comment 1
    #[rustfmt::skip]
                        #[allow(non_snake_case)]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

Note that when impl Foo is replaced by fn foo in the above code then both give the same output which is compatible with the updated version output for impl:

fn Foo() {
    // Comment 1
    #[rustfmt::skip]
                        #[allow(non_snake_case)]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

@calebcartwright
Copy link
Member

Thank you for checking, although it"s not entirely clear to me what the input what as at this point given the many subsequent file changes.

The leftward shift of the skip attribute back to the block indented position it would be placed in if the item wasn"t being skipped is not correct. That"s an outer attribute on the item that is being skipped, so assuming an input of this:

impl Foo {
                        // Comment 1
                        #[rustfmt::skip]
                        #[allow(non_snake_case)]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

I would expect an output of:

impl Foo {
    // Comment 1
                        #[rustfmt::skip]
                        #[allow(non_snake_case)]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

The first comment has to be realigned because from the AST and span info it"s just an arbitrary comment within the Foo impl that needs to be block indented, but everything that"s contained within the full span of the function foo item would be left as-is

@davidBar-On
Copy link
Contributor Author

... so assuming an input of this:

Yes, this is the input I was referring to.

... everything that"s contained within the full span of the function foo item would be left as-is

As far as I understand and checked this is not the current functionality. Currently the first line of the block is properly indented. I assumed this is done with the expectation that the skip is the first block line, and that the intention was that the skip line itself will be indented. Something like that the skip is indicating "please do not format the following lines".

Wasn"t that the intention? Should the logic be changed to not format the whole block?

1 similar comment
@davidBar-On
Copy link
Contributor Author

... so assuming an input of this:

Yes, this is the input I was referring to.

... everything that"s contained within the full span of the function foo item would be left as-is

As far as I understand and checked this is not the current functionality. Currently the first line of the block is properly indented. I assumed this is done with the expectation that the skip is the first block line, and that the intention was that the skip line itself will be indented. Something like that the skip is indicating "please do not format the following lines".

Wasn"t that the intention? Should the logic be changed to not format the whole block?

@calebcartwright
Copy link
Member

Something like that the skip is indicating "please do not format the following lines".

Remember that attributes are always applied to the corresponding item, expression, etc. not arbitrary lines in a file. In these cases there"s a set of outer attributes, which includes the rustfmt attr, added to the foo function item. So that skip attribute is applied to the entire function item, it does not just apply to the "lines" below which is why it doesn"t matter where the skip attribute is placed within the collection of attributes.

The indentation level within the top level impl item is correct yes when reformatting things within that impl, such as inner attributes, comments, items, etc. However, since the user has added skip attribute to the function item that means rustfmt should not touch that function item at all. The reformatting of the first attribute is happening in these most recent snippets with current rustfmt versions because the skipped function item is the very first element within that top level impl.

e.g. compare this (doesn"t matter whether there"s another item, comment, etc., just something before the skipped item):

impl Foo {
    fn bar(   ) {}
                        #[rustfmt::skip]
                        #[allow(non_snake_case)]
                   // Comment 2
                        #[allow(non_snake_case)]
                                pub fn foo(&self) {}
}

vs the very first node within the top level Foo impl being skipped:

impl Foo {

                        #[rustfmt::skip]
                        #[allow(non_snake_case)]
       // Comment 2
                        #[allow(non_snake_case)]
       pub fn foo(&self) {}
}

This is incorrect and inconsistent (the indentation of the first attribute of a skipped item should not depend on that items position in the sequence of subitems within the containing item), but is an existing issue that can and should be addressed separately. However, my concern here is that you said that the formatting/indentation changes are happening even if there are preceding items/comments/etc., which seems to be trading one problem for another.

It"s really important to remember that stable/consistent formatting is a critical concern for rustfmt, so we really want to avoid a scenario where folks would upgrade to the next minor/patch version of rustfmt and have their formatting checks start failing on skipped things because the updated rustfmt now wants to move those skipped elements. That would make for a frustrating user experience and also run afoul of our stability guarantee.

So while the core change here is something we"ll certainly want to incorporate, we"ll also need to resolve the indentation changes it"s currently causing (i.e. as shown in this previous example, and above with the additional bar function item preceding the skipped foo item).

Does that make sense?

@davidBar-On
Copy link
Contributor Author

With these detailed explanations I believe I now understand much better the approach. However there are still two issues I don"t understand. The following explanation of my issues repeat some of the examples above, but I hope it will make the issues more clear.

The main issue is if there is a difference between formatting fn and impl with skip inside. Currently (1.x, 2.x) the following fn with skip inside:

fn foo() {
                        // Comment 1
                        #[rustfmt::skip]
                        #[allow(non_snake_case)]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

is formatted with the skip line indented:

fn foo() {
    // Comment 1
    #[rustfmt::skip]
                        #[allow(non_snake_case)]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

However, from your example here, replacing fn with impl:

impl Foo {
                        // Comment 1
                        #[rustfmt::skip]
                        #[allow(non_snake_case)]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

should be formatted without indenting the skip line:

impl Foo {
    // Comment 1
                        #[rustfmt::skip]
                        #[allow(non_snake_case)]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

Why the skip line should be indented for fn but not for impl? Should there be such difference? If yes, is it because of backward compatibility or because there is a difference between fn and impl that I don"t understand?

The second issue is the indentation of the first line in attributes block that includes a skip (as in 1.x, 2.x). In the above fn example, the skip line is the first, so it is indented. However, if skip is the second line in the block:

fn foo() {
                        // Comment 1
                        #[allow(non_snake_case)]
                        #[rustfmt::skip]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

is formatted with the new first line indented:

fn foo() {
    // Comment 1
    #[allow(non_snake_case)]
                        #[rustfmt::skip]
                        // Comment 2
                        #[allow(non_snake_case)]
                        pub fn foo(&self) {}
}

From your explanations I understand the indentation of the first line in the attributes block may be a bug. Therefore, I am not sure what is the preferred approach for this PR formatting inside impl with skip:

  1. Indent the first block line as done for fn?
  2. Don"t indent the first block line only for impl (because of backward compatibility)?
  3. Don"t indent the first block line for both fn and impl?

@calebcartwright
Copy link
Member

Why the skip line should be indented for fn but not for impl? Should there be such difference?

You"ve honed in on the important question: should vs. does

Why does it get formatted differently within a function block? Because those are different AST that end up getting processed and formatted by different portions of the codebase; it"s not an intentional "feature".

Again, important to think about it from the end user perspective. User says "don"t touch this thing", and rustfmt currently says "sure, but I"m going to steal the first attribute and reformat it anyway". The end result can be pretty jarring in these sorts of cases, since the attribute ends up being so disconnected from the element it is attributing. It"s not right regardless of whatever the skipped item/expr/whatever is nested within.

So to summarize explicitly:

  • (1) rustfmt should not change the indentation of the first attribute on a skipped item nested within an item (e.g. an impl, the original issue driving this pr)
    • (1a) rustfmt does currently do this, but only if the skipped item is the first nested element
  • (2) rustfmt should not change the indentation of the first attribute of a skipped element within a block
    • (2a) rustfmt does currently do this
    • (2b) this is a separate, technically unrelated issue, which just happens to present with a nearly identical set of symptoms
    • (2c) addressing this would likely be fairly tricky, and is a very low priority

We"re really only focused on 1/1a here in this PR, and the fn aspects, especially 2a which you"ve raised, are orthogonal.

@davidBar-On davidBar-On force-pushed the issue-4499-skip-in-struct-impl branch from 0a16936 to 42e38e2 Compare March 30, 2021 10:30
@davidBar-On
Copy link
Contributor Author

Changed so first line of attributes block i not indented only inside impl. In all other cases the first line of attributes block is indented, to keep backward compatibility. Initially I thought that first line should not be indented also inside trait, but did not did that as 1.x properly handles skip inside trait.

Also rebased and combined all PR branches into one branch (assuming will help the review, as the previous changes were minor compared to the last change).

@calebcartwright calebcartwright changed the base branch from master to rustfmt-2.0.0-rc.2 April 30, 2021 00:24
@ytmimi ytmimi added pr-targeting-2.0 This PR is targeting the 2.0 branch p-low labels Jul 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants