Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: explicitly set shell in lint & format scripts #1391

Merged
merged 6 commits into from
Dec 23, 2022

Conversation

jthegedus
Copy link
Contributor

@jthegedus jthegedus commented Dec 21, 2022

Summary

In #1313 I ran some analysis of our Shellcheck errors with the Shell set explicitly to POSIX. Having done this I thought it might be worth updating our scripts/* to explicitly check specific files for specific Shell adherence (something discussed in #1313).

What I discovered was that we were not testing some files or dirs, and also setting bats in Shellcheck introduced a whole set of errors which have clearly been missed with auto-detection.

What I want with regards to review discussion is whether we actually capture asdf.sh as a POSIX file or whether we should capture it in the .bash linting? If we consider it Bash, should we rename it asdf.sh -> asdf.bash? Same applies to /lib/asdf.sh

TODO:

  • use long name flags in scripts/*
  • separate shfmt and shellcheck for explicitly scanning .sh, .bats and .bash files with the correct Shell settings.
  • add --indent flag on shfmt. shfmt has integration to read from .editorconfig, but it seems if you add --language-dialect flag it no longer reads formatting settings from .editorconfig, so we must match it manually with flags (as was done in the past).
    • we're still using 2-space indent 🤮
  • add scripts/format.bash to run shfmt with correct settings across the codebase en masse.

Related: #1313, #1349

Other Information

@jthegedus jthegedus requested a review from a team as a code owner December 21, 2022 05:16
@jthegedus jthegedus changed the title chore: specify explicit shell in scripts/ fix: explicitly set shell in lint & format scripts Dec 21, 2022
@hyperupcall
Copy link
Contributor

hyperupcall commented Dec 21, 2022

I saw that the tests/* directory wasn't being ran through shellcheck whatsoever, so thanks for getting to this before I was able to.

It looks like /lib/asdf.sh is sourced by /asdf.sh so I'd say the file extension shouldn't be Bash.

EDIT: Also for the files that end in .sh, there should be a shellcheck declaration like # shellcheck shell=sh on the first line so it can lint properly (in the editor)

@jthegedus
Copy link
Contributor Author

jthegedus commented Dec 21, 2022

I have certainly missed things, still correcting and updating. This is a rabbit hole uncovering a lot of issues 😱 Haven't even started on resolving the issues.

EDIT:

It looks like /lib/asdf.sh is sourced by /asdf.sh so I'd say the file extension shouldn't be Bash.

Both asdf.sh & lib/asdf.sh throw errors when running Shellcheck with sh, because they're using Bash features. So should we change their names, or remove the Bash from the code, or do nothing?

@hyperupcall
Copy link
Contributor

Both asdf.sh & lib/asdf.sh throw errors when running Shellcheck with sh, because they're using Bash features. So should we change their names, or remove the Bash from the code, or do nothing?

I'd say we keep their names and keep the declaration at the top. Since we want CI to pass for everyone, we can just add # shellcheck disable=... lines before each Bashism for now, until we decide what to do with them later.

@jthegedus
Copy link
Contributor Author

until we decide what to do with them later

Right, I was wondering if we should just solve that now. Happy to ignore specifics for these files now.

@hyperupcall
Copy link
Contributor

hyperupcall commented Dec 21, 2022

Oh wow yeah, there are 5 real violations in those files, yeah. In my opinion most of the file should be reworked anyways. The variables ASDF_BIN ASDF_USER_SHIMS shouldn't be capitalized and not all the variables are prefixed with an underscore. And I'm not sure why we're munging PATH - if really we don't want to append to the path twice, we can just set an environment variable with values (but a naive approach will break in several circumstances). So I guess that's to say I'm for doing everything separately, since there wil be more eyes and especially since if there is one mistake in this crutial code path, everything likely could break

@jthegedus
Copy link
Contributor Author

jthegedus commented Dec 21, 2022

Yes, there are a lot of errors here.

@Stratus3D My suggested path forward is this:

  • merge fix: lint errors from scripts/checkstyle.py #1385
  • rebase this PR
  • disable the check of asdf.sh & lib/asdf.sh
  • go through and fix all reported issues (I can do this)
  • merge this PR
  • rebase all other PRs
  • unlock and tackle asdf.sh & lib/asdf.sh in a separate PR

What do you think?

@jthegedus
Copy link
Contributor Author

jthegedus commented Dec 23, 2022

I have commented out the updates to scripts/* that cause new errors, this way we can introduce them piecemeal over the next month or so. This way the branch won't have sweeping changes and become stale and reduce disruption to other PRs.

I will create issues to track the uncommenting of these scripts.

@jthegedus jthegedus changed the title fix: explicitly set shell in lint & format scripts ci: explicitly set shell in lint & format scripts Dec 23, 2022
@jthegedus jthegedus merged commit ea18e96 into master Dec 23, 2022
@jthegedus jthegedus deleted the chore-scripts-explicitly-declare-bash branch December 23, 2022 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants