GitHub Actions tutorial: Problem with equal version names when uploading to test.pypi #804
Comments
|
Agreed. We worked hard to fix the version issue, which solved it for a time, until we filled up the space on test pypi. I don't see much purpose in using test pypi at all at this point. If it either allowed me to mark versions as temporary somehow so they were auto-deleted after a period of time or allowed me to do batch deletions it would be very helpful. I really like the idea of pushing to test pypi on every push to make sure nothing is breaking -- that's very much in line with standard CI practice, but it currently does not work. |
|
Another possible solution I can think of would be to only publish tagged github releases with "rc" (release candidate) in the in the name. In that way you can still test your release by making release candidates with version tags like "v1.2.0rc2" and then only publish the versions without "rc" to pypi. Of course this all depends on the release workflow which might differ per project. However I think the way it's now shown in the instructions does not work nicely out of the box. So would be great to change this to something that does not have these issues or add some disclaimers and possible solutions. |
|
Agreed. I didn't demonstrate how to solve the version clash there. I wanted to start with something simple but clearly, the tutorial should be improved. Personally, I rely on setuptools-scm to generate versions and I apply "no local version part" strategy to upload per commit (example: https://github.com/ansible/pylibssh/blob/a648cd7/.github/workflows/build-test-n-publish.yml#L337-L341). As for filling up the space, I think having a note in the tutorial is a good idea but there's nothing else we can do about it right now. |
webknjaz
added
type: enhancement
This is based on https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#defining-a-workflow-job-environment . It's the sample config from there, with versions updated and our project name substituted in the appropriate places. I dropped the publish-to-testpypi bit because of pypa/packaging.python.org#804 , and left the Github release part left out for now. We can add that later if we like, but we never published releases to Github before, so it doesn't seem required yet. I also tweaked the conditionals a bit to avoid running the build job on forks and publish only tags that start with 'v', as that's our convention for versions. Signed-off-by: Adam Williamson <[email protected]>
This is based on https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#defining-a-workflow-job-environment . It's the sample config from there, with versions updated and our project name substituted in the appropriate places. I dropped the publish-to-testpypi bit because of pypa/packaging.python.org#804 , and left the Github release part left out for now. We can add that later if we like, but we never published releases to Github before, so it doesn't seem required yet. I also tweaked the conditionals a bit to avoid running the build job on forks and publish only tags that start with 'v', as that's our convention for versions. Signed-off-by: Adam Williamson <[email protected]>
This is based on https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#defining-a-workflow-job-environment . It's the sample config from there, with versions updated and our project name substituted in the appropriate places. I dropped the publish-to-testpypi bit because of pypa/packaging.python.org#804 , and left the Github release part left out for now. We can add that later if we like, but we never published releases to Github before, so it doesn't seem required yet. I also tweaked the conditionals a bit to avoid running the build job on forks and publish only tags that start with 'v', as that's our convention for versions. Signed-off-by: Adam Williamson <[email protected]>
|
There's another problem with the guide as-is - it doesn't use any conditional to restrict the publish-to-testpypi job to pushes to the main repo, so the job will also run on pushes to forks which have actions enabled. Of course, it won't work, because the pypi project won't be configured to allow the fork to publish, so you just get an annoying failure email. Theoretically this problem also exists for the publish-to-pypi job, but in practice it's less likely to be an issue because it's less common to push tags to a fork... Since 2021 it seems github kinda defaults to disabling actions on newly-created forks (though it gives you a 'helpful' button to press to enable them), but forks created before then seem to have actions enabled by default. |
This is based on https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#defining-a-workflow-job-environment . It's the sample config from there, with versions updated and our project name substituted in the appropriate places. I dropped the publish-to-testpypi bit because of pypa/packaging.python.org#804 , and left the Github release part left out for now. We can add that later if we like, but we never published releases to Github before, so it doesn't seem required yet. I also tweaked the conditionals a bit to avoid running the build job on forks and publish only tags that start with 'v', as that's our convention for versions. Signed-off-by: Adam Williamson <[email protected]>
This is based on https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#defining-a-workflow-job-environment . It's the sample config from there, with versions updated and our project name substituted in the appropriate places. I dropped the publish-to-testpypi bit because of pypa/packaging.python.org#804 , and left the Github release part left out for now. We can add that later if we like, but we never published releases to Github before, so it doesn't seem required yet. I also tweaked the conditionals a bit to avoid running the build job on forks and publish only tags that start with 'v', as that's our convention for versions. Signed-off-by: Adam Williamson <[email protected]>
This is based on https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#defining-a-workflow-job-environment . It's the sample config from there, with versions updated and our project name substituted in the appropriate places. I dropped the publish-to-testpypi bit because of pypa/packaging.python.org#804 , and left the Github release part left out for now. We can add that later if we like, but we never published releases to Github before, so it doesn't seem required yet. I also tweaked the conditionals a bit to avoid running the build job on forks and publish only tags that start with 'v', as that's our convention for versions. Signed-off-by: Adam Williamson <[email protected]>
This is based on https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#defining-a-workflow-job-environment . It's the sample config from there, with versions updated and our project name substituted in the appropriate places. I dropped the publish-to-testpypi bit because of pypa/packaging.python.org#804 , and left the Github release part left out for now. We can add that later if we like, but we never published releases to Github before, so it doesn't seem required yet. I also tweaked the conditionals a bit to avoid running the build job on forks and publish only tags that start with 'v', as that's our convention for versions. Signed-off-by: Adam Williamson <[email protected]>
This is based on https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#defining-a-workflow-job-environment . It's the sample config from there, with versions updated and our project name substituted in the appropriate places. I dropped the publish-to-testpypi bit because of pypa/packaging.python.org#804 , and left the Github release part left out for now. We can add that later if we like, but we never published releases to Github before, so it doesn't seem required yet. I also tweaked the conditionals a bit to avoid running the build job on forks and publish only tags that start with 'v', as that's our convention for versions. Signed-off-by: Adam Williamson <[email protected]>
|
I'm only starting to play with CI/CD so I might be missing something, but doesn't this solve the issue? I had success on my side adding the mentionned flag: with:
repository-url: https://test.pypi.org/legacy/
skip-existing: true |
|
@rderollepot people wanted that toggle so I added it but personally, I consider it to be a footgun. Instead, use plugins like |
and others
When following this guide I ran into a problem related to #777.
The point is that you cannot upload packages to pypi (or test.pypi) with the same (version) name. This definitely happens when uploading to test.pypi on every push. The workflow from this tutorial therefore fails even when running it for a second time with the following message:
This filename has already been used, use a different version.A solution is given by @bhazelton with the script from RadioAstronomySoftwareGroup/pyuvdata#931: https://gist.github.com/plaplant/0902f09e59166bac742bbd554f3cd2f9/raw/make_dev_version.sh
However this raises another issue (the one of filling up test pypi space) which is reported in #777.
Another solution (as implemented in the same PR mentioned above) is to only publish on test.pypi for tagged versions, same as for pypi. However this defeats the purpose of publishing to test.pypi as the same package is also directly published to pypi.
The text was updated successfully, but these errors were encountered: