Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Celery v5.5.0 Release #9140

Open
6 tasks
Nusnus opened this issue Jul 21, 2024 · 3 comments
Open
6 tasks

Celery v5.5.0 Release #9140

Nusnus opened this issue Jul 21, 2024 · 3 comments

Comments

@Nusnus
Copy link
Member

Nusnus commented Jul 21, 2024

Minor Release Overview: v5.5.0

This issue will summarize the status and discussion in preparation for the new release. It will be used to track the progress of the release and to ensure that all the necessary steps are taken. It will serve as a checklist for the release and will be used to communicate the status of the release to the community.

⚠️ Warning: The release checklist is a living document. It will be updated as the release progresses. Please check back often to ensure that you are up to date with the latest information.

Checklist

  • Codebase Stability
  • Breaking Changes Validation
  • Compile Changelog
  • Release
  • Release Announcement

Release Details

The release manager is responsible for completing the release end-to-end ensuring that all the necessary steps are taken and that the release is completed in a timely manner. This is usually the owner of the release issue but may be assigned to a different maintainer if necessary.

  • Release Manager: Tomer Nosrati (@Nusnus)
  • Release Date: TBD
  • Release Branch: main

Release Steps

The release manager is expected to execute the checklist below. The release manager is also responsible for ensuring that the checklist is updated as the release progresses. Any changes or issues should be communicated under this issue for centralized tracking.

Potential Release Blockers

1. Codebase Stability

  • The main branch build passes

    Build Status

2. Breaking Changes Validation

A patch release should not contain any breaking changes. The release manager is responsible for reviewing all of the merged PRs since the last release to ensure that there are no breaking changes. If there are any breaking changes, the release manager should discuss with the maintainers to determine the best course of action if an obvious solution is not apparent.

3. Compile Changelog

The release changelog is set in two different places:

  1. The Changelog.rst that uses the RST format.
  2. The GitHub Release auto-generated changelog that uses the Markdown format. This is auto-generated by the GitHub Draft Release UI.

⚠️ Warning: The pre-commit changes should not be included in the changelog.

To generate the changelog automatically, draft a new release on GitHub using a fake new version tag for the automatic changelog generation. Notice the actual tag creation is done on publish so we can use that to generate the changelog and then delete the draft release without publishing it thus avoiding creating a new tag.

  • Create a new tag
CleanShot 2023-09-05 at 22 06 24@2x
  • Generate Markdown release notes
CleanShot 2023-09-05 at 22 13 39@2x
  • Copy the generated release notes.

  • Delete the draft release without publishing it.

3.1 Changelog.rst

Once you have the actual changes, you need to convert it to rst format and add it to the Changelog.rst file. The new version block needs to follow the following format:

.. _version-x.y.z:

x.y.z
=====

:release-date: YYYY-MM-DD HH:MM P.M/A.M TimeZone
:release-by: Release Manager Name

Changes list in RST format.

These changes will reflect in the Change history section of the documentation.

3.2 Changelog PR

The changes to the Changelog.rst file should be submitted as a PR. This will PR should be the last merged PR before the release.

4. Release

4.1 Prepare releasing environment

Before moving forward with the release, the release manager should ensure that bumpversion and twine are installed. These are required to publish the release.

4.2 Bump version

The release manager should bump the version using the following command:

bumpversion patch

The changes should be pushed directly to main by the release manager.

At this point, the git log should appear somewhat similar to this:

commit XXX (HEAD -> main, tag: vX.Y.Z, upstream/main, origin/main)
Author: Release Manager
Date:   YYY

    Bump version: a.b.c → x.y.z

commit XXX
Author: Release Manager
Date:   YYY

    Added changelog for vX.Y.Z (#1234)

If everything looks good, the bump version commit can be directly pushed to main:

git push origin main --tags

4.3 Publish release to PyPI

The release manager should publish the release to PyPI using the following commands running under the root directory of the repository:

python setup.py clean build sdist bdist_wheel

If the build is successful, the release manager should publish the release to PyPI using the following command:

twine upload dist/celery-X.Y.Z*

⚠️ Warning: The release manager should double check that the release details are correct (project/version) before publishing the release to PyPI.

⚠️ Critical Reminder: Should the released package prove to be faulty or need retraction for any reason, do not delete it from PyPI. The appropriate course of action is to "yank" the release.

Release Announcement

After the release is published, the release manager should create a new GitHub Release and set it as the latest release.

CleanShot 2023-09-05 at 22 51 24@2x

Add Release Notes

On a per-case basis, the release manager may also attach an additional release note to the auto-generated release notes. This is usually done when there are important changes that are not reflected in the auto-generated release notes.

OpenCollective Update

After successfully publishing the new release, the release manager is responsible for announcing it on the project's OpenCollective page. This is to engage with the community and keep backers and sponsors in the loop.

@Nusnus Nusnus added this to the 5.5 milestone Jul 21, 2024
@Nusnus Nusnus self-assigned this Jul 21, 2024
@Nusnus
Copy link
Member Author

Nusnus commented Jul 24, 2024

Celery v5.5.0b1 released.

@JockeTF
Copy link

JockeTF commented Jul 29, 2024

I've been playing around with quorum queues, and they seem really nice! I had set up a small RabbitMQ 3.13 cluster using a simplified (upgraded, without the entrypoint script, and with manual cluster joining) version of serkodev/rabbitmq-cluster-docker. I threw 65 536 tasks at it over the cause of an hour while this little thing was running.

while true; do
  for i in {1..3}; do
    docker kill "rabbitmq$i"
    sleep 30
    docker start "rabbitmq$i"
    sleep 30
  done
done

It seems 16 tasks were lost from two of the kills (I don't know why yet), but the system remained stable. The worst thing I saw beyond that was a handful of our response times slowing down by half a second after a kill. I also noticed that non-default queues for tasks were being created as classic queues instead of quorum queues.

Thanks for this release! It is looking pretty great so far!

@Nusnus
Copy link
Member Author

Nusnus commented Aug 6, 2024

Celery v5.5.0b1 released.

Celery v5.5.0b2 released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants