Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modify parameters of a tasks with ack late wich is reenqueued #8716

Open
bdoublet91 opened this issue Dec 12, 2023 · 1 comment
Open

Modify parameters of a tasks with ack late wich is reenqueued #8716

bdoublet91 opened this issue Dec 12, 2023 · 1 comment
Labels
Issue Type: Feature Request

Comments

@bdoublet91
Copy link

bdoublet91 commented Dec 12, 2023 •
edited
Loading

Brief Summary

When a task with acks_late to true is not ack (because of celery restart for example), the task is pushed at the end of the queue without the possibility to modify the parameters like priority with a retry handler or something else.
For our use case, we have a chain of tasks with a callback task and if some tasks are reenqueued without higher priority, the callback task will be delayed because of new tasks queued.

For example

queue default: 1000 tasks

group_tasks_job1: 2 tasks   callback task

group_tasks_job1 -> celery_process_tasks_job1.1 -> success -> ACK
                                 -> celery_process_tasks_job1.2 -> celery_restart -> NOT_ACK -> re-queued
tasks_job1.2 is the 999th task in the queue default so the group_tasks_job1 have to wait for all the 998 tasks to be processed and then start the callback task for this group.

I didn't find any solution to trigger this event and change the priority ?

Thanks for your help
@ShaheedHaque
Copy link
Contributor

When re-queueing, shouldn't the default behaviour be to use ALL the settings it was originally created with?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Issue Type: Feature Request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ShaheedHaque @bdoublet91