-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improved cylc release
task matching.
#5752
base: master
Are you sure you want to change the base?
Conversation
d9c7682
to
9595276
Compare
Reviewers, an easy way to test this branch: [scheduling]
cycling mode = integer
initial cycle point = 1
final cycle point = 3
[[graph]]
P1 = """
holder => a & b & c => d & e & f
"""
[runtime]
[[holder]]
script = """
P=${CYLC_TASK_CYCLE_POINT}
cylc hold ${CYLC_WORKFLOW_ID} //$P/a //$P/b //$P/c //$P/d //$P/e //$P/f
"""
[[FAM, BAM]]
[[a,b]]
inherit = FAM
[[d,e]]
inherit = BAM
[[c,f]] So, Now mess about with |
Good spot! |
(I've just come back to this, will un-draft when done) |
676d374
to
2042dcd
Compare
538a70f
to
a027e52
Compare
a027e52
to
87a2ad5
Compare
Ready for review (if tests are good). I've pushed it forward to 8.3.0 - not critical but useful and probably easy to review and test. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, works for me:
before
2024-04-08T20:22:07 12:00 INFO - Command "release" received. ID=4294af93-ed58-461f-b262-b432641f4fc7
release(tasks=['1/d'])
2024-04-08T20:22:08 12:00 INFO - Command "release" actioned. ID=4294af93-ed58-461f-b262-b432641f4fc7
2024-04-08T20:22:57 12:00 INFO - Command "release" received. ID=08777829-a3c4-458e-9ef8-b7eeb4fa0529
release(tasks=['*/d'])
2024-04-08T20:22:58 12:00 WARNING - No active tasks matching: */d
2024-04-08T20:22:58 12:00 INFO - Command "release" actioned with 1 warnings. ID=08777829-a3c4-458e-9ef8-b7eeb4fa0529
after
2024-04-08T20:25:28 12:00 INFO - Command "release" received. ID=0f774513-6307-437d-890a-807c846ea142
release(tasks=['1/d'])
2024-04-08T20:25:29 12:00 INFO - Task hold list:
* 3/b
* 2/f
* 1/b
* 2/e
* 1/e
* 3/e
* 3/c
* 3/a
* 3/d
* 3/f
* 2/c
* 2/a
* 2/d
* 1/c
* 1/a
* 1/f
* 2/b
2024-04-08T20:25:29 12:00 INFO - Command "release" actioned. ID=0f774513-6307-437d-890a-807c846ea142
2024-04-08T20:26:11 12:00 INFO - Command "release" received. ID=256e25a9-ff87-4639-a1d1-3aecc70c1263
release(tasks=['*/d'])
2024-04-08T20:26:11 12:00 INFO - Task hold list:
* 3/b
* 2/f
* 1/b
* 2/e
* 1/e
* 3/e
* 3/c
* 3/a
* 3/f
* 2/c
* 2/a
* 1/c
* 1/a
* 1/f
* 2/b
quite verbose .. but not really an issue I suppose, maybe for large workflows it might be a bit much (?)..
We don't have another way to expose the complete list of all held and to-be-held tasks to users yet, so I figured the log was best for the moment. Could put it to DEBUG mode I guess. |
The release command doesn't need to do generic matching of future tasks, because we can only release tasks that are already held - and those are all recorded in the tasks-to-hold list.
That being the case, we can glob-match in both the task pool and the tasks-to-hold list, to identify tasks to release.
Close #5750
Bonus -
cylc remove
now discards removed tasks from the tasks-to-hold list.Motivation: one way of handling graph rewind is:
The new flow should not get held when it reaches the previously-removed tasks from the original flow.
Check List
CONTRIBUTING.md
and added my name as a Code Contributor.setup.cfg
(andconda-environment.yml
if present).CHANGES.md
entry included if this is a change that can affect users?.?.x
branch.