Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data store: add absolute graph edges #6103

Merged
merged 2 commits into from
Jul 19, 2024

Conversation

oliver-sanders
Copy link
Member

@oliver-sanders oliver-sanders commented May 14, 2024

Closes #5845

Include absolute graph edges into the data store.

At present, absolute graph parents apply only when the parent is in the same cycle as the child.

E.G. Taking this example:

R1 = build
P1 = build[^] => run

The build => run graph edge would only exist in the R1 cycle (the one in which the build task was scheduled to run) rather than all subsequent cycles to which it applied.

The cause of the bug was a bit of code that was purposefully filtering out absolute graph edges which point to tasks in cycles other than the one that the child task is in. The code contained this comment which suggests doing this resolved some other issue:

# If 'foo[^] => bar' only spawn off of '^'.

Taking this code out fixes #5845, but I'm worried that this code served a purpose and that by deleting it I'm solving one problem by introducing another.

The comment first appeared here:

416240f#diff-e67a65e5cc6c5da815c97f19bf1afc269d56a7f032351aa72364a414ae726048R84

I can't track down why it was added by scanning through #3811. Long shot, @dwsutherland you don't have any memory of this (3 years ago!)?

Check List

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
  • Tests are included (or explain why tests are not needed).
  • CHANGES.md entry included if this is a change that can affect users
  • Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
  • If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

@oliver-sanders oliver-sanders added this to the 8.x milestone May 14, 2024
@oliver-sanders oliver-sanders added the bug Something is wrong :( label May 14, 2024
@oliver-sanders oliver-sanders changed the title data store: add aboluste graph edges data store: add absolute graph edges May 14, 2024
@oliver-sanders oliver-sanders self-assigned this May 14, 2024
@dwsutherland
Copy link
Member

I might have shifted some code about also.. I guess this is separate to the issue of spawning off ^ ? (where you'd spawn out to the RH limit if this dependency was in more than just R1)

@dwsutherland
Copy link
Member

dwsutherland commented May 21, 2024

Actually yeah, so I've only included it in the same cycle (like you said)... it makes sense WRT limiting the edges to ^ cycle, but it does feel artificial ..

Copy link
Member

@dwsutherland dwsutherland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine, tests passing.. Probably desirable for NIWA site also..

I suspect it may have tripped up the old graph walk in some way? hard to tell now.

@oliver-sanders oliver-sanders modified the milestones: 8.x, 8.4.0 May 22, 2024
@oliver-sanders oliver-sanders modified the milestones: 8.4.0, 8.3.3 Jul 15, 2024
@oliver-sanders oliver-sanders force-pushed the data-store.add-aboluste-graph-edges branch from 2957e50 to ec7e16d Compare July 15, 2024 15:23
@oliver-sanders oliver-sanders changed the base branch from master to 8.3.x July 15, 2024 15:26
@oliver-sanders oliver-sanders marked this pull request as ready for review July 15, 2024 15:27
@oliver-sanders oliver-sanders force-pushed the data-store.add-aboluste-graph-edges branch from ec7e16d to 5e6e741 Compare July 15, 2024 15:28
@oliver-sanders
Copy link
Member Author

Added a test and changelog, rebased onto 8.3.x.

@wxtim Assigned for review 2, well worth a good play with the graph view (static cylc graph and live in the GUI).

6103.fix.md Outdated Show resolved Hide resolved
Co-authored-by: Tim Pillinger <26465611 [email protected]>
@wxtim wxtim merged commit e8118d8 into cylc:8.3.x Jul 19, 2024
27 checks passed
@oliver-sanders oliver-sanders deleted the data-store.add-aboluste-graph-edges branch July 19, 2024 08:40
@MetRonnie
Copy link
Member

What about this equivalent bit in generate_graph_children()?

if is_abs and trigger.get_parent_point(point) != point:
# If 'foo[^] => bar' only spawn off of '^'.
continue

@oliver-sanders
Copy link
Member Author

oliver-sanders commented Aug 7, 2024

What about this equivalent bit in generate_graph_children()?

(haven't forgotten about this, still catching up with my inbox).

A good point that I either don't have an answer to or simply can't remember.

I don't think I've seen any consequences of this (abs deps seem to work fine, perhaps this is the graph walking algorithm doing its job?).

Opened an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is wrong :(
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tdef: graph parents - absolute dependencies ignored
4 participants