Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for internal runtime usage in build and PRs #35133

Merged
merged 10 commits into from
Aug 17, 2021

Conversation

mmitche
Copy link
Member

@mmitche mmitche commented Aug 6, 2021

  • Don't conditionalize queue adds based on use of open queues.
    In eng/targets/Helix.targets, these are replaced with the non-open versions automatically
  • Don't use default error detection for the repo tasks build in source build, as this may
    pick up the error messages from the first attempts to install an internal runtime from a public location (it later succeeds using the private location.)

- Don't conditionalize queue adds based on use of open queues.
  In eng/targets/Helix.targets, these are replaced with the non-open versions automatically
- Don't use default error detection for the repo tasks build in source build, as this may
  pick up the error messages from the first attempts to install an internal runtime from a public location (it later succeeds using the private location.)
@mmitche mmitche requested a review from a team as a code owner August 6, 2021 22:08
@ghost ghost added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Aug 6, 2021
@wtgodbe wtgodbe requested a review from HaoK August 6, 2021 22:16
@Tratcher
Copy link
Member

Tratcher commented Aug 9, 2021

What do we expect to happen to this build queue?
https://dev.azure.com/dnceng/internal/_build?definitionId=928

It's intent is to only run Windows.10.Amd64.ClientPre.VS2019.Pre. Will it start running the non-open versions of all queues now?

@mmitche
Copy link
Member Author

mmitche commented Aug 9, 2021

What do we expect to happen to this build queue?
https://dev.azure.com/dnceng/internal/_build?definitionId=928

It's intent is to only run Windows.10.Amd64.ClientPre.VS2019.Pre. Will it start running the non-open versions of all queues now?

There is replacement logic in the repo that will replace '.Open' with the empty string when running on the internal project, where the .Open queues are not available. In public, the .Open queues are only available, and nothing changes.

@Tratcher
Copy link
Member

Tratcher commented Aug 9, 2021

We don't want all of those jobs running on that internal queue, there's a public queue that runs them separately.
https://dev.azure.com/dnceng/public/_build?definitionId=837

Is there another internal queue where you're trying to enable these jobs? Do we need a different mechanic to disable these for the existing internal queue?

@mmitche
Copy link
Member Author

mmitche commented Aug 9, 2021

We don't want all of those jobs running on that internal queue, there's a public queue that runs them separately.
https://dev.azure.com/dnceng/public/_build?definitionId=837

Is there another internal queue where you're trying to enable these jobs? Do we need a different mechanic to disable these for the existing internal queue?

This is for internal PRs only, not for enabling on official builds or whatnot. Right now no Helix tests are running on internal PRs. The intention is to mirror the public PRs in this case.

@Tratcher
Copy link
Member

Tratcher commented Aug 9, 2021

But as a side-effect, they'll all start running on https://dev.azure.com/dnceng/internal/_build?definitionId=928, right? We'd want a way to suppress that.

@mmitche
Copy link
Member Author

mmitche commented Aug 9, 2021

But as a side-effect, they'll all start running on https://dev.azure.com/dnceng/internal/_build?definitionId=928, right? We'd want a way to suppress that.

Good point. I'll look into getting that fixed.

@mmitche
Copy link
Member Author

mmitche commented Aug 16, 2021

@Tratcher Fixed this up. The previous logic on queue additions is a bit hard to follow (since the ci.yml queues ended up getting used for the helix-matrix and quarantine cases). I fixed it up by making the first block specific to PRs only and duplicating those 3 queues into the daily blocks in the right places.

If this queue matrix ever gets more complicated it probably deserves a rework so that it's not a mishmash of conditionals. In these cases I tend to favor using some kind of enumeration identifier for each distinct scenario and then using the Choose construct, making each scenario have an explicit queue list. It does result in duplication of lists, but it's a lot easier to follow and results in less unintended side effects when someone who doesn't understand subtleties in the conditionals attempts to change something.

@Tratcher Tratcher requested a review from HaoK August 16, 2021 20:52
@Tratcher
Copy link
Member

Thanks @mmitche, sounds good to me. I've pinged Hao for a fresh review of the implementation.

eng/SourceBuild.props Outdated Show resolved Hide resolved
eng/targets/Helix.Common.props Outdated Show resolved Hide resolved
eng/targets/Helix.Common.props Outdated Show resolved Hide resolved
Copy link
Member

@HaoK HaoK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable to me, eventually switching to explicit lists for each pipeline/usage does seem inevitable at this point

mmitche and others added 3 commits August 17, 2021 07:37
Co-authored-by: Doug Bunting <6431421 [email protected]>
Co-authored-by: Doug Bunting <6431421 [email protected]>
Co-authored-by: Doug Bunting <6431421 [email protected]>
@mmitche mmitche enabled auto-merge (squash) August 17, 2021 14:38
@mmitche mmitche merged commit 45d2c24 into dotnet:main Aug 17, 2021
@ghost ghost added this to the 6.0-rc1 milestone Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants