-
Notifications
You must be signed in to change notification settings - Fork 520
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"unlisted" spammed over and over during package resolution/restore #2149
Comments
@optical I fixed the spamming in Paket 3 (master branch). I think slow resolution is basically because of yesterday's hotfix where I disabled a heuristic if "lowest matching" is used. I assume with "normal" resolution it would be much faster. I didn't take to much time for yesterday's fix since Paket 4 didn't had that bug. @cloudRoutine this is an interesting case. As describe above I fixed it in master. Interstingly Paket 4 resolver runs into StackOverflow on this. |
Ok scratch that. There is no StackOverflow. I forgot to use Release mode. Regarding slowness: I now think it's because of the very tight restrictions. But using max resolver seems a bit faster. |
Thanks for the fix. Unfortunately the speed is too slow to be practically useful in my current setup. I've worked around this by pinning some of the packages paket appears to be having a rough time with, which isn't ideal but does work. For some context I am currently migrating a very large, monolithic source shared class library from using checked in binaries over to Nuget packages using paket to resolve. This assembly is shared amongst many applications via a subrepository and they include it as a csproj. This results in a paket.lock for each application, so we do end up doing these initial resolves a fair bit. Its worth noting, as you probably saw yourself the bulk of the restore is spent with the CPU pegged at 100%, so its not the sheer number of network requests that make it slow. Is this something that paket 4 will eventually cope with better? Admittedly I don't know much about V4, but just dropping the latest release into my solution and running it didn't work so well, looks like some of the resolver & framework settings are either being ignored or renamed, so I wasn't able to see if it was any faster. |
In general you want to reduce restrictions as much as possible to get more
speed. Otherwise the solution space is super weird shaped and it's hard to
find a valid solution.
What did not work in paket 4 that worked in paket 3?
Am 23.02.2017 8:27 nachm. schrieb "Jared Klopper" <[email protected]
:
Thanks for the fix. Unfortunately the speed is too slow to be practically
useful in my current setup. I've worked around this by pinning some of the
packages paket appears to be having a rough time with, which isn't ideal
but does work. For some context I am currently migrating a very large,
monolithic source shared class library from using checked in binaries over
to Nuget packages using paket to resolve. This assembly is shared amongst
many applications via a subrepository and they include it as a csproj. This
results in a paket.lock for each application, so we do end up doing these
initial resolves a fair bit.
Is this something that paket 4 will eventually cope with better? Admittedly
I don't know much about V4, but just dropping the latest release into my
solution and running it didn't work so well, looks like some of the
resolver & framework settings are either being ignored or renamed, so I
wasn't able to see if it was any faster.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#2149 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADgNL8UkLX9jD9GanDT1DhWoslz31FDks5rfd2XgaJpZM4MJhMH>
.
|
Also resolver strategy min is not really optimized. We don't use that very
much ourselves.
Am 23.02.2017 8:33 nachm. schrieb "Steffen Forkmann" <[email protected]>:
… In general you want to reduce restrictions as much as possible to get more
speed. Otherwise the solution space is super weird shaped and it's hard to
find a valid solution.
What did not work in paket 4 that worked in paket 3?
Am 23.02.2017 8:27 nachm. schrieb "Jared Klopper" <
***@***.***>:
Thanks for the fix. Unfortunately the speed is too slow to be practically
useful in my current setup. I've worked around this by pinning some of the
packages paket appears to be having a rough time with, which isn't ideal
but does work. For some context I am currently migrating a very large,
monolithic source shared class library from using checked in binaries over
to Nuget packages using paket to resolve. This assembly is shared amongst
many applications via a subrepository and they include it as a csproj. This
results in a paket.lock for each application, so we do end up doing these
initial resolves a fair bit.
Is this something that paket 4 will eventually cope with better?
Admittedly I don't know much about V4, but just dropping the latest release
into my solution and running it didn't work so well, looks like some of the
resolver & framework settings are either being ignored or renamed, so I
wasn't able to see if it was any faster.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#2149 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADgNL8UkLX9jD9GanDT1DhWoslz31FDks5rfd2XgaJpZM4MJhMH>
.
|
Thanks for the reply. The advice to not pin anything seems to be a winning strategy, I dropped all the pins and retained the min strategies and things are going about twice as fast as they were before, while resolving the same graph. Its taking less than a minute and not pegging my CPU, so I'll stick with this approach and see how it pans out once I shift all our dependencies over. I retested with paket4 and found I had made a mistake in my setup which caused it to behave differently - sorry for the confusion. |
Thaks for the feedback. What's the real reason for min strategy? |
It was driven by several factors, the tipping point being #2134 stinging us during deployment due to the missing transitive dependency. The other major factor is that this is a project shared by probably 50 other devs, and since we're switching from checked in binaries (I know...) which provide absolute guarantees around what version you're getting, over to potentially floating versions, there was a lot of fear about potentially getting bumped up to a version which introduces a regression etc. Additionally we have a few dependencies, some of which are internal that aren't following proper SemVer, so that is why we had some pinned packages. Given #2134 came up we decided to go with min to try minimize risk of aggressively floating up, and try ensure all applications sharing this monolith end up with as close to the same versions of its dependencies. The fact that Nuget utilises min by default was also an argument that came into play. Hopefully that makes some sense into the decision for min over max. |
Tbh I don't understand the issue in #2134 yet. Is it about missing binding
redirects? Maybe we should really try to analyze that one deeply.
In general I'm personally very unhappy with nuget's default. It just makes
no sense to me. Regarding fears about floating. The workflow is that we
look very careful into the different of the lock file after paket update.
Am 23.02.2017 9:32 nachm. schrieb "Jared Klopper" <[email protected]
…:
It was driven by several factors, the tipping point being #2134
<#2134> stinging us during
deployment due to the missing transitive dependency. The other major factor
is that this is a project shared by probably 50 other devs, and since
we're switching from checked in binaries (I know...) which provide absolute
guarantees around what version you're getting, over to potentially floating
versions, there was a lot of fear about potentially getting bumped up to a
version which introduces a regression etc. Additionally we have a few
dependencies, some of which are internal that aren't following proper
SemVer, so that is why we had some pinned packages. Given #2134
<#2134> came up we decided to
go with min to try minimize risk of aggressively floating up, and try
ensure all applications sharing this monolith end up with as close to the
same versions of its dependencies. The fact that Nuget utilises min by
default was also an argument that came into play.
Hopefully that makes some sense into the decision for min over max.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#2149 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AADgNDn_KnuEBX28DTUlx8KrLMBbaU0Eks5rfezZgaJpZM4MJhMH>
.
|
I am not sure what the solution with #2134 is, but adding binding redirects is one. I need to setup a test scenario which causes Nuget to float the transitive dep up to v15 and see how it copes with the situation, but I haven't got around to it, mostly since we decided to just go with the Min strategy. I'll see if I can get some time this weekend to setup that situation. |
Description
Not sure why paket logs "unlisted" when doing a restore, but after some dependencies I've noticed paket goes abit off the chain when doing the initial resolution of the dependency graph, spamming "unlisted" thousands of times, taking a very very long time (10 min ) to eventually succeed.
Repro steps
Please provide the steps required to reproduce the problem
runKlondike.bat
batch file to startup the nuget repositorypaket install
Expected behavior
Paket eventually succeeds with a meaningful log output
Actual behavior
Paket succeeds after a very long time and thousands upon thousands of noisy, seemingly meaningless log lines
Known workarounds
None known
The text was updated successfully, but these errors were encountered: