Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.20.1: paket install takes 40 seconds while 0.18 took 2 seconds #487

Closed
vasily-kirichenko opened this issue Dec 30, 2014 · 26 comments
Closed

Comments

@vasily-kirichenko
Copy link
Contributor

No description provided.

@forki
Copy link
Member

forki commented Dec 30, 2014

The new version invalidates the cache. So the first install will create a lot more nuget traffic. Is the second install faster?

@vasily-kirichenko
Copy link
Contributor Author

No, all installs are equally slow.

@forki
Copy link
Member

forki commented Dec 30, 2014

Ok. That's unfortunate. I'll investigate tomorrow.
On Dec 30, 2014 3:33 PM, "Vasily Kirichenko" [email protected]
wrote:

No, all installs are equally slow.


Reply to this email directly or view it on GitHub
#487 (comment).

@forki
Copy link
Member

forki commented Dec 30, 2014

Do you see this kind of performance issue also on open source project? I think I need something to run the profiler.

@forki
Copy link
Member

forki commented Dec 30, 2014

Also: how many projects / references files do you have?

@forki
Copy link
Member

forki commented Dec 30, 2014

And I forgot to mention: we redefined the install process. Now it performs a selective update. This means you can edit the dependencies file or references files and "install" will try to make it work with the minimal changes in the lockfile. In order to do so it runs the resolver.
So it is expected to be a bot slower (maybe not that slow). If you just want to restore the packages, there is also the "restore" command.

That said: the first shot for this selective update is not really optimized and I will try to make it faster. Maybe we could also skip the resolver if we detect no changes. Or things like that.

@forki
Copy link
Member

forki commented Dec 30, 2014

/cc @haf @agross

@haf
Copy link
Member

haf commented Dec 30, 2014

I've used it about 30 times today and I think it's really fast. But right now I'm only working towards the public nuget server.

@isaacabraham
Copy link
Contributor

If it's something to do with multiple nuget sources there's already #331 for that.

@vasily-kirichenko
Copy link
Contributor Author

36 seconds is not fast enough.
Yes, the vast majority of that time is taken by "exploring" our private nuget TeamCity server.

@vasily-kirichenko
Copy link
Contributor Author

This is our paket.dependencies. The solution contains ~180 projects.

source https://www.nuget.org/api/v2
source http://private/guestAuth/app/nuget/v1/FeedService.svc

nuget AutoFixture
nuget bootstrap 3.3.1
nuget BouncyCastle 1.7.0
nuget CassandraCSharpDriver
nuget Castle.FactorySupportFacility
nuget CommonServiceLocator 1.0.0.0
nuget DotNetZip
nuget EasyNetQ.Management.Client
nuget ExtCore
nuget FAKE
nuget Fizzler
nuget Fizzler.Systems.HtmlAgilityPack
nuget FluentNHibernate
nuget FontAwesome
nuget Foq
nuget FParsec
nuget FsCheck
nuget FsCheck.NUnit
nuget FSharp.Collections.ParallelSeq
nuget FSharp.Configuration
nuget FSharp.Control.Reactive
nuget FSharp.Data
nuget FSharp.Data.SqlClient
nuget FSharp.Management
nuget FSharpx.Core
nuget FSharpx.TypeProviders.Regex
nuget FsPickler
nuget FsUnit
nuget FsWebAddRegistryEntries 0.1.0.0
nuget fszmq
nuget HtmlAgilityPack 1.4.6
nuget Http.fs
nuget jquery.TypeScript.DefinitelyTyped 1.6.0
nuget knockout.TypeScript.DefinitelyTyped 0.6.7
nuget knockoutjs 3.2.0
nuget Microsoft.AspNet.Mvc 5.2.2
nuget Microsoft.AspNet.Web.Optimization 1.1.3
nuget Microsoft.AspNet.WebApi 5.0.0
nuget Moment.js 2.8.4
nuget moment.TypeScript.DefinitelyTyped 0.9.7
nuget mongocsharpdriver 1.9.2
nuget Moq
nuget MSBuild.Extension.Pack 1.3.0
nuget MSDN.FSharpChart.dll
nuget NEST 0.12.0.0
nuget NUnit 2.6.3
nuget NUnit.Runners 2.6.3
nuget Paket
nuget PowerShellTypeProvider
nuget protobuf-net 2.0.0.640
nuget Quartz
nuget RabbitMQ.Client
nuget RegexProvider
nuget RhinoMocks
nuget rickshaw 1.4.6
nuget rickshaw.TypeScript.DefinitelyTyped 0.0.2
nuget Selenium.WebDriver 2.41.0
nuget Selenium.WebDriver.ChromeDriver 2.9.0.1
nuget SevenZipSharp
nuget SQLProvider
nuget SSH.NET
nuget Strathweb.CacheOutput.WebApi2 0.6
nuget Suave
nuget underscore.js 1.7.0
nuget underscore.TypeScript.DefinitelyTyped 0.4.9
nuget UnionArgParser
nuget Unity 2.1.505.2
nuget Unquote
nuget WindowsAzure.Storage 3.0.2.0
nuget ZetaLongPaths
nuget NuGet.CommandLine
  9 private packages

@forki
Copy link
Member

forki commented Dec 30, 2014

Ok. That's sounds a bit like we broke the request caching.
On Dec 30, 2014 5:02 PM, "Vasily Kirichenko" [email protected]
wrote:

36 seconds is not fast enough.
Yes, the vast majority of that time is taken by "exploring" our private
nuget TeamCity server.


Reply to this email directly or view it on GitHub
#487 (comment).

@vasily-kirichenko
Copy link
Contributor Author

14 seconds if I remove all the private packages. Still too slow.

@isaacabraham
Copy link
Contributor

14 seconds for an install isn't (IMHO) too bad considering it's doing the dependency resolution etc.. Install shouldn't be the standard command to use though - a restore should be much quicker. But if it was 2 seconds before....

@forki
Copy link
Member

forki commented Dec 30, 2014

Sounds like a good benchmark project ;-)
On Dec 30, 2014 5:07 PM, "Vasily Kirichenko" [email protected]
wrote:

14 seconds if I remove all the private packages. Still too slow.


Reply to this email directly or view it on GitHub
#487 (comment).

@forki
Copy link
Member

forki commented Dec 30, 2014

There is still A LOT room for improvement...

@haf
Copy link
Member

haf commented Dec 30, 2014

Cool! Two of my projects in there ;)

@forki
Copy link
Member

forki commented Dec 31, 2014

could you please run the install with 0.20.6?

The second install should be a lot faster - I broke the caching in 0.20.0.

But I still found some other low hanging fruits as a goal for tomorrow.

@vasily-kirichenko
Copy link
Contributor Author

25 seconds first run, 17 seconds subsequent runs. If I remove all private packages, it works really fast (2 seconds).

Do I understand right that TeamCity support nuget v1 only and it causes the terrible performance?

@forki
Copy link
Member

forki commented Dec 31, 2014

Awesome.

There is still one improvement I need to do. Caching with more than one
server seems to be still an issue and might decrease perf. I think I can
fix that, too. But not in 2014.
On Dec 31, 2014 5:05 PM, "Vasily Kirichenko" [email protected]
wrote:

25 seconds first run, 17 seconds subsequent runs. If I remove all private
packages, it works really fast (2 seconds).

Do I understand right that TeamCity support nuget v1 only and it causes
the terrible performance?


Reply to this email directly or view it on GitHub
#487 (comment).

@isaacabraham
Copy link
Contributor

AFAIK - maybe no longer the case? is that every external NuGet feed causes a perf cost as the whole dependency chain must be searched across all feeds.

@forki
Copy link
Member

forki commented Dec 31, 2014

Yes that's a separate problem. But here I'm talking about subsequent
install processes (selective update with all versions pinned) without any
changes in the dependencies file or the references files. In theory there
should be no http request in this case. Unfortunately we are still doing a
lot.

If you do a full update it's still probing all feeds.
On Dec 31, 2014 5:21 PM, "Isaac Abraham" [email protected] wrote:

AFAIK - maybe no longer the case? is that every external NuGet feed causes
a perf cost as the whole dependency chain must be searched across all feeds.


Reply to this email directly or view it on GitHub
#487 (comment).

@forki
Copy link
Member

forki commented Jan 1, 2015

please retry with 0.20.10
as before the second install should be a lot faster, even with multiple feeds

@vasily-kirichenko
Copy link
Contributor Author

32 seconds first paket install (after updating to 0.20.11), 5 seconds subsequent installs.

Great job!

@forki
Copy link
Member

forki commented Jan 2, 2015

should be even faster now.
Could you please retry with 0.20.21?

@vasily-kirichenko
Copy link
Contributor Author

4 seconds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants