Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V2 is different than V3 #2632

Merged
merged 9 commits into from
Aug 21, 2017
Merged

V2 is different than V3 #2632

merged 9 commits into from
Aug 21, 2017

Conversation

matthid
Copy link
Member

@matthid matthid commented Aug 19, 2017

For example consider https://api.nuget.org/v3-flatcontainer/machine.specifications/index.json?semVerLevel=2.0.0 (and V3 API)
it returns "0.10.0-unstable0014". Paket uses this string currently on a V2 API,

returns "0.10.0-Unstable0014". And in fact https://www.nuget.org/api/v2/Packages(Id='machine.specifications',Version='0.10.0-Unstable0014') works

The problem with this change is in fact that it degrades performance. As the V3 GetPackageDetails needs to API calls to get the required information, which the V2 API generally is only a single call.

@matthid
Copy link
Member Author

matthid commented Aug 19, 2017

Also this change uncovers some bugs on our v3 implementation, which were hidden because it was never actually used. For example if I let it run on FAKE (paket update) with v3 urls:

Paket failed with
-> The NuGet source http://www.nuget.org/api/v2 for package Microsoft.Bcl was not found in the paket.dependencies file with sources [NuGetV3 {Url = "https://api.nuget.org/v3/index.json";
             Authentication = None;};
    NuGetV2 {Url = "https://ci.appveyor.com/nuget/fake";
             Authentication = None;};
    NuGetV2 {Url = "https://ci.appveyor.com/nuget/fsharp-formatting";
             Authentication = None;}]

As well as some "Unable to parse framework .NETPlatform5.4" (and other warnings). It seems like V3 returns some (including Portable-) profiles in a format we cannot currently parse.

@matthid
Copy link
Member Author

matthid commented Aug 19, 2017

The new warning:

$ ../Paket/bin/paket.exe update --group main
Paket version 5.85.8
Updating group main in C:\Proj\FAKE\paket.dependencies
Resolving packages for group Main:
 - Microsoft.Web.Administration is pinned to 7.0.0
 - Argu is pinned to 3.3.0
 - Microsoft.AspNet.Razor is pinned to 2.0.30506
 - Microsoft.AspNet.WebPages is pinned to 2.0.30506
Could not detect any platforms from '.NETPortable4.5-Profile259' in NETStandard.Library 2.0.0
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0
 - log4net is pinned to 2.0.5
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
 - serilog.sinks.nlog is pinned to 1.5.4
Could not detect any platforms from '.NETPortable4.5-Profile259' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0-preview2-25401-01
 - Machine.Specifications is pinned to 0.9.3
Could not detect any platforms from '.NETPortable4.5-Profile259' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0-preview1-25301-01
 - Machine.Specifications.Should is pinned to 0.8
 - RavenDB.Client 2.5.25023
 - Mono.Cecil 0.10.0-beta6
 - Paket.Core 5.86.0
 - bootstrap 3.3.7
 - FluentMigrator.Runner 1.12.0
 - FsCheck 2.9.0
 - FsCheck.Xunit 2.9.0
 - FSharp.Compiler.Service 13.0.0
 - FSharp.Core 4.2.3
 - HashLib 2.0.1
 - Knockout 0.0.1
 - Knockout.Mapping 2.4.0
 - Machine.Specifications.Runner.Console 0.9.3
 - Microsoft.DotNet.PlatformAbstractions 2.0.0
 - Microsoft.Net.Http 2.2.29
Could not detect any platforms from '.NETPortable0.0-Profile7' in Microsoft.DiaSymReader.PortablePdb 1.2.0
Could not detect any platforms from '.NETPortable0.0-Profile7' in Microsoft.DiaSymReader.PortablePdb 1.2.0
Could not detect any platforms from '.NETPortable0.0-Profile7' in Microsoft.DiaSymReader.PortablePdb 1.2.0
 - Mono.Web.Xdt 1.0.0
 - Nancy.Authentication.Forms 1.4.1
 - Nancy.Authentication.Stateless 1.4.1
Could not detect any platforms from '.NETPortable0.0-Profile7' in System.Reflection.Metadata 1.5.0
Could not detect any platforms from '.NETPortable4.5-Profile111' in System.Reflection.Metadata 1.5.0
 - Nancy.Hosting.Aspnet 1.4.1
 - Nancy.Hosting.Self 1.4.1
 - Nancy.Serialization.JsonNet 1.4.1
 - Nancy.Testing 1.4.1
 - Nancy.Viewengines.Razor 1.4.3
 - Newtonsoft.Json 10.0.3
 - NuGet.CommandLine 4.1.0
 - Nuget.Core 2.14.0
 - NUnit 3.7.1
 - NUnit.Console 3.7.0
 - Octokit 0.24.0
 - RavenDB.Server 3.5.4
 - SSH.NET 2016.0.0
 - System.Web.Razor.Unofficial 2.0.2
 - toastr 2.1.1
 - Unquote 3.2.0
 - xunit 2.2.0
 - xunit.runner.console 2.2.0
 - xunit.extensibility.execution 2.2.0
 - jQuery 3.1.1
 - Serilog 1.5.14
 - xunit.assert 2.2.0
 - xunit.core 2.2.0
 - SshNet.Security.Cryptography 1.2.0
 - System.ValueTuple 4.4.0
 - runtime.native.System 4.3.0
 - runtime.native.System.IO.Compression 4.3.0
 - System.Buffers 4.4.0
 - System.Collections 4.3.0
 - System.Diagnostics.Debug 4.3.0
 - System.Diagnostics.Tracing 4.3.0
 - System.Globalization 4.3.0
 - System.IO 4.3.0
 - System.IO.Compression 4.3.0
 - System.Linq 4.3.0
 - System.Reflection 4.3.0
 - System.Reflection.Emit 4.3.0
 - System.Reflection.Extensions 4.3.0
 - System.Reflection.Primitives 4.3.0
 - System.Reflection.TypeExtensions 4.4.0
 - System.Resources.ResourceManager 4.3.0
 - System.Runtime 4.3.0
 - System.Runtime.Extensions 4.3.0
 - System.Runtime.Handles 4.3.0
 - System.Runtime.InteropServices 4.3.0
 - System.Security.Cryptography.Algorithms 4.3.0
 - System.Text.Encoding 4.3.0
 - System.Text.Encoding.Extensions 4.3.0
 - System.Threading 4.3.0
 - System.Threading.Tasks 4.3.0
 - Microsoft.CSharp 4.4.0
 - System.ComponentModel.TypeConverter 4.3.0
 - System.Runtime.Serialization.Formatters 4.3.0
 - System.Runtime.Serialization.Primitives 4.3.0
 - System.Xml.XmlDocument 4.3.0
 - System.Runtime.Loader 4.3.0
 - System.Threading.Thread 4.3.0
 - System.Diagnostics.FileVersionInfo 4.3.0
 - System.Diagnostics.Process 4.3.0
 - System.Diagnostics.TraceSource 4.3.0
 - System.Security.Cryptography.ProtectedData 4.4.0
 - System.Xml.XDocument 4.3.0
 - System.Xml.XPath.XDocument 4.3.0
 - System.Xml.XPath.XmlDocument 4.3.0
 - System.Linq.Expressions 4.3.0
 - System.Text.RegularExpressions 4.3.0
 - System.AppContext 4.3.0
 - System.Net.Sockets 4.3.0
 - System.Net.Requests 4.3.0
 - System.Threading.ThreadPool 4.3.0
 - System.Diagnostics.Tools 4.3.0
 - System.Linq.Queryable 4.3.0
 - System.Runtime.Numerics 4.3.0
 - System.Threading.Tasks.Parallel 4.3.0
 - System.Threading.Timer 4.3.0
 - System.IO.FileSystem 4.3.0
 - System.IO.FileSystem.Primitives 4.3.0
 - System.Console 4.3.0
 - AspNetMvc 4.0.20710
 - System.Runtime.InteropServices.RuntimeInformation 4.3.0
 - System.Security.Cryptography.Csp 4.3.0
 - System.Net.NameResolution 4.3.0
 - NUnit.ConsoleRunner 3.7.0
 - NUnit.Extension.NUnitV2Driver 3.7.0
 - NUnit.Extension.NUnitProjectLoader 3.6.0
 - NUnit.Extension.NUnitV2ResultWriter 3.6.0
 - NUnit.Extension.VSProjectLoader 3.6.0
 - NLog 4.4.12
 - Microsoft.Web.Xdt 2.1.1
 - xunit.abstractions 2.0.1
 - knockoutjs 3.4.2
 - FluentMigrator.Runner 1.6.2
 - FluentMigrator 1.6.2
 - NETStandard.Library 2.0.0
 - Nancy 1.4.4
 - System.Reflection.Metadata 1.5.0
 - CsQuery 1.3.4
 - System.Collections.Immutable 1.4.0
 - Microsoft.DiaSymReader.PortablePdb 1.2.0
 - Microsoft.Bcl 1.1.10
 - Microsoft.DiaSymReader 1.1.0
 - Microsoft.NETCore.Platforms 2.0.0
 - Microsoft.NETCore.Targets 2.0.0
 - Microsoft.Bcl.Build 1.0.21
 - NUnit.Extension.TeamCityEventListener 1.0.2
 - Microsoft.Web.Infrastructure 1.0.0
 - Microsoft.CompilerServices.AsyncTargetingPack 1.0.1
 - Chessie 0.6.0
 - FSharp.Compiler.Tools 4.1.23
 - Microsoft.AspNet.Mvc 4.0.40804
 - xunit.extensibility.core 2.2.0
 - System.Net.Http 4.3.2
 - System.Dynamic.Runtime 4.3.0
 - Microsoft.Win32.Primitives 4.3.0
 - System.Collections.Concurrent 4.3.0
 - System.Globalization.Calendars 4.3.0
 - System.IO.Compression.ZipFile 4.3.0
 - System.Net.Primitives 4.3.0
 - System.ObjectModel 4.3.0
 - System.Security.Cryptography.Encoding 4.3.0
 - System.Security.Cryptography.Primitives 4.3.0
 - System.Security.Cryptography.X509Certificates 4.3.0
 - System.Xml.ReaderWriter 4.3.0
 - System.ComponentModel 4.3.0
 - System.ComponentModel.Primitives 4.3.0
 - System.Reflection.Emit.ILGeneration 4.3.0
 - System.Reflection.Emit.Lightweight 4.3.0
 - System.Security.Principal.Windows 4.4.0
 - runtime.native.System.Security.Cryptography.Apple 4.3.0
 - runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple 4.3.0
 - runtime.native.System.Net.Http 4.3.0
 - System.Diagnostics.DiagnosticSource 4.4.1
 - System.Globalization.Extensions 4.3.0
 - System.Security.Cryptography.OpenSsl 4.3.0
 - System.Security.Cryptography.Cng 4.4.0
 - System.Security.Claims 4.3.0
 - System.Security.Principal 4.3.0
C:\Proj\FAKE\paket.lock is already up-to-date
Installing into projects:
 - Creating model and downloading packages.
Downloading FSharp.Formatting 3.0.0-beta07 (Build)
Downloading FSharp.Formatting.CommandTool 3.0.0-beta07 (Build)
Performance:
 - Resolver: 1 minute, 22 seconds (1 runs)
    - Runtime: 904 milliseconds
    - Blocked (retrieving package details): 1 minute, 11 seconds (55 times)
    - Blocked (retrieving package versions): 9 seconds (14 times)
    - Not Blocked (retrieving package details): 119 times
    - Not Blocked (retrieving package versions): 249 times
 - Average Request Time: 581 milliseconds
 - Number of Requests: 838
 - Runtime: 1 minute, 26 seconds
Paket failed with
-> The NuGet source http://www.nuget.org/api/v2 for package FSharp.Core was not found in the paket.dependencies file with sources [NuGetV3 {Url = "https://api.nuget.org/v3/index.json";
             Authentication = None;};
    NuGetV2 {Url = "https://ci.appveyor.com/nuget/fake";
             Authentication = None;};
    NuGetV2 {Url = "https://ci.appveyor.com/nuget/fsharp-formatting";
             Authentication = None;}]

@matthid
Copy link
Member Author

matthid commented Aug 19, 2017

I think now we have too many warnings again :)

@matthid
Copy link
Member Author

matthid commented Aug 19, 2017

Ok the error message from above is an unrelated bug: #2634

@matthid matthid changed the title [WIP] V2 is different than V3 V2 is different than V3 Aug 19, 2017
@matthid
Copy link
Member Author

matthid commented Aug 19, 2017

This is basically ready (maybe some integration tests need to be updated -> CI), however I expect it to degrade performance a bit.

None
else Some { Name = path; Platforms = platforms })

let extractPlatforms warn path =
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general guideline for warning parameter is:

  • true is always OK
  • false is only OK when either the calling method prints a better warning OR when the calling method has it's own warning parameter

This way warnings should not get lost. I don't know if there is better way to do this.

@matthid
Copy link
Member Author

matthid commented Aug 19, 2017

@dmunch just FYI I removed this monoandroid44w thing. Is there any actual package using this? Because afaics nuget doesn't even support it either: http://nugettoolsdev.azurewebsites.net/4.3.0-preview4/parse-framework?framework=monoandroid44w

@forki
Copy link
Member

forki commented Aug 20, 2017

what's the status here?

@matthid
Copy link
Member Author

matthid commented Aug 20, 2017

Failing test is spurious, no idea what's going on

@matthid
Copy link
Member Author

matthid commented Aug 20, 2017

OK found the problem: There is no wpa80 apparently, fix is under way.

- Add net50 to project files ( update baselines)
- Add test to ensure KnownTargetProfiles-lists are updated
- Fix wpa parsing: Was parsed as wpa80 but that one doesn't exist (at
least not in NuGet). wpa81 is the correct one.
- Add hack for uap10.1 to keep project files constant for '.NetCore' (I
don't know the correct condition to generate anyway)
let tagReader = FSharp.Reflection.FSharpValue.PreComputeUnionTagReader(typeof<'t>)
let cases = FSharp.Reflection.FSharpType.GetUnionCases(typeof<'t>)
checkListEx tagReader cases l
checkList KnownTargetProfiles.DotNetFrameworkVersions
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:)

@matthid
Copy link
Member Author

matthid commented Aug 20, 2017

I feel like this is ready.

@dmunch
Copy link
Contributor

dmunch commented Aug 20, 2017 via email

@dmunch
Copy link
Contributor

dmunch commented Aug 20, 2017 via email

@matthid
Copy link
Member Author

matthid commented Aug 20, 2017

@dmunch technically the string is not a valid tfm. Nuget.exe is not detecting it either.

I don't want to break anything, but can you show an actual package using this string and is nuget.exe working with it?

@0x53A
Copy link
Contributor

0x53A commented Aug 20, 2017

Note: uap10.1 is also detected as unsupported by paket, but IS detected by nuget:
http://nugettoolsdev.azurewebsites.net/4.3.0-preview4/parse-framework?framework=uap10.1

@matthid
Copy link
Member Author

matthid commented Aug 20, 2017

@0x53A I thought I added that with this PR?

[<Test>]
let ``Can detect uap10.1``() =
let p = PlatformMatching.forceExtractPlatforms "UAP10.1"
p.ToTargetProfile false |> shouldEqual (Some (SinglePlatform (FrameworkIdentifier.UAP UAPVersion.V10_1)))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like this.

Copy link
Contributor

@0x53A 0x53A Aug 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, sorry, I just saw the warning in another build log, and searching in github found the log at the top, where it was also missing, but I didn't actually look at the code.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no problem. This pr "fixes" tfm parsing to be more like nuget, but it makes supporting monoandroid44w a bit cumbersome. That's why I have removed it.

@matthid matthid deleted the v2_vs_v3_data branch July 2, 2019 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants