Intermittent Paket.Restore.props error in 5.181.1 - Index was outside the bounds of the array #3404

yaakov-h · 2018-10-30T05:18:13Z

Description

After upgrading Paket to 5.181.1 from 5.173.1 I get an intermittent build error in Paket.Restore.props.

Repro steps

Run .paket\paket.exe restore
Run dotnet.exe restore MyProject.csproj --no-dependencies

Expected behavior

Packages are restored

Actual behavior

 Checking Paket version (version 5.181.1 requested)...
  Paket.exe 5.181.1 is up to date.
  Paket version 5.181.1
  The last restore is still up to date. Nothing left to do.
  Performance:
   - Runtime: 104 milliseconds
C:\path\to\project\.paket\Paket.Restore.targets(147,5): error MSB4184: The expression ""System.String[]".GetValue(5)" cannot be evaluated. Index was outside the bounds of the array.

Known workarounds

Delete all un-source-controlled files and rebuild.

The text was updated successfully, but these errors were encountered:

yaakov-h · 2018-10-30T05:40:55Z

On further investigation this appears to be caused by having paket files from 5.173.1 lying around when running the 5.181.1 restore process. If I scorch the project (tfpt scorch) then it builds fine.

yaakov-h · 2018-10-30T05:56:10Z

It seems to be reliably trigger-able by a dirty workspace.

In my case, Project\obj\Project.csproj.netstandard2.0.paket.resolved, Project\obj\Project.csproj.net46.paket.resolved and similar are CSV files with only 5 columns, not 6.

e.g.

Newtonsoft.Json,11.0.2,Transitive,Main,false
Microsoft.AspNet.WebApi.Client,5.2.6,Direct,Main,false

I assume that either:

Paket did not know to invalidate this and recompute this intermediary file, or
Paket does know to invalidate and recompute, but does that at some point after the MSBuild parsing runs and crashes.

j-alexander · 2018-11-05T21:27:25Z

It seems to be reliably trigger-able by a dirty workspace.

In my case, Project\obj\Project.csproj.netstandard2.0.paket.resolved, Project\obj\Project.csproj.net46.paket.resolved and similar are CSV files with only 5 columns, not 6.

Aha! Yes, and running git clean -xdf from the root of my project and running through it again works. I made the mistake of copying the repo from my windows VM and then trying to build in Visual Studio for mac.

forki · 2018-11-06T07:06:07Z

Can someone can send a pr to the targets file with proper error handling around that? Am Mo., 5. Nov. 2018, 22:27 hat Jonathan Leaver <[email protected]> geschrieben:

…

It seems to be reliably trigger-able by a dirty workspace. In my case, Project\obj\Project.csproj.netstandard2.0.paket.resolved, Project\obj\Project.csproj.net46.paket.resolved and similar are CSV files with only 5 columns, not 6. Aha! Yes, and running git -xdf from the root of my project and running through it again works. I made the mistake of copying the repo from my windows VM and then trying to build in Visual Studio for mac. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3404 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNE2VkanF9VzMKHynFUzD9F1gqUAgks5usK0-gaJpZM4YBMwo> .

yaakov-h · 2018-11-06T07:53:47Z

I think I can make a get-or-default fairly easily, but should the default be true, false, or some other behaviour entirely?

forki · 2018-11-06T08:17:22Z

maybe just make it a nicer error message?

yaakov-h · 2018-11-06T10:57:02Z

“Aaaaaaaah please delete all your obj/ folders and try again”? 🙃

forki · 2018-11-06T11:00:06Z

Well maybe go with a default of false then 😂 Am Di., 6. Nov. 2018, 11:57 hat Yaakov <[email protected]> geschrieben:

…

“Aaaaaaaah please delete all your obj/ folders and try again”? 🙃 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3404 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNP0IxBObiteW9rKABdggyEwL6Pcrks5usWr_gaJpZM4YBMwo> .

enricosada · 2018-11-06T14:26:02Z

The paket target file need to add msbuild condition, to check array length after split

Paket/src/Paket.Core/embedded/Paket.Restore.targets

Line 154 in 9f05288

 <PackageName>$([System.String]::Copy('%(PaketReferencesFileLines.Identity)').Split(',')[0])</PackageName> 

because was trying to load a csv with fewer columns, generated by previous paket version

yaakov-h · 2018-11-07T00:40:25Z

I can't quite make sense of 373c969:

A new column was inserted as the second-last column in the file.
The new column is AllPrivateAssets.
The old column was AllPrivateAssets and is now CopyLocal.

This doesn't leave me with a simple answer to default value.

Can we perhaps force packet.exe to regenerate this intermediary file if the column count is off?

forki · 2018-11-07T06:30:49Z

Ok then just give the error with info about cleaning /obj. Sorry Am Mi., 7. Nov. 2018, 01:40 hat Yaakov <[email protected]> geschrieben:

…

I can't quite make sense of 373c969 <373c969> : A new column was inserted as the second-last column in the file. The new column is AllPrivateAssets. The old column was AllPrivateAssets and is now CopyLocal. This doesn't leave me with a simple answer to default value. Can we perhaps force packet.exe to regenerate this intermediary file if the column count is off? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3404 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNH5qnDU-Wvxyc2Q-XJtCxLf83Q9eks5usiv5gaJpZM4YBMwo> .

yaakov-h · 2018-11-07T06:42:23Z

I'd rather have Paket self-recover than stall and require the user to take action. Particularly because I'd have to manually clean up a lot of obj folders.

Another possible fix:

If there are 5 columns, use old behaviour in props.
If there are 6 columns, use new behaviour in props.

Would you be OK with a bit of back-compat in the props file?

forki · 2018-11-07T06:58:00Z

Git clean -xdf and run build again. It does not matter how many obj you have. The problem with the fallback is that it's doing wrong behavior if you already use the properties. I think I'd rather see it come up with a better error message. Am Mi., 7. Nov. 2018, 07:42 hat Yaakov <[email protected]> geschrieben:

…

I'd rather have Paket self-recover than stall and require the user to take action. Particularly because I'd have to manually clean up a *lot* of obj folders. Another possible fix: If there are 5 columns, use old behaviour in props. If there are 6 columns, use new behaviour in props. Would you be OK with a bit of back-compat in the props file? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3404 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNOg8qB2bvJ1eAMoMlrcCSMdoIqVPks5usoDPgaJpZM4YBMwo> .

yaakov-h · 2018-11-07T07:00:00Z

Git clean -xdf and run build again. It does not matter how many obj you
have.

Try about 80 different TFVC repositories times 40 build servers, and I'd have to clean all serversat the time where paket is upgraded, in each repository. If I clean too early, the old paket will write the old files back out again.

(No, the build servers don't auto-clean between builds yet 😢)

forki · 2018-11-07T07:10:37Z

Ok but these issues are unrelated to paket. They will always bite you with all the others tools as well. I mean I can certainly feel your pain, but I don't see a good solution here that won't make things potentially worse. Am Mi., 7. Nov. 2018, 08:00 hat Yaakov <[email protected]> geschrieben:

…

Git clean -xdf and run build again. It does not matter how many obj you have. Try about 80 different TFVC repositories times 40 build servers, and I'd have to clean all serversat the time where paket is upgraded, in each repository. If I clean too early, the old paket will write the old files back out again. (No, the build servers don't auto-clean between builds yet 😢) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3404 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNIogSI-kt9T0tgGatGxQf1s5ZqnOks5usoTygaJpZM4YBMwo> .

yaakov-h · 2018-11-07T07:11:51Z

Possibly, but at the same time, I've never seen a similar tool that requires major intervention on a minor version upgrade. It should be able to either deal with it, or self-heal.

forki · 2018-11-07T07:20:53Z

That is actually a good point. I fucked up here. Let me try to fix it myself. Am Mi., 7. Nov. 2018, 08:11 hat Yaakov <[email protected]> geschrieben:

…

Possibly, but at the same time, I've never seen a similar tool that requires major intervention on a minor version upgrade. It should be able to either deal with it, or self-heal. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3404 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNBuKOyLebmToiFFZfUzy5SnQ_YYUks5usoe4gaJpZM4YBMwo> .

forki · 2018-11-07T10:44:23Z

can you please retry with latest - it also comes with slightly changed targets file

yaakov-h · 2018-11-08T04:23:50Z

I retried with 5.186.0, but now Paket doesn't include my dependencies from paket.references. I get build errors about missing classes/namespaces, and the dependencies don't show under the "Dependencies" node in Visual Studio's Solution Explorer.

If I scorch/clean the repository and rebuild, it works, suggesting that some leftover data from the old version is still being (mis-)used.

matthid · 2018-11-11T00:03:16Z

I'm not sure this is solved. I'm seeing something similar in the fake release process.

I'd expect that the following scenario works:

Restore with an older paket version
Continue the build with a newer paket version (consider the user updating the version in paket.dependencies for example)

As the following build shows this scenario is not working:

D:\a\1\s\.paket\Paket.Restore.targets(157,9): error MSB4184: The expression ""System.String[]".GetValue(5)" cannot be evaluated. Index was outside the bounds of the array. [D:\a\1\s\src\app\Fake.Core.Context\Fake.Core.Context.fsproj]

I feel like we need to be much more careful when updating caching files as this scenario is kind of important for fake to work properly together with paket.

forki closed this as completed in 828f9e4 Nov 7, 2018

matthid reopened this Nov 11, 2018

matthid mentioned this issue Nov 11, 2018

add compat fallback in case of older cache files. #3417

Merged

forki closed this as completed in #3417 Nov 13, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Intermittent Paket.Restore.props error in 5.181.1 - Index was outside the bounds of the array #3404

Intermittent Paket.Restore.props error in 5.181.1 - Index was outside the bounds of the array #3404

yaakov-h commented Oct 30, 2018 •

edited

Loading

yaakov-h commented Oct 30, 2018 •

edited

Loading

yaakov-h commented Oct 30, 2018

j-alexander commented Nov 5, 2018 •

edited

Loading

forki commented Nov 6, 2018 via email

yaakov-h commented Nov 6, 2018

forki commented Nov 6, 2018

yaakov-h commented Nov 6, 2018

forki commented Nov 6, 2018 via email

enricosada commented Nov 6, 2018

yaakov-h commented Nov 7, 2018

forki commented Nov 7, 2018 via email

yaakov-h commented Nov 7, 2018

forki commented Nov 7, 2018 via email

yaakov-h commented Nov 7, 2018

forki commented Nov 7, 2018 via email

yaakov-h commented Nov 7, 2018

forki commented Nov 7, 2018 via email

forki commented Nov 7, 2018

yaakov-h commented Nov 8, 2018

matthid commented Nov 11, 2018 •

edited

Loading

Intermittent Paket.Restore.props error in 5.181.1 - Index was outside the bounds of the array #3404

Intermittent Paket.Restore.props error in 5.181.1 - Index was outside the bounds of the array #3404

Comments

yaakov-h commented Oct 30, 2018 • edited Loading

Description

Repro steps

Expected behavior

Actual behavior

Known workarounds

yaakov-h commented Oct 30, 2018 • edited Loading

yaakov-h commented Oct 30, 2018

j-alexander commented Nov 5, 2018 • edited Loading

forki commented Nov 6, 2018 via email

yaakov-h commented Nov 6, 2018

forki commented Nov 6, 2018

yaakov-h commented Nov 6, 2018

forki commented Nov 6, 2018 via email

enricosada commented Nov 6, 2018

yaakov-h commented Nov 7, 2018

forki commented Nov 7, 2018 via email

yaakov-h commented Nov 7, 2018

forki commented Nov 7, 2018 via email

yaakov-h commented Nov 7, 2018

forki commented Nov 7, 2018 via email

yaakov-h commented Nov 7, 2018

forki commented Nov 7, 2018 via email

forki commented Nov 7, 2018

yaakov-h commented Nov 8, 2018

matthid commented Nov 11, 2018 • edited Loading

yaakov-h commented Oct 30, 2018 •

edited

Loading

yaakov-h commented Oct 30, 2018 •

edited

Loading

j-alexander commented Nov 5, 2018 •

edited

Loading

matthid commented Nov 11, 2018 •

edited

Loading