Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[build] reproducible builds issue from readdir #1625

Closed
bmwiedemann opened this issue Aug 14, 2023 · 3 comments
Closed

[build] reproducible builds issue from readdir #1625

bmwiedemann opened this issue Aug 14, 2023 · 3 comments

Comments

@bmwiedemann
Copy link

bmwiedemann commented Aug 14, 2023 •
edited
Loading

While working on reproducible builds for openSUSE, I found that our warp package varied across builds from non-determinism introduced by filesystem readdir order in

https://github.com/briansmith/ring/blob/98638b3/build.rs#L818

In python, a recursive walk usually needs some sorting of dir+file entries before processing to make it deterministic:
https://www.riverbankcomputing.com/pipermail/pyqt/2019-June/041854.html

edit: I get such diff:

--- old /usr/bin/warp (objdump)
+++ new /usr/bin/warp (objdump)
@@ -368906,10 +368906,10 @@
  5a17a0 2f3e3c2f 673e3c2f 7376673e 00002875  /></g></svg>..(u
  5a17b0 75617929 64726579 2f000000 14000000  uay)drey/.......
  5a17c0 7363616c 61626c65 2f000000 0b000000  scalable/.......
- 5a17d0 61637469 6f6e732f 06000000 12000000  actions/........
- 5a17e0 08000000 03000000 05000000 10000000  ................
- 5a17f0 07000000 16000000 02000000 13000000  ................
- 5a1800 00000000 0d000000 6170702f 09000000  ........app/....
+ 5a17d0 61637469 6f6e732f 02000000 03000000  actions/........
+ 5a17e0 0d000000 08000000 07000000 16000000  ................
+ 5a17f0 06000000 00000000 05000000 13000000  ................
+ 5a1800 12000000 10000000 6170702f 09000000  ........app/....
  5a1810 686f7269 7a6f6e74 616c2d61 72726f77  horizontal-arrow
  5a1820 732d6f6e 652d7761 792d7379 6d626f6c  s-one-way-symbol
  5a1830 69632e73 76670000 7b040000 00000000  ic.svg..{.......
@@ -371819,8 +371819,8 @@
  5acdb0 20302220 66696c6c 3d222332 65333433   0" fill="#2e343
  5acdc0 36222066 696c6c2d 72756c65 3d226576  6" fill-rule="ev
  5acdd0 656e6f64 64222f3e 3c2f7376 673e0000  enodd"/></svg>..
- 5acde0 28757561 79295761 72702f00 01000000  (uuay)Warp/.....
- 5acdf0 0f000000 04000000 11000000 68656c70  ............help
+ 5acde0 28757561 79295761 72702f00 04000000  (uuay)Warp/.....
+ 5acdf0 0f000000 11000000 01000000 68656c70  ............help
  5ace00 5f6f7665 726c6179 2e756900 00000000  _overlay.ui.....
  5ace10 f5070000 01000000 78dacd55 c16ec230  ........x..U.n.0
  5ace20 0cfd9528 f732b8ed d0861bec b80da61d  ...(.2..........
@bmwiedemann
Copy link
Author

I tried such a patch, https://github.com/rust-lang/git2-rs/pull/619/files but it did not help.

@briansmith
Copy link
Owner

Thanks for reporting this @bmwiedemann I definitely want to get to the point where ring"s build system is reproducable by default.

It makes sense that we need to do things in a deterministic order. I would be happy to review and merge a PR that (just) does the sorting and adds a comment, even if it doesn"t fully fix the issue. (There is another issue about embedded paths in the object/lib files that also needs to be addressed separately, IIRC, so it isn"t surprising that the sorting isn"t enough.)

@briansmith
Copy link
Owner

I looked at this and I don"t think our use of read_dir is likely to change the build output because it just emits cargo:rerun-if-changed lines.

Please comment if I"ve overlooked something and I"ll gladly reopen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@briansmith @bmwiedemann