Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.77 produced release binaries die w/ SIGKILL on macos 14.4 #122902

Closed
devjgm opened this issue Mar 22, 2024 · 14 comments
Closed

1.77 produced release binaries die w/ SIGKILL on macos 14.4 #122902

devjgm opened this issue Mar 22, 2024 · 14 comments
Labels
C-bug Category: This is a bug. O-macos Operating system: macOS T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@devjgm
Copy link

devjgm commented Mar 22, 2024

Setup

$ cargo new demo
     Created binary (application) `demo` package
$ cd demo/

This command works with 1.76

$ cargo  1.76 run --release
   Compiling demo v0.1.0 (/Users/greg/scratch/demo)
    Finished release [optimized] target(s) in 0.63s
     Running `target/release/demo`
Hello, world!

This command fails with SIGKILL with 1.77

$ cargo  1.77 run --release
   Compiling demo v0.1.0 (/Users/greg/scratch/demo)
    Finished release [optimized] target(s) in 0.47s
     Running `target/release/demo`
Killed: 9
$ ./target/release/demo 
Killed: 9

I expected to see this happen: I expected to see "Hello, world!" printed.

Instead, this happened: The attempt to execute target/release/demo was killed by SIGKILL.

Note: This problem only seems to happen with --release profile builds. cargo 1.77 run --profile dev works fine.

Meta

rustc --version --verbose:

$ rustc --version --verbose
rustc 1.77.0 (aedd173a2 2024-03-17)
binary: rustc
commit-hash: aedd173a2c086e558c2b66d3743b344f977621a7
commit-date: 2024-03-17
host: aarch64-apple-darwin
release: 1.77.0
LLVM version: 17.0.6

$ sw_vers 
ProductName:            macOS
ProductVersion:         14.4
BuildVersion:           23E214
@devjgm devjgm added the C-bug Category: This is a bug. label Mar 22, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Mar 22, 2024
@jieyouxu jieyouxu added O-macos Operating system: macOS T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 22, 2024
@devjgm
Copy link
Author

devjgm commented Mar 22, 2024

The issue seems to have started in nightly-2024-01-18:

$ cargo  nightly-2024-01-18 run --release 
    Finished release [optimized] target(s) in 0.16s
     Running `target/release/demo`
Killed: 9

But this one works using nightly-2024-01-17

$ cargo  nightly-2024-01-17 run --release 
    Finished release [optimized] target(s) in 0.16s
     Running `target/release/demo`
Hello, world!

@joboet
Copy link
Contributor

joboet commented Mar 22, 2024

I have the exact same software versions, but I am unable to reproduce this. Could you try running the binaries under gdb/lldb to see where the SIGKILL gets generated?

@devjgm
Copy link
Author

devjgm commented Mar 22, 2024

lldb reports that it's an invalid mach-o file.

$ cargo  1.77 run --release
    Finished release [optimized] target(s) in 0.02s
     Running `target/release/demo`
Killed: 9
$ lldb target/release/demo
(lldb) target create "target/release/demo"
Current executable set to '/Users/greg/scratch/demo/target/release/demo' (arm64).
(lldb) r
error: Malformed Mach-o file
(lldb) 

However, objdump seems to show that the mach-o file is fine:

$ file target/release/demo
target/release/demo: Mach-O executable arm64
$ objdump -f target/release/demo

target/release/demo:     file format mach-o-arm64
architecture: aarch64, flags 0x00000012:
EXEC_P, HAS_SYMS
start address 0x0000000000001ad0

lldb works with a --profile dev binary

$ cargo  1.77 run --profile dev
    Finished dev [unoptimized   debuginfo] target(s) in 0.03s
     Running `target/debug/demo`
Hello, world!
$ lldb target/debug/demo
(lldb) target create "target/debug/demo"
Current executable set to '/Users/greg/scratch/demo/target/debug/demo' (arm64).
(lldb) r
Process 14925 launched: '/Users/greg/scratch/demo/target/debug/demo' (arm64)
Hello, world!
Process 14925 exited with status = 0 (0x00000000) 
(lldb) 

@joboet
Copy link
Contributor

joboet commented Mar 22, 2024

That looks like a linker issue. Do you have any linker overrides (e.g. in ~/.cargo/config.toml)?

@devjgm
Copy link
Author

devjgm commented Mar 22, 2024

Nothing there. I rm -rf ~/.cargo ~/.rustup and start from scratch installing from rustup.rs.

$ rustup show
Default host: aarch64-apple-darwin
rustup home:  /Users/greg/.rustup

installed toolchains
--------------------

stable-aarch64-apple-darwin (default)
1.76-aarch64-apple-darwin
1.77-aarch64-apple-darwin

active toolchain
----------------

stable-aarch64-apple-darwin (default)
rustc 1.77.0 (aedd173a2 2024-03-17)
/Users/greg/.cargo
├── bin
│   ├── cargo
│   ├── cargo-clippy
│   ├── cargo-fmt
│   ├── cargo-miri
│   ├── clippy-driver
│   ├── rls
│   ├── rust-analyzer
│   ├── rust-gdb
│   ├── rust-gdbgui
│   ├── rust-lldb
│   ├── rustc
│   ├── rustdoc
│   ├── rustfmt
│   └── rustup
├── env
└── env.fish

2 directories, 16 files

Still see the same SIGKILL from the OP.


In case it matters, on a MacBook Apple M2 Pro

@joboet
Copy link
Contributor

joboet commented Mar 22, 2024

Ok, so it's the default linker. What does ld -v say for you?

@devjgm
Copy link
Author

devjgm commented Mar 22, 2024

@(#)PROGRAM:ld PROJECT:ld-1053.12
BUILD 15:45:29 Feb  3 2024
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
will use ld-classic for: armv6 armv7 armv7s arm64_32 i386 armv6m armv7k armv7m armv7em
LTO support using: LLVM version 15.0.0 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 15.0.0 (tapi-1500.3.2.2)

@ehuss
Copy link
Contributor

ehuss commented Mar 22, 2024

I have a suspicion it might involve having the wrong strip in PATH, which can happen with homebrew. which strip should be /usr/bin/strip.

@devjgm
Copy link
Author

devjgm commented Mar 22, 2024

Good suspicion! Indeed, a homebrew strip is found first. However, tweaking the path doesn't seem to fix the problem:

$ which strip
/opt/homebrew/opt/binutils/bin/strip
$ PATH=/usr/bin:$PATH which strip
/usr/bin/strip
$ PATH=/usr/bin:$PATH cargo  1.77 run --release
    Finished release [optimized] target(s) in 0.02s
     Running `target/release/demo`
Killed: 9

@ehuss
Copy link
Contributor

ehuss commented Mar 22, 2024

You will need to run cargo clean before trying to run cargo run again.

@devjgm
Copy link
Author

devjgm commented Mar 22, 2024

UPDATE: I forgot to do a clean before re-doing the build. Indeed changing the path for strip does fix the issue:

$ cargo clean
     Removed 17 files, 1.0MiB total
$ PATH=/usr/bin:$PATH cargo  1.77 run --release
   Compiling demo v0.1.0 (/Users/greg/scratch/demo)
    Finished release [optimized] target(s) in 0.78s
     Running `target/release/demo`
Hello, world!

Thank you!

@devjgm
Copy link
Author

devjgm commented Mar 22, 2024

In case it's useful, here's a diff of the good (left) vs bad (right) binaries.

image

One thing I noticed is that the bad one (right) has a magic number of 0xfeedface, which I believe should be used for 32-bit executables. So maybe that's the problem, not sure.


Thank you both very much for helping debug this! Much appreciated!

@CrazyHer
Copy link

CrazyHer commented Mar 23, 2024

It might be related to this issue:
https://blogs.oracle.com/java/post/java-on-macos-14-4
This can explain why you received SIGKILL

@devjgm
Copy link
Author

devjgm commented Mar 23, 2024

It might be related to this issue: https://blogs.oracle.com/java/post/java-on-macos-14-4 This can explain why you received SIGKILL

I think that's a different issue.

The issue that @joboet and @ehuss helped track down was due to using my machine using strip from homebrew. That strip from homebrew binutils does not work correctly on macOS, apparently. Though it seems to have worked until Rust 1.77 (likely until rust-lang/cargo#13257).

It may be nice if cargo warned if /usr/bin/strip is not used, but I'll let you folks decide if there's anything to be done about this. For now, my problem has been diagnosed and fixed on my end.

Thanks again for the help, folks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. O-macos Operating system: macOS T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

6 participants