Unable to Run cargo check with `--package xla --no-default-features --features "trn"` #14620

xanderdunn · 2023-04-21T12:45:40Z

xanderdunn
Apr 21, 2023

I want rust-analyzer to build and check like this: cargo check --package xla --no-default-features --features "trn". This command successfully runs from the root of my rust repo on the command line.

Log of coc.nvim

2023-04-20T07:35:55.073 INFO (pid:430621) [plugin] - coc.nvim initialized with node: v18.15.0 after 180
2023-04-20T07:35:55.074 INFO (pid:430621) [services] - LanguageClient Rust Analyzer Language Server state change: stopped => starting
2023-04-20T07:35:55.576 INFO (pid:430621) [language-client-index] - Language server "rust-analyzer" started with 430677
2023-04-20T07:35:55.587 INFO (pid:430621) [services] - LanguageClient Rust Analyzer Language Server state change: starting => running
2023-04-20T07:35:55.620 INFO (pid:430621) [services] - service rust-analyzer started

:CocCommand rust-analyzer.serverVersion

[coc.nvim] rust-analyzer 0.3.1481-standalone

Question

I haven't been able to figure out how to successfully configure rust-analyzer to build only one package with no default features. There are certain packages which only build when a GPU is present, which is why --package xla is necessary. There are certain features which only build when a GPU is present, which is why we need --no-default-features --features "trn". I attempted to reproduce these flags for rust-analyzer in :CocConfig:

{
    "rust-analyzer.check.noDefaultFeatures": true,
    "rust-analyzer.check.features": [ "trn" ],
    "rust-analyzer.check.extraEnv": { "CPATH": "/opt/aws/neuron/include", "LIBRARY_PATH": "/opt/aws/neuron/lib", "LD_LIBRARY_PATH": "/opt/aws/neuron/lib" },
    "rust-analyzer.cargo.noDefaultFeatures": true,
    "rust-analyzer.cargo.features": [ "trn" ],
    "rust-analyzer.check.extraArgs": [ "--package", "xla" ],
    "rust-analyzer.cargo.extraEnv": { "CPATH": "/opt/aws/neuron/include", "LIBRARY_PATH": "/opt/aws/neuron/lib", "LD_LIBRARY_PATH": "/opt/aws/neuron/lib" },
    "rust-analyzer.trace.server": "messages",
    "rust-analyzer.check.invocationLocation": "/home/ubuntu/dev/Kholinar/xla/",
    "coc.preferences.formatOnSaveFiletypes": ["rust"]
}

But the config changes appear to have no effect. CocCommand workspace.showOutput rust-analyzer shows:

[ERROR rust_analyzer::main_loop] FetchBuildDataError:
error: failed to run custom build command for `kernels v0.1.0 (/home/ubuntu/dev/Kholinar/kernels)`

Caused by:
  process didn't exit successfully: `/home/ubuntu/dev/Kholinar/target/debug/build/kernels-4c4d8a7354bdd89c/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=kernels/kernels/saxpy.cu
  TARGET = Some("x86_64-unknown-linux-gnu")
  HOST = Some("x86_64-unknown-linux-gnu")
  cargo:rerun-if-env-changed=CXX_x86_64-unknown-linux-gnu
  CXX_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CXX_x86_64_unknown_linux_gnu
  CXX_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CXX
  HOST_CXX = None
  cargo:rerun-if-env-changed=CXX
  CXX = None
  cargo:rerun-if-env-changed=NVCC_x86_64-unknown-linux-gnu
  NVCC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=NVCC_x86_64_unknown_linux_gnu
  NVCC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_NVCC
  HOST_NVCC = None
  cargo:rerun-if-env-changed=NVCC
  NVCC = None
  cargo:rerun-if-env-changed=CXXFLAGS_x86_64-unknown-linux-gnu
  CXXFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CXXFLAGS_x86_64_unknown_linux_gnu
  CXXFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CXXFLAGS
  HOST_CXXFLAGS = None
  cargo:rerun-if-env-changed=CXXFLAGS
  CXXFLAGS = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  running: "nvcc" "-ccbin=c  " "-Xcompiler" "-O2" "-Xcompiler" "-ffunction-sections" "-Xcompiler" "-fdata-sections" "-Xcompiler" "-fPIC" "-m64" "-Xcompiler" "-Wall" "-Xcompiler" "-Wextra" "-cudart=shared" "-gencode" "arch=compute_80,code=sm_80" "-o" "/home/ubuntu/dev/Kholinar/target/debug/build/kernels-8531f2e94c23d756/out/kernels/saxpy.o" "-c" "kernels/saxpy.cu"

  --- stderr


  error occurred: Failed to find tool. Is `nvcc` installed?


error: failed to run custom build command for `nccl v0.1.0 (/home/ubuntu/dev/Kholinar/nccl)`

Caused by:
  process didn't exit successfully: `/home/ubuntu/dev/Kholinar/target/debug/build/nccl-229f99c26e15ea6a/build-script-build` (exit status: 101)
  --- stdout
  cargo:rustc-link-lib=dylib=cudart
  cargo:rerun-if-changed=nccl_wrapper.h
  cargo:rustc-link-lib=dylib=nccl

  --- stderr
  nccl_wrapper.h:1:10: fatal error: 'nccl.h' file not found
  thread 'main' panicked at 'Unable to generate bindings: ClangDiagnostic("nccl_wrapper.h:1:10: fatal error: 'nccl.h' file not found\n")', nccl/build.rs:53:10
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:575:5
     1: core::panicking::panic_fmt
               at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/panicking.rs:64:14
     2: core::result::unwrap_failed
               at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/result.rs:1790:5
     3: core::result::Result<T,E>::expect
               at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/result.rs:1069:23
     4: build_script_build::generate_bindings
               at ./build.rs:44:20
     5: build_script_build::main
               at ./build.rs:66:5
     6: core::ops::function::FnOnce::call_once
               at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/ops/function.rs:250:5
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

We see it is trying to build the kernels module, which requires GPU. This isn't going to work, and the combination of --package xla and --no-default-features would prevent this, as it does when I run cargo check --package xla --no-default-features --features "trn" on the command line. Is there any way for me to see exactly what cargo check command is being issued by rust-analyzer? The Cargo.toml at the root of my repo looks like this:

[workspace]
resolver = "2"
members = [
    "examples/*",
    "kernels",
    "logging",
    ... other unrelated members
    "nn",
    "pytest",
    "s3",
    "utils",
    "xla",
]

[workspace.dependencies]
...

Do you see any issues with my coc-settings.json above? What's the right way to build only a specific package in my workspace with no default features and one specific feature enabled?

Answered by xanderdunn

Apr 23, 2023

I switched to VSCode to remove vim and the coc-rust-analyzer plugin as a variable here. My settings.json:

{
    "rust-analyzer.cargo.extraArgs": [
        "--package", "xla"
    ],
    "rust-analyzer.cargo.noDefaultFeatures": true,
    "rust-analyzer.cargo.features": [
        "trn"
    ],
}

However, rust-analyzer logs show it is still trying to build the kernels package. I believe this is because rust-analyzer is issuing this command:

cd "/Users/xander/dev/Kholinar" && "/Users/xander/.cargo/bin/cargo" "check" "--workspace" "--message-format=json-diagnostic-rendered-ansi" "--manifest-path" "/Users/xander/dev/Kholinar/Cargo.toml" "--all-targets" "--no-default-features" "--features" "trn" "…

View full answer

lnicola · 2023-04-22T13:02:41Z

lnicola
Apr 22, 2023
Collaborator

Does setting rust-analyzer.cargo.extraArgs help?

3 replies

xanderdunn Apr 22, 2023
Author

Thanks for suggesting this. I see in #14328 this was introduced to allow passing flags when building build scripts. It is indeed a build script that rust analyzer's cargo invocation is failing on, so I will likely need rust-analyzer.cargo.extraArgs to fix this problem. However, it's unclear if the neovim plugin for rust-analyzer supports cargo.extraArgs, see fannheyward/coc-rust-analyzer#1134.

lnicola Apr 22, 2023
Collaborator

Good point. As far as I know, that plugin is a fork of the Code extension, but this setting is used strictly by the server. As long as the Vim plugin doesn't outright reject unknown settings, it should work.

xanderdunn Apr 22, 2023
Author

I tried it and it had no effect.

xanderdunn · 2023-04-23T11:47:48Z

xanderdunn
Apr 23, 2023
Author

I switched to VSCode to remove vim and the coc-rust-analyzer plugin as a variable here. My settings.json:

{
    "rust-analyzer.cargo.extraArgs": [
        "--package", "xla"
    ],
    "rust-analyzer.cargo.noDefaultFeatures": true,
    "rust-analyzer.cargo.features": [
        "trn"
    ],
}

However, rust-analyzer logs show it is still trying to build the kernels package. I believe this is because rust-analyzer is issuing this command:

cd "/Users/xander/dev/Kholinar" && "/Users/xander/.cargo/bin/cargo" "check" "--workspace" "--message-format=json-diagnostic-rendered-ansi" "--manifest-path" "/Users/xander/dev/Kholinar/Cargo.toml" "--all-targets" "--no-default-features" "--features" "trn" "--package" "xla"

Notice the --workspace. If I run this command on the command line but remove --workspace, then it behaves as desired. It builds the xla package and its dependencies and nothing else. So, ultimately, this is the config that worked:

    "rust-analyzer.cargo.buildScripts.overrideCommand": ["/home/ubuntu/.cargo/bin/cargo", "check", "--message-format=json", "--all-targets" , "--package", "xla", "--no-default-features", "--features", "trn"],
    "rust-analyzer.check.overrideCommand": ["/home/ubuntu/.cargo/bin/cargo", "check", "--message-format=json", "--all-targets" , "--package", "xla", "--no-default-features", "--features", "trn"]

I had to manually override the entire build commands because by default rust-analyzer passes --workspace, which builds all the packages even if I specify --package xla elsewhere.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to Run cargo check with `--package xla --no-default-features --features "trn"` #14620

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 3 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Unable to Run cargo check with --package xla --no-default-features --features "trn" #14620

xanderdunn Apr 21, 2023

Log of coc.nvim

:CocCommand rust-analyzer.serverVersion

Question

Replies: 2 comments · 3 replies

lnicola Apr 22, 2023 Collaborator

xanderdunn Apr 22, 2023 Author

lnicola Apr 22, 2023 Collaborator

xanderdunn Apr 22, 2023 Author

xanderdunn Apr 23, 2023 Author

Unable to Run cargo check with `--package xla --no-default-features --features "trn"` #14620

xanderdunn
Apr 21, 2023

Replies: 2 comments 3 replies

lnicola
Apr 22, 2023
Collaborator

xanderdunn Apr 22, 2023
Author

lnicola Apr 22, 2023
Collaborator

xanderdunn Apr 22, 2023
Author

xanderdunn
Apr 23, 2023
Author