Display embedded man pages for built-in commands. #8456

ehuss · 2020-07-06T02:16:11Z

This changes cargo help COMMAND to display the man page for the given command. cargo COMMAND --help continues to show the basic clap output.

The man pages are embedded in the executable in a compressed format. There's also a copy of the man pages in text format for platforms that do not have the man executable (like Windows).

It is unfortunate to check in more pre-generated files. I hope in the future that the usage of asciidoc can be replaced with something else (possibly a custom markdown-based solution).

cc #6104

rust-highfive · 2020-07-06T02:16:14Z

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton

Seems pretty reasonable to me!

Is it possible though to add some form of CI checks to the auto-generated files? Something which ensures that what's in the repo matches what they'd look like if they were generated on CI

alexcrichton · 2020-07-06T14:52:09Z

src/bin/cargo/commands/help.rs

+/// Extracts the given man page from the compressed archive.
+///
+/// Returns None if the command wasn't found.
+fn extract_man(subcommand: &str, extension: &str) -> CargoResult<Option<Vec<u8>>> {


Should this return just Option since it unwrap()s internally anyway?

(also mind documenting the unwraps?)

alexcrichton · 2020-07-06T14:53:10Z

src/bin/cargo/commands/help.rs

+/// Write the contents of a man page to disk and spawn the given command to
+/// display it.
+fn write_and_spawn(contents: &[u8], command: &str) -> CargoResult<()> {
+    let mut tmp = tempfile::Builder::new().prefix("cargo-man").tempfile()?;


Is the tempfile required here? Programs like less I think we can just write everything to a pipe?

I'm not actually sure what other common CLI apps do here.

Unfortunately, AFAIK, the man program does not support piped content.

ehuss · 2020-07-06T18:01:16Z

I've been messing with checking on CI, and came to the realization that man (and sometimes asciidoctor itself) produces different output on different systems.

I think that emphasizes that it will be increasingly difficult for people to contribute to the documentation (if they don't have the exact same setup). I've been concerned about it in the past, but I feel like this is a tipping point.

Would you be opposed to a custom solution using Rust and markdown (to replace the asciidoctor stuff)? I don't know exactly what that would look like, or how much code it would be. Comparing to the html renderer and asciidoctor's manpage converter, I would hope it is roughly the same magnitude (under 1,000 lines).

I would probably still lean towards pre-generated files (instead of embedding pulldown-cmark into cargo itself, and rendering at runtime). I would probably just add a utility crate that can generate the rendered files.

alexcrichton · 2020-07-06T21:10:49Z

Gah bummer!

I don't think it's the end of the world to have a custom solution here, but it's sort of a bummer that we have to make something just for Cargo. Ideally it'd be shared by other CLI tooling elsewhere or such. (Cargo doesn't seem particularly unique in this regard).

As you mentioned though it's probably not a huge feature or implementation, so I think vendoring our own should be ok.

Mark-Simulacrum · 2020-07-06T21:34:22Z

It seems reasonable at least to perhaps have it out-of-repo (or make that an explicit "goal") -- I imagine at the very least, we'll probably want similar-ish tooling perhaps for rust{doc,c} --help eventually.

Add mdman for generating man pages. This introduces a new utility called `mdman` that converts a markdown-formatted document to a man page. This replaces asciidoctor, with the intent to make it easier to contribute, easier to have consistent formatting across platforms, and easier to generate plain-text documents for use on Windows (for #8456). This also includes a number of formatting fixes. There is some documentation in the `mdman/doc` directory explaining how to use it, and the docs in `src/doc/README.md` have been updated (this explains the structure of the files). The Makefile has been replaced with a simple shell script. CI has been updated to verify the checked-in docs are up-to-date. Perhaps in the future, these can be generated automatically (perhaps by `build.rs`?), but since that requires a bit of build system work (like upstream rust), this is deferred till later.

bors · 2020-08-03T19:03:39Z

☔ The latest upstream changes (presumably #8577) made this pull request unmergeable. Please resolve the merge conflicts.

ehuss · 2020-08-03T20:03:19Z

OK, I have rebased after #8577, and I think this should be ready to go. I haven't done much testing on less-common platforms and terminals (just mac/windows/linux/freebsd with a variety of common terminals). I'm kinda hoping if people run into problems, they can file an issue.

alexcrichton · 2020-08-03T21:20:42Z

@bors: r

Looks great to me 👍

bors · 2020-08-03T21:20:43Z

📌 Commit 0d6881c has been approved by alexcrichton

bors · 2020-08-03T21:20:51Z

⌛ Testing commit 0d6881c with merge 2732539...

bors · 2020-08-03T21:42:12Z

☀️ Test successful - checks-actions
Approved by: alexcrichton
Pushing 2732539 to master...

Update cargo 8 commits in 2d5c2381e4e50484bf281fc1bfe19743aa9eb37a..1653f354644834073d6d2541e27fae94588e685e 2020-07-31 21:56:08 0000 to 2020-08-04 23:14:37 0000 - Fix close_output test. (rust-lang/cargo#8587) - clippy fixes, use matches! macro in more places (rust-lang/cargo#8575) - Display embedded man pages for built-in commands. (rust-lang/cargo#8456) - Add mdman for generating man pages. (rust-lang/cargo#8577) - Fix typo 'more then' -> 'more than' in error and comments (rust-lang/cargo#8581) - cargo login: make login message less ambiguous (rust-lang/cargo#8579) - Fix broken link in Build Cache chapter. (rust-lang/cargo#8578) - Fix intra-doc tests for renamed lint. (rust-lang/cargo#8576)

Pkgsrc changes: * Remove patches now integrated upstream, many related to SunOS / Illumos. * The LLVM fix for powerpc is also now integrated upstream. * Adapt those patches where the source has moved or parts are integrated. * The randomness patches no longer applies, and I could not find where those files went... * Provide a separate bootstrap for NetBSD/powerpc 9.0, since apparently the C ABI is different from 8.0. Yes, this appears to be specific to the NetBSD powerpc ports. Upstream changes: Version 1.47.0 (2020-10-08) ========================== Language -------- - [Closures will now warn when not used.][74869] Compiler -------- - [Stabilized the `-C control-flow-guard` codegen option][73893], which enables [Control Flow Guard][1.47.0-cfg] for Windows platforms, and is ignored on other platforms. - [Upgraded to LLVM 11.][73526] - [Added tier 3\* support for the `thumbv4t-none-eabi` target.][74419] - [Upgrade the FreeBSD toolchain to version 11.4][75204] - [`RUST_BACKTRACE`'s output is now more compact.][75048] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [`CStr` now implements `Index<RangeFrom<usize>>`.][74021] - [Traits in `std`/`core` are now implemented for arrays of any length, not just those of length less than 33.][74060] - [`ops::RangeFull` and `ops::Range` now implement Default.][73197] - [`panic::Location` now implements `Copy`, `Clone`, `Eq`, `Hash`, `Ord`, `PartialEq`, and `PartialOrd`.][73583] Stabilized APIs --------------- - [`Ident::new_raw`] - [`Range::is_empty`] - [`RangeInclusive::is_empty`] - [`Result::as_deref`] - [`Result::as_deref_mut`] - [`Vec::leak`] - [`pointer::offset_from`] - [`f32::TAU`] - [`f64::TAU`] The following previously stable APIs have now been made const. - [The `new` method for all `NonZero` integers.][73858] - [The `checked_add`,`checked_sub`,`checked_mul`,`checked_neg`, `checked_shl`, `checked_shr`, `saturating_add`, `saturating_sub`, and `saturating_mul` methods for all integers.][73858] - [The `checked_abs`, `saturating_abs`, `saturating_neg`, and `signum` for all signed integers.][73858] - [The `is_ascii_alphabetic`, `is_ascii_uppercase`, `is_ascii_lowercase`, `is_ascii_alphanumeric`, `is_ascii_digit`, `is_ascii_hexdigit`, `is_ascii_punctuation`, `is_ascii_graphic`, `is_ascii_whitespace`, and `is_ascii_control` methods for `char` and `u8`.][73858] Cargo ----- - [`build-dependencies` are now built with opt-level 0 by default.][cargo/8500] You can override this by setting the following in your `Cargo.toml`. ```toml [profile.release.build-override] opt-level = 3 ``` - [`cargo-help` will now display man pages for commands rather just the `--help` text.][cargo/8456] - [`cargo-metadata` now emits a `test` field indicating if a target has tests enabled.][cargo/8478] - [`workspace.default-members` now respects `workspace.exclude`.][cargo/8485] - [`cargo-publish` will now use an alternative registry by default if it's the only registry specified in `package.publish`.][cargo/8571] Misc ---- - [Added a help button beside Rustdoc's searchbar that explains rustdoc's type based search.][75366] - [Added the Ayu theme to rustdoc.][71237] Compatibility Notes ------------------- - [Bumped the minimum supported Emscripten version to 1.39.20.][75716] - [Fixed a regression parsing `{} && false` in tail expressions.][74650] - [Added changes to how proc-macros are expanded in `macro_rules!` that should help to preserve more span information.][73084] These changes may cause compiliation errors if your macro was unhygenic or didn't correctly handle `Delimiter::None`. - [Moved support for the CloudABI target to tier 3.][75568] - [`linux-gnu` targets now require minimum kernel 2.6.32 and glibc 2.11.][74163] Internal Only -------- - [Improved default settings for bootstrapping in `x.py`.][73964] You can read details about this change in the ["Changes to `x.py` defaults"](https://blog.rust-lang.org/inside-rust/2020/08/30/changes-to-x-py-defaults.html) post on the Inside Rust blog. - [Added the `rustc-docs` component.][75560] This allows you to install and read the documentation for the compiler internal APIs. (Currently only available for `x86_64-unknown-linux-gnu`.) [1.47.0-cfg]: https://docs.microsoft.com/en-us/windows/win32/secbp/control-flow-guard [76980]: rust-lang/rust#76980 [75048]: rust-lang/rust#75048 [74163]: rust-lang/rust#74163 [71237]: rust-lang/rust#71237 [74869]: rust-lang/rust#74869 [73858]: rust-lang/rust#73858 [75716]: rust-lang/rust#75716 [75908]: rust-lang/rust#75908 [75516]: rust-lang/rust#75516 [75560]: rust-lang/rust#75560 [75568]: rust-lang/rust#75568 [75366]: rust-lang/rust#75366 [75204]: rust-lang/rust#75204 [74650]: rust-lang/rust#74650 [74419]: rust-lang/rust#74419 [73964]: rust-lang/rust#73964 [74021]: rust-lang/rust#74021 [74060]: rust-lang/rust#74060 [73893]: rust-lang/rust#73893 [73526]: rust-lang/rust#73526 [73583]: rust-lang/rust#73583 [73084]: rust-lang/rust#73084 [73197]: rust-lang/rust#73197 [72488]: rust-lang/rust#72488 [cargo/8456]: rust-lang/cargo#8456 [cargo/8478]: rust-lang/cargo#8478 [cargo/8485]: rust-lang/cargo#8485 [cargo/8500]: rust-lang/cargo#8500 [cargo/8571]: rust-lang/cargo#8571 [`Ident::new_raw`]: https://doc.rust-lang.org/nightly/proc_macro/struct.Ident.html#method.new_raw [`Range::is_empty`]: https://doc.rust-lang.org/nightly/std/ops/struct.Range.html#method.is_empty [`RangeInclusive::is_empty`]: https://doc.rust-lang.org/nightly/std/ops/struct.RangeInclusive.html#method.is_empty [`Result::as_deref_mut`]: https://doc.rust-lang.org/nightly/std/result/enum.Result.html#method.as_deref_mut [`Result::as_deref`]: https://doc.rust-lang.org/nightly/std/result/enum.Result.html#method.as_deref [`TypeId::of`]: https://doc.rust-lang.org/nightly/std/any/struct.TypeId.html#method.of [`Vec::leak`]: https://doc.rust-lang.org/nightly/std/vec/struct.Vec.html#method.leak [`f32::TAU`]: https://doc.rust-lang.org/nightly/std/f32/consts/constant.TAU.html [`f64::TAU`]: https://doc.rust-lang.org/nightly/std/f64/consts/constant.TAU.html [`pointer::offset_from`]: https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.offset_from

rust-highfive assigned alexcrichton Jul 6, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 6, 2020

alexcrichton reviewed Jul 6, 2020

View reviewed changes

ehuss added S-waiting-on-author Status: The marked PR is awaiting some action (such as code changes) from the PR author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 17, 2020

ehuss mentioned this pull request Aug 1, 2020

Add mdman for generating man pages. #8577

Merged

ehuss added 3 commits August 3, 2020 12:17

Create a dedicated module for help tests.

9138d65

Display embedded man pages for built-in commands.

0e26eae

Change extract_man to not return a Result (not needed).

0d6881c

ehuss force-pushed the embedded-man branch from 1dd0684 to 0d6881c Compare August 3, 2020 19:37

ehuss removed the S-waiting-on-author Status: The marked PR is awaiting some action (such as code changes) from the PR author. label Aug 3, 2020

bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Aug 3, 2020

bors merged commit 2732539 into rust-lang:master Aug 3, 2020

ehuss mentioned this pull request Aug 5, 2020

Update cargo rust-lang/rust#75169

Merged

ehuss added this to the 1.47.0 milestone Feb 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Display embedded man pages for built-in commands. #8456

Display embedded man pages for built-in commands. #8456

ehuss commented Jul 6, 2020

rust-highfive commented Jul 6, 2020

alexcrichton left a comment

alexcrichton Jul 6, 2020

alexcrichton Jul 6, 2020

ehuss Jul 6, 2020

ehuss commented Jul 6, 2020

alexcrichton commented Jul 6, 2020

Mark-Simulacrum commented Jul 6, 2020

bors commented Aug 3, 2020

ehuss commented Aug 3, 2020

alexcrichton commented Aug 3, 2020

bors commented Aug 3, 2020

bors commented Aug 3, 2020

bors commented Aug 3, 2020

Display embedded man pages for built-in commands. #8456

Display embedded man pages for built-in commands. #8456

Conversation

ehuss commented Jul 6, 2020

rust-highfive commented Jul 6, 2020

alexcrichton left a comment

Choose a reason for hiding this comment

alexcrichton Jul 6, 2020

Choose a reason for hiding this comment

alexcrichton Jul 6, 2020

Choose a reason for hiding this comment

ehuss Jul 6, 2020

Choose a reason for hiding this comment

ehuss commented Jul 6, 2020

alexcrichton commented Jul 6, 2020

Mark-Simulacrum commented Jul 6, 2020

bors commented Aug 3, 2020

ehuss commented Aug 3, 2020

alexcrichton commented Aug 3, 2020

bors commented Aug 3, 2020

bors commented Aug 3, 2020

bors commented Aug 3, 2020