Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decide on schedule for bumping minimum supported Rust version #709

Closed
Minoru opened this issue Dec 25, 2019 · 17 comments · Fixed by #788
Closed

Decide on schedule for bumping minimum supported Rust version #709

Minoru opened this issue Dec 25, 2019 · 17 comments · Fixed by #788
Labels
meta This discussion relates to how Newsboat is managed and run as a project
Milestone

Comments

@Minoru
Copy link
Member

Minoru commented Dec 25, 2019

When I wrote our first Rust code, I used the earliest Rust version I could—it was 1.24. Later on, some important crate updates made me bump that to 1.26. We"re still using that version today.

This situation is great for our users, because it increases the chance that Newsboat can be built with whatever the user has in their distro.

At the same time, this situation is less great for the developers:

  • newer compilers warn about deprecated functions, polluting our CI logs and increasing our chances of missing something truly important;
  • we have to code against Rust 2015, which is sometimes more verbose (e.g. requiring ref in pattern matching arms) and doesn"t have some potentially useful things (I expect async/.await to be a great boon for our Reloader class).

Warnings can be suppressed, and lack of features can be tolerated to a point, but we can"t be stuck with 1.26 forever. We need a schedule by which we will bump this minimum supported version.

My next comment will outline my proposed solution.

@Minoru Minoru added the meta This discussion relates to how Newsboat is managed and run as a project label Dec 25, 2019
@Minoru
Copy link
Member Author

Minoru commented Dec 25, 2019

According to this Repology page, most distributions ship one of the three latest Rust releases. This old-ish Mozilla page says that Firefox requires stable Rust 2 weeks after it"s been released—but since Firefox uses a train release model, it means that by the time a release is promoted to "stable", its associated Rust version is already 2+6+6=14 weeks, or 3 months old (one train track takes 6 weeks for Firefox).

Extra data points:

With all of that said, I"m proposing what I outlined in #576 (comment):

  • right after releasing Newsboat, I"ll be setting minimum supported Rust version to current_stable-2. This should be comfortable enough for our developers, and also for bleeding-edge users—both these groups are unlikely to be stuck on some older Rust version. By the time of next Newsboat release, two Rust releases will happen, so each Newsboat release effectively supports the last five Rust releases
  • patch releases will support whatever version their major.minor supported—i.e. X.Y.100 will be buildable with the same Rust as X.Y.0, even though these two releases might be months apart, and new Rust versions came out in the meantime
  • our next major release (2.19, in March 2020) will still support 1.26, but I"ll announce that we"re switching to scheduled bumps starting with Newsboat 2.20 (expected in June 2020)

Perhaps a table would explain it better (dates and versions are approximate—I won"t commit for Newsboat, and I obviously can"t commit for Rust):

Newsboat release Release date Min. supported Rust version Rust stable at that time
TODAY: master branch requires 1.26 or newer
2.19 March 2020 1.26 1.42
master branch requires 1.40 or newer
2.20 June 2020 1.40 1.44
master branch requires 1.42 or newer
2.21 September 2020 1.42 1.46
master branch requires 1.44 or newer
2.22 December 2020 1.44 1.48
master branch requires 1.46 or newer
2.23 March 2021 1.46 1.50 or 1.51
master branch requires 1.48 or newer

Does this seem reasonable? Would that schedule create any problems for your workflow?

Let"s decide this by 1 March 2020.

Notification cannon for all the people who contributed in the last 6 months: @RomeuG @SeanBE @der-lyse @mjsir911 @rnestler @tsipinakis @t6 @khardix @noctux @juhopp @LucasParsy @kmws (if you ever run make, this issue concerns you; please respond!)

@khardix
Copy link
Contributor

khardix commented Dec 26, 2019

From the developer perspective, this seems fine to me, I can use rustup to install whatever toolchain version is required.

However, from the package maintainer (Fedora) perspective, this can get more complicated, and largely depends on which rust version is shipped in which Fedora release; they are unlikely to be version-bumped once released. On the other hand, supporting current stable-2 should be fine.

@tsipinakis
Copy link
Member

tsipinakis commented Dec 26, 2019

Does this seem reasonable?

On the Debian side this seems a bit too restrictive,
Current Rust stable is 1.40, Debian testing is still at 1.37 and stable at 1.34 (and more likely stable will not be updated until the next release in 2021).

Which has the following implications:

  • Anyone wanting a recent version of newsboat with Debian stable will have to manually install a more recent version of Rust
  • Possible delay of getting new versions of newsboat into testing

Would that schedule create any problems for your workflow?

I fit into the first point mentioned so with that restriction I"ll either have to look into backporting rust or switch to using an unstable chroot for newsboat development.

@der-lyse
Copy link
Contributor

I have always compiled and installed Newsb{euter,oat} myself and never used a version from the Debian packages (cannot remember why, maybe because there weren"t any at the time, not sure). So I reckon the proposed roadmap should be alright with me. Obviously, it would be nice if I could use the default Debian-shipped tools to build Newsboat, but getting the required Rust release manually is not the end of the world to me.

@noctux
Copy link
Contributor

noctux commented Dec 26, 2019

As (currently) an archlinux user, I"m slightly biased towards "should be buildable with latest rust" because this is what my distribution offers, but this does not necessarily have a say concerning the "least supported" version (which hopefully the CI will check for the folks running more recent rustc versions). As development mostly concerns current git and after forking you made a point of frequent releases (and thus we are at current_stable-2 most of the time) this sounds fine to me :)

Two things that crossed my mind though (I do not know enough about rust to really answer them though):

  • Forward-incompatible changes, similar to the deprecation warnings you mentioned: I for instance quite like to programm in Haskell, however, every major GHC-release breaks half of the libraries I depend on, making my distro-ghc unusable. However, Haskells motto is "Avoid success at all costs" to be able to deliberatly "break" the language if it seems useful, the rust situation might be different. But If this is relevant to rust, how would we deal with this, if there were breaking inconsistencies within the range current_stable-2 .. current_stable (so it"s an either or situation)?
  • Installing newer versions on older systems is often easier than doing it the other way round. I deal with a lot of "research-grade" software, and getting those to compile on arch is often a hassle (stricter compilers, newer python versions, etc.pp.). Doing porting in the other direction (stuff that works on my arch laptop to the debian based infrastructure on debian) is usually easier and in the case of cases means building individual libraries by hand (in constrast to building a substantial part of an older userland, as the old version of libx also requires an older version of liby...), mostly because the more recent versions of a dependency do the right thing during ./configure. I do not think that this is a major problem for newsboat currently, but I would prefer not to be forced into using containers/vms or nix to do the compilation during dev :)

Merry christmas,
Simon

@ghost
Copy link

ghost commented Dec 26, 2019

It should be fine for FreeBSD since we currently do a good job of updating our Rust package every 6 weeks on release day or close to it. And stable-2 or stable-4 should give enough slack if something goes wrong with that.

@rnestler
Copy link
Contributor

I use Arch on my private laptop and latest Ubuntu LTS at work, but I compile newsboat myself anyways 🙂 So I"m fine with the stable - 2 minimum required Rust version.

@Minoru
Copy link
Member Author

Minoru commented Jan 12, 2020

Thank you all for your responses! \o/

To sum up, the vast majority is fine with the proposed changes, but there are some concerns:

@khardix and @tsipinakis (paraphrased by me)

Users won"t be able to compile our newest releases on already-released distributions (Fedora release, Debian stable etc.), because those ship a fixed version of Rust that won"t be updated until a new distro release comes out. These users include @tsipinakis, who does Newsboat development on Debian stable.

I think the only way to really resolve this is to pin our schedule to those distributions, not to time. For example, we could support the last stable release of every major distro according to DistroWatch. Repology doesn"t have info on all of them, though:

  • Linux Mint is missing entirely
  • MX Linux: no info on stable release
  • OpenSUSE: I"m lost between "Leap 42" and "Leap 15" releases

Among the remaining ones, the lowest version is 1.31 (in CentOS 8).

That"s a pretty reasonable policy, but I wonder if we won"t be doing our users a dis-service by adopting it. Newsboat directly depends on 19 crates. I took a look at their policies regarding minimum supported versions:

section_testing is not maintained, doesn"t specify anything, and doesn"t even test anything. Yeah, that"s the crate we write all our tests with. Yeah.

With all transitive dependencies, Newsboat depends on 102 crates. Two questions arise:

  1. How many of those compile with 1.26 by pure chance, like tempfile does?
  2. How soon we"ll see a RUSTSEC announcement against one of those 102 crates, forcing us to upgrade, in turn forcing us to bump our minimum required Rust version? Given the list above, that bump will be somewhere around current_stable-10 or higher—i.e. some stable distributions won"t have a "new enough" compiler anymore.

The list of dependencies will only grow: we"ll need some creates to handle JSON and XML, and eventually HTTP and UI as well.

Seems like Rust ecosystem is not ready (doesn"t want to be ready?) to provide long-term support that we"ll need for "distro-based" policy.

Apropos to this: some users might use our Snap instead. Developers might use Docker containers that we prepare for CI. I understand both of these have drawbacks of their own, but at least that gives people some options to choose from.

Possible delay of getting new versions of newsboat into testing

I think this is the same problem as with stable distro releases, just on a smaller scale: weeks or months instead of years. I guess it will be resolved, one way or another, when we tackle the bigger problem.

@noctux

the "least supported" version (which hopefully the CI will check for the folks running more recent rustc versions)

Definitely. It will be checked just like we"re checking for 1.26 today.

But If this is relevant to rust, how would we deal with this, if there were breaking inconsistencies within the range current_stable-2 .. current_stable (so it"s an either or situation)?

It shouldn"t be relevant: with Rust 1.0 release, Rust team made a promise not to change stabilized APIs in a backward-incompatible ways, except to fix bugs. Over the last five years, the only such change that I can remember is a new borrow checker that rejects some programs that were accepted by previous compilers. But, to my knowledge, the code can be written in a way that is accepted by both versions, so it"s not exactly the kind of problem that you mention.

But if some Rust release X.Y does break backwards compatibility, and we can"t get around it with conditional compilation or any other trick, I think I would:

  1. immediately cut a new release, with lowest supported Rust as per our schedule;
  2. set our lowest supported version to X.Y, and carry on with the development.

I"m not sure what your other point was about. Newsboat will always be compilable with latest Rust version, so you shouldn"t need a container/VM as long as you"re on Arch or have rustup. The lowest version will always be checked by CI, and our experience with 1.26 shows that the problems are infrequent—you can avoid installing 1.26 locally most of the time. You might need that in case of some tricky bugs, but the same is already true for C++ compilers we test on CI—e.g. you might need Clang 5 even though your distro is already on Clang 8 or something.

@khardix
Copy link
Contributor

khardix commented Jan 13, 2020

@Minoru To be completely honest, the Rust situation on Fedora is currently very broken (at least from what I can tell), so I"m currently not able to update newsboat on any stable Fedoras (sigh). So feel free to ignore Fedora for now 😞

@noctux
Copy link
Contributor

noctux commented Jan 13, 2020

Hey,

thanks for the response :)

I"m not sure what your other point was about. Newsboat will always be compilable with latest Rust version, so you shouldn"t need a container/VM as long as you"re on Arch or have rustup. The lowest version will always be checked by CI, and our experience with 1.26 shows that the problems are infrequent—you can avoid installing 1.26 locally most of the time. You might need that in case of some tricky bugs, but the same is already true for C++ compilers we test on CI—e.g. you might need Clang 5 even though your distro is already on Clang 8 or something.

Yeah, that resolves the second point, thanks.

@rnestler
Copy link
Contributor

@Minoru Regarding supporting the Rust version of stable releases of Distros: In my opinion this is not worth the trouble, since packages of the stable releases won"t get updated anyway except for security bugs, so there is no point in keeping support for them in newer versions. Users of these stable distros which build newsboat themselves may as well use rustup.

@Minoru
Copy link
Member Author

Minoru commented Jan 13, 2020

@khardix, okay! Though at this point, I view Fedora and Debian more as examples of "point release" approach (in contrast to "rolling release" approach) and the challenges it creates. It"s fine if any of those distros have some problems that aren"t related to the release model itself.

@rnestler, my understanding is that not all people like Rustup. I"m aware of only two arguments against it: 1) installation via infamous curl | sh, since barely any distro has it packaged; 2) rustup mostly sidesteps the rest of the distro, which adds burden when troubleshooting, and generally makes it harder to install something and keep it updated. I"d avoid making Rustup the only option if I can (at the moment I suspect I won"t be able to avoid that, but the discussion is still open—perhaps my worry will be proven unbased).

@kpcyrd
Copy link
Contributor

kpcyrd commented Jan 14, 2020

I"m one of the debian rust maintainers and we encourage people to use rustup instead. The compiler in debian stable is mostly to build the rust software shipped in debian itself.

Ironically, tracking outdated libraries make it even harder for us to upload it to debian unstable since we would need to port it to recent dependencies downstream. 😿

@Minoru Minoru pinned this issue Jan 23, 2020
@Minoru
Copy link
Member Author

Minoru commented Feb 7, 2020

This issue has been pinned for two weeks now, and it"s been a week since I asked for more input over on the mailing list and on Mastodon. This only yielded a single response, with a suggestion for the Ubuntu .deb to install our Snap.

I guess it"s time to make a decision, then.

My original motivation for this was to avoid stagnation, but after looking at our dependencies in #709 (comment), the overriding concern became to avoid depending on unmaintained, possibly vulnerable versions of crates. BTW, this really made me understand why some people still consider Rust an immature language; with just a few months of support, it kinda is. I still believe that Rust"s safety guarantees outweigh this, and the situation is bound to improve as libraries mature; so I don"t plan to switch away from Rust anytime soon.

Anyway. I"m going through with my original proposal from #709 (comment). This means I don"t really resolve problems with point-release distros (outlined in #709 (comment)), but I plan to allay them for users through better documentation: our README will explain how to:

  1. install from distro"s repository. This will be a simple mention, not specific command-line invocation—as I said numerous times before, I don"t want to maintain a growing list of commands for every distro under the Sun
  2. install from Snap. Just a link to Snap site on how to install the Snap itself, plus a command to run to install Newsboat through Snap
  3. build from source using Docker. That"ll be a pointer our existing Docker guide, perhaps with some edits to make it more accessible to non-programmers
  4. build from source in a chroot. Just a mention of such a possibility, with an invitation to research if user"s distro provides any helper programs like debootstrap in Debian
  5. build from source. Basically a copy of the existing "Dependencies" and "Installation" sections, plus a mention of rustup

This still leaves the possibility of delays in Debian testing (our newest release might be stuck in queue waiting for new enough Rust to appear in testing), but that"s a risk we"ll have to take. I"m pretty sure that this will become less and less likely as Debian polishes its processes.

I"ll make a PR with these changes. If you have anything to add/tweak, or just general thoughts, please share here.

And thanks once again for all the responses! It"d be hard to think this through without your perspectives.

@pickfire
Copy link
Contributor

It shouldn"t be relevant: with Rust 1.0 release, Rust team made a promise not to change stabilized APIs in a backward-incompatible ways, except to fix bugs. Over the last five years, the only such change that I can remember is a new borrow checker that rejects some programs that were accepted by previous compilers. But, to my knowledge, the code can be written in a way that is accepted by both versions, so it"s not exactly the kind of problem that you mention.

But if some Rust release X.Y does break backwards compatibility, and we can"t get around it with conditional compilation or any other trick, I think I would:

  1. immediately cut a new release, with lowest supported Rust as per our schedule;
  2. set our lowest supported version to X.Y, and carry on with the development.

Regarding that, I think having #[cfg(version(..))] which is currently still in development may help, as in to prevent breaking with conditional compilation based on rust version, we could then fuse out the old parts as our minimum supported rust version changed. rust-lang/rfcs#2523

@Minoru
Copy link
Member Author

Minoru commented Feb 23, 2020

@pickfire Thanks for that, but I noted explicitly that I"m talking of a case when conditional compilation can"t help us.

@Minoru Minoru unpinned this issue Feb 25, 2020
Minoru added a commit that referenced this issue Feb 25, 2020
@Minoru
Copy link
Member Author

Minoru commented Feb 26, 2020

I submitted #788 that adds all the docs I promised. Please take a look if you"re interested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meta This discussion relates to how Newsboat is managed and run as a project
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants