Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Upgrade format output to a type more efficient than String #2156

Open
marcusklaas opened this issue Nov 15, 2017 · 4 comments
Open

RFC: Upgrade format output to a type more efficient than String #2156

marcusklaas opened this issue Nov 15, 2017 · 4 comments

Comments

@marcusklaas
Copy link
Contributor

Rustfmt spends a lot of time concatenating strings. This involves many allocations, memcpys and deallocations. Common operations on types like ropes have much better time complexity than standard strings. Unfortunately, rustfmt is fairly fixed on using these String values to represent formatted code. Ideally, the formatting code should be more flexible as to its output type. That way, it would be very easy to experiment with other types and observe their effects on performance. Below is a rough proposal on how this could be achieved gradually, without a huge overhaul or breakage.

  1. gradually replace the return type of formatting functions from Option<String> to Option<FmtString>, where FmtString is just an alias for String
  2. introduce a custom trait abstracting common string operations like to_owned, push_str and push and implement it for String
  3. replace method calls to String specific functions with calls to methods in the abstract trait in the
    relevant functions
  4. the format! macro is also highly specific to Strings. Replace these invocations by a custom macro call doing string-like concatenation using the new trait

After this, switching to a new output type should be as simple as implementing this custom trait for your favourite string representation and setting the FmtString alias to it.

This will still be a very big undertaking, but I believe that steps 1 through 4 can be done without breaking anything. Some steps can even be done partially. And it should be worth the effort, as there are many performance gains to be had.

Proof of concept

@marcusklaas marcusklaas changed the title Upgrade format output to a type more efficient than String RFC: Upgrade format output to a type more efficient than String Nov 15, 2017
@nrc
Copy link
Member

nrc commented Nov 15, 2017

I think we could get some of the way by just using a more efficient String in the FileMap and continuing using Rust strings in the visitors. That would be much easier to implement and should show the scale of the speedup we can expect

@nrc
Copy link
Member

nrc commented Nov 15, 2017

cc #338

@marcusklaas
Copy link
Contributor Author

Doesn't FileMap already use a string variant that is at least fairly efficient for its task, namely StringBuffer?

@nrc
Copy link
Member

nrc commented Nov 19, 2017

It does, or at least tries, but it might in fact not be very efficient, or we might be able to do better. Not sure if the profiling tells us about that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants