Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Less TokenTree cloning #114115

Merged
merged 8 commits into from
Jul 28, 2023
Merged

Less TokenTree cloning #114115

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift click to select a range
53379a7
Simplify the `ttdelim_span` test.
nnethercote Jul 26, 2023
a9d8459
Replace `into_trees` with `trees` in a test.
nnethercote Jul 27, 2023
103bd4a
Use `TokenStream::trees` instead of `into_trees` for attributes.
nnethercote Jul 27, 2023
55a7324
Make `TokenTree::uninterpolate` take `&self` and return a `Cow`.
nnethercote Jul 27, 2023
853f453
Avoid some token cloning in `filter_tokens_from_list`.
nnethercote Jul 27, 2023
d2f7f67
Avoid some token tree cloning in decl macro parsing.
nnethercote Jul 27, 2023
ee6ed60
Avoid `into_trees` usage in rustfmt.
nnethercote Jul 27, 2023
4ebf2be
Remove `Iterator` impl for `TokenTreeCursor`.
nnethercote Jul 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Make TokenTree::uninterpolate take &self and return a Cow. 
Making it similar to `Token::uninterpolate`. This avoids some more token
tree cloning.
  • Loading branch information
@nnethercote
nnethercote committed Jul 27, 2023
commit 55a732461dcd104311b24cc048dcef2ed0c9d57c
14 changes: 7 additions & 7 deletions compiler/rustc_ast/src/attr/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,12 290,12 @@ impl MetaItem {
I: Iterator<Item = &'a TokenTree>,
{
// FIXME: Share code with `parse_path`.
let path = match tokens.next().map(|tt| TokenTree::uninterpolate(tt.clone())) {
Some(TokenTree::Token(
Token { kind: kind @ (token::Ident(..) | token::ModSep), span },
let path = match tokens.next().map(|tt| TokenTree::uninterpolate(tt)).as_deref() {
Some(&TokenTree::Token(
Token { kind: ref kind @ (token::Ident(..) | token::ModSep), span },
_,
)) => 'arm: {
let mut segments = if let token::Ident(name, _) = kind {
let mut segments = if let &token::Ident(name, _) = kind {
if let Some(TokenTree::Token(Token { kind: token::ModSep, .. }, _)) =
tokens.peek()
{
Expand All @@ -308,8 308,8 @@ impl MetaItem {
thin_vec![PathSegment::path_root(span)]
};
loop {
if let Some(TokenTree::Token(Token { kind: token::Ident(name, _), span }, _)) =
tokens.next().map(|tt| TokenTree::uninterpolate(tt.clone()))
if let Some(&TokenTree::Token(Token { kind: token::Ident(name, _), span }, _)) =
tokens.next().map(|tt| TokenTree::uninterpolate(tt)).as_deref()
{
segments.push(PathSegment::from_ident(Ident::new(name, span)));
} else {
Expand All @@ -326,7 326,7 @@ impl MetaItem {
let span = span.with_hi(segments.last().unwrap().ident.span.hi());
Path { span, segments, tokens: None }
}
Some(TokenTree::Token(Token { kind: token::Interpolated(nt), .. }, _)) => match &*nt {
Some(TokenTree::Token(Token { kind: token::Interpolated(nt), .. }, _)) => match &**nt {
token::Nonterminal::NtMeta(item) => return item.meta(item.path.span),
token::Nonterminal::NtPath(path) => (**path).clone(),
_ => return None,
Expand Down
12 changes: 7 additions & 5 deletions compiler/rustc_ast/src/tokenstream.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 25,7 @@ use rustc_serialize::{Decodable, Decoder, Encodable, Encoder};
use rustc_span::{Span, DUMMY_SP};
use smallvec::{smallvec, SmallVec};

use std::borrow::Cow;
use std::{fmt, iter, mem};

/// When the main Rust parser encounters a syntax-extension invocation, it
Expand Down Expand Up @@ -98,12 99,13 @@ impl TokenTree {
TokenTree::Token(Token::new(kind, span), Spacing::Joint)
}

pub fn uninterpolate(self) -> TokenTree {
pub fn uninterpolate(&self) -> Cow<'_, TokenTree> {
match self {
TokenTree::Token(token, spacing) => {
TokenTree::Token(token.uninterpolate().into_owned(), spacing)
}
tt => tt,
TokenTree::Token(token, spacing) => match token.uninterpolate() {
Cow::Owned(token) => Cow::Owned(TokenTree::Token(token, *spacing)),
Cow::Borrowed(_) => Cow::Borrowed(self),
},
_ => Cow::Borrowed(self),
}
}
}
Expand Down