Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking Issue for macro_metavar_expr_concat #124225

Open
2 of 9 tasks
c410-f3r opened this issue Apr 21, 2024 · 13 comments
Open
2 of 9 tasks

Tracking Issue for macro_metavar_expr_concat #124225

c410-f3r opened this issue Apr 21, 2024 · 13 comments
Labels
B-experimental Blocker: In-tree experiment; RFC pending or unneeded. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. F-macro_metavar_expr_concat `#![feature(macro_metavar_expr_concat)]` S-tracking-unimplemented Status: The feature has not been implemented. T-lang Relevant to the language team, which will review and decide on the PR/issue.

Comments

@c410-f3r
Copy link
Contributor

c410-f3r commented Apr 21, 2024

This is a tracking issue for the experimental feature macro metavariable expression concat (RFC pending).
The feature gate for the issue is #![feature(macro_metavar_expr_concat)].

Provides a modern alternative to the concat_idents! macro.

#![feature(macro_metavar_expr_concat)]

macro_rules! many_idents {
    ($a:ident, $c:ident) => {
        const ${concat($a, B, $c, D)}: i32 = 1;
    };
}

fn main() {
    many_idents!(A, C);
    assert_eq!(ABCD, 1);
}

About experimental features

An experimental feature is one that has not yet had an RFC. The idea is to allow implementation work to proceed to better inform an upcoming RFC. Experimental features cannot be stabilized without first having an RFC. The existence of an experimental feature does not indicate lang team consensus that the feature is desirable, only that there is a problem that is worthy of being solved and that the idea has enough merit to consider exploring. See the lang team process page for more details.

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps / History

Unresolved Questions

References

@c410-f3r c410-f3r added the C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. label Apr 21, 2024
@fmease fmease added T-lang Relevant to the language team, which will review and decide on the PR/issue. B-experimental Blocker: In-tree experiment; RFC pending or unneeded. F-macro_metavar_expr_concat `#![feature(macro_metavar_expr_concat)]` S-tracking-unimplemented Status: The feature has not been implemented. labels Apr 21, 2024
@fmease
Copy link
Member

fmease commented Apr 21, 2024

This still needs a T-lang shepherd, right?

@c410-f3r
Copy link
Contributor Author

This still needs a T-lang shepherd, right?

Unfortunately I am not familiar or aware of this procedure.

@fmease
Copy link
Member

fmease commented Apr 21, 2024

All language features have to go through the RFC process eventually. However, the implementation can be done before an RFC has been written by following T-lang's experimental feature process which requires a T-lang member to support — shepherd — the venture. Only “small” language (not compiler) “fixes” don't need to follow the RFC process, they just need a T-lang FCP.

@fmease
Copy link
Member

fmease commented Apr 21, 2024

T-lang, this feature is seeking a shepherd, therefore I'm nominating this for you.

An implementation is already under way (#118958). This feature adds a new kind of macro metavariable expression (the latter (overarching) feature has been approved by the merged RFC 3086, see also its tracking issue and stabilization PR (proposed FCP, disposition: merge)). The feature represents a more potent alternative / potential successor to the unstable built-in macro concat_idents (the latter has stalled, blocked on design concerns, see its tracking issue).

Please consult the tracking issue description for more context.

@fmease fmease added the I-lang-nominated The issue / PR has been nominated for discussion during a lang team meeting. label Apr 21, 2024
@joshtriplett
Copy link
Member

Happy to second this with my T-lang hat on.

@c410-f3r Good luck, and please reach out to #t-lang on Zulip if you have any questions. Feel free to @ me there if needed.

@joshtriplett joshtriplett removed the I-lang-nominated The issue / PR has been nominated for discussion during a lang team meeting. label May 15, 2024
@c410-f3r
Copy link
Contributor Author

Thank you @joshtriplett

bors added a commit to rust-lang-ci/rust that referenced this issue Jun 14, 2024
Add a new concat metavar expr

Revival of rust-lang#111930

Giving it another try now that rust-lang#117050 was merged.

With the new rules, meta-variable expressions must be referenced with a dollar sign (`$`) and this can cause misunderstands with `$concat`.

```rust
macro_rules! foo {
    ( $bar:ident ) => {
        const ${concat(VAR, bar)}: i32 = 1;
    };
}

// Will produce `VARbar` instead of `VAR_123`
foo!(_123);
```

In other words, forgetting the dollar symbol can produce undesired outputs.

cc rust-lang#29599
cc rust-lang#124225
@UserIsntAvailable
Copy link
Contributor

Sorry if this is the wrong place to discuss this, but should ${concat} provide its own case-conversion mechanism (just like paste! does), or should we instead provide a standalone ${case} metavar expr to handle this?

If we where to follow paste! syntax, then it would look like:

macro_rules! many_idents {
    ($a:ident, $c:ident) => {
        const ${concat($a:upper, B, $c:lower, D)}: i32 = 1;
    };
}

fn main() {
    many_idents!(a, C);
    assert_eq!(ABcD, 1);
}

Or with a future ${case} metavar expr:

macro_rules! many_idents {
    ($a:ident, $c:ident) => {
        const ${concat(${case($a, upper)}, B, ${case($c, lower)}, D)}: i32 = 1;
    };
}

fn main() {
    many_idents!(a, C);
    assert_eq!(ABcD, 1);
}

Which I think is way too verbose, but maybe I'm just too used to paste! simplicity. The advantage of ${case} over ${concat} would be that, currently, ${concat} requires 2 operands to be able to be used (this could technically be changed), so ${case} would allow:

use std::ops::Add;

pub struct Wrapper<T>(T);

macro_rules! impl_op {
    ($Op:ident => $Ty:ty) => {
        impl $Op<$Ty> for Wrapper<$Ty> {
            type Output = Self;

            // alternative syntax; `:` instead of `,`.
            fn ${case($Op:snake)}(self, rhs: $Ty) -> Self::Output {
                Self(self.0.${case($Op:snake)}(rhs))
            }
        }
    };
}

impl_op!(Add => u8);

With that said, maybe there is a world where both approaches are accepted, but I think there is probably less motivation to do that?

cc @c410-f3r @joshtriplett for opinions about this.

@c410-f3r
Copy link
Contributor Author

Well, I personally don't mind discussing potential designs in a tracking issue.

${case}

The introduction of ${case} as a new metavariable expression that can be used with or without ${concat} looks worth pursuing and semantically correct but IFAICT, inputs currently only accept ad-hoc parameters. More specifically, ${concat} only accepts literal identifiers and variables.

// Same problem with multiples `${concat}`s

macro_rules! foo {
    ($a:ident, $b:ident, $c:ident) => {
        let ${concat(a, ${concat(b, c)})}: () = (); // ERROR !
    }
}

Perhaps the evaluation of nested metaravariables expressions will become a thing someday?

$ident:modifier

Looks feasible in regards to a possible implementation but semantically awkward IMO. I am not aware of something similar in the macro system.

If acceptable, then it should probably be restricted to a subset of operations.

macro_rules! foo {
    ($a:ident) => {
        // Is it OK?
        let $a:upper = ();
    }
}

These are just my opinions. A theoretical stabilization of ${concat} with or without case modifiers is fine to me.

@UserIsntAvailable
Copy link
Contributor

UserIsntAvailable commented Jun 16, 2024

The introduction of ${case} as a new metavariable expression that can be used with or without ${concat} looks worth pursuing and semantically correct but IFAICT, inputs currently only accept ad-hoc parameters. More specifically, ${concat} only accepts literal identifiers and variables.

// Same problem with multiples `${concat}`s

macro_rules! foo {
    ($a:ident, $b:ident, $c:ident) => {
        let ${concat(a, ${concat(b, c)})}: () = (); // ERROR !
    }
}

Perhaps the evaluation of nested metaravariables expressions will become a thing someday?

Oh I wasn't aware of that limitation. I imagine that something like that would be needed down the line. I don't much about the internals of the macro codebase, so I wouldn't know if this would be hard to implement.

As a side note, I wonder if something like ${concat($a, concat($b, $c)} would be able to be made non ambiguous; I would imagine this kind of analysis would be hard to pull off, since you can't no longer look for ${} to know that the next thing is a metavar expr identifier.

$ident:modifier

Looks feasible in regards to a possible implementation but semantically awkward IMO. I am not aware of something similar in the macro system.

If acceptable, then it should probably be restricted to a subset of operations.

macro_rules! foo {
    ($a:ident) => {
        // Is it OK?
        let $a:upper = ();
    }
}

Do note that if $ident:modifier is accepted as the final syntax, it would only work inside of ${concat} or ${case}, which would remove the ambiguity for let $a:upper = ();; identifiers can't use the : symbol, so let ${case($a:upper)} = (); is non ambiguous.

With that said, you made me realize that with this change some metavar exprs would have "special syntax" attached to them, which might not be the desired design, specially because most users would think about metavar exprs as functions rather than "fancier macros" (although accepting this change would contradict this).

Another option would be to allow a standalone syntax ${$ident:modifier} (without writing case), but I don't know if case-conversion is important enough to have its own syntax (I also don't know what other modifiers you could really add to these).

These are just my opinions. A theoretical stabilization of ${concat} with or without case modifiers is fine to me.

This is something that can go under "Future possibilities" on the final RFC. I don't particularly think that stabilization should be blocked on this, since we can always add it later. :)

github-actions bot pushed a commit to rust-lang/miri that referenced this issue Jun 17, 2024
Add a new concat metavar expr

Revival of #111930

Giving it another try now that #117050 was merged.

With the new rules, meta-variable expressions must be referenced with a dollar sign (`$`) and this can cause misunderstands with `$concat`.

```rust
macro_rules! foo {
    ( $bar:ident ) => {
        const ${concat(VAR, bar)}: i32 = 1;
    };
}

// Will produce `VARbar` instead of `VAR_123`
foo!(_123);
```

In other words, forgetting the dollar symbol can produce undesired outputs.

cc #29599
cc rust-lang/rust#124225
bors added a commit to rust-lang/rust-analyzer that referenced this issue Jun 20, 2024
Add a new concat metavar expr

Revival of #111930

Giving it another try now that #117050 was merged.

With the new rules, meta-variable expressions must be referenced with a dollar sign (`$`) and this can cause misunderstands with `$concat`.

```rust
macro_rules! foo {
    ( $bar:ident ) => {
        const ${concat(VAR, bar)}: i32 = 1;
    };
}

// Will produce `VARbar` instead of `VAR_123`
foo!(_123);
```

In other words, forgetting the dollar symbol can produce undesired outputs.

cc #29599
cc rust-lang/rust#124225
@crumblingstatue
Copy link
Contributor

crumblingstatue commented Jun 20, 2024

Is it a known issue that this doesn't seem to work with identifiers repeating at a depth?

#![feature(macro_metavar_expr_concat)]

macro_rules! many_idents {
    ($a:ident, $c:ident) => {
        const ${concat($a, B, $c, D)}: i32 = 1;
    };
}

// Paste implementation included for reference
macro_rules! many_idents_paste {
    ($a:ident, $c:ident) => {
        paste::paste! {
            const [<$a B $c D>]: i32 = 2;
        }
    };
}

macro_rules! many_idents_multi_metavar {
    ($($a:ident, $c:ident;)*) => {
        $(
            const ${concat($a, B, $c, D)}: i32 = 3;
        )*
    };
}

// Paste implementation included for reference
macro_rules! many_idents_multi_paste {
    ($($a:ident, $c:ident;)*) => {
        $(
            paste::paste! {
                const [<$a B $c D>]: i32 = 3;
            }
        )*
    };
}

fn main() {
    many_idents!(A, C);
    assert_eq!(ABCD, 1);
    many_idents_paste!(F, G);
    assert_eq!(FBGD, 2);
    many_idents_multi_paste! {
        H, I;
        J, K;
        L, M;
    }
    assert_eq!(HBID, 3);
    assert_eq!(JBKD, 3);
    assert_eq!(LBMD, 3);
    many_idents_multi_metavar! {
        N, O;
        P, Q;
        R, S;
    }
}
error: attempted to repeat an expression containing no syntax variables matched as repeating at this depth
  --> src/main.rs:19:10
   |
19 |           $(
   |  __________^
20 | |             const ${concat($a, B, $c, D)}: i32 = 3;
21 | |         )*
   | |_________^

The paste version works without errors.

I can go a bit further with ignore

macro_rules! many_idents_multi_metavar {
    ($($a:ident, $c:ident;)*) => {
        $(
            ${ignore($a)}
            ${ignore($c)}
            const ${concat($a, B, $c, D)}: i32 = 3;
        )*
    };
}

But then I get:

error: `${concat(..)}` currently only accepts identifiers or meta-variables as parameters
  --> src/main.rs:22:29
   |
22 |             const ${concat($a, B, $c, D)}: i32 = 3;
   |     

This seems like a major limitation.

@c410-f3r
Copy link
Contributor Author

Looks like a bug to me. I will try to investigate it in the following weeks

flip1995 pushed a commit to flip1995/rust-clippy that referenced this issue Jun 28, 2024
Add a new concat metavar expr

Revival of #111930

Giving it another try now that #117050 was merged.

With the new rules, meta-variable expressions must be referenced with a dollar sign (`$`) and this can cause misunderstands with `$concat`.

```rust
macro_rules! foo {
    ( $bar:ident ) => {
        const ${concat(VAR, bar)}: i32 = 1;
    };
}

// Will produce `VARbar` instead of `VAR_123`
foo!(_123);
```

In other words, forgetting the dollar symbol can produce undesired outputs.

cc #29599
cc rust-lang/rust#124225
jieyouxu added a commit to jieyouxu/rust that referenced this issue Jul 8, 2024
[`macro_metavar_expr_concat`] Add support for literals

Adds support for things like `${concat($variable, 123)}` or `${concat("hello", "_world")}` .

cc rust-lang#124225
jieyouxu added a commit to jieyouxu/rust that referenced this issue Jul 8, 2024
[`macro_metavar_expr_concat`] Add support for literals

Adds support for things like `${concat($variable, 123)}` or `${concat("hello", "_world")}` .

cc rust-lang#124225
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Jul 8, 2024
Rollup merge of rust-lang#126841 - c410-f3r:concat-again, r=petrochenkov

[`macro_metavar_expr_concat`] Add support for literals

Adds support for things like `${concat($variable, 123)}` or `${concat("hello", "_world")}` .

cc rust-lang#124225
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Jul 17, 2024
[`macro_metavar_expr_concat`] Add support for literals

Adds support for literals in macro parameters.

```rust
macro_rules! with_literal {
    ($literal:literal) => {
        const ${concat(FOO, $literal)}: i32 = 1;
    }
}

fn main() {
    with_literal!("_BAR");
    assert_eq!(FOO_BAR, 1);
}
```

cc rust-lang#124225

r? `@petrochenkov`
tgross35 added a commit to tgross35/rust that referenced this issue Jul 18, 2024
[`macro_metavar_expr_concat`] Add support for literals

Adds support for literals in macro parameters.

```rust
macro_rules! with_literal {
    ($literal:literal) => {
        const ${concat(FOO, $literal)}: i32 = 1;
    }
}

fn main() {
    with_literal!("_BAR");
    assert_eq!(FOO_BAR, 1);
}
```

cc rust-lang#124225

r? ``@petrochenkov``
@tgross35
Copy link
Contributor

It would be nice if there was some way to allow turning ${concat(c, $litstr)} into c"some string" for constructing C string literals when a Rust string is provided. That is, a replacement for cstr!. But I can't think of a good syntax here.

#127542 adds support for literals, but the above would get turned into (invalid) identifier csome string.

@tgross35
Copy link
Contributor

Regarding syntax, I would prefer a space-separated form like ${concat(get_ $item _mut)}, rather than the current comma-separated ${concat(get_, $item, _mut)}. That just makes it easier to read imo, and matches paste.

rust-timer added a commit to rust-lang-ci/rust that referenced this issue Jul 18, 2024
Rollup merge of rust-lang#127542 - c410-f3r:concat-again, r=petrochenkov

[`macro_metavar_expr_concat`] Add support for literals

Adds support for literals in macro parameters.

```rust
macro_rules! with_literal {
    ($literal:literal) => {
        const ${concat(FOO, $literal)}: i32 = 1;
    }
}

fn main() {
    with_literal!("_BAR");
    assert_eq!(FOO_BAR, 1);
}
```

cc rust-lang#124225

r? ``@petrochenkov``
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Aug 2, 2024
…gjubilee

[`macro_metavar_expr_concat`] Dogfooding

cc rust-lang#124225

Starts inner usage to test the robustness of the implementation.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Aug 2, 2024
Rollup merge of rust-lang#128491 - c410-f3r:unlock-rfc-2011, r=workingjubilee

[`macro_metavar_expr_concat`] Dogfooding

cc rust-lang#124225

Starts inner usage to test the robustness of the implementation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
B-experimental Blocker: In-tree experiment; RFC pending or unneeded. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. F-macro_metavar_expr_concat `#![feature(macro_metavar_expr_concat)]` S-tracking-unimplemented Status: The feature has not been implemented. T-lang Relevant to the language team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

6 participants