Edition 2021 reserved prefixed tokens are accepted

[ehuss]

The 2021 edition reserved several forms of prefixed literals that proc-macro2 is currently accepting. I'm not sure what the edition policy of proc-macro2 is, but I would somewhat expect these to be rejected.

use std::str::FromStr;

fn main() {
    for input in [
        "prefix#identifier",
        "prefix\"string\"",
        "prefix'c'",
        "prefix#123",
        "'prefix#",
    ] {
        println!("{:?}", proc_macro2::TokenStream::from_str(input));
    }
}

displays the following:

Ok(TokenStream [Ident { sym: prefix }, Punct { char: '#', spacing: Alone }, Ident { sym: identifier }])
Ok(TokenStream [Ident { sym: prefix }, Literal { lit: "string" }])
Ok(TokenStream [Ident { sym: prefix }, Literal { lit: 'c' }])
Ok(TokenStream [Ident { sym: prefix }, Punct { char: '#', spacing: Alone }, Literal { lit: 123 }])
Err(LexError { span: Span })

The edition chapter on the reservation: https://doc.rust-lang.org/nightly/edition-guide/rust-2021/reserved-syntax.html. This was from RFC 3101. The exact grammar from the reference is:

RESERVED_TOKEN_DOUBLE_QUOTE ->
    IDENTIFIER_OR_KEYWORD _except `b` or `c` or `r` or `br` or `cr`_ `"`

RESERVED_TOKEN_SINGLE_QUOTE ->
    IDENTIFIER_OR_KEYWORD _except `b`_ `'`

RESERVED_TOKEN_POUND ->
    IDENTIFIER_OR_KEYWORD _except `r` or `br` or `cr`_ `#`

RESERVED_TOKEN_LIFETIME ->
    `'` IDENTIFIER_OR_KEYWORD _except `r`_ `#`

(Note, proc-macro2 is handling the lifetime tokens correctly.)

Using proc-macro2 1.0.106