Repository navigation
fix(semantic): reject a pattern that follows an expr placeholder with a token it would swallow - #10305
Conversation
5f7d17c to
9b8111a
Compare
9b8111a to
46bd0ab
Compare
PR SummaryMedium Risk Overview Pattern well-formedness checking gains Reviewed by Cursor Bugbot for commit 9d3faab. Bugbot is set up for automated code reviews on this repo. Configure here. |
46bd0ab to
929912f
Compare
929912f to
85b8353
Compare
7cc0ce6 to
fd19862
Compare
65c3f43 to
304290b
Compare
orizi
left a comment
There was a problem hiding this comment.
@orizi+AGNT made 6 comments and resolved 6 discussions.
Reviewable status: 0 of 4 files reviewed, all discussions resolved (waiting on TomerStarkware).
crates/cairo-lang-semantic/src/diagnostic.rs line 1258 at r1 (raw file):
Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…
the allowed set is spelled out here and in
EXPR_FOLLOW_SET- two places to keep in sync. make the constpub(crate)and build this from it, e.g.EXPR_FOLLOW_SET.map(|t| format!("{t}")).join(", ").
Done - EXPR_FOLLOW_SET is pub(crate) and the message is built from it.
crates/cairo-lang-semantic/src/expr/test_data/inline_macros line 3576 at r1 (raw file):
Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…
this contradicts the golden right below it: there is no
E2158here, so the rule does match.parse_expron1 2stops at1, the2token then matches the pattern's2, and the call only fails to expand becauseerris set. rejecting$x:expr 2is forward-compat (as in rustc), not "this call breaks today" - say that instead.
Rewrote the comment to the forward-compat rationale - the call matches today and only fails to expand because the rule is marked defective.
crates/cairo-lang-semantic/src/items/macro_declaration.rs line 303 at r1 (raw file):
Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…
Followerholds nothing that isn't already in the node, andSyntaxNodeisCopy. drop the struct and itsimpl, useVec<SyntaxNode<'db>>, and take the text/ptr at the single place they're used:if EXPR_FOLLOW_SET.contains(&follower.get_text_without_trivia(db).long(db).as_str()) { continue; } res = Err(diagnostics.report( follower.stable_ptr(db), SemanticDiagnosticKind::MacroExprPlaceholderFollower { name, follower: follower.get_text_without_trivia(db), }, ));
Dropped Follower; bare SyntaxNodes, text/ptr taken at the report site.
crates/cairo-lang-semantic/src/items/macro_declaration.rs line 344 at r1 (raw file):
Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…
body_first.is_empty()is not "the body can match nothing". a body made only of zero-able repetitions has a non-empty FIRST and still matches nothing, so a+over it is treated as guaranteed non-empty and the scan stops early.($x:expr $($(a)?)+ b)never reportsb. this is a second incompleteness beyond the one the PR description documents - rustc doesn't have it, it carriesmaybe_emptyon the set. return it here too:/// ... and whether `elements` can match no input at all. fn first_followers<'db>( db: &'db dyn Database, elements: &[ast::MacroElement<'db>], outer: &[Follower<'db>], ) -> (Vec<Follower<'db>>, bool) { let mut res = vec![]; for element in elements { match element { ast::MacroElement::Token(token) => { res.push(Follower::new(db, token.as_syntax_node())); return (res, false); } ast::MacroElement::Param(param) => { res.push(Follower::new(db, param.as_syntax_node())); return (res, false); } ast::MacroElement::Subtree(subtree) => { res.push(Follower::new(db, subtree_open_delimiter(db, &subtree.subtree(db)))); return (res, false); } ast::MacroElement::Repetition(repetition) => { let (body_first, body_may_match_nothing) = first_followers(db, &repetition.elements(db).elements_vec(db), &[]); res.extend(body_first); if !body_may_match_nothing && !matches!( repetition.operator(db), ast::MacroRepetitionOperator::ZeroOrOne(_) | ast::MacroRepetitionOperator::ZeroOrMore(_) ) { return (res, false); } } } } res.extend(outer.iter().cloned()); (res, true) }
Fixed as suggested - first_followers returns (Vec, bool) with maybe_empty carried per rustc; a body of zero-able repetitions no longer stops the scan.
crates/cairo-lang-semantic/src/items/macro_declaration.rs line 369 at r1 (raw file):
Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…
third recursive walk over
MacroElementin this file with the same four arms (collect_pattern_placeholder_names,check_pattern_elements, now this). fold it intocheck_pattern_elements- take&[ast::MacroElement<'db>]plusouterthere - so a pattern is walked once and the call site stays a single.and.
Folded - check_pattern_elements takes the slice plus outer and carries the expr-follow check; single walk, single .and at the call site.
crates/cairo-lang-semantic/src/items/macro_declaration.rs line 417 at r1 (raw file):
Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…
nothing covers this arm actually reporting -
gridonly exercises the valid path, and test 3 covers a subtree as a follower, not a violation inside one. dropping this recursive call fails no test. add a negative golden, e.g.macro m { ([$x:expr 2]) => { $x }; }.
Added ([$x:expr 2]) - E2209 from inside the subtree.
304290b to
df4f07e
Compare
df4f07e to
96f2495
Compare
96f2495 to
e1b8458
Compare
817e9e3 to
e2134ff
Compare
8fd1650 to
b657e7f
Compare
b657e7f to
4bc47fa
Compare
4bc47fa to
42e0536
Compare
42e0536 to
e20bda1
Compare
e20bda1 to
7a2e763
Compare
… a token it would swallow An `expr` placeholder captures by parsing the longest expression the call's tokens start with, so whatever the pattern puts after it is only ever reached when the expression grammar cannot extend over it. Only `,`, `;` and `=>` can never continue an expression, so only they may follow one - the same set rustc allows after its own `expr` fragment. Anything else makes the rule match far less than it reads as matching, or nothing at all. Reported as E2209 at the offending follower, as rustc reports it, and folded into the rule's `err` so a diagnosed rule does not still expand. The follow set is computed the way rustc's `check_matcher_core` does, including its one incompleteness: a separator-less repetition does not wrap its body's FIRST set around, so `($($x:expr)*)` stays accepted. The third negative-control golden pins that parity. In-tree violations, grepped over corelib/, tests/bug_samples/ and the semantic golden dirs (and, beyond those, the whole repo): 3, all in the one macro `add_exprs` of corelib/src/test/language_features/macro_test.cairo - `($x:expr 2)`, `($x:expr $y:expr)` and `(abc $x:expr $y:expr)`. Zero in production corelib code, zero in tests/bug_samples/, zero in the goldens. All three are given a legal separator here, keeping every rule and every assertion of `test_add_exprs`; the two illegal shapes are pinned as E2209 goldens instead. A hard error rather than a warning is warranted because user-defined inline macros are still behind `are_user_defined_inline_macros_enabled`, nothing on the stable surface breaks, and a warning could not satisfy the requirement that the rule stop expanding. Note for future changes here: no gate in CI other than the corelib suite itself compiles corelib/src/test/, so `cairo-test -- corelib/` has to be run explicitly when macro declaration semantics change.
7a2e763 to
9d3faab
Compare

An
exprplaceholder captures by parsing the longest expression the call'stokens start with, so whatever the pattern puts after it is only ever reached
when the expression grammar cannot extend over it. Only
,,;and=>cannever continue an expression, so only they may follow one - the same set rustc
allows after its own
exprfragment. Anything else makes the rule match farless than it reads as matching, or nothing at all.
Reported as E2209 at the offending follower, as rustc reports it, and folded
into the rule's
errso a diagnosed rule does not still expand.The follow set is computed the way rustc's
check_matcher_coredoes, includingits one incompleteness: a separator-less repetition does not wrap its body's
FIRST set around, so
($($x:expr)*)stays accepted. The third negative-controlgolden pins that parity.
In-tree violations, grepped over corelib/, tests/bug_samples/ and the semantic
golden dirs (and, beyond those, the whole repo): 3, all in the one macro
add_exprsof corelib/src/test/language_features/macro_test.cairo -($x:expr 2),($x:expr $y:expr)and(abc $x:expr $y:expr). Zero inproduction corelib code, zero in tests/bug_samples/, zero in the goldens. All
three are given a legal separator here, keeping every rule and every assertion
of
test_add_exprs; the two illegal shapes are pinned as E2209 goldens instead.A hard error rather than a warning is warranted because user-defined inline
macros are still behind
are_user_defined_inline_macros_enabled, nothing on thestable surface breaks, and a warning could not satisfy the requirement that the
rule stop expanding.
Note for future changes here: no gate in CI other than the corelib suite itself
compiles corelib/src/test/, so
cairo-test -- corelib/has to be runexplicitly when macro declaration semantics change.