Lines Matching defs:tokens
560 // There is no information about the tokens that can follow the last one
594 // Now compute the visiable tokens that can follow this rule, according to context
781 /// During rule invocation, the parser pushes the set of tokens that can
784 /// enclosing rule. This local follow set only includes tokens
792 /// thing to do is to consume tokens until you see something that
794 /// You don't want the exact set of viable next tokens because the
796 /// rest of the input looking for one of the missing tokens.
808 /// At each rule invocation, the set of tokens that could follow
833 /// all context-sensitive FOLLOW sets--the set of all tokens that
835 /// resync to one of those tokens. Note that FOLLOW(c)='^' and if
882 /// viable tokens that can possibly come next (look ahead depth 1)
890 /// FOLLOW(r) is the set of all tokens that can possibly follow
894 /// than covering superset) set of following tokens.
1173 // You may get this if there are not more tokens and more are needed
1228 // possible tokens at this point, but we did not see any
1233 // What tokens could we have accepted at this point in the
1243 // However many tokens we could have dealt with here, it is usually
1315 // Used to compute the follow set of tokens
1372 // Consume tokens until we have resynced to something in the follows set
1417 /// is in the set of tokens that can follow the ')' token
1547 // Neither deleting nor inserting tokens allows recovery
1615 /// both. No tokens are consumed to recover from insertions. Return
1670 /* First we need to know which of the available tokens are viable
1683 /* We can now see if the current token works with the set of tokens
1713 /// Eat tokens from the input stream until we get one of JUST the right type
1753 // Start eating tokens until we get to the one we want.
1762 /// Eat tokens from the input stream until we find one that
1803 // Start eating tokens until we get to one we want.
1832 toStrings (pANTLR3_BASE_RECOGNIZER recognizer, pANTLR3_HASH_TABLE tokens)
2192 // We don't yet have a token factory for making tokens