Lines Matching refs:tokens
439 /// method should have no observable side-effect on the lexed tokens.
498 /// for each macro argument, the list of tokens that were provided to the
571 // If this macro expands to no tokens, don't bother to push it onto the
605 // Update the tokens location to include both its expansion and physical
666 /// vector of tokens in NewTokens. The new number of arguments will be placed
736 // Copy over argument tokens
778 // Read arguments as unexpanded tokens. This avoids issues, e.g., where
783 // ArgTokens - Build up a list of tokens that make up each argument. Each
808 // Read arguments as unexpanded tokens. This avoids issues, e.g., where
1045 /// Keeps macro expanded tokens for TokenLexers.
1047 /// Works like a stack; a TokenLexer adds the macro expanded tokens that is
1048 /// going to lex in the cache and when it finishes the tokens are removed
1051 ArrayRef<Token> tokens) {
1053 if (tokens.empty())
1057 bool cacheNeedsToGrow = tokens.size() >
1059 MacroExpandedTokens.append(tokens.begin(), tokens.end());
1080 // Pop the cached macro expanded tokens from the end.
1087 /// the identifier tokens inserted.