Lines Matching defs:tokens
432 /// method should have no observable side-effect on the lexed tokens.
489 /// for each macro argument, the list of tokens that were provided to the
562 // If this macro expands to no tokens, don't bother to push it onto the
596 // Update the tokens location to include both its expansion and physical
657 /// vector of tokens in NewTokens. The new number of arguments will be placed
727 // Copy over argument tokens
769 // Read arguments as unexpanded tokens. This avoids issues, e.g., where
774 // ArgTokens - Build up a list of tokens that make up each argument. Each
799 // Read arguments as unexpanded tokens. This avoids issues, e.g., where
1032 /// Keeps macro expanded tokens for TokenLexers.
1034 /// Works like a stack; a TokenLexer adds the macro expanded tokens that is
1035 /// going to lex in the cache and when it finishes the tokens are removed
1038 ArrayRef<Token> tokens) {
1040 if (tokens.empty())
1044 bool cacheNeedsToGrow = tokens.size() >
1046 MacroExpandedTokens.append(tokens.begin(), tokens.end());
1067 // Pop the cached macro expanded tokens from the end.
1074 /// the identifier tokens inserted.