Lines Matching defs:tokens
67 /* The regexp is parsed into an array of tokens in postfix form. Some tokens
211 the constraints corresponding to the special tokens previously defined. */
290 charclass *charclasses; /* Array of character sets for CSET tokens. */
295 token *tokens; /* Postfix parse array. */
296 int tindex; /* Index for adding new tokens. */
297 int talloc; /* Number of tokens currently allocated. */
309 if tokens[i] < NOTCHAR
310 bit 1 : tokens[i] is a singlebyte character, or the last-byte of
312 bit 0 : tokens[i] is a singlebyte character, or the 1st-byte of
314 if tokens[i] = MBCSET
318 tokens