Lines Matching defs:tokens
47 /* The regexp is parsed into an array of tokens in postfix form. Some tokens
192 the constraints corresponding to the special tokens previously defined. */
271 charclass *charclasses; /* Array of character sets for CSET tokens. */
276 token *tokens; /* Postfix parse array. */
277 int tindex; /* Index for adding new tokens. */
278 int talloc; /* Number of tokens currently allocated. */
290 if tokens[i] < NOTCHAR
291 bit 1 : tokens[i] is a singlebyte character, or the last-byte of
293 bit 0 : tokens[i] is a singlebyte character, or the 1st-byte of
295 if tokens[i] = MBCSET
299 tokens