flex/MISC/flex.man

FLEX(1)                  USER COMMANDS                    FLEX(1)


NAME
     flex - fast lexical analyzer generator

SYNOPSIS
     flex [-bcdfhilnpstvwBFILTV78+? -C[aefFmr] -ooutput  -Pprefix
     -Sskeleton] [--help --version] [filename ...]

OVERVIEW
     This manual describes flex, a tool for  generating  programs
     that  perform pattern-matching on text.  The manual includes
     both tutorial and reference sections:

         Description
             a brief overview of the tool

         Some Simple Examples

         Format Of The Input File

         Patterns
             the extended regular expressions used by flex

         How The Input Is Matched
             the rules for determining what has been matched

         Actions
             how to specify what to do when a pattern is matched

         The Generated Scanner
             details regarding the scanner that flex produces;
             how to control the input source

         Start Conditions
             introducing context into your scanners, and
             managing "mini-scanners"

         Multiple Input Buffers
             how to manipulate multiple input sources; how to
             scan from strings instead of files

         End-of-file Rules
             special rules for matching the end of the input

         Miscellaneous Macros
             a summary of macros available to the actions

         Values Available To The User
             a summary of values available to the actions

         Interfacing With Yacc
             connecting flex scanners together with yacc parsers


Version 2.5          Last change: April 1995                    1


FLEX(1)                  USER COMMANDS                    FLEX(1)


         Options
             flex command-line options, and the "%option"
             directive

         Performance Considerations
             how to make your scanner go as fast as possible

         Generating C++ Scanners
             the (experimental) facility for generating C++
             scanner classes

         Incompatibilities With Lex And POSIX
             how flex differs from AT&T lex and the POSIX lex
             standard

         Diagnostics
             those error messages produced by flex (or scanners
             it generates) whose meanings might not be apparent

         Files
             files used by flex

         Deficiencies / Bugs
             known problems with flex

         See Also
             other documentation, related tools

         Author
             includes contact information


DESCRIPTION
     flex is a  tool  for  generating  scanners:  programs  which
     recognized  lexical  patterns in text.  flex reads the given
     input files, or its standard input  if  no  file  names  are
     given,  for  a  description  of  a scanner to generate.  The
     description is in the form of pairs of  regular  expressions
     and  C  code,  called  rules.  flex  generates as output a C
     source file, lex.yy.c, which defines a routine yylex(). This
     file is compiled and linked with the -lfl library to produce
     an executable.  When the executable is run, it analyzes  its
     input  for occurrences of the regular expressions.  Whenever
     it finds one, it executes the corresponding C code.

SOME SIMPLE EXAMPLES
     First some simple examples to get the flavor of how one uses
     flex.  The  following  flex  input specifies a scanner which
     whenever it encounters the string "username" will replace it
     with the user's login name:

         %%


Version 2.5          Last change: April 1995                    2


FLEX(1)                  USER COMMANDS                    FLEX(1)


         username    printf( "%s", getlogin() );

     By default, any text not matched by a flex scanner is copied
     to  the output, so the net effect of this scanner is to copy
     its input file to its output with each occurrence of  "user-
     name"  expanded.   In  this  input,  there is just one rule.
     "username" is the pattern and the "printf"  is  the  action.
     The "%%" marks the beginning of the rules.

     Here's another simple example:

                 int num_lines = 0, num_chars = 0;

         %%
         \n      ++num_lines; ++num_chars;
         .       ++num_chars;

         %%
         main()
                 {
                 yylex();
                 printf( "# of lines = %d, # of chars = %d\n",
                         num_lines, num_chars );
                 }

     This scanner counts the number of characters and the  number
     of  lines in its input (it produces no output other than the
     final report on the counts).  The first  line  declares  two
     globals,  "num_lines"  and "num_chars", which are accessible
     both inside yylex() and in the main() routine declared after
     the  second  "%%".  There are two rules, one which matches a
     newline ("\n") and increments both the line  count  and  the
     character  count,  and one which matches any character other
     than a newline (indicated by the "." regular expression).

     A somewhat more complicated example:

         /* scanner for a toy Pascal-like language */

         %{
         /* need this for the call to atof() below */
         #include <math.h>
         %}

         DIGIT    [0-9]
         ID       [a-z][a-z0-9]*

         %%

         {DIGIT}+    {
                     printf( "An integer: %s (%d)\n", yytext,
                             atoi( yytext ) );


Version 2.5          Last change: April 1995                    3


FLEX(1)                  USER COMMANDS                    FLEX(1)


                     }

         {DIGIT}+"."{DIGIT}*        {
                     printf( "A float: %s (%g)\n", yytext,
                             atof( yytext ) );
                     }

         if|then|begin|end|procedure|function        {
                     printf( "A keyword: %s\n", yytext );
                     }

         {ID}        printf( "An identifier: %s\n", yytext );

         "+"|"-"|"*"|"/"   printf( "An operator: %s\n", yytext );

         "{"[^}\n]*"}"     /* eat up one-line comments */

         [ \t\n]+          /* eat up whitespace */

         .           printf( "Unrecognized character: %s\n", yytext );

         %%

         main( argc, argv )
         int argc;
         char **argv;
             {
             ++argv, --argc;  /* skip over program name */
             if ( argc > 0 )
                     yyin = fopen( argv[0], "r" );
             else
                     yyin = stdin;

             yylex();
             }

     This is the beginnings of a simple scanner  for  a  language
     like  Pascal.   It  identifies different types of tokens and
     reports on what it has seen.

     The details of this example will be explained in the follow-
     ing sections.

FORMAT OF THE INPUT FILE
     The flex input file consists of three sections, separated by
     a line with just %% in it:

         definitions
         %%
         rules
         %%
         user code


Version 2.5          Last change: April 1995                    4


FLEX(1)                  USER COMMANDS                    FLEX(1)


     The definitions section contains declarations of simple name
     definitions  to  simplify  the  scanner  specification,  and
     declarations of start conditions, which are explained  in  a
     later section.

     Name definitions have the form:

         name definition

     The "name" is a word beginning with a letter  or  an  under-
     score  ('_')  followed by zero or more letters, digits, '_',
     or '-' (dash).  The definition is  taken  to  begin  at  the
     first  non-white-space character following the name and con-
     tinuing to the end of the line.  The definition  can  subse-
     quently  be referred to using "{name}", which will expand to
     "(definition)".  For example,

         DIGIT    [0-9]
         ID       [a-z][a-z0-9]*

     defines "DIGIT" to be a regular expression which  matches  a
     single  digit,  and  "ID"  to  be a regular expression which
     matches a letter followed by zero-or-more letters-or-digits.
     A subsequent reference to

         {DIGIT}+"."{DIGIT}*

     is identical to

         ([0-9])+"."([0-9])*

     and matches one-or-more digits followed by a '.' followed by
     zero-or-more digits.

     The rules section of the flex input  contains  a  series  of
     rules of the form:

         pattern   action

     where the pattern must be unindented  and  the  action  must
     begin on the same line.

     See below for a further description of patterns and actions.

     Finally, the user code section is simply copied to  lex.yy.c
     verbatim.   It  is used for companion routines which call or
     are called by the scanner.  The presence of this section  is
     optional;  if it is missing, the second %% in the input file
     may be skipped, too.

     In the definitions and rules sections, any indented text  or
     text  enclosed in %{ and %} is copied verbatim to the output


Version 2.5          Last change: April 1995                    5


FLEX(1)                  USER COMMANDS                    FLEX(1)


     (with the %{}'s removed).  The %{}'s must appear  unindented
     on lines by themselves.

     In the rules section, any indented  or  %{}  text  appearing
     before the first rule may be used to declare variables which
     are local to the scanning routine and  (after  the  declara-
     tions)  code  which  is to be executed whenever the scanning
     routine is entered.  Other indented or %{} text in the  rule
     section  is  still  copied to the output, but its meaning is
     not well-defined and it may well cause  compile-time  errors
     (this feature is present for POSIX compliance; see below for
     other such features).

     In the definitions section (but not in the  rules  section),
     an  unindented comment (i.e., a line beginning with "/*") is
     also copied verbatim to the output up to the next "*/".

PATTERNS
     The patterns in the input are written using an extended  set
     of regular expressions.  These are:

         x          match the character 'x'
         .          any character (byte) except newline
         [xyz]      a "character class"; in this case, the pattern
                      matches either an 'x', a 'y', or a 'z'
         [abj-oZ]   a "character class" with a range in it; matches
                      an 'a', a 'b', any letter from 'j' through 'o',
                      or a 'Z'
         [^A-Z]     a "negated character class", i.e., any character
                      but those in the class.  In this case, any
                      character EXCEPT an uppercase letter.
         [^A-Z\n]   any character EXCEPT an uppercase letter or
                      a newline
         r*         zero or more r's, where r is any regular expression
         r+         one or more r's
         r?         zero or one r's (that is, "an optional r")
         r{2,5}     anywhere from two to five r's
         r{2,}      two or more r's
         r{4}       exactly 4 r's
         {name}     the expansion of the "name" definition
                    (see above)
         "[xyz]\"foo"
                    the literal string: [xyz]"foo
         \X         if X is an 'a', 'b', 'f', 'n', 'r', 't', or 'v',
                      then the ANSI-C interpretation of \x.
                      Otherwise, a literal 'X' (used to escape
                      operators such as '*')
         \0         a NUL character (ASCII code 0)
         \123       the character with octal value 123
         \x2a       the character with hexadecimal value 2a
         (r)        match an r; parentheses are used to override
                      precedence (see below)


Version 2.5          Last change: April 1995                    6


FLEX(1)                  USER COMMANDS                    FLEX(1)


         rs         the regular expression r followed by the
                      regular expression s; called "concatenation"


         r|s        either an r or an s


         r/s        an r but only if it is followed by an s.  The
                      text matched by s is included when determining
                      whether this rule is the "longest match",
                      but is then returned to the input before
                      the action is executed.  So the action only
                      sees the text matched by r.  This type
                      of pattern is called trailing context".
                      (There are some combinations of r/s that flex
                      cannot match correctly; see notes in the
                      Deficiencies / Bugs section below regarding
                      "dangerous trailing context".)
         ^r         an r, but only at the beginning of a line (i.e.,
                      which just starting to scan, or right after a
                      newline has been scanned).
         r$         an r, but only at the end of a line (i.e., just
                      before a newline).  Equivalent to "r/\n".

                    Note that flex's notion of "newline" is exactly
                    whatever the C compiler used to compile flex
                    interprets '\n' as; in particular, on some DOS
                    systems you must either filter out \r's in the
                    input yourself, or explicitly use r/\r\n for "r$".


         <s>r       an r, but only in start condition s (see
                      below for discussion of start conditions)
         <s1,s2,s3>r
                    same, but in any of start conditions s1,
                      s2, or s3
         <*>r       an r in any start condition, even an exclusive one.


         <<EOF>>    an end-of-file
         <s1,s2><<EOF>>
                    an end-of-file when in start condition s1 or s2

     Note that inside of a character class, all  regular  expres-
     sion  operators  lose  their  special  meaning except escape
     ('\') and the character class operators, '-', ']',  and,  at
     the beginning of the class, '^'.

     The regular expressions listed above are  grouped  according
     to  precedence, from highest precedence at the top to lowest
     at the bottom.   Those  grouped  together  have  equal  pre-
     cedence.  For example,


Version 2.5          Last change: April 1995                    7


FLEX(1)                  USER COMMANDS                    FLEX(1)


         foo|bar*

     is the same as

         (foo)|(ba(r*))

     since the '*' operator has higher precedence than concatena-
     tion, and concatenation higher than alternation ('|').  This
     pattern therefore matches either the  string  "foo"  or  the
     string "ba" followed by zero-or-more r's.  To match "foo" or
     zero-or-more "bar"'s, use:

         foo|(bar)*

     and to match zero-or-more "foo"'s-or-"bar"'s:

         (foo|bar)*


     In addition to characters and ranges of characters,  charac-
     ter  classes  can  also contain character class expressions.
     These are expressions enclosed inside [: and  :]  delimiters
     (which themselves must appear between the '[' and ']' of the
     character class; other elements may occur inside the charac-
     ter class, too).  The valid expressions are:

         [:alnum:] [:alpha:] [:blank:]
         [:cntrl:] [:digit:] [:graph:]
         [:lower:] [:print:] [:punct:]
         [:space:] [:upper:] [:xdigit:]

     These  expressions  all  designate  a  set   of   characters
     equivalent  to  the corresponding standard C isXXX function.
     For example, [:alnum:] designates those characters for which
     isalnum()  returns  true  - i.e., any alphabetic or numeric.
     Some  systems  don't  provide  isblank(),  so  flex  defines
     [:blank:] as a blank or a tab.

     For  example,  the  following  character  classes  are   all
     equivalent:

         [[:alnum:]]
         [[:alpha:][:digit:]
         [[:alpha:]0-9]
         [a-zA-Z0-9]

     If your scanner is  case-insensitive  (the  -i  flag),  then
     [:upper:] and [:lower:] are equivalent to [:alpha:].

     Some notes on patterns:

     -    A negated character class such as the example  "[^A-Z]"


Version 2.5          Last change: April 1995                    8


FLEX(1)                  USER COMMANDS                    FLEX(1)


          above   will   match  a  newline  unless  "\n"  (or  an
          equivalent escape sequence) is one  of  the  characters
          explicitly  present  in  the  negated  character  class
          (e.g., "[^A-Z\n]").  This is unlike how many other reg-
          ular  expression tools treat negated character classes,
          but unfortunately  the  inconsistency  is  historically
          entrenched.   Matching  newlines  means  that a pattern
          like [^"]* can match the entire  input  unless  there's
          another quote in the input.

     -    A rule can have at most one instance of  trailing  con-
          text (the '/' operator or the '$' operator).  The start
          condition, '^', and "<<EOF>>" patterns can  only  occur
          at the beginning of a pattern, and, as well as with '/'
          and '$', cannot be grouped inside parentheses.   A  '^'
          which  does  not  occur at the beginning of a rule or a
          '$' which does not occur at the end of a rule loses its
          special  properties  and is treated as a normal charac-
          ter.

          The following are illegal:

              foo/bar$
              <sc1>foo<sc2>bar

          Note  that  the  first  of  these,   can   be   written
          "foo/bar\n".

          The following will result in '$' or '^'  being  treated
          as a normal character:

              foo|(bar$)
              foo|^bar

          If what's wanted is a  "foo"  or  a  bar-followed-by-a-
          newline,  the  following could be used (the special '|'
          action is explained below):

              foo      |
              bar$     /* action goes here */

          A similar trick will work for matching a foo or a  bar-
          at-the-beginning-of-a-line.

HOW THE INPUT IS MATCHED
     When the generated scanner is run,  it  analyzes  its  input
     looking  for strings which match any of its patterns.  If it
     finds more than one match, it takes  the  one  matching  the
     most  text  (for  trailing  context rules, this includes the
     length of the trailing part, even though  it  will  then  be
     returned  to the input).  If it finds two or more matches of
     the same length, the rule listed first  in  the  flex  input


Version 2.5          Last change: April 1995                    9


FLEX(1)                  USER COMMANDS                    FLEX(1)


     file is chosen.

     Once the match is determined, the text corresponding to  the
     match  (called  the  token)  is made available in the global
     character pointer yytext,  and  its  length  in  the  global
     integer yyleng. The action corresponding to the matched pat-
     tern is  then  executed  (a  more  detailed  description  of
     actions  follows),  and  then the remaining input is scanned
     for another match.

     If no match is found, then the default rule is executed: the
     next character in the input is considered matched and copied
     to the standard output.  Thus, the simplest legal flex input
     is:

         %%

     which generates a scanner that simply copies its input  (one
     character at a time) to its output.

     Note that yytext can  be  defined  in  two  different  ways:
     either  as  a character pointer or as a character array. You
     can control which definition flex uses by including  one  of
     the  special  directives  %pointer  or  %array  in the first
     (definitions) section of your flex input.   The  default  is
     %pointer, unless you use the -l lex compatibility option, in
     which case yytext will be an array.  The advantage of  using
     %pointer  is  substantially  faster  scanning  and no buffer
     overflow when matching very large tokens (unless you run out
     of  dynamic  memory).  The disadvantage is that you are res-
     tricted in how your actions can modify yytext (see the  next
     section),  and  calls  to  the unput() function destroys the
     present contents of yytext,  which  can  be  a  considerable
     porting headache when moving between different lex versions.

     The advantage of %array is that you can then  modify  yytext
     to your heart's content, and calls to unput() do not destroy
     yytext (see  below).   Furthermore,  existing  lex  programs
     sometimes access yytext externally using declarations of the
     form:
         extern char yytext[];
     This definition is erroneous when used  with  %pointer,  but
     correct for %array.

     %array defines yytext to be an array of  YYLMAX  characters,
     which  defaults to a fairly large value.  You can change the
     size by simply #define'ing YYLMAX to a  different  value  in
     the  first  section of your flex input.  As mentioned above,
     with %pointer yytext grows dynamically to accommodate  large
     tokens.  While this means your %pointer scanner can accommo-
     date very large tokens (such as matching  entire  blocks  of
     comments),  bear  in  mind  that  each time the scanner must


Version 2.5          Last change: April 1995                   10


FLEX(1)                  USER COMMANDS                    FLEX(1)


     resize yytext it also must rescan the entire token from  the
     beginning,  so  matching such tokens can prove slow.  yytext
     presently does not dynamically grow if  a  call  to  unput()
     results  in too much text being pushed back; instead, a run-
     time error results.

     Also note that  you  cannot  use  %array  with  C++  scanner
     classes (the c++ option; see below).

ACTIONS
     Each pattern in a rule has a corresponding action, which can
     be any arbitrary C statement.  The pattern ends at the first
     non-escaped whitespace character; the remainder of the  line
     is  its  action.  If the action is empty, then when the pat-
     tern is matched the input token is  simply  discarded.   For
     example,  here  is  the  specification  for  a program which
     deletes all occurrences of "zap me" from its input:

         %%
         "zap me"

     (It will copy all other characters in the input to the  out-
     put since they will be matched by the default rule.)

     Here is a program which compresses multiple blanks and  tabs
     down  to a single blank, and throws away whitespace found at
     the end of a line:

         %%
         [ \t]+        putchar( ' ' );
         [ \t]+$       /* ignore this token */


     If the action contains a '{', then the action spans till the
     balancing  '}'  is  found, and the action may cross multiple
     lines.  flex knows about C strings and comments and won't be
     fooled  by braces found within them, but also allows actions
     to begin with %{ and will consider the action to be all  the
     text up to the next %} (regardless of ordinary braces inside
     the action).

     An action consisting solely of a vertical  bar  ('|')  means
     "same  as  the  action for the next rule."  See below for an
     illustration.

     Actions can  include  arbitrary  C  code,  including  return
     statements  to  return  a  value  to whatever routine called
     yylex(). Each time yylex() is called it continues processing
     tokens  from  where it last left off until it either reaches
     the end of the file or executes a return.


Version 2.5          Last change: April 1995                   11


FLEX(1)                  USER COMMANDS                    FLEX(1)


     Actions are free to modify yytext except for lengthening  it
     (adding  characters  to  its end--these will overwrite later
     characters in the input  stream).   This  however  does  not
     apply  when  using  %array (see above); in that case, yytext
     may be freely modified in any way.

     Actions are free to modify yyleng except they should not  do
     so if the action also includes use of yymore() (see below).

     There are a  number  of  special  directives  which  can  be
     included within an action:

     -    ECHO copies yytext to the scanner's output.

     -    BEGIN followed by the name of a start condition  places
          the  scanner  in the corresponding start condition (see
          below).

     -    REJECT directs the scanner to proceed on to the "second
          best"  rule which matched the input (or a prefix of the
          input).  The rule is chosen as described above in  "How
          the  Input  is  Matched",  and yytext and yyleng set up
          appropriately.  It may either be one which  matched  as
          much  text as the originally chosen rule but came later
          in the flex input file, or one which matched less text.
          For example, the following will both count the words in
          the input  and  call  the  routine  special()  whenever
          "frob" is seen:

                      int word_count = 0;
              %%

              frob        special(); REJECT;
              [^ \t\n]+   ++word_count;

          Without the REJECT, any "frob"'s in the input would not
          be  counted  as  words, since the scanner normally exe-
          cutes only one action per token.  Multiple REJECT's are
          allowed,  each  one finding the next best choice to the
          currently active rule.  For example, when the following
          scanner  scans the token "abcd", it will write "abcdab-
          caba" to the output:

              %%
              a        |
              ab       |
              abc      |
              abcd     ECHO; REJECT;
              .|\n     /* eat up any unmatched character */

          (The first three rules share the fourth's action  since
          they   use   the  special  '|'  action.)  REJECT  is  a


Version 2.5          Last change: April 1995                   12


FLEX(1)                  USER COMMANDS                    FLEX(1)


          particularly expensive feature in terms of scanner per-
          formance; if it is used in any of the scanner's actions
          it will  slow  down  all  of  the  scanner's  matching.
          Furthermore,  REJECT cannot be used with the -Cf or -CF
          options (see below).

          Note also that unlike the other special actions, REJECT
          is  a  branch;  code  immediately  following  it in the
          action will not be executed.

     -    yymore() tells  the  scanner  that  the  next  time  it
          matches  a  rule,  the  corresponding  token  should be
          appended onto the current value of yytext  rather  than
          replacing  it.   For  example,  given  the input "mega-
          kludge" the following will write "mega-mega-kludge"  to
          the output:

              %%
              mega-    ECHO; yymore();
              kludge   ECHO;

          First "mega-" is matched  and  echoed  to  the  output.
          Then  "kludge"  is matched, but the previous "mega-" is
          still hanging around at the beginning of yytext so  the
          ECHO  for  the "kludge" rule will actually write "mega-
          kludge".

     Two notes regarding use of yymore(). First, yymore() depends
     on  the value of yyleng correctly reflecting the size of the
     current token, so you must not  modify  yyleng  if  you  are
     using  yymore().  Second,  the  presence  of yymore() in the
     scanner's action entails a minor performance penalty in  the
     scanner's matching speed.

     -    yyless(n) returns all but the first n characters of the
          current token back to the input stream, where they will
          be rescanned when the scanner looks for the next match.
          yytext  and  yyleng  are  adjusted appropriately (e.g.,
          yyleng will now be equal to n ).  For example,  on  the
          input  "foobar"  the  following will write out "foobar-
          bar":

              %%
              foobar    ECHO; yyless(3);
              [a-z]+    ECHO;

          An argument of  0  to  yyless  will  cause  the  entire
          current  input  string  to  be  scanned  again.  Unless
          you've changed how the scanner will  subsequently  pro-
          cess  its  input  (using BEGIN, for example), this will
          result in an endless loop.


Version 2.5          Last change: April 1995                   13


FLEX(1)                  USER COMMANDS                    FLEX(1)


     Note that yyless is a macro and can only be used in the flex
     input file, not from other source files.

     -    unput(c) puts the  character  c  back  onto  the  input
          stream.   It  will  be the next character scanned.  The
          following action will take the current token and  cause
          it to be rescanned enclosed in parentheses.

              {
              int i;
              /* Copy yytext because unput() trashes yytext */
              char *yycopy = strdup( yytext );
              unput( ')' );
              for ( i = yyleng - 1; i >= 0; --i )
                  unput( yycopy[i] );
              unput( '(' );
              free( yycopy );
              }

          Note that since each unput() puts the  given  character
          back at the beginning of the input stream, pushing back
          strings must be done back-to-front.

     An important potential problem when using unput() is that if
     you are using %pointer (the default), a call to unput() des-
     troys the contents of yytext, starting  with  its  rightmost
     character  and devouring one character to the left with each
     call.  If you need the value of  yytext  preserved  after  a
     call  to  unput() (as in the above example), you must either
     first copy it elsewhere, or build your scanner using  %array
     instead (see How The Input Is Matched).

     Finally, note that you cannot put back  EOF  to  attempt  to
     mark the input stream with an end-of-file.

     -    input() reads the next character from the input stream.
          For  example, the following is one way to eat up C com-
          ments:

              %%
              "/*"        {
                          register int c;

                          for ( ; ; )
                              {
                              while ( (c = input()) != '*' &&
                                      c != EOF )
                                  ;    /* eat up text of comment */

                              if ( c == '*' )
                                  {
                                  while ( (c = input()) == '*' )


Version 2.5          Last change: April 1995                   14


FLEX(1)                  USER COMMANDS                    FLEX(1)


                                      ;
                                  if ( c == '/' )
                                      break;    /* found the end */
                                  }

                              if ( c == EOF )
                                  {
                                  error( "EOF in comment" );
                                  break;
                                  }
                              }
                          }

          (Note that if the scanner is compiled using  C++,  then
          input()  is  instead referred to as yyinput(), in order
          to avoid a name clash with the C++ stream by  the  name
          of input.)

     -    YY_FLUSH_BUFFER flushes the scanner's  internal  buffer
          so  that  the next time the scanner attempts to match a
          token, it will first refill the buffer  using  YY_INPUT
          (see  The  Generated Scanner, below).  This action is a
          special case  of  the  more  general  yy_flush_buffer()
          function, described below in the section Multiple Input
          Buffers.

     -    yyterminate() can be used in lieu of a return statement
          in  an action.  It terminates the scanner and returns a
          0 to the scanner's caller, indicating "all  done".   By
          default,  yyterminate()  is also called when an end-of-
          file is encountered.  It is a macro and  may  be  rede-
          fined.

THE GENERATED SCANNER
     The output of flex is the file lex.yy.c, which contains  the
     scanning  routine yylex(), a number of tables used by it for
     matching tokens, and a number of auxiliary routines and mac-
     ros.  By default, yylex() is declared as follows:

         int yylex()
             {
             ... various definitions and the actions in here ...
             }

     (If your environment supports function prototypes,  then  it
     will  be  "int  yylex(  void  )".)   This  definition may be
     changed by defining the "YY_DECL" macro.  For  example,  you
     could use:

         #define YY_DECL float lexscan( a, b ) float a, b;

     to give the scanning routine the name lexscan,  returning  a


Version 2.5          Last change: April 1995                   15


FLEX(1)                  USER COMMANDS                    FLEX(1)


     float, and taking two floats as arguments.  Note that if you
     give  arguments  to  the  scanning  routine  using  a   K&R-
     style/non-prototyped  function  declaration,  you  must ter-
     minate the definition with a semi-colon (;).

     Whenever yylex() is called, it scans tokens from the  global
     input  file  yyin  (which  defaults to stdin).  It continues
     until it either reaches an end-of-file (at  which  point  it
     returns the value 0) or one of its actions executes a return
     statement.

     If the scanner reaches an end-of-file, subsequent calls  are
     undefined  unless either yyin is pointed at a new input file
     (in which case scanning continues from that file), or yyres-
     tart()  is called.  yyrestart() takes one argument, a FILE *
     pointer (which can be nil, if you've set up YY_INPUT to scan
     from  a  source  other  than yyin), and initializes yyin for
     scanning from that file.  Essentially there is no difference
     between  just  assigning  yyin  to a new input file or using
     yyrestart() to do so; the latter is available  for  compati-
     bility with previous versions of flex, and because it can be
     used to switch input files in the middle  of  scanning.   It
     can  also be used to throw away the current input buffer, by
     calling it with an argument of yyin; but better  is  to  use
     YY_FLUSH_BUFFER (see above).  Note that yyrestart() does not
     reset the start condition to INITIAL (see Start  Conditions,
     below).

     If yylex() stops scanning due to executing a  return  state-
     ment  in  one of the actions, the scanner may then be called
     again and it will resume scanning where it left off.

     By default (and for purposes  of  efficiency),  the  scanner
     uses  block-reads  rather  than  simple getc() calls to read
     characters from yyin. The nature of how it  gets  its  input
     can   be   controlled   by   defining  the  YY_INPUT  macro.
     YY_INPUT's           calling           sequence           is
     "YY_INPUT(buf,result,max_size)".   Its action is to place up
     to max_size characters in the character array buf and return
     in  the integer variable result either the number of charac-
     ters read or the constant YY_NULL (0  on  Unix  systems)  to
     indicate  EOF.   The  default YY_INPUT reads from the global
     file-pointer "yyin".

     A sample definition of YY_INPUT (in the definitions  section
     of the input file):

         %{
         #define YY_INPUT(buf,result,max_size) \
             { \
             int c = getchar(); \
             result = (c == EOF) ? YY_NULL : (buf[0] = c, 1); \


Version 2.5          Last change: April 1995                   16


FLEX(1)                  USER COMMANDS                    FLEX(1)


             }
         %}

     This definition will change the input  processing  to  occur
     one character at a time.

     When the scanner receives  an  end-of-file  indication  from
     YY_INPUT, it then checks the yywrap() function.  If yywrap()
     returns false (zero), then it is assumed that  the  function
     has  gone  ahead  and  set up yyin to point to another input
     file, and scanning continues.   If  it  returns  true  (non-
     zero),  then  the  scanner  terminates,  returning  0 to its
     caller.  Note that  in  either  case,  the  start  condition
     remains unchanged; it does not revert to INITIAL.

     If you do not supply your own version of yywrap(), then  you
     must  either use %option noyywrap (in which case the scanner
     behaves as though yywrap() returned 1),  or  you  must  link
     with  -lfl  to  obtain  the  default version of the routine,
     which always returns 1.

     Three routines are available  for  scanning  from  in-memory
     buffers     rather     than     files:     yy_scan_string(),
     yy_scan_bytes(), and yy_scan_buffer(). See the discussion of
     them below in the section Multiple Input Buffers.

     The scanner writes its  ECHO  output  to  the  yyout  global
     (default, stdout), which may be redefined by the user simply
     by assigning it to some other FILE pointer.

START CONDITIONS
     flex  provides  a  mechanism  for  conditionally  activating
     rules.   Any rule whose pattern is prefixed with "<sc>" will
     only be active when the scanner is in  the  start  condition
     named "sc".  For example,

         <STRING>[^"]*        { /* eat up the string body ... */
                     ...
                     }

     will be active only when the  scanner  is  in  the  "STRING"
     start condition, and

         <INITIAL,STRING,QUOTE>\.        { /* handle an escape ... */
                     ...
                     }

     will be active only when  the  current  start  condition  is
     either "INITIAL", "STRING", or "QUOTE".

     Start conditions are declared  in  the  definitions  (first)
     section  of  the input using unindented lines beginning with


Version 2.5          Last change: April 1995                   17


FLEX(1)                  USER COMMANDS                    FLEX(1)


     either %s or %x followed by a list  of  names.   The  former
     declares  inclusive  start  conditions, the latter exclusive
     start conditions.  A start condition is activated using  the
     BEGIN  action.   Until  the  next  BEGIN action is executed,
     rules with the given start  condition  will  be  active  and
     rules  with other start conditions will be inactive.  If the
     start condition is inclusive, then rules with no start  con-
     ditions  at  all  will  also be active.  If it is exclusive,
     then only rules qualified with the start condition  will  be
     active.   A  set  of  rules contingent on the same exclusive
     start condition describe a scanner which is  independent  of
     any  of the other rules in the flex input.  Because of this,
     exclusive start conditions make it easy  to  specify  "mini-
     scanners"  which scan portions of the input that are syntac-
     tically different from the rest (e.g., comments).

     If the distinction between  inclusive  and  exclusive  start
     conditions  is still a little vague, here's a simple example
     illustrating the connection between the  two.   The  set  of
     rules:

         %s example
         %%

         <example>foo   do_something();

         bar            something_else();

     is equivalent to

         %x example
         %%

         <example>foo   do_something();

         <INITIAL,example>bar    something_else();

     Without the <INITIAL,example> qualifier, the bar pattern  in
     the second example wouldn't be active (i.e., couldn't match)
     when in start condition example. If we just  used  <example>
     to  qualify  bar,  though,  then  it would only be active in
     example and not in INITIAL, while in the first example  it's
     active  in  both,  because  in the first example the example
     startion condition is an inclusive (%s) start condition.

     Also note that the  special  start-condition  specifier  <*>
     matches  every  start  condition.   Thus,  the above example
     could also have been written;

         %x example
         %%


Version 2.5          Last change: April 1995                   18


FLEX(1)                  USER COMMANDS                    FLEX(1)


         <example>foo   do_something();

         <*>bar    something_else();


     The default rule (to ECHO any unmatched  character)  remains
     active in start conditions.  It is equivalent to:

         <*>.|\n     ECHO;


     BEGIN(0) returns to the original state where only the  rules
     with no start conditions are active.  This state can also be
     referred   to   as   the   start-condition   "INITIAL",   so
     BEGIN(INITIAL)  is  equivalent to BEGIN(0). (The parentheses
     around the start condition name are  not  required  but  are
     considered good style.)

     BEGIN actions can also be given  as  indented  code  at  the
     beginning  of the rules section.  For example, the following
     will cause the scanner to enter the "SPECIAL"  start  condi-
     tion  whenever  yylex()  is  called  and the global variable
     enter_special is true:

                 int enter_special;

         %x SPECIAL
         %%
                 if ( enter_special )
                     BEGIN(SPECIAL);

         <SPECIAL>blahblahblah
         ...more rules follow...


     To illustrate the  uses  of  start  conditions,  here  is  a
     scanner  which  provides  two different interpretations of a
     string like "123.456".  By default it will treat it as three
     tokens,  the  integer  "123",  a  dot ('.'), and the integer
     "456".  But if the string is preceded earlier in the line by
     the  string  "expect-floats"  it  will  treat it as a single
     token, the floating-point number 123.456:

         %{
         #include <math.h>
         %}
         %s expect

         %%
         expect-floats        BEGIN(expect);

         <expect>[0-9]+"."[0-9]+      {


Version 2.5          Last change: April 1995                   19


FLEX(1)                  USER COMMANDS                    FLEX(1)


                     printf( "found a float, = %f\n",
                             atof( yytext ) );
                     }
         <expect>\n           {
                     /* that's the end of the line, so
                      * we need another "expect-number"
                      * before we'll recognize any more
                      * numbers
                      */
                     BEGIN(INITIAL);
                     }

         [0-9]+      {
                     printf( "found an integer, = %d\n",
                             atoi( yytext ) );
                     }

         "."         printf( "found a dot\n" );

     Here is a scanner which recognizes (and discards) C comments
     while maintaining a count of the current input line.

         %x comment
         %%
                 int line_num = 1;

         "/*"         BEGIN(comment);

         <comment>[^*\n]*        /* eat anything that's not a '*' */
         <comment>"*"+[^*/\n]*   /* eat up '*'s not followed by '/'s */
         <comment>\n             ++line_num;
         <comment>"*"+"/"        BEGIN(INITIAL);

     This scanner goes to a bit of trouble to match as much  text
     as  possible with each rule.  In general, when attempting to
     write a high-speed scanner try to match as much possible  in
     each rule, as it's a big win.

     Note that start-conditions names are really  integer  values
     and  can  be  stored  as  such.   Thus,  the  above could be
     extended in the following fashion:

         %x comment foo
         %%
                 int line_num = 1;
                 int comment_caller;

         "/*"         {
                      comment_caller = INITIAL;
                      BEGIN(comment);
                      }


Version 2.5          Last change: April 1995                   20


FLEX(1)                  USER COMMANDS                    FLEX(1)


         ...

         <foo>"/*"    {
                      comment_caller = foo;
                      BEGIN(comment);
                      }

         <comment>[^*\n]*        /* eat anything that's not a '*' */
         <comment>"*"+[^*/\n]*   /* eat up '*'s not followed by '/'s */
         <comment>\n             ++line_num;
         <comment>"*"+"/"        BEGIN(comment_caller);

     Furthermore, you can  access  the  current  start  condition
     using  the  integer-valued YY_START macro.  For example, the
     above assignments to comment_caller could instead be written

         comment_caller = YY_START;

     Flex provides YYSTATE as an alias for YY_START  (since  that
     is what's used by AT&T lex).

     Note that start conditions do not have their own name-space;
     %s's   and  %x's  declare  names  in  the  same  fashion  as
     #define's.

     Finally, here's an example of how to  match  C-style  quoted
     strings using exclusive start conditions, including expanded
     escape sequences (but not including checking  for  a  string
     that's too long):

         %x str

         %%
                 char string_buf[MAX_STR_CONST];
                 char *string_buf_ptr;


         \"      string_buf_ptr = string_buf; BEGIN(str);

         <str>\"        { /* saw closing quote - all done */
                 BEGIN(INITIAL);
                 *string_buf_ptr = '\0';
                 /* return string constant token type and
                  * value to parser
                  */
                 }

         <str>\n        {
                 /* error - unterminated string constant */
                 /* generate error message */
                 }


Version 2.5          Last change: April 1995                   21


FLEX(1)                  USER COMMANDS                    FLEX(1)


         <str>\\[0-7]{1,3} {
                 /* octal escape sequence */
                 int result;

                 (void) sscanf( yytext + 1, "%o", &result );

                 if ( result > 0xff )
                         /* error, constant is out-of-bounds */

                 *string_buf_ptr++ = result;
                 }

         <str>\\[0-9]+ {
                 /* generate error - bad escape sequence; something
                  * like '\48' or '\0777777'
                  */
                 }

         <str>\\n  *string_buf_ptr++ = '\n';
         <str>\\t  *string_buf_ptr++ = '\t';
         <str>\\r  *string_buf_ptr++ = '\r';
         <str>\\b  *string_buf_ptr++ = '\b';
         <str>\\f  *string_buf_ptr++ = '\f';

         <str>\\(.|\n)  *string_buf_ptr++ = yytext[1];

         <str>[^\\\n\"]+        {
                 char *yptr = yytext;

                 while ( *yptr )
                         *string_buf_ptr++ = *yptr++;
                 }


     Often, such as in some of the examples above,  you  wind  up
     writing  a  whole  bunch  of  rules all preceded by the same
     start condition(s).  Flex makes this  a  little  easier  and
     cleaner  by introducing a notion of start condition scope. A
     start condition scope is begun with:

         <SCs>{

     where SCs is a list of one or more start conditions.  Inside
     the  start condition scope, every rule automatically has the
     prefix <SCs> applied to it, until a '}'  which  matches  the
     initial '{'. So, for example,

         <ESC>{
             "\\n"   return '\n';
             "\\r"   return '\r';
             "\\f"   return '\f';
             "\\0"   return '\0';


Version 2.5          Last change: April 1995                   22


FLEX(1)                  USER COMMANDS                    FLEX(1)


         }

     is equivalent to:

         <ESC>"\\n"  return '\n';
         <ESC>"\\r"  return '\r';
         <ESC>"\\f"  return '\f';
         <ESC>"\\0"  return '\0';

     Start condition scopes may be nested.

     Three routines are  available  for  manipulating  stacks  of
     start conditions:

     void yy_push_state(int new_state)
          pushes the current start condition onto the top of  the
          start  condition  stack  and  switches  to new_state as
          though you had used BEGIN new_state (recall that  start
          condition names are also integers).

     void yy_pop_state()
          pops the top of the stack and switches to it via BEGIN.

     int yy_top_state()
          returns the top  of  the  stack  without  altering  the
          stack's contents.

     The start condition stack grows dynamically and  so  has  no
     built-in  size  limitation.  If memory is exhausted, program
     execution aborts.

     To use start condition stacks, your scanner must  include  a
     %option stack directive (see Options below).

MULTIPLE INPUT BUFFERS
     Some scanners (such as those which support "include"  files)
     require   reading  from  several  input  streams.   As  flex
     scanners do a large amount of buffering, one cannot  control
     where  the  next input will be read from by simply writing a
     YY_INPUT  which  is  sensitive  to  the  scanning   context.
     YY_INPUT  is only called when the scanner reaches the end of
     its buffer, which may be a long time after scanning a state-
     ment such as an "include" which requires switching the input
     source.

     To negotiate  these  sorts  of  problems,  flex  provides  a
     mechanism  for creating and switching between multiple input
     buffers.  An input buffer is created by using:

         YY_BUFFER_STATE yy_create_buffer( FILE *file, int size )

     which takes a FILE pointer and a size and creates  a  buffer


Version 2.5          Last change: April 1995                   23


FLEX(1)                  USER COMMANDS                    FLEX(1)


     associated with the given file and large enough to hold size
     characters (when in doubt, use YY_BUF_SIZE  for  the  size).
     It  returns  a  YY_BUFFER_STATE  handle,  which  may then be
     passed to other routines (see below).   The  YY_BUFFER_STATE
     type is a pointer to an opaque struct yy_buffer_state struc-
     ture, so you may safely initialize YY_BUFFER_STATE variables
     to  ((YY_BUFFER_STATE) 0) if you wish, and also refer to the
     opaque structure in order to correctly declare input buffers
     in  source files other than that of your scanner.  Note that
     the FILE pointer in the call  to  yy_create_buffer  is  only
     used  as the value of yyin seen by YY_INPUT; if you redefine
     YY_INPUT so it no longer uses yyin, then you can safely pass
     a nil FILE pointer to yy_create_buffer. You select a partic-
     ular buffer to scan from using:

         void yy_switch_to_buffer( YY_BUFFER_STATE new_buffer )

     switches the scanner's input  buffer  so  subsequent  tokens
     will  come  from new_buffer. Note that yy_switch_to_buffer()
     may be used by yywrap() to set things up for continued scan-
     ning, instead of opening a new file and pointing yyin at it.
     Note  also  that  switching   input   sources   via   either
     yy_switch_to_buffer()  or yywrap() does not change the start
     condition.

         void yy_delete_buffer( YY_BUFFER_STATE buffer )

     is used to reclaim the storage associated with a buffer.   (
     buffer  can be nil, in which case the routine does nothing.)
     You can also clear the current contents of a buffer using:

         void yy_flush_buffer( YY_BUFFER_STATE buffer )

     This function discards the buffer's contents,  so  the  next
     time  the scanner attempts to match a token from the buffer,
     it will first fill the buffer anew using YY_INPUT.

     yy_new_buffer() is an alias for yy_create_buffer(), provided
     for  compatibility  with  the  C++ use of new and delete for
     creating and destroying dynamic objects.

     Finally,   the    YY_CURRENT_BUFFER    macro    returns    a
     YY_BUFFER_STATE handle to the current buffer.

     Here is an example of using these  features  for  writing  a
     scanner  which expands include files (the <<EOF>> feature is
     discussed below):

         /* the "incl" state is used for picking up the name
          * of an include file
          */
         %x incl


Version 2.5          Last change: April 1995                   24


FLEX(1)                  USER COMMANDS                    FLEX(1)


         %{
         #define MAX_INCLUDE_DEPTH 10
         YY_BUFFER_STATE include_stack[MAX_INCLUDE_DEPTH];
         int include_stack_ptr = 0;
         %}

         %%
         include             BEGIN(incl);

         [a-z]+              ECHO;
         [^a-z\n]*\n?        ECHO;

         <incl>[ \t]*      /* eat the whitespace */
         <incl>[^ \t\n]+   { /* got the include file name */
                 if ( include_stack_ptr >= MAX_INCLUDE_DEPTH )
                     {
                     fprintf( stderr, "Includes nested too deeply" );
                     exit( 1 );
                     }

                 include_stack[include_stack_ptr++] =
                     YY_CURRENT_BUFFER;

                 yyin = fopen( yytext, "r" );

                 if ( ! yyin )
                     error( ... );

                 yy_switch_to_buffer(
                     yy_create_buffer( yyin, YY_BUF_SIZE ) );

                 BEGIN(INITIAL);
                 }

         <<EOF>> {
                 if ( --include_stack_ptr < 0 )
                     {
                     yyterminate();
                     }

                 else
                     {
                     yy_delete_buffer( YY_CURRENT_BUFFER );
                     yy_switch_to_buffer(
                          include_stack[include_stack_ptr] );
                     }
                 }

     Three routines are available for setting  up  input  buffers
     for  scanning  in-memory  strings  instead of files.  All of
     them create a new input buffer for scanning the string,  and
     return  a  corresponding  YY_BUFFER_STATE  handle (which you


Version 2.5          Last change: April 1995                   25


FLEX(1)                  USER COMMANDS                    FLEX(1)


     should delete with yy_delete_buffer() when  done  with  it).
     They    also    switch    to    the    new    buffer   using
     yy_switch_to_buffer(), so the  next  call  to  yylex()  will
     start scanning the string.

     yy_scan_string(const char *str)
          scans a NUL-terminated string.

     yy_scan_bytes(const char *bytes, int len)
          scans len bytes (including possibly NUL's) starting  at
          location bytes.

     Note that both of these functions create and scan a copy  of
     the  string or bytes.  (This may be desirable, since yylex()
     modifies the contents of the buffer it  is  scanning.)   You
     can avoid the copy by using:

     yy_scan_buffer(char *base, yy_size_t size)
          which scans in place the buffer starting at base,  con-
          sisting of size bytes, the last two bytes of which must
          be YY_END_OF_BUFFER_CHAR (ASCII NUL).  These  last  two
          bytes  are  not  scanned;  thus,  scanning  consists of
          base[0] through base[size-2], inclusive.

          If you fail to set up base in this manner (i.e., forget
          the   final   two  YY_END_OF_BUFFER_CHAR  bytes),  then
          yy_scan_buffer()  returns  a  nil  pointer  instead  of
          creating a new input buffer.

          The type yy_size_t is an integral type to which you can
          cast  an  integer expression reflecting the size of the
          buffer.

END-OF-FILE RULES
     The special rule "<<EOF>>" indicates actions which are to be
     taken  when  an  end-of-file  is  encountered  and  yywrap()
     returns non-zero (i.e., indicates no further files  to  pro-
     cess).  The action must finish by doing one of four things:

     -    assigning yyin to a new input file  (in  previous  ver-
          sions  of  flex,  after doing the assignment you had to
          call the special action YY_NEW_FILE; this is no  longer
          necessary);

     -    executing a return statement;

     -    executing the special yyterminate() action;

     -    or,    switching    to    a    new     buffer     using
          yy_switch_to_buffer() as shown in the example above.


Version 2.5          Last change: April 1995                   26


FLEX(1)                  USER COMMANDS                    FLEX(1)


     <<EOF>> rules may not be used with other patterns; they  may
     only  be  qualified  with a list of start conditions.  If an
     unqualified <<EOF>> rule is given, it applies to  all  start
     conditions  which  do  not already have <<EOF>> actions.  To
     specify an <<EOF>> rule for only the  initial  start  condi-
     tion, use

         <INITIAL><<EOF>>


     These rules are useful for  catching  things  like  unclosed
     comments.  An example:

         %x quote
         %%

         ...other rules for dealing with quotes...

         <quote><<EOF>>   {
                  error( "unterminated quote" );
                  yyterminate();
                  }
         <<EOF>>  {
                  if ( *++filelist )
                      yyin = fopen( *filelist, "r" );
                  else
                     yyterminate();
                  }


MISCELLANEOUS MACROS
     The macro YY_USER_ACTION can be defined to provide an action
     which is always executed prior to the matched rule's action.
     For example, it could be #define'd to call a routine to con-
     vert  yytext to lower-case.  When YY_USER_ACTION is invoked,
     the variable yy_act gives the number  of  the  matched  rule
     (rules  are  numbered starting with 1).  Suppose you want to
     profile how often each of your rules is matched.   The  fol-
     lowing would do the trick:

         #define YY_USER_ACTION ++ctr[yy_act]

     where ctr is an array to hold the counts for  the  different
     rules.   Note  that  the  macro YY_NUM_RULES gives the total
     number of rules (including the default rule, even if you use
     -s), so a correct declaration for ctr is:

         int ctr[YY_NUM_RULES];


     The macro YY_USER_INIT may be defined to provide  an  action
     which  is  always executed before the first scan (and before


Version 2.5          Last change: April 1995                   27


FLEX(1)                  USER COMMANDS                    FLEX(1)


     the scanner's internal initializations are done).  For exam-
     ple,  it  could  be used to call a routine to read in a data
     table or open a logging file.

     The macro yy_set_interactive(is_interactive) can be used  to
     control  whether  the  current buffer is considered interac-
     tive. An interactive buffer is processed  more  slowly,  but
     must  be  used  when  the  scanner's  input source is indeed
     interactive to avoid problems due to waiting to fill buffers
     (see the discussion of the -I flag below).  A non-zero value
     in the macro invocation marks the buffer as  interactive,  a
     zero  value as non-interactive.  Note that use of this macro
     overrides  %option  always-interactive  or  %option   never-
     interactive  (see Options below).  yy_set_interactive() must
     be invoked prior to beginning to scan the buffer that is (or
     is not) to be considered interactive.

     The macro yy_set_bol(at_bol) can be used to control  whether
     the  current  buffer's  scanning  context for the next token
     match is done as though at the beginning of a line.  A  non-
     zero macro argument makes rules anchored with

     The macro YY_AT_BOL() returns true if the next token scanned
     from  the  current  buffer will have '^' rules active, false
     otherwise.

     In the generated scanner, the actions are  all  gathered  in
     one  large  switch  statement  and separated using YY_BREAK,
     which may be redefined.  By default, it is simply a "break",
     to  separate  each  rule's action from the following rule's.
     Redefining  YY_BREAK  allows,  for  example,  C++  users  to
     #define  YY_BREAK  to  do  nothing (while being very careful
     that every rule ends with a "break" or a "return"!) to avoid
     suffering  from unreachable statement warnings where because
     a rule's action ends with "return", the YY_BREAK is inacces-
     sible.

VALUES AVAILABLE TO THE USER
     This section summarizes the various values available to  the
     user in the rule actions.

     -    char *yytext holds the text of the current  token.   It
          may  be  modified but not lengthened (you cannot append
          characters to the end).

          If the special directive %array appears  in  the  first
          section  of  the  scanner  description,  then yytext is
          instead declared char yytext[YYLMAX], where YYLMAX is a
          macro  definition  that  you  can redefine in the first
          section if you don't like the default value  (generally
          8KB).    Using   %array   results  in  somewhat  slower
          scanners, but the value of  yytext  becomes  immune  to


Version 2.5          Last change: April 1995                   28


FLEX(1)                  USER COMMANDS                    FLEX(1)


          calls to input() and unput(), which potentially destroy
          its value when yytext  is  a  character  pointer.   The
          opposite of %array is %pointer, which is the default.

          You cannot  use  %array  when  generating  C++  scanner
          classes (the -+ flag).

     -    int yyleng holds the length of the current token.

     -    FILE *yyin is the file  which  by  default  flex  reads
          from.   It  may  be  redefined  but doing so only makes
          sense before scanning begins or after an EOF  has  been
          encountered.  Changing it in the midst of scanning will
          have unexpected results since flex buffers  its  input;
          use  yyrestart()  instead.   Once  scanning  terminates
          because an end-of-file has been seen,  you  can  assign
          yyin  at  the  new input file and then call the scanner
          again to continue scanning.

     -    void yyrestart( FILE *new_file ) may be called to point
          yyin at the new input file.  The switch-over to the new
          file is immediate (any previously buffered-up input  is
          lost).   Note  that calling yyrestart() with yyin as an
          argument thus throws away the current input buffer  and
          continues scanning the same input file.

     -    FILE *yyout is the file to which ECHO actions are done.
          It can be reassigned by the user.

     -    YY_CURRENT_BUFFER returns a YY_BUFFER_STATE  handle  to
          the current buffer.

     -    YY_START returns an integer value corresponding to  the
          current start condition.  You can subsequently use this
          value with BEGIN to return to that start condition.

INTERFACING WITH YACC
     One of the main uses of flex is as a companion to  the  yacc
     parser-generator.   yacc  parsers  expect  to call a routine
     named yylex() to find the next input token.  The routine  is
     supposed  to  return  the  type of the next token as well as
     putting any associated value in the global  yylval.  To  use
     flex  with  yacc,  one  specifies  the  -d option to yacc to
     instruct it to generate the file y.tab.h containing  defini-
     tions  of all the %tokens appearing in the yacc input.  This
     file is then included in the flex scanner.  For example,  if
     one of the tokens is "TOK_NUMBER", part of the scanner might
     look like:

         %{
         #include "y.tab.h"
         %}


Version 2.5          Last change: April 1995                   29


FLEX(1)                  USER COMMANDS                    FLEX(1)


         %%

         [0-9]+        yylval = atoi( yytext ); return TOK_NUMBER;


OPTIONS
     flex has the following options:

     -b   Generate backing-up information to lex.backup. This  is
          a  list  of scanner states which require backing up and
          the input characters on which they do  so.   By  adding
          rules   one  can  remove  backing-up  states.   If  all
          backing-up states are eliminated  and  -Cf  or  -CF  is
          used, the generated scanner will run faster (see the -p
          flag).  Only users who wish to squeeze every last cycle
          out  of  their  scanners  need worry about this option.
          (See the section on Performance Considerations below.)

     -c   is a do-nothing, deprecated option included  for  POSIX
          compliance.

     -d   makes the generated scanner run in debug  mode.   When-
          ever   a   pattern   is   recognized   and  the  global
          yy_flex_debug is non-zero (which is the  default),  the
          scanner will write to stderr a line of the form:

              --accepting rule at line 53 ("the matched text")

          The line number refers to the location of the  rule  in
          the  file defining the scanner (i.e., the file that was
          fed to flex).  Messages are  also  generated  when  the
          scanner backs up, accepts the default rule, reaches the
          end of its input buffer (or encounters a NUL;  at  this
          point,  the  two  look the same as far as the scanner's
          concerned), or reaches an end-of-file.

     -f   specifies fast scanner. No table  compression  is  done
          and  stdio  is bypassed.  The result is large but fast.
          This option is equivalent to -Cfr (see below).

     -h   generates a "help" summary of flex's options to  stdout
          and then exits.  -? and --help are synonyms for -h.

     -i   instructs flex to generate a case-insensitive  scanner.
          The  case  of  letters given in the flex input patterns
          will be ignored,  and  tokens  in  the  input  will  be
          matched  regardless of case.  The matched text given in
          yytext will have the preserved case (i.e., it will  not
          be folded).

     -l   turns on maximum compatibility with the  original  AT&T
          lex  implementation.  Note that this does not mean full


Version 2.5          Last change: April 1995                   30


FLEX(1)                  USER COMMANDS                    FLEX(1)


          compatibility.  Use of this option costs a considerable
          amount  of  performance, and it cannot be used with the
          -+, -f, -F, -Cf, or -CF options.  For  details  on  the
          compatibilities  it provides, see the section "Incompa-
          tibilities With Lex And POSIX" below.  This option also
          results  in the name YY_FLEX_LEX_COMPAT being #define'd
          in the generated scanner.

     -n   is another do-nothing, deprecated option included  only
          for POSIX compliance.

     -p   generates a performance report to stderr.   The  report
          consists  of  comments  regarding  features of the flex
          input file which will cause a serious loss  of  perfor-
          mance  in  the resulting scanner.  If you give the flag
          twice, you will also get  comments  regarding  features
          that lead to minor performance losses.

          Note that the use  of  REJECT,  %option  yylineno,  and
          variable  trailing context (see the Deficiencies / Bugs
          section  below)  entails  a   substantial   performance
          penalty;  use  of  yymore(), the ^ operator, and the -I
          flag entail minor performance penalties.

     -s   causes the default rule (that unmatched  scanner  input
          is  echoed to stdout) to be suppressed.  If the scanner
          encounters input that does not match any of its  rules,
          it  aborts  with  an  error.  This option is useful for
          finding holes in a scanner's rule set.

     -t   instructs flex to write the  scanner  it  generates  to
          standard output instead of lex.yy.c.

     -v   specifies that flex should write to stderr a summary of
          statistics regarding the scanner it generates.  Most of
          the statistics are meaningless to the casual flex user,
          but the first line identifies the version of flex (same
          as reported by -V), and the next line  the  flags  used
          when  generating  the scanner, including those that are
          on by default.

     -w   suppresses warning messages.

     -B   instructs flex to generate a batch scanner,  the  oppo-
          site  of  interactive  scanners  generated  by  -I (see
          below).  In general, you use -B when  you  are  certain
          that your scanner will never be used interactively, and
          you want to squeeze a little more  performance  out  of
          it.   If your goal is instead to squeeze out a lot more
          performance, you  should   be  using  the  -Cf  or  -CF
          options  (discussed  below), which turn on -B automati-
          cally anyway.


Version 2.5          Last change: April 1995                   31


FLEX(1)                  USER COMMANDS                    FLEX(1)


     -F   specifies that the fast  scanner  table  representation
          should  be used (and stdio bypassed).  This representa-
          tion is about as fast as the full table  representation
          (-f),  and  for some sets of patterns will be consider-
          ably smaller (and for others, larger).  In general,  if
          the  pattern  set contains both "keywords" and a catch-
          all, "identifier" rule, such as in the set:

              "case"    return TOK_CASE;
              "switch"  return TOK_SWITCH;
              ...
              "default" return TOK_DEFAULT;
              [a-z]+    return TOK_ID;

          then you're better off using the full table representa-
          tion.  If only the "identifier" rule is present and you
          then use a hash table or some such to detect  the  key-
          words, you're better off using -F.

          This option is equivalent to -CFr (see below).  It can-
          not be used with -+.

     -I   instructs flex to generate an interactive scanner.   An
          interactive  scanner  is  one  that only looks ahead to
          decide what token has been  matched  if  it  absolutely
          must.  It turns out that always looking one extra char-
          acter ahead, even  if  the  scanner  has  already  seen
          enough text to disambiguate the current token, is a bit
          faster than only looking  ahead  when  necessary.   But
          scanners  that always look ahead give dreadful interac-
          tive performance; for example, when a user types a new-
          line,  it  is  not  recognized as a newline token until
          they enter another token, which often means  typing  in
          another whole line.

          Flex scanners default to interactive unless you use the
          -Cf  or  -CF  table-compression  options  (see  below).
          That's because if you're looking  for  high-performance
          you  should  be  using  one of these options, so if you
          didn't, flex assumes you'd rather trade off  a  bit  of
          run-time    performance   for   intuitive   interactive
          behavior.  Note also that you cannot use -I in conjunc-
          tion  with  -Cf or -CF. Thus, this option is not really
          needed; it is on by default  for  all  those  cases  in
          which it is allowed.

          You can force a scanner to not be interactive by  using
          -B (see above).

     -L   instructs  flex  not  to  generate  #line   directives.
          Without this option, flex peppers the generated scanner
          with #line directives so error messages in the  actions


Version 2.5          Last change: April 1995                   32


FLEX(1)                  USER COMMANDS                    FLEX(1)


          will  be  correctly  located with respect to either the
          original flex input file (if the errors are due to code
          in  the  input  file),  or  lex.yy.c (if the errors are
          flex's fault -- you should report these sorts of errors
          to the email address given below).

     -T   makes flex run in trace mode.  It will generate  a  lot
          of  messages to stderr concerning the form of the input
          and the resultant non-deterministic  and  deterministic
          finite  automata.   This  option  is  mostly for use in
          maintaining flex.

     -V   prints the version number to stdout and exits.   --ver-
          sion is a synonym for -V.

     -7   instructs flex to generate a 7-bit scanner,  i.e.,  one
          which  can  only  recognized  7-bit  characters  in its
          input.  The advantage of using -7 is that the scanner's
          tables  can  be  up to half the size of those generated
          using the -8 option (see below).  The  disadvantage  is
          that  such  scanners often hang or crash if their input
          contains an 8-bit character.

          Note, however, that unless you  generate  your  scanner
          using  the -Cf or -CF table compression options, use of
          -7 will save only a small amount of  table  space,  and
          make  your  scanner considerably less portable.  Flex's
          default behavior is to generate an 8-bit scanner unless
          you  use the -Cf or -CF, in which case flex defaults to
          generating 7-bit scanners unless your site  was  always
          configured to generate 8-bit scanners (as will often be
          the case with non-USA sites).   You  can  tell  whether
          flex  generated a 7-bit or an 8-bit scanner by inspect-
          ing the flag summary in  the  -v  output  as  described
          above.

          Note that if you use -Cfe or -CFe (those table compres-
          sion  options,  but  also  using equivalence classes as
          discussed see below), flex still defaults to generating
          an  8-bit scanner, since usually with these compression
          options full 8-bit tables are not much  more  expensive
          than 7-bit tables.

     -8   instructs flex to generate an 8-bit scanner, i.e.,  one
          which  can  recognize  8-bit  characters.  This flag is
          only needed for scanners generated using -Cf or -CF, as
          otherwise  flex defaults to generating an 8-bit scanner
          anyway.

          See the discussion  of  -7  above  for  flex's  default
          behavior  and  the  tradeoffs  between  7-bit and 8-bit
          scanners.


Version 2.5          Last change: April 1995                   33


FLEX(1)                  USER COMMANDS                    FLEX(1)


     -+   specifies that you want flex to generate a C++  scanner
          class.   See  the  section  on  Generating C++ Scanners
          below for details.

     -C[aefFmr]
          controls the degree of table compression and, more gen-
          erally,  trade-offs  between  small  scanners  and fast
          scanners.

          -Ca ("align") instructs flex to trade off larger tables
          in the generated scanner for faster performance because
          the elements of  the  tables  are  better  aligned  for
          memory  access and computation.  On some RISC architec-
          tures, fetching  and  manipulating  longwords  is  more
          efficient  than with smaller-sized units such as short-
          words.  This option can double the size of  the  tables
          used by your scanner.

          -Ce directs  flex  to  construct  equivalence  classes,
          i.e.,  sets  of characters which have identical lexical
          properties (for example,  if  the  only  appearance  of
          digits  in  the  flex  input  is in the character class
          "[0-9]" then the digits '0', '1', ..., '9' will all  be
          put   in  the  same  equivalence  class).   Equivalence
          classes usually give dramatic reductions in  the  final
          table/object file sizes (typically a factor of 2-5) and
          are pretty cheap performance-wise  (one  array  look-up
          per character scanned).

          -Cf specifies that the full scanner  tables  should  be
          generated - flex should not compress the tables by tak-
          ing advantages of similar transition functions for dif-
          ferent states.

          -CF specifies that the alternate fast scanner represen-
          tation  (described  above  under the -F flag) should be
          used.  This option cannot be used with -+.

          -Cm directs flex to construct meta-equivalence classes,
          which  are  sets of equivalence classes (or characters,
          if equivalence classes are not  being  used)  that  are
          commonly  used  together.  Meta-equivalence classes are
          often a big win when using compressed tables, but  they
          have  a  moderate  performance  impact (one or two "if"
          tests and one array look-up per character scanned).

          -Cr causes the generated scanner to bypass use  of  the
          standard  I/O  library  (stdio)  for input.  Instead of
          calling fread() or getc(), the  scanner  will  use  the
          read()  system  call,  resulting  in a performance gain
          which varies from system to system, but in  general  is
          probably  negligible  unless  you are also using -Cf or


Version 2.5          Last change: April 1995                   34


FLEX(1)                  USER COMMANDS                    FLEX(1)


          -CF. Using -Cr can cause strange behavior if, for exam-
          ple,  you  read  from yyin using stdio prior to calling
          the scanner (because the  scanner  will  miss  whatever
          text  your  previous  reads  left  in  the  stdio input
          buffer).

          -Cr has no effect if you define YY_INPUT (see The  Gen-
          erated Scanner above).

          A lone -C specifies that the scanner tables  should  be
          compressed  but  neither  equivalence classes nor meta-
          equivalence classes should be used.

          The options -Cf or  -CF  and  -Cm  do  not  make  sense
          together - there is no opportunity for meta-equivalence
          classes if the table is not being  compressed.   Other-
          wise  the  options may be freely mixed, and are cumula-
          tive.

          The default setting is -Cem, which specifies that  flex
          should   generate   equivalence   classes   and   meta-
          equivalence classes.  This setting provides the highest
          degree   of  table  compression.   You  can  trade  off
          faster-executing scanners at the cost of larger  tables
          with the following generally being true:

              slowest & smallest
                    -Cem
                    -Cm
                    -Ce
                    -C
                    -C{f,F}e
                    -C{f,F}
                    -C{f,F}a
              fastest & largest

          Note that scanners with the smallest tables are usually
          generated and compiled the quickest, so during develop-
          ment you will usually want to use the default,  maximal
          compression.

          -Cfe is often a good compromise between speed and  size
          for production scanners.

     -ooutput
          directs flex to write the scanner to  the  file  output
          instead  of  lex.yy.c.  If  you  combine -o with the -t
          option, then the scanner is written to stdout  but  its
          #line directives (see the -L option above) refer to the
          file output.

     -Pprefix


Version 2.5          Last change: April 1995                   35


FLEX(1)                  USER COMMANDS                    FLEX(1)


          changes the default yy prefix  used  by  flex  for  all
          globally-visible variable and function names to instead
          be prefix. For  example,  -Pfoo  changes  the  name  of
          yytext  to  footext.  It  also  changes the name of the
          default output file from lex.yy.c  to  lex.foo.c.  Here
          are all of the names affected:

              yy_create_buffer
              yy_delete_buffer
              yy_flex_debug
              yy_init_buffer
              yy_flush_buffer
              yy_load_buffer_state
              yy_switch_to_buffer
              yyin
              yyleng
              yylex
              yylineno
              yyout
              yyrestart
              yytext
              yywrap

          (If you are using a C++ scanner, then only  yywrap  and
          yyFlexLexer  are affected.) Within your scanner itself,
          you can still refer to the global variables  and  func-
          tions  using  either  version of their name; but exter-
          nally, they have the modified name.

          This option lets you easily link together multiple flex
          programs  into the same executable.  Note, though, that
          using this option also renames  yywrap(),  so  you  now
          must either provide your own (appropriately-named) ver-
          sion of the routine for your scanner,  or  use  %option
          noyywrap,  as  linking with -lfl no longer provides one
          for you by default.

     -Sskeleton_file
          overrides the default skeleton  file  from  which  flex
          constructs its scanners.  You'll never need this option
          unless you are doing flex maintenance or development.

     flex also  provides  a  mechanism  for  controlling  options
     within  the  scanner  specification itself, rather than from
     the flex command-line.  This is done  by  including  %option
     directives  in  the  first section of the scanner specifica-
     tion.  You  can  specify  multiple  options  with  a  single
     %option directive, and multiple directives in the first sec-
     tion of your flex input file.

     Most options are given simply as names, optionally  preceded
     by  the word "no" (with no intervening whitespace) to negate


Version 2.5          Last change: April 1995                   36


FLEX(1)                  USER COMMANDS                    FLEX(1)


     their meaning.  A number are equivalent  to  flex  flags  or
     their negation:

         7bit            -7 option
         8bit            -8 option
         align           -Ca option
         backup          -b option
         batch           -B option
         c++             -+ option

         caseful or
         case-sensitive  opposite of -i (default)

         case-insensitive or
         caseless        -i option

         debug           -d option
         default         opposite of -s option
         ecs             -Ce option
         fast            -F option
         full            -f option
         interactive     -I option
         lex-compat      -l option
         meta-ecs        -Cm option
         perf-report     -p option
         read            -Cr option
         stdout          -t option
         verbose         -v option
         warn            opposite of -w option
                         (use "%option nowarn" for -w)

         array           equivalent to "%array"
         pointer         equivalent to "%pointer" (default)

     Some %option's provide features otherwise not available:

     always-interactive
          instructs flex to generate a scanner which always  con-
          siders  its input "interactive".  Normally, on each new
          input file the scanner calls isatty() in an attempt  to
          determine   whether   the  scanner's  input  source  is
          interactive and thus should be read a  character  at  a
          time.   When this option is used, however, then no such
          call is made.

     main directs flex to provide a default  main()  program  for
          the  scanner,  which  simply calls yylex(). This option
          implies noyywrap (see below).

     never-interactive
          instructs flex to generate a scanner which  never  con-
          siders  its input "interactive" (again, no call made to


Version 2.5          Last change: April 1995                   37


FLEX(1)                  USER COMMANDS                    FLEX(1)


          isatty()). This is the opposite of always-interactive.

     stack
          enables the use of start condition  stacks  (see  Start
          Conditions above).

     stdinit
          if set (i.e., %option  stdinit)  initializes  yyin  and
          yyout  to  stdin  and stdout, instead of the default of
          nil.  Some  existing  lex  programs  depend   on   this
          behavior,  even though it is not compliant with ANSI C,
          which does not require stdin and stdout to be  compile-
          time constant.

     yylineno
          directs flex to generate a scanner that  maintains  the
          number  of  the current line read from its input in the
          global variable yylineno. This  option  is  implied  by
          %option lex-compat.

     yywrap
          if unset (i.e., %option noyywrap),  makes  the  scanner
          not  call  yywrap()  upon  an  end-of-file,  but simply
          assume that there are no more files to scan (until  the
          user  points  yyin  at  a  new  file  and calls yylex()
          again).

     flex scans your rule actions to determine  whether  you  use
     the  REJECT  or  yymore()  features.   The reject and yymore
     options are available to override its decision as to whether
     you  use  the options, either by setting them (e.g., %option
     reject) to indicate the feature is indeed used, or unsetting
     them  to  indicate  it  actually  is not used (e.g., %option
     noyymore).

     Three options take string-delimited values, offset with '=':

         %option outfile="ABC"

     is equivalent to -oABC, and

         %option prefix="XYZ"

     is equivalent to -PXYZ. Finally,

         %option yyclass="foo"

     only applies when generating a C++ scanner ( -+ option).  It
     informs  flex  that  you  have  derived foo as a subclass of
     yyFlexLexer, so flex will place your actions in  the  member
     function  foo::yylex()  instead  of yyFlexLexer::yylex(). It
     also generates a yyFlexLexer::yylex() member  function  that


Version 2.5          Last change: April 1995                   38


FLEX(1)                  USER COMMANDS                    FLEX(1)


     emits      a      run-time      error      (by      invoking
     yyFlexLexer::LexerError()) if called.   See  Generating  C++
     Scanners, below, for additional information.

     A number of options are available for lint purists who  want
     to  suppress the appearance of unneeded routines in the gen-
     erated scanner.  Each of  the  following,  if  unset  (e.g.,
     %option  nounput ), results in the corresponding routine not
     appearing in the generated scanner:

         input, unput
         yy_push_state, yy_pop_state, yy_top_state
         yy_scan_buffer, yy_scan_bytes, yy_scan_string

     (though yy_push_state()  and  friends  won't  appear  anyway
     unless you use %option stack).

PERFORMANCE CONSIDERATIONS
     The main design goal of  flex  is  that  it  generate  high-
     performance  scanners.   It  has  been optimized for dealing
     well with large sets of rules.  Aside from  the  effects  on
     scanner  speed  of the table compression -C options outlined
     above, there are a number of options/actions  which  degrade
     performance.  These are, from most expensive to least:

         REJECT
         %option yylineno
         arbitrary trailing context

         pattern sets that require backing up
         %array
         %option interactive
         %option always-interactive

         '^' beginning-of-line operator
         yymore()

     with the first three all being quite expensive and the  last
     two  being  quite  cheap.   Note also that unput() is imple-
     mented as a routine call that potentially does quite  a  bit
     of  work,  while yyless() is a quite-cheap macro; so if just
     putting back some excess text you scanned, use yyless().

     REJECT should be avoided at all costs  when  performance  is
     important.  It is a particularly expensive option.

     Getting rid of backing up is messy and often may be an enor-
     mous  amount  of work for a complicated scanner.  In princi-
     pal,  one  begins  by  using  the  -b  flag  to  generate  a
     lex.backup file.  For example, on the input

         %%


Version 2.5          Last change: April 1995                   39


FLEX(1)                  USER COMMANDS                    FLEX(1)


         foo        return TOK_KEYWORD;
         foobar     return TOK_KEYWORD;

     the file looks like:

         State #6 is non-accepting -
          associated rule line numbers:
                2       3
          out-transitions: [ o ]
          jam-transitions: EOF [ \001-n  p-\177 ]

         State #8 is non-accepting -
          associated rule line numbers:
                3
          out-transitions: [ a ]
          jam-transitions: EOF [ \001-`  b-\177 ]

         State #9 is non-accepting -
          associated rule line numbers:
                3
          out-transitions: [ r ]
          jam-transitions: EOF [ \001-q  s-\177 ]

         Compressed tables always back up.

     The first few lines tell us that there's a scanner state  in
     which  it  can  make  a  transition on an 'o' but not on any
     other character,  and  that  in  that  state  the  currently
     scanned text does not match any rule.  The state occurs when
     trying to match the rules found at lines  2  and  3  in  the
     input  file.  If the scanner is in that state and then reads
     something other than an 'o', it will have to back up to find
     a  rule  which is matched.  With a bit of headscratching one
     can see that this must be the state it's in when it has seen
     "fo".   When  this  has  happened,  if  anything  other than
     another 'o' is seen, the scanner will have  to  back  up  to
     simply match the 'f' (by the default rule).

     The comment regarding State #8 indicates there's  a  problem
     when  "foob"  has  been  scanned.   Indeed, on any character
     other than an 'a', the scanner  will  have  to  back  up  to
     accept  "foo".  Similarly, the comment for State #9 concerns
     when "fooba" has been scanned and an 'r' does not follow.

     The final comment reminds us that there's no point going  to
     all the trouble of removing backing up from the rules unless
     we're using -Cf or -CF, since there's  no  performance  gain
     doing so with compressed scanners.

     The way to remove the backing up is to add "error" rules:

         %%


Version 2.5          Last change: April 1995                   40


FLEX(1)                  USER COMMANDS                    FLEX(1)


         foo         return TOK_KEYWORD;
         foobar      return TOK_KEYWORD;

         fooba       |
         foob        |
         fo          {
                     /* false alarm, not really a keyword */
                     return TOK_ID;
                     }


     Eliminating backing up among a list of keywords can also  be
     done using a "catch-all" rule:

         %%
         foo         return TOK_KEYWORD;
         foobar      return TOK_KEYWORD;

         [a-z]+      return TOK_ID;

     This is usually the best solution when appropriate.

     Backing up messages tend to cascade.  With a complicated set
     of  rules it's not uncommon to get hundreds of messages.  If
     one can decipher them, though, it often only takes  a  dozen
     or so rules to eliminate the backing up (though it's easy to
     make a mistake and have an error rule accidentally  match  a
     valid  token.   A  possible  future  flex feature will be to
     automatically add rules to eliminate backing up).

     It's important to keep in mind that you gain the benefits of
     eliminating  backing up only if you eliminate every instance
     of backing up.  Leaving just one means you gain nothing.

     Variable trailing context (where both the leading and trail-
     ing  parts  do  not  have a fixed length) entails almost the
     same performance loss as  REJECT  (i.e.,  substantial).   So
     when possible a rule like:

         %%
         mouse|rat/(cat|dog)   run();

     is better written:

         %%
         mouse/cat|dog         run();
         rat/cat|dog           run();

     or as

         %%
         mouse|rat/cat         run();


Version 2.5          Last change: April 1995                   41


FLEX(1)                  USER COMMANDS                    FLEX(1)


         mouse|rat/dog         run();

     Note that here the special '|' action does not  provide  any
     savings,  and can even make things worse (see Deficiencies /
     Bugs below).

     Another area where the user can increase a scanner's perfor-
     mance  (and  one that's easier to implement) arises from the
     fact that the longer the  tokens  matched,  the  faster  the
     scanner will run.  This is because with long tokens the pro-
     cessing of most input characters takes place in the  (short)
     inner  scanning  loop, and does not often have to go through
     the additional work of setting up the  scanning  environment
     (e.g.,  yytext)  for  the  action.  Recall the scanner for C
     comments:

         %x comment
         %%
                 int line_num = 1;

         "/*"         BEGIN(comment);

         <comment>[^*\n]*
         <comment>"*"+[^*/\n]*
         <comment>\n             ++line_num;
         <comment>"*"+"/"        BEGIN(INITIAL);

     This could be sped up by writing it as:

         %x comment
         %%
                 int line_num = 1;

         "/*"         BEGIN(comment);

         <comment>[^*\n]*
         <comment>[^*\n]*\n      ++line_num;
         <comment>"*"+[^*/\n]*
         <comment>"*"+[^*/\n]*\n ++line_num;
         <comment>"*"+"/"        BEGIN(INITIAL);

     Now instead of each  newline  requiring  the  processing  of
     another  action,  recognizing  the newlines is "distributed"
     over the other rules to keep the matched  text  as  long  as
     possible.   Note  that  adding  rules does not slow down the
     scanner!  The speed of the scanner  is  independent  of  the
     number  of  rules or (modulo the considerations given at the
     beginning of this section) how  complicated  the  rules  are
     with regard to operators such as '*' and '|'.

     A final example in speeding up a scanner: suppose  you  want
     to  scan through a file containing identifiers and keywords,


Version 2.5          Last change: April 1995                   42


FLEX(1)                  USER COMMANDS                    FLEX(1)


     one per line and with no other  extraneous  characters,  and
     recognize all the keywords.  A natural first approach is:

         %%
         asm      |
         auto     |
         break    |
         ... etc ...
         volatile |
         while    /* it's a keyword */

         .|\n     /* it's not a keyword */

     To eliminate the back-tracking, introduce a catch-all rule:

         %%
         asm      |
         auto     |
         break    |
         ... etc ...
         volatile |
         while    /* it's a keyword */

         [a-z]+   |
         .|\n     /* it's not a keyword */

     Now, if it's guaranteed that there's exactly  one  word  per
     line,  then  we  can reduce the total number of matches by a
     half by merging in the recognition of newlines with that  of
     the other tokens:

         %%
         asm\n    |
         auto\n   |
         break\n  |
         ... etc ...
         volatile\n |
         while\n  /* it's a keyword */

         [a-z]+\n |
         .|\n     /* it's not a keyword */

     One has to be careful here,  as  we  have  now  reintroduced
     backing  up  into the scanner.  In particular, while we know
     that there will never be any characters in the input  stream
     other  than letters or newlines, flex can't figure this out,
     and it will plan for possibly needing to back up when it has
     scanned  a  token like "auto" and then the next character is
     something other than a newline or a letter.   Previously  it
     would  then  just match the "auto" rule and be done, but now
     it has no "auto" rule, only a "auto\n" rule.   To  eliminate
     the possibility of backing up, we could either duplicate all


Version 2.5          Last change: April 1995                   43


FLEX(1)                  USER COMMANDS                    FLEX(1)


     rules but without final newlines, or, since we never  expect
     to  encounter  such  an  input  and therefore don't how it's
     classified, we can introduce one more catch-all  rule,  this
     one which doesn't include a newline:

         %%
         asm\n    |
         auto\n   |
         break\n  |
         ... etc ...
         volatile\n |
         while\n  /* it's a keyword */

         [a-z]+\n |
         [a-z]+   |
         .|\n     /* it's not a keyword */

     Compiled with -Cf, this is about as fast as one  can  get  a
     flex scanner to go for this particular problem.

     A final note: flex is slow when matching NUL's, particularly
     when  a  token  contains multiple NUL's.  It's best to write
     rules which match short amounts of text if it's  anticipated
     that the text will often include NUL's.

     Another final note regarding performance: as mentioned above
     in  the section How the Input is Matched, dynamically resiz-
     ing yytext to accommodate huge  tokens  is  a  slow  process
     because  it presently requires that the (huge) token be res-
     canned from the beginning.  Thus if  performance  is  vital,
     you  should  attempt to match "large" quantities of text but
     not "huge" quantities, where the cutoff between the  two  is
     at about 8K characters/token.

GENERATING C++ SCANNERS
     flex provides two different ways to  generate  scanners  for
     use  with C++.  The first way is to simply compile a scanner
     generated by flex using a C++ compiler instead of a  C  com-
     piler.   You  should  not  encounter any compilations errors
     (please report any you find to the email  address  given  in
     the  Author  section  below).   You can then use C++ code in
     your rule actions instead of C code.  Note that the  default
     input  source  for  your  scanner  remains yyin, and default
     echoing is still done to yyout. Both of these remain FILE  *
     variables and not C++ streams.

     You can also use flex to generate a C++ scanner class, using
     the  -+  option  (or,  equivalently,  %option c++), which is
     automatically specified if the name of the  flex  executable
     ends  in a '+', such as flex++. When using this option, flex
     defaults to generating the scanner  to  the  file  lex.yy.cc
     instead  of  lex.yy.c.  The  generated  scanner includes the


Version 2.5          Last change: April 1995                   44


FLEX(1)                  USER COMMANDS                    FLEX(1)


     header file FlexLexer.h, which defines the interface to  two
     C++ classes.

     The first class, FlexLexer, provides an abstract base  class
     defining  the  general scanner class interface.  It provides
     the following member functions:

     const char* YYText()
          returns the text of the most  recently  matched  token,
          the equivalent of yytext.

     int YYLeng()
          returns the length of the most recently matched  token,
          the equivalent of yyleng.

     int lineno() const
          returns the current  input  line  number  (see  %option
          yylineno), or 1 if %option yylineno was not used.

     void set_debug( int flag )
          sets the debugging flag for the scanner, equivalent  to
          assigning  to  yy_flex_debug  (see  the Options section
          above).  Note that you must  build  the  scanner  using
          %option debug to include debugging information in it.

     int debug() const
          returns the current setting of the debugging flag.

     Also   provided   are   member   functions   equivalent   to
     yy_switch_to_buffer(),  yy_create_buffer() (though the first
     argument is an istream* object pointer  and  not  a  FILE*),
     yy_flush_buffer(),   yy_delete_buffer(),   and   yyrestart()
     (again, the first argument is a istream* object pointer).

     The second class  defined  in  FlexLexer.h  is  yyFlexLexer,
     which  is  derived  from FlexLexer. It defines the following
     additional member functions:

     yyFlexLexer( istream* arg_yyin = 0, ostream* arg_yyout = 0 )
          constructs a yyFlexLexer object using the given streams
          for  input  and  output.  If not specified, the streams
          default to cin and cout, respectively.

     virtual int yylex()
          performs the same role is  yylex()  does  for  ordinary
          flex  scanners:  it  scans  the input stream, consuming
          tokens, until a rule's action returns a value.  If  you
          derive a subclass S from yyFlexLexer and want to access
          the member functions and variables of S inside yylex(),
          then you need to use %option yyclass="S" to inform flex
          that you will be using that subclass instead of yyFlex-
          Lexer.   In   this   case,   rather   than   generating


Version 2.5          Last change: April 1995                   45


FLEX(1)                  USER COMMANDS                    FLEX(1)


          yyFlexLexer::yylex(), flex  generates  S::yylex()  (and
          also  generates a dummy yyFlexLexer::yylex() that calls
          yyFlexLexer::LexerError() if called).

     virtual void switch_streams(istream* new_in = 0,
          ostream* new_out = 0)  reassigns  yyin  to  new_in  (if
          non-nil)  and  yyout  to  new_out (ditto), deleting the
          previous input buffer if yyin is reassigned.

     int yylex( istream* new_in, ostream* new_out = 0 )
          first switches the input  streams  via  switch_streams(
          new_in,  new_out  )  and  then  returns  the  value  of
          yylex().

     In addition, yyFlexLexer  defines  the  following  protected
     virtual  functions which you can redefine in derived classes
     to tailor the scanner:

     virtual int LexerInput( char* buf, int max_size )
          reads up to max_size characters into  buf  and  returns
          the  number  of  characters  read.  To indicate end-of-
          input, return 0 characters.   Note  that  "interactive"
          scanners  (see  the  -B  and -I flags) define the macro
          YY_INTERACTIVE. If you redefine LexerInput()  and  need
          to  take  different actions depending on whether or not
          the scanner might  be  scanning  an  interactive  input
          source,  you can test for the presence of this name via
          #ifdef.

     virtual void LexerOutput( const char* buf, int size )
          writes out size characters from the buffer buf,  which,
          while NUL-terminated, may also contain "internal" NUL's
          if the scanner's rules can match  text  with  NUL's  in
          them.

     virtual void LexerError( const char* msg )
          reports a fatal error message.  The default version  of
          this function writes the message to the stream cerr and
          exits.

     Note that a yyFlexLexer object contains its entire  scanning
     state.   Thus  you  can use such objects to create reentrant
     scanners.  You can instantiate  multiple  instances  of  the
     same  yyFlexLexer  class,  and you can also combine multiple
     C++ scanner classes together in the same program  using  the
     -P option discussed above.

     Finally, note that the %array feature is  not  available  to
     C++ scanner classes; you must use %pointer (the default).

     Here is an example of a simple C++ scanner:


Version 2.5          Last change: April 1995                   46


FLEX(1)                  USER COMMANDS                    FLEX(1)


             // An example of using the flex C++ scanner class.

         %{
         int mylineno = 0;
         %}

         string  \"[^\n"]+\"

         ws      [ \t]+

         alpha   [A-Za-z]
         dig     [0-9]
         name    ({alpha}|{dig}|\$)({alpha}|{dig}|[_.\-/$])*
         num1    [-+]?{dig}+\.?([eE][-+]?{dig}+)?
         num2    [-+]?{dig}*\.{dig}+([eE][-+]?{dig}+)?
         number  {num1}|{num2}

         %%

         {ws}    /* skip blanks and tabs */

         "/*"    {
                 int c;

                 while((c = yyinput()) != 0)
                     {
                     if(c == '\n')
                         ++mylineno;

                     else if(c == '*')
                         {
                         if((c = yyinput()) == '/')
                             break;
                         else
                             unput(c);
                         }
                     }
                 }

         {number}  cout << "number " << YYText() << '\n';

         \n        mylineno++;

         {name}    cout << "name " << YYText() << '\n';

         {string}  cout << "string " << YYText() << '\n';

         %%

         int main( int /* argc */, char** /* argv */ )
             {
             FlexLexer* lexer = new yyFlexLexer;


Version 2.5          Last change: April 1995                   47


FLEX(1)                  USER COMMANDS                    FLEX(1)


             while(lexer->yylex() != 0)
                 ;
             return 0;
             }
     If you want to create multiple  (different)  lexer  classes,
     you  use  the -P flag (or the prefix= option) to rename each
     yyFlexLexer to some other xxFlexLexer. You then can  include
     <FlexLexer.h>  in  your  other sources once per lexer class,
     first renaming yyFlexLexer as follows:

         #undef yyFlexLexer
         #define yyFlexLexer xxFlexLexer
         #include <FlexLexer.h>

         #undef yyFlexLexer
         #define yyFlexLexer zzFlexLexer
         #include <FlexLexer.h>

     if, for example, you used %option  prefix="xx"  for  one  of
     your scanners and %option prefix="zz" for the other.

     IMPORTANT: the present form of the scanning class is experi-
     mental and may change considerably between major releases.

INCOMPATIBILITIES WITH LEX AND POSIX
     flex is a rewrite of the AT&T Unix lex tool (the two  imple-
     mentations  do not share any code, though), with some exten-
     sions and incompatibilities, both of which are of concern to
     those who wish to write scanners acceptable to either imple-
     mentation.  Flex is  fully  compliant  with  the  POSIX  lex
     specification,   except   that   when  using  %pointer  (the
     default), a call to unput() destroys the contents of yytext,
     which is counter to the POSIX specification.

     In this section we discuss all of the known areas of  incom-
     patibility  between flex, AT&T lex, and the POSIX specifica-
     tion.

     flex's -l option turns on  maximum  compatibility  with  the
     original  AT&T  lex  implementation,  at the cost of a major
     loss in the generated scanner's performance.  We note  below
     which incompatibilities can be overcome using the -l option.

     flex is fully compatible with lex with the following  excep-
     tions:

     -    The undocumented lex scanner internal variable yylineno
          is not supported unless -l or %option yylineno is used.

          yylineno should be maintained on  a  per-buffer  basis,
          rather  than  a  per-scanner  (single  global variable)
          basis.


Version 2.5          Last change: April 1995                   48


FLEX(1)                  USER COMMANDS                    FLEX(1)


          yylineno is not part of the POSIX specification.

     -    The input() routine is not redefinable, though  it  may
          be  called  to  read  characters following whatever has
          been matched by a rule.  If input() encounters an  end-
          of-file  the  normal  yywrap()  processing  is done.  A
          ``real'' end-of-file is returned by input() as EOF.

          Input is instead controlled by  defining  the  YY_INPUT
          macro.

          The flex restriction that input() cannot  be  redefined
          is  in  accordance  with the POSIX specification, which
          simply does not specify  any  way  of  controlling  the
          scanner's input other than by making an initial assign-
          ment to yyin.

     -    The unput() routine is not redefinable.  This  restric-
          tion is in accordance with POSIX.

     -    flex scanners are not as reentrant as lex scanners.  In
          particular,  if  you have an interactive scanner and an
          interrupt handler which long-jumps out of the  scanner,
          and  the  scanner is subsequently called again, you may
          get the following message:

              fatal flex scanner internal error--end of buffer missed

          To reenter the scanner, first use

              yyrestart( yyin );

          Note that this call will throw away any buffered input;
          usually  this  isn't  a  problem  with  an  interactive
          scanner.

          Also note that flex C++ scanner classes are  reentrant,
          so  if  using  C++ is an option for you, you should use
          them instead.  See "Generating C++ Scanners" above  for
          details.

     -    output() is not supported.  Output from the ECHO  macro
          is done to the file-pointer yyout (default stdout).

          output() is not part of the POSIX specification.

     -    lex does not support exclusive start  conditions  (%x),
          though they are in the POSIX specification.

     -    When definitions are expanded, flex  encloses  them  in
          parentheses.  With lex, the following:


Version 2.5          Last change: April 1995                   49


FLEX(1)                  USER COMMANDS                    FLEX(1)


              NAME    [A-Z][A-Z0-9]*
              %%
              foo{NAME}?      printf( "Found it\n" );
              %%

          will not match the string "foo" because when the  macro
          is  expanded  the rule is equivalent to "foo[A-Z][A-Z0-
          9]*?" and the precedence is such that the '?' is  asso-
          ciated  with  "[A-Z0-9]*".  With flex, the rule will be
          expanded to "foo([A-Z][A-Z0-9]*)?" and  so  the  string
          "foo" will match.

          Note that if the definition begins with ^ or ends  with
          $  then  it  is not expanded with parentheses, to allow
          these operators to appear in definitions without losing
          their  special  meanings.   But the <s>, /, and <<EOF>>
          operators cannot be used in a flex definition.

          Using -l results in the lex behavior of no  parentheses
          around the definition.

          The POSIX  specification  is  that  the  definition  be
          enclosed in parentheses.

     -    Some implementations of lex allow a  rule's  action  to
          begin  on  a  separate  line, if the rule's pattern has
          trailing whitespace:

              %%
              foo|bar<space here>
                { foobar_action(); }

          flex does not support this feature.

     -    The lex %r (generate a Ratfor scanner)  option  is  not
          supported.  It is not part of the POSIX specification.

     -    After a call to unput(), yytext is undefined until  the
          next  token  is  matched,  unless the scanner was built
          using %array. This is not the  case  with  lex  or  the
          POSIX specification.  The -l option does away with this
          incompatibility.

     -    The precedence of the {} (numeric  range)  operator  is
          different.   lex  interprets  "abc{1,3}" as "match one,
          two, or  three  occurrences  of  'abc'",  whereas  flex
          interprets  it  as "match 'ab' followed by one, two, or
          three occurrences of 'c'".  The latter is in  agreement
          with the POSIX specification.

     -    The precedence of the ^  operator  is  different.   lex
          interprets  "^foo|bar"  as  "match  either 'foo' at the


Version 2.5          Last change: April 1995                   50


FLEX(1)                  USER COMMANDS                    FLEX(1)


          beginning of a line, or 'bar' anywhere",  whereas  flex
          interprets  it  as "match either 'foo' or 'bar' if they
          come at the beginning of a line".   The  latter  is  in
          agreement with the POSIX specification.

     -    The special table-size declarations  such  as  %a  sup-
          ported  by  lex are not required by flex scanners; flex
          ignores them.

     -    The name FLEX_SCANNER is #define'd so scanners  may  be
          written  for use with either flex or lex. Scanners also
          include YY_FLEX_MAJOR_VERSION and YY_FLEX_MINOR_VERSION
          indicating  which version of flex generated the scanner
          (for example, for the 2.5 release, these defines  would
          be 2 and 5 respectively).

     The following flex features are not included in lex  or  the
     POSIX specification:

         C++ scanners
         %option
         start condition scopes
         start condition stacks
         interactive/non-interactive scanners
         yy_scan_string() and friends
         yyterminate()
         yy_set_interactive()
         yy_set_bol()
         YY_AT_BOL()
         <<EOF>>
         <*>
         YY_DECL
         YY_START
         YY_USER_ACTION
         YY_USER_INIT
         #line directives
         %{}'s around actions
         multiple actions on a line

     plus almost all of the flex flags.  The last feature in  the
     list  refers to the fact that with flex you can put multiple
     actions on the same line, separated with semi-colons,  while
     with lex, the following

         foo    handle_foo(); ++num_foos_seen;

     is (rather surprisingly) truncated to

         foo    handle_foo();

     flex does not truncate the action.   Actions  that  are  not
     enclosed  in  braces are simply terminated at the end of the


Version 2.5          Last change: April 1995                   51


FLEX(1)                  USER COMMANDS                    FLEX(1)


     line.

DIAGNOSTICS
     warning, rule cannot be matched  indicates  that  the  given
     rule  cannot  be matched because it follows other rules that
     will always match the same text as it.  For example, in  the
     following  "foo" cannot be matched because it comes after an
     identifier "catch-all" rule:

         [a-z]+    got_identifier();
         foo       got_foo();

     Using REJECT in a scanner suppresses this warning.

     warning, -s option given but default  rule  can  be  matched
     means  that  it  is  possible  (perhaps only in a particular
     start condition) that the default  rule  (match  any  single
     character)  is  the  only  one  that will match a particular
     input.  Since -s was given, presumably this is not intended.

     reject_used_but_not_detected          undefined           or
     yymore_used_but_not_detected  undefined  -  These errors can
     occur at compile time.  They indicate that the scanner  uses
     REJECT  or yymore() but that flex failed to notice the fact,
     meaning that flex scanned the first two sections looking for
     occurrences  of  these  actions  and failed to find any, but
     somehow you snuck some in (via a #include  file,  for  exam-
     ple).   Use  %option reject or %option yymore to indicate to
     flex that you really do use these features.

     flex scanner jammed - a scanner compiled with -s has encoun-
     tered  an  input  string  which wasn't matched by any of its
     rules.  This error can also occur due to internal problems.

     token too large, exceeds YYLMAX - your scanner  uses  %array
     and one of its rules matched a string longer than the YYLMAX
     constant (8K bytes by default).  You can increase the  value
     by  #define'ing  YYLMAX  in  the definitions section of your
     flex input.

     scanner requires -8 flag to use the  character  'x'  -  Your
     scanner specification includes recognizing the 8-bit charac-
     ter 'x' and you did  not  specify  the  -8  flag,  and  your
     scanner  defaulted  to 7-bit because you used the -Cf or -CF
     table compression options.  See the  discussion  of  the  -7
     flag for details.

     flex scanner push-back overflow - you used unput()  to  push
     back  so  much text that the scanner's buffer could not hold
     both the pushed-back text and the current token  in  yytext.
     Ideally  the scanner should dynamically resize the buffer in
     this case, but at present it does not.


Version 2.5          Last change: April 1995                   52


FLEX(1)                  USER COMMANDS                    FLEX(1)


     input buffer overflow, can't enlarge buffer because  scanner
     uses  REJECT  -  the  scanner  was  working  on  matching an
     extremely large token and needed to expand the input buffer.
     This doesn't work with scanners that use REJECT.

     fatal flex scanner internal error--end of  buffer  missed  -
     This  can  occur  in  an  scanner which is reentered after a
     long-jump has jumped out (or over) the scanner's  activation
     frame.  Before reentering the scanner, use:

         yyrestart( yyin );

     or, as noted above, switch to using the C++ scanner class.

     too many start conditions in <> you listed more start condi-
     tions  in a <> construct than exist (so you must have listed
     at least one of them twice).

FILES
     -lfl library with which scanners must be linked.

     lex.yy.c
          generated scanner (called lexyy.c on some systems).

     lex.yy.cc
          generated C++ scanner class, when using -+.

     <FlexLexer.h>
          header file defining the C++ scanner base class,  Flex-
          Lexer, and its derived class, yyFlexLexer.

     flex.skl
          skeleton scanner.  This file is only used when building
          flex, not when flex executes.

     lex.backup
          backing-up information for -b flag (called  lex.bck  on
          some systems).

DEFICIENCIES / BUGS
     Some trailing context patterns cannot  be  properly  matched
     and  generate  warning  messages  ("dangerous  trailing con-
     text").  These are patterns where the ending  of  the  first
     part  of  the rule matches the beginning of the second part,
     such as "zx*/xy*", where the 'x*' matches  the  'x'  at  the
     beginning  of  the  trailing  context.  (Note that the POSIX
     draft states that the text matched by such patterns is unde-
     fined.)

     For some trailing context rules, parts  which  are  actually
     fixed-length  are  not  recognized  as  such, leading to the
     abovementioned performance loss.  In particular, parts using


Version 2.5          Last change: April 1995                   53


FLEX(1)                  USER COMMANDS                    FLEX(1)


     '|'   or  {n}  (such  as  "foo{3}")  are  always  considered
     variable-length.

     Combining trailing context with the special '|'  action  can
     result  in fixed trailing context being turned into the more
     expensive variable trailing context.  For  example,  in  the
     following:

         %%
         abc      |
         xyz/def


     Use of unput() invalidates yytext  and  yyleng,  unless  the
     %array directive or the -l option has been used.

     Pattern-matching  of  NUL's  is  substantially  slower  than
     matching other characters.

     Dynamic resizing of the input buffer is slow, as it  entails
     rescanning  all the text matched so far by the current (gen-
     erally huge) token.

     Due to both buffering of input and  read-ahead,  you  cannot
     intermix  calls to <stdio.h> routines, such as, for example,
     getchar(), with flex rules and  expect  it  to  work.   Call
     input() instead.

     The total table entries listed by the -v flag  excludes  the
     number  of  table  entries needed to determine what rule has
     been matched.  The number of entries is equal to the  number
     of  DFA states if the scanner does not use REJECT, and some-
     what greater than the number of states if it does.

     REJECT cannot be used with the -f or -F options.

     The flex internal algorithms need documentation.

SEE ALSO
     lex(1), yacc(1), sed(1), awk(1).

     John Levine,  Tony  Mason,  and  Doug  Brown,  Lex  &  Yacc,
     O'Reilly and Associates.  Be sure to get the 2nd edition.

     M. E. Lesk and E. Schmidt, LEX - Lexical Analyzer Generator

     Alfred Aho, Ravi Sethi and Jeffrey Ullman, Compilers:  Prin-
     ciples,   Techniques   and   Tools,  Addison-Wesley  (1986).
     Describes  the  pattern-matching  techniques  used  by  flex
     (deterministic finite automata).


Version 2.5          Last change: April 1995                   54


FLEX(1)                  USER COMMANDS                    FLEX(1)


AUTHOR
     Vern Paxson, with the help of many ideas and  much  inspira-
     tion  from Van Jacobson.  Original version by Jef Poskanzer.
     The fast table representation is a partial implementation of
     a  design done by Van Jacobson.  The implementation was done
     by Kevin Gong and Vern Paxson.

     Thanks to the many flex beta-testers, feedbackers, and  con-
     tributors,  especially Francois Pinard, Casey Leedom, Robert
     Abramovitz,  Stan  Adermann,  Terry  Allen,  David   Barker-
     Plummer,  John  Basrai,  Neal  Becker,  Nelson  H.F.  Beebe,
     benson@odi.com, Karl Berry, Peter A. Bigot, Simon Blanchard,
     Keith  Bostic,  Frederic Brehm, Ian Brockbank, Kin Cho, Nick
     Christopher, Brian Clapper, J.T.  Conklin,  Jason  Coughlin,
     Bill  Cox,  Nick  Cropper, Dave Curtis, Scott David Daniels,
     Chris  G.  Demetriou,  Theo  Deraadt,  Mike  Donahue,  Chuck
     Doucette,  Tom  Epperly,  Leo  Eskin,  Chris  Faylor,  Chris
     Flatters, Jon Forrest, Jeffrey Friedl, Joe Gayda,  Kaveh  R.
     Ghazi,  Wolfgang  Glunz, Eric Goldman, Christopher M. Gould,
     Ulrich Grepel, Peer Griebel, Jan  Hajic,  Charles  Hemphill,
     NORO  Hideo,  Jarkko  Hietaniemi, Scott Hofmann, Jeff Honig,
     Dana Hudes, Eric Hughes,  John  Interrante,  Ceriel  Jacobs,
     Michal Jaegermann, Sakari Jalovaara, Jeffrey R. Jones, Henry
     Juengst, Klaus Kaempf, Jonathan I. Kamens, Terrence O  Kane,
     Amir  Katz, ken@ken.hilco.com, Kevin B. Kenny, Steve Kirsch,
     Winfried Koenig, Marq  Kole,  Ronald  Lamprecht,  Greg  Lee,
     Rohan  Lenard, Craig Leres, John Levine, Steve Liddle, David
     Loffredo, Mike Long, Mohamed el Lozy, Brian  Madsen,  Malte,
     Joe Marshall, Bengt Martensson, Chris Metcalf, Luke Mewburn,
     Jim Meyering,  R.  Alexander  Milowski,  Erik  Naggum,  G.T.
     Nicol,  Landon  Noll,  James  Nordby,  Marc  Nozell, Richard
     Ohnemus, Karsten Pahnke, Sven Panne,  Roland  Pesch,  Walter
     Pelissero,  Gaumond  Pierre, Esmond Pitt, Jef Poskanzer, Joe
     Rahmeh, Jarmo Raiha, Frederic Raimbault,  Pat  Rankin,  Rick
     Richardson,  Kevin  Rodgers,  Kai  Uwe  Rommel, Jim Roskind,
     Alberto Santini,  Andreas  Scherer,  Darrell  Schiebel,  Raf
     Schietekat,  Doug  Schmidt,  Philippe  Schnoebelen,  Andreas
     Schwab, Larry Schwimmer, Alex Siegel, Eckehard  Stolz,  Jan-
     Erik  Strvmquist, Mike Stump, Paul Stuart, Dave Tallman, Ian
     Lance Taylor, Chris Thewalt, Richard M. Timoney, Jodi  Tsai,
     Paul  Tuinenga,  Gary  Weik, Frank Whaley, Gerhard Wilhelms,
     Kent Williams, Ken Yap,  Ron  Zellar,  Nathan  Zelle,  David
     Zuhn,  and  those whose names have slipped my marginal mail-
     archiving skills but whose contributions are appreciated all
     the same.

     Thanks to Keith Bostic, Jon  Forrest,  Noah  Friedman,  John
     Gilmore, Craig Leres, John Levine, Bob Mulcahy, G.T.  Nicol,
     Francois Pinard, Rich Salz, and Richard  Stallman  for  help
     with various distribution headaches.


Version 2.5          Last change: April 1995                   55


FLEX(1)                  USER COMMANDS                    FLEX(1)


     Thanks to Esmond Pitt and Earle Horton for  8-bit  character
     support; to Benson Margulies and Fred Burke for C++ support;
     to Kent Williams and Tom Epperly for C++ class  support;  to
     Ove  Ewerlid  for  support  of NUL's; and to Eric Hughes for
     support of multiple buffers.

     This work was primarily done when I was with the  Real  Time
     Systems  Group at the Lawrence Berkeley Laboratory in Berke-
     ley, CA.  Many  thanks  to  all  there  for  the  support  I
     received.

     Send comments to vern@ee.lbl.gov.


Version 2.5          Last change: April 1995                   56