| NEWS (2258) | NEWS (16514) |
|---|---|
| 1Changes between release 2.5.3 (29May96) and release 2.5.2: 2 3 - Some serious bugs in yymore() have been fixed. In particular, 4 when using AT&T-lex-compatibility or %array, you can intermix 5 calls to input(), unput(), and yymore(). (This still doesn't 6 work for %pointer, and isn't likely to in the future.) 7 8 - A bug in handling NUL's in the input stream of scanners using 9 REJECT has been fixed. 10 11 - The default main() in libfl.a now repeatedly calls yylex() until 12 it returns 0, rather than just calling it once. 13 14 - Minor tweak for Windows NT Makefile, MISC/NT/Makefile. 15 16 17Changes between release 2.5.2 (25Apr95) and release 2.5.1: 18 19 - The --prefix configuration option now works. 20 21 - A bug that completely broke the "-Cf" table compression 22 option has been fixed. 23 24 - A major headache involving "const" declarators and Solaris 25 systems has been fixed. 26 27 - An octal escape sequence in a flex regular expression must 28 now contain only the digits 0-7. 29 30 - You can now use "--" on the flex command line to mark the 31 end of flex options. 32 33 - You can now specify the filename '-' as a synonym for stdin. 34 35 - By default, the scanners generated by flex no longer 36 statically initialize yyin and yyout to stdin and stdout. 37 This change is necessary because in some ANSI environments, 38 stdin and stdout are not compile-time constant. You can 39 force the initialization using "%option stdinit" in the first 40 section of your flex input. 41 42 - "%option nounput" now correctly omits the unput() routine 43 from the output. 44 45 - "make clean" now removes config.log, config.cache, and the 46 flex binary. The fact that it removes the flex binary means 47 you should take care if making changes to scan.l, to make 48 sure you don't wind up in a bootstrap problem. 49 50 - In general, the Makefile has been reworked somewhat (thanks 51 to Francois Pinard) for added flexibility - more changes will 52 follow in subsequent releases. 53 54 - The .texi and .info files in MISC/texinfo/ have been updated, 55 thanks also to Francois Pinard. 56 57 - The FlexLexer::yylex(istream* new_in, ostream* new_out) method 58 now does not have a default for the first argument, to disambiguate 59 it from FlexLexer::yylex(). 60 61 - A bug in destructing a FlexLexer object before doing any scanning 62 with it has been fixed. 63 64 - A problem with including FlexLexer.h multiple times has been fixed. 65 66 - The alloca() chud necessary to accommodate bison has grown 67 even uglier, but hopefully more correct. 68 69 - A portability tweak has been added to accommodate compilers that 70 use char* generic pointers. 71 72 - EBCDIC contact information in the file MISC/EBCDIC has been updated. 73 74 - An OS/2 Makefile and config.h for flex 2.5 is now available in 75 MISC/OS2/, contributed by Kai Uwe Rommel. 76 77 - The descrip.mms file for building flex under VMS has been updated, 78 thanks to Pat Rankin. 79 80 - The notes on building flex for the Amiga have been updated for 81 flex 2.5, contributed by Andreas Scherer. 82 83 84Changes between release 2.5.1 (28Mar95) and release 2.4.7: 85 86 - A new concept of "start condition" scope has been introduced. 87 A start condition scope is begun with: 88 89 <SCs>{ 90 91 where SCs is a list of one or more start conditions. Inside 92 the start condition scope, every rule automatically has the 93 prefix <SCs> applied to it, until a '}' which matches the 94 initial '{'. So, for example: 95 96 <ESC>{ 97 "\\n" return '\n'; 98 "\\r" return '\r'; 99 "\\f" return '\f'; 100 "\\0" return '\0'; 101 } 102 103 is equivalent to: 104 105 <ESC>"\\n" return '\n'; 106 <ESC>"\\r" return '\r'; 107 <ESC>"\\f" return '\f'; 108 <ESC>"\\0" return '\0'; 109 110 As indicated in this example, rules inside start condition scopes 111 (and any rule, actually, other than the first) can be indented, 112 to better show the extent of the scope. 113 114 Start condition scopes may be nested. 115 116 - The new %option directive can be used in the first section of 117 a flex scanner to control scanner-generation options. Most 118 options are given simply as names, optionally preceded by the 119 word "no" (with no intervening whitespace) to negate their 120 meaning. Some are equivalent to flex flags, so putting them 121 in your scanner source is equivalent to always specifying 122 the flag (%option's take precedence over flags): 123 124 7bit -7 option 125 8bit -8 option 126 align -Ca option 127 backup -b option 128 batch -B option 129 c++ -+ option 130 caseful opposite of -i option (caseful is the default); 131 case-sensitive same as above 132 caseless -i option; 133 case-insensitive same as above 134 debug -d option 135 default opposite of -s option 136 ecs -Ce option 137 fast -F option 138 full -f option 139 interactive -I option 140 lex-compat -l option 141 meta-ecs -Cm option 142 perf-report -p option 143 read -Cr option 144 stdout -t option 145 verbose -v option 146 warn opposite of -w option (so use "%option nowarn" for -w) 147 148 array equivalent to "%array" 149 pointer equivalent to "%pointer" (default) 150 151 Some provide new features: 152 153 always-interactive generate a scanner which always 154 considers its input "interactive" (no call to isatty() 155 will be made when the scanner runs) 156 main supply a main program for the scanner, which 157 simply calls yylex(). Implies %option noyywrap. 158 never-interactive generate a scanner which never 159 considers its input "interactive" (no call to isatty() 160 will be made when the scanner runs) 161 stack if set, enable start condition stacks (see below) 162 stdinit if unset ("%option nostdinit"), initialize yyin 163 and yyout statically to nil FILE* pointers, instead 164 of stdin and stdout 165 yylineno if set, keep track of the current line 166 number in global yylineno (this option is expensive 167 in terms of performance). The line number is available 168 to C++ scanning objects via the new member function 169 lineno(). 170 yywrap if unset ("%option noyywrap"), scanner does not 171 call yywrap() upon EOF but simply assumes there 172 are no more files to scan 173 174 Flex scans your rule actions to determine whether you use the 175 REJECT or yymore features (this is not new). Two %options can be 176 used to override its decision, either by setting them to indicate 177 the feature is indeed used, or unsetting them to indicate it 178 actually is not used: 179 180 reject 181 yymore 182 183 Three %option's take string-delimited values, offset with '=': 184 185 outfile="<name>" equivalent to -o<name> 186 prefix="<name>" equivalent to -P<name> 187 yyclass="<name>" set the name of the C++ scanning class 188 (see below) 189 190 A number of %option's are available for lint purists who 191 want to suppress the appearance of unneeded routines in 192 the generated scanner. Each of the following, if unset, 193 results in the corresponding routine not appearing in the 194 generated scanner: 195 196 input, unput 197 yy_push_state, yy_pop_state, yy_top_state 198 yy_scan_buffer, yy_scan_bytes, yy_scan_string 199 200 You can specify multiple options with a single %option directive, 201 and multiple directives in the first section of your flex input file. 202 203 - The new function: 204 205 YY_BUFFER_STATE yy_scan_string( const char *str ) 206 207 returns a YY_BUFFER_STATE (which also becomes the current input 208 buffer) for scanning the given string, which occurs starting 209 with the next call to yylex(). The string must be NUL-terminated. 210 A related function: 211 212 YY_BUFFER_STATE yy_scan_bytes( const char *bytes, int len ) 213 214 creates a buffer for scanning "len" bytes (including possibly NUL's) 215 starting at location "bytes". 216 217 Note that both of these functions create and scan a *copy* of 218 the string/bytes. (This may be desirable, since yylex() modifies 219 the contents of the buffer it is scanning.) You can avoid the 220 copy by using: 221 222 YY_BUFFER_STATE yy_scan_buffer( char *base, yy_size_t size ) 223 224 which scans in place the buffer starting at "base", consisting 225 of "size" bytes, the last two bytes of which *must* be 226 YY_END_OF_BUFFER_CHAR (these bytes are not scanned; thus, scanning 227 consists of base[0] through base[size-2], inclusive). If you 228 fail to set up "base" in this manner, yy_scan_buffer returns a 229 nil pointer instead of creating a new input buffer. 230 231 The type yy_size_t is an integral type to which you can cast 232 an integer expression reflecting the size of the buffer. 233 234 - Three new routines are available for manipulating stacks of 235 start conditions: 236 237 void yy_push_state( int new_state ) 238 239 pushes the current start condition onto the top of the stack 240 and BEGIN's "new_state" (recall that start condition names are 241 also integers). 242 243 void yy_pop_state() 244 245 pops the top of the stack and BEGIN's to it, and 246 247 int yy_top_state() 248 249 returns the top of the stack without altering the stack's 250 contents. 251 252 The start condition stack grows dynamically and so has no built-in 253 size limitation. If memory is exhausted, program execution 254 is aborted. 255 256 To use start condition stacks, your scanner must include 257 a "%option stack" directive. 258 259 - flex now supports POSIX character class expressions. These 260 are expressions enclosed inside "[:" and ":]" delimiters (which 261 themselves must appear between the '[' and ']' of a character 262 class; other elements may occur inside the character class, too). 263 The expressions flex recognizes are: 264 265 [:alnum:] [:alpha:] [:blank:] [:cntrl:] [:digit:] [:graph:] 266 [:lower:] [:print:] [:punct:] [:space:] [:upper:] [:xdigit:] 267 268 These expressions all designate a set of characters equivalent to 269 the corresponding isXXX function (for example, [:alnum:] designates 270 those characters for which isalnum() returns true - i.e., any 271 alphabetic or numeric). Some systems don't provide isblank(), 272 so flex defines [:blank:] as a blank or a tab. 273 274 For example, the following character classes are all equivalent: 275 276 [[:alnum:]] 277 [[:alpha:][:digit:] 278 [[:alpha:]0-9] 279 [a-zA-Z0-9] 280 281 If your scanner is case-insensitive (-i flag), then [:upper:] 282 and [:lower:] are equivalent to [:alpha:]. 283 284 - The promised rewrite of the C++ FlexLexer class has not yet 285 been done. Support for FlexLexer is limited at the moment to 286 fixing show-stopper bugs, so, for example, the new functions 287 yy_scan_string() & friends are not available to FlexLexer 288 objects. 289 290 - The new macro 291 292 yy_set_interactive(is_interactive) 293 294 can be used to control whether the current buffer is considered 295 "interactive". An interactive buffer is processed more slowly, 296 but must be used when the scanner's input source is indeed 297 interactive to avoid problems due to waiting to fill buffers 298 (see the discussion of the -I flag in flex.1). A non-zero value 299 in the macro invocation marks the buffer as interactive, a zero 300 value as non-interactive. Note that use of this macro overrides 301 "%option always-interactive" or "%option never-interactive". 302 303 yy_set_interactive() must be invoked prior to beginning to 304 scan the buffer. 305 306 - The new macro 307 308 yy_set_bol(at_bol) 309 310 can be used to control whether the current buffer's scanning 311 context for the next token match is done as though at the 312 beginning of a line (non-zero macro argument; makes '^' anchored 313 rules active) or not at the beginning of a line (zero argument, 314 '^' rules inactive). 315 316 - Related to this change, the mechanism for determining when a scan is 317 starting at the beginning of a line has changed. It used to be 318 that '^' was active iff the character prior to that at which the 319 scan started was a newline. The mechanism now is that '^' is 320 active iff the last token ended in a newline (or the last call to 321 input() returned a newline). For most users, the difference in 322 mechanisms is negligible. Where it will make a difference, 323 however, is if unput() or yyless() is used to alter the input 324 stream. When in doubt, use yy_set_bol(). 325 326 - The new beginning-of-line mechanism involved changing some fairly 327 twisted code, so it may have introduced bugs - beware ... 328 329 - The macro YY_AT_BOL() returns true if the next token scanned from 330 the current buffer will have '^' rules active, false otherwise. 331 332 - The new function 333 334 void yy_flush_buffer( struct yy_buffer_state* b ) 335 336 flushes the contents of the current buffer (i.e., next time 337 the scanner attempts to match a token using b as the current 338 buffer, it will begin by invoking YY_INPUT to fill the buffer). 339 This routine is also available to C++ scanners (unlike some 340 of the other new routines). 341 342 The related macro 343 344 YY_FLUSH_BUFFER 345 346 flushes the contents of the current buffer. 347 348 - A new "-ooutput" option writes the generated scanner to "output". 349 If used with -t, the scanner is still written to stdout, but 350 its internal #line directives (see previous item) use "output". 351 352 - Flex now generates #line directives relating the code it 353 produces to the output file; this means that error messages 354 in the flex-generated code should be correctly pinpointed. 355 356 - When generating #line directives, filenames with embedded '\'s 357 have those characters escaped (i.e., turned into '\\'). This 358 feature helps with reporting filenames for some MS-DOS and OS/2 359 systems. 360 361 - The FlexLexer class includes two new public member functions: 362 363 virtual void switch_streams( istream* new_in = 0, 364 ostream* new_out = 0 ) 365 366 reassigns yyin to new_in (if non-nil) and yyout to new_out 367 (ditto), deleting the previous input buffer if yyin is 368 reassigned. It is used by: 369 370 int yylex( istream* new_in = 0, ostream* new_out = 0 ) 371 372 which first calls switch_streams() and then returns the value 373 of calling yylex(). 374 375 - C++ scanners now have yy_flex_debug as a member variable of 376 FlexLexer rather than a global, and member functions for testing 377 and setting it. 378 379 - When generating a C++ scanning class, you can now use 380 381 %option yyclass="foo" 382 383 to inform flex that you have derived "foo" as a subclass of 384 yyFlexLexer, so flex will place your actions in the member 385 function foo::yylex() instead of yyFlexLexer::yylex(). It also 386 generates a yyFlexLexer::yylex() member function that generates a 387 run-time error if called (by invoking yyFlexLexer::LexerError()). 388 This feature is necessary if your subclass "foo" introduces some 389 additional member functions or variables that you need to access 390 from yylex(). 391 392 - Current texinfo files in MISC/texinfo, contributed by Francois 393 Pinard. 394 395 - You can now change the name "flex" to something else (e.g., "lex") 396 by redefining $(FLEX) in the Makefile. 397 398 - Two bugs (one serious) that could cause "bigcheck" to fail have 399 been fixed. 400 401 - A number of portability/configuration changes have been made 402 for easier portability. 403 404 - You can use "YYSTATE" in your scanner as an alias for YY_START 405 (for AT&T lex compatibility). 406 407 - input() now maintains yylineno. 408 409 - input() no longer trashes yytext. 410 411 - interactive scanners now read characters in YY_INPUT up to a 412 newline, a large performance gain. 413 414 - C++ scanner objects now work with the -P option. You include 415 <FlexLexer.h> once per scanner - see comments in <FlexLexer.h> 416 (or flex.1) for details. 417 418 - C++ FlexLexer objects now use the "cerr" stream to report -d output 419 instead of stdio. 420 421 - The -c flag now has its full glorious POSIX interpretation (do 422 nothing), rather than being interpreted as an old-style -C flag. 423 424 - Scanners generated by flex now include two #define's giving 425 the major and minor version numbers (YY_FLEX_MAJOR_VERSION, 426 YY_FLEX_MINOR_VERSION). These can then be tested to see 427 whether certain flex features are available. 428 429 - Scanners generated using -l lex compatibility now have the symbol 430 YY_FLEX_LEX_COMPAT #define'd. 431 432 - When initializing (i.e., yy_init is non-zero on entry to yylex()), 433 generated scanners now set yy_init to zero before executing 434 YY_USER_INIT. This means that you can set yy_init back to a 435 non-zero value in YY_USER_INIT if you need the scanner to be 436 reinitialized on the next call. 437 438 - You can now use "#line" directives in the first section of your 439 scanner specification. 440 441 - When generating full-table scanners (-Cf), flex now puts braces 442 around each row of the 2-d array initialization, to silence warnings 443 on over-zealous compilers. 444 445 - Improved support for MS-DOS. The flex sources have been successfully 446 built, unmodified, for Borland 4.02 (all that's required is a 447 Borland Makefile and config.h file, which are supplied in 448 MISC/Borland - contributed by Terrence O Kane). 449 450 - Improved support for Macintosh using Think C - the sources should 451 build for this platform "out of the box". Contributed by Scott 452 Hofmann. 453 454 - Improved support for VMS, in MISC/VMS/, contributed by Pat Rankin. 455 456 - Support for the Amiga, in MISC/Amiga/, contributed by Andreas 457 Scherer. Note that the contributed files were developed for 458 flex 2.4 and have not been tested with flex 2.5. 459 460 - Some notes on support for the NeXT, in MISC/NeXT, contributed 461 by Raf Schietekat. 462 463 - The MISC/ directory now includes a preformatted version of flex.1 464 in flex.man, and pre-yacc'd versions of parse.y in parse.{c,h}. 465 466 - The flex.1 and flexdoc.1 manual pages have been merged. There 467 is now just one document, flex.1, which includes an overview 468 at the beginning to help you find the section you need. 469 470 - Documentation now clarifies that start conditions persist across 471 switches to new input files or different input buffers. If you 472 want to e.g., return to INITIAL, you must explicitly do so. 473 474 - The "Performance Considerations" section of the manual has been 475 updated. 476 477 - Documented the "yy_act" variable, which when YY_USER_ACTION is 478 invoked holds the number of the matched rule, and added an 479 example of using yy_act to profile how often each rule is matched. 480 481 - Added YY_NUM_RULES, a definition that gives the total number 482 of rules in the file, including the default rule (even if you 483 use -s). 484 485 - Documentation now clarifies that you can pass a nil FILE* pointer 486 to yy_create_buffer() or yyrestart() if you've arrange YY_INPUT 487 to not need yyin. 488 489 - Documentation now clarifies that YY_BUFFER_STATE is a pointer to 490 an opaque "struct yy_buffer_state". 491 492 - Documentation now stresses that you gain the benefits of removing 493 backing-up states only if you remove *all* of them. 494 495 - Documentation now points out that traditional lex allows you 496 to put the action on a separate line from the rule pattern if 497 the pattern has trailing whitespace (ugh!), but flex doesn't 498 support this. 499 500 - A broken example in documentation of the difference between 501 inclusive and exclusive start conditions is now fixed. 502 503 - Usage (-h) report now goes to stdout. 504 505 - Version (-V) info now goes to stdout. 506 507 - More #ifdef chud has been added to the parser in attempt to 508 deal with bison's use of alloca(). 509 510 - "make clean" no longer deletes emacs backup files (*~). 511 512 - Some memory leaks have been fixed. 513 514 - A bug was fixed in which dynamically-expanded buffers were 515 reallocated a couple of bytes too small. 516 517 - A bug was fixed which could cause flex to read and write beyond 518 the end of the input buffer. 519 520 - -S will not be going away. 521 522 |
|
| 1Changes between release 2.4.7 (03Aug94) and release 2.4.6: 2 3 - Fixed serious bug in reading multiple files. 4 5 - Fixed bug in scanning NUL's. 6 7 - Fixed bug in input() returning 8-bit characters. 8 --- 10 unchanged lines hidden (view full) --- 19 - Linking with -lfl no longer required if your program includes 20 its own yywrap() and main() functions. (This change will cause 21 problems if you have a non-ANSI compiler on a system for which 22 sizeof(int) != sizeof(void*) or sizeof(int) != sizeof(size_t).) 23 24 - The use of 'extern "C++"' in FlexLexer.h has been modified to 25 get around an incompatibility with g++'s header files. 26 | 523Changes between release 2.4.7 (03Aug94) and release 2.4.6: 524 525 - Fixed serious bug in reading multiple files. 526 527 - Fixed bug in scanning NUL's. 528 529 - Fixed bug in input() returning 8-bit characters. 530 --- 10 unchanged lines hidden (view full) --- 541 - Linking with -lfl no longer required if your program includes 542 its own yywrap() and main() functions. (This change will cause 543 problems if you have a non-ANSI compiler on a system for which 544 sizeof(int) != sizeof(void*) or sizeof(int) != sizeof(size_t).) 545 546 - The use of 'extern "C++"' in FlexLexer.h has been modified to 547 get around an incompatibility with g++'s header files. 548 |
| 27 | |
| 28Changes between release 2.4.5 (11Dec93) and release 2.4.4: 29 30 - Fixed bug breaking C++ scanners that use REJECT or variable 31 trailing context. 32 33 - Fixed serious input problem for interactive scanners on 34 systems for which char is unsigned. 35 --- 255 unchanged lines hidden (view full) --- 291 292 - yyless() can now be used in the third (user action) section 293 of a scanner specification, thanks to Ceriel Jacobs. yyless() 294 remains a macro and cannot be used outside of the scanner source. 295 296 - The skeleton file is no longer opened at run-time, but instead 297 compiled into a large string array (thanks to John Gilmore and 298 friends at Cygnus). You can still use the -S flag to point flex | 549Changes between release 2.4.5 (11Dec93) and release 2.4.4: 550 551 - Fixed bug breaking C++ scanners that use REJECT or variable 552 trailing context. 553 554 - Fixed serious input problem for interactive scanners on 555 systems for which char is unsigned. 556 --- 255 unchanged lines hidden (view full) --- 812 813 - yyless() can now be used in the third (user action) section 814 of a scanner specification, thanks to Ceriel Jacobs. yyless() 815 remains a macro and cannot be used outside of the scanner source. 816 817 - The skeleton file is no longer opened at run-time, but instead 818 compiled into a large string array (thanks to John Gilmore and 819 friends at Cygnus). You can still use the -S flag to point flex |
| 299 at a different skeleton file, though if you use this option let 300 me know, as I plan to otherwise do away with -S in the near 301 future. | 820 at a different skeleton file. |
| 302 303 - flex no longer uses a temporary file to store the scanner's 304 actions. 305 306 - A number of changes have been made to decrease porting headaches. 307 In particular, flex no longer uses memset() or ctime(), and 308 provides a single simple mechanism for dealing with C compilers 309 that still define malloc() as returning char* instead of void*. --- 394 unchanged lines hidden --- | 821 822 - flex no longer uses a temporary file to store the scanner's 823 actions. 824 825 - A number of changes have been made to decrease porting headaches. 826 In particular, flex no longer uses memset() or ctime(), and 827 provides a single simple mechanism for dealing with C compilers 828 that still define malloc() as returning char* instead of void*. --- 394 unchanged lines hidden --- |