1264790SbaptTom Shields, March 17, 2014 2264790Sbapt 3264790SbaptPARKING LOT ISSUES: 4264790Sbapt------------------- 5264790Sbapt- verify debian packaging still works? 6264790Sbapt 7264790Sbapt- there are no #line directives in y.tab.i, other than those that come 8264790Sbapt from the input file and the skeleton file; to fix this, would need to 9264790Sbapt count output lines in externs_file and add 'write_externs_lineno()' 10264790Sbapt similar to 'write_code_lineno()' 11264790Sbapt- if there are no defined symbols, the .tab.h file isn't empty (weird case, 12264790Sbapt may not be worth fixing) 13264790Sbapt- consider: treat []-actions identical to {}-actions if not processing a 14264790Sbapt backtracking parser (avoids test case error)? 15264790Sbapt 16264790Sbapt 17264790SbaptBTYACC CHANGES CURRENTLY DEFERRED, BY FILE: 18264790Sbapt------------------------------------------- 19264790Sbapt 20264790Sbaptpush.skel 21264790Sbapt- skeleton for a 'push' parser 22264790Sbapt- needs to be upgraded match the structure of yaccpar.skel 23264790Sbapt 24264790Sbaptdefs.h 25264790Sbapt- adopt '%include' changes 26264790Sbapt- adopt '%define'/'%ifdef'/'%endif' 27264790Sbapt- adopt -E flag to print preprocessed grammar to stdout 28264790Sbapt 29264790Sbapterror.c 30264790Sbapt- adopt '%include' changes 31264790Sbapt- NOTE: there is a btyacc change that might be worth adopting in byacc 32264790Sbapt [FileError() refactoring to eliminate duplicated code in most of the 33264790Sbapt error message functions] 34264790Sbapt 35264790Sbaptmain.c 36264790Sbapt- adopt '%define' changes 37264790Sbapt- adopt '-DNAME' command line option to define preprocessor variable NAME 38264790Sbapt- adopt -E flag to print preprocessed grammar to stdout 39264790Sbapt- adopt '-S skeleton_file' command line option to select an alternate parser 40264790Sbapt skeleton file 41264790Sbapt- the skeleton file named by the -S flag is used as provided to open the 42264790Sbapt file; consider a change to this behavior to check whether the named file 43264790Sbapt has a path prefix, and if not, look in 'installation' directory if the 44264790Sbapt file is not found in the working directory 45264790Sbapt 46264790Sbaptoutput.c 47264790Sbapt- adopt '%include' changes 48264790Sbapt 49264790Sbaptreader.c 50264790Sbapt- adopt '%include' changes 51264790Sbapt- adopt '%define'/'%ifdef'/'%endif' changes 52264790Sbapt- adopt -E flag to print preprocessed grammar to stdout 53264790Sbapt- NOTE: there is a btyacc change that might be worth adopting in byacc 54264790Sbapt [copy_string() & copy_comment() refactoring to eliminate duplicated 55264790Sbapt code in copy_text() and copy_union()] 56264790Sbapt 57264790Sbaptwarshall.c 58264790Sbapt- NOTE: there is a btyacc change that might be worth adopting in byacc 59264790Sbapt [shifting 'mask' incrementally rather than literal '1' by a variable 60264790Sbapt amount each time thru the loop] 61264790Sbapt 62264790Sbapt 63264790Sbapt================================================================================ 64264790Sbapt 65264790Sbapt 66264790Sbaptnew files: 67264790Sbapt---------- 68264790Sbapt 69264790Sbaptskel2c 70264790Sbapt- modified from btyacc distribution: don't generate #include defs.h 71264790Sbapt- extended syntax recognized to include '%% insert VERSION here', generating 72264790Sbapt the defines for YYMAJOR, YYMINOR and YYPATCH at that point 73264790Sbapt- made generated tables type 'const char *const' to match skelton.c from 74264790Sbapt byacc-20130925 baseline 75264790Sbapt- added code to append text for write_section() to end of generated skeleton.c 76264790Sbapt- remove conversion of tab to \t in generated skeleton.c 77264790Sbapt- extended syntax recognized to include '%%ifdef', '%%ifndef', '%%else' and 78264790Sbapt '%%endif'; used in yaccpar.skel to bracket code that is specific to 79264790Sbapt backtracking 80264790Sbapt 81264790Sbaptyaccpar.skel.old 82264790Sbapt- created from skeleton.c in byacc-20140101 baseline; use of this skeleton 83264790Sbapt will create a version of skeleton.c that is close to that in the 84264790Sbapt byacc-20140101 baseline 85264790Sbapt- eliminated 'body_3' and 'trailer_2' skeleton segments - no need to generate 86264790Sbapt yyerror() invocation dynamically; YYERROR_CALL() is already generated 87264790Sbapt earlier, and so can be used in the skeleton to simplify 88264790Sbapt- added 'const' to types in '%% tables' section to match what skel2c, 89264790Sbapt start_int_table() and state_str_table() generate 90264790Sbapt- added a few cosmetic changes (e.g., added some additional comments, 91264790Sbapt reworded debugging output to match yaccpar.skel, changed yygrowstack() 92264790Sbapt to return YYENOMEM for 'out of memory' error, rather than -1, to match 93264790Sbapt yaccpar.skel; changed yyparse() return value from 1 to 2 for the 94264790Sbapt 'out of memory' error to match yaccpar.skel) 95264790Sbapt- added '#ifndef'/'#endif' around '#define YYINITSTACKSIZE 200' to allow 96264790Sbapt the value to be changed at compile time 97264790Sbapt- changed 'printf(' to 'fprintf(stderr, '; added stack depth (yydepth) to 98264790Sbapt debugging output from yaccpar.skel 99264790Sbapt- use 'YYINT' rather than 'short' for integer table types 100264790Sbapt 101264790Sbaptyaccpar.skel 102264790Sbapt- renamed from btyaccpa.ske, merged with btyacc-c.ske 103264790Sbapt- modified from btyacc distribution to match the latest byacc-20140101 104264790Sbapt skeleton structure & data structures 105264790Sbapt- make local functions static 106264790Sbapt- change "virtual memory exceeded" to "memory exhausted" for bison 107264790Sbapt compatibility 108264790Sbapt- change debug output generation from printf/puts/putc onto stdout to use 109264790Sbapt fprintf/fputs/fputc onto stderr; include 110264790Sbapt stack depth and whether or not in trial parsing 111264790Sbapt- changed types of generated string tables to be 'const pointer to const char' 112264790Sbapt- check all malloc()/realloc() return values, ensure return value of 113264790Sbapt yyparse() = 2 if parsing failed due to memory exhaustion 114264790Sbapt- change YYDBPR() macro to YYSTYPE_TOSTRING(); define semantics as delivering 115264790Sbapt a char* value representing a semantic value (e.g., yylval or yyval, or the 116264790Sbapt contents of an entry on the semantic stack); additional parameter passed: 117264790Sbapt grammar symbol # (to assist interpretation of semantic value) 118264790Sbapt- change YYPOSN to YYLTYPE and yyposn to yylloc (position corresponding to 119264790Sbapt yylval) for bison compatibility; add yyloc (corresponding to yyval) 120264790Sbapt- move default definition of YYLTYPE into output.c, generating a typedef 121264790Sbapt- add '#if defined(YYLTYPE) || defined(YYLTYPE_IS_DECLARED)'/'#endif' around 122264790Sbapt all lines specific to position processing 123264790Sbapt- add '#if defined(YYDESTRUCT_CALL)'/'#endif' around all lines specific to 124264790Sbapt semantic & position stack processing to reclaim memory associated with 125264790Sbapt discarded symbols 126264790Sbapt- add '%%ifdef YYBTYACC'/'%%endif' around all lines specific to backtrack 127264790Sbapt parsing; converted by skel2c into '#if defined(YYBTYACC)'/'#endif' 128264790Sbapt- distinguish between "yacc stack overflow" and "memory exhausted" situations 129264790Sbapt- consolidated termination cleanup code; introduced yyreturn, set to 2 after 130264790Sbapt labels yyoverflow/yyenomem, set to 1 after label yyabort, set to 0 after 131264790Sbapt label yyaccept; all termination cases jump to label yyreturn, which does 132264790Sbapt any cleanup then returns yyreturn value 133264790Sbapt- replaced YYDELETEVAL & YYDELETEPOSN user-supplied macro capability by 134264790Sbapt implementation of byacc-generated yydestruct() as defined by bison 135264790Sbapt compatible %destructor mechanism 136264790Sbapt- moved invocation of 'YYREDUCEPOSNFUNC' macro to immediately prior to, rather 137264790Sbapt than after, execution of final rule action (so that, at some future 138264790Sbapt date, implementation extensions can be added to enable custom calculation 139264790Sbapt of locations associated with non-terminals within rule actions); deleted 140264790Sbapt unnecessary flag 'reduce_posn'; deleted 'YYCALLREDUCEPOSN' macro; deleted 141264790Sbapt C++ variant of 'YYREDUCEPOSNFUNC' invocation 142264790Sbapt- adopt approach similar to bison for default computation of yyloc; change 143264790Sbapt macro 'YYREDUCEPOSNFUNC' name to 'YYLLOC_DEFAULT' for bison compatibility; 144264790Sbapt added 'yyerror_loc_range[2]' to hold start & end locations for error 145264790Sbapt situations that pop the stack 146264790Sbapt- use 'YYINT' rather than 'short' for integer table types, and for indexing 147264790Sbapt parser tables 148264790Sbapt 149264790Sbaptreadskel.c 150264790Sbapthttp://www.verisign.com/index.html- replaced error() with fprintf() 151264790Sbapt 152264790Sbaptmstring.h 153264790Sbapt- moved contents of mstring.h to defs.h - mstring.h is obsolete 154264790Sbapt 155264790Sbaptmstring.c 156264790Sbapt- replaced include of mstring.h with defs.h 157264790Sbapt- changed 'START' to 'HEAD' to remove conflict with 'START' used for 158264790Sbapt the start symbol defined in defs.h 159264790Sbapt 160264790Sbaptmodified byacc files: 161264790Sbapt--------------------- 162264790Sbapt 163264790Sbaptskeleton.c 164264790Sbapt- skeleton.c is now generated from the appropriate skeleton file by 'skel2c' 165264790Sbapt 166264790Sbaptconfigure.in 167264790Sbapt- added configuration for --enable-btyacc option; if 'yes' add '-DYYBTYACC' 168264790Sbapt to DEFINES in makefile.in; --enable-btyacc defaults to 'no' 169264790Sbapt- added configuration for --with-max-table-size option; if present, 170264790Sbapt overrides the value of MAXTABLE defined in defs.h 171264790Sbapt- regenerate configure using autoconf 172264790Sbapt 173264790Sbaptmakefile.in 174264790Sbapt- added mstring.c to C_FILES 175264790Sbapt- added mstring$o to OBJS 176264790Sbapt- added @DEFINES@ as value of DEFINES make variable 177264790Sbapt- added new make variable SKELETON with value 'yaccpar.skel' 178264790Sbapt- added rule to generate skeleton.c from $(SKELETON), depending on skel2c 179264790Sbapt and makefile 180264790Sbapt- added rm -f skeleton.c distclean rule 181264790Sbapt- moved dependency on makefile from only main$o & skeleton$o to $(OBJS), 182264790Sbapt since if ./configure is run changing, for example, from --enable-btyacc 183264790Sbapt to --disable-btyacc, all files must be recompiled to ensure a clean 184264790Sbapt executable 185264790Sbapt- add @MAXTABLE@ for optional '-DMAXTABLE=nnn' if configured using 186264790Sbapt --with-max-table-size=nnn 187264790Sbapt- changed 'cd test && rn 0f test-*'to 'rm -f $(testdir)/test-*' 188264790Sbapt 189264790Sbapttest/run_test.sh 190264790Sbapt- ??? 191264790Sbapt 192264790Sbapttest/run_make.sh 193264790Sbapt- ??? 194264790Sbapt 195264790Sbaptdefs.h 196264790Sbapt- moved contents of mstring.h to defs.h - mstring.h is obsolete 197264790Sbapt- added <limits.h> to get the various system defined machine limits; 198264790Sbapt changed definitions of MAXCHAR, MAXSHORT, MINSHORT & BITS_PER_WORD to use 199264790Sbapt defines from <limits.h>; changed definitions of BIT and SETBIT to use 200264790Sbapt value of BITS_PER_WORD 201264790Sbapt- added typedef for __compar_fn_t, conditioned on _COMPAR_FN_T being 202264790Sbapt undefined (at least for Mac OSX environment) 203264790Sbapt- adopt new symbol class values ACTION and ARGUMENT 204264790Sbapt- adopt changes/additions used by inherited attribute processing 205264790Sbapt- clean up locations of extern function definitions to match where they 206264790Sbapt actually live in source files 207264790Sbapt- adopt error functions from inherited attribute processing; added new error 208264790Sbapt functions 209264790Sbapt- added keyword code LOCATIONS for %locations 210264790Sbapt- added keyword code DESTRUCTOR for %destructor 211264790Sbapt- added extern decl for 'int locations'; true if %locations present 212264790Sbapt- added extern decl for 'int backtrack'; initialized to 0 (= false), set to 213264790Sbapt 1 (= true) if -B flag is present 214264790Sbapt- added extern decl for 'int destructor'; true if at least one %destructor 215264790Sbapt present in grammar spec file 216264790Sbapt- define 'YYINT' as the smallest C type that can be used to address a 217264790Sbapt table of size 'MAXTABLE'; define 'YYINT' based on the value of 218264790Sbapt 'MAXTABLE' using the standard system type size definitions from <limits.h>; 219264790Sbapt define 'MAXYYINT' and 'MINYYINT' accordingly 220264790Sbapt- change 'Value_t' and 'Index_t' to 'YYINT' from 'short' 221264790Sbapt- allow 'MAXTABLE' to be defined by '-DMAXTABLE=nnn' at compile-time 222264790Sbapt 223264790Sbaptclosure.c 224264790Sbapt- changed print_closure(), print_EFF() and print_first_derives() to 'static'; 225264790Sbapt added fwd declarations 226264790Sbapt- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already 227264790Sbapt used for variables/parameters that were related to variables/parameters 228264790Sbapt declared as 'short' 229264790Sbapt 230264790Sbapterror.c 231264790Sbapt- adopt error functions from inherited attribute processing; added a few 232264790Sbapt additional inherited attribute error functions 233264790Sbapt 234264790Sbaptgraph.c 235264790Sbapt- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already 236264790Sbapt used for variables/parameters that were related to variables/parameters 237264790Sbapt declared as 'short' 238264790Sbapt 239264790Sbaptlalr.c 240264790Sbapt- changed MAXSHORT to MAXYYINT 241264790Sbapt 242264790Sbaptlr0.c 243264790Sbapt- changed MAXSHORT to MAXYYINT 244264790Sbapt- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already 245264790Sbapt used for variables/parameters that were related to variables/parameters 246264790Sbapt declared as 'short' 247264790Sbapt 248264790Sbaptmain.c 249264790Sbapt- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already 250264790Sbapt used for variables/parameters that were related to variables/parameters 251264790Sbapt declared as 'short' 252264790Sbapt 253264790Sbaptmkpar.c 254264790Sbapt- backtracking attempts to resolve shift/reduce and reduce/reduce conflicts 255264790Sbapt 256264790Sbaptoutput.c 257264790Sbapt- generate prefix & YYPREFIX defines into externs file (-i, .tab.i) and 258264790Sbapt code file (-r, .code.c); generate into output file (.tab.c) only if not 259264790Sbapt using -r option; eliminates doubled output of prefix aliases if -r with 260264790Sbapt no -i in y.tab.c and y.code.c or if -r & -i in y.tab.i and y.code.c 261264790Sbapt- changed types of generated string tables to be 'const pointer to const char' 262264790Sbapt- adopt backtracking as an alternative in cases where otherwise we have a 263264790Sbapt conflict in the parsing actions (3, rather than 2, choices) 264264790Sbapt- wrap defines file with (where "yy" is value of 'symbol_prefix') 265264790Sbapt #ifndef __yy_defines_h_ 266264790Sbapt #define _yy_defines_h_ 267264790Sbapt 268264790Sbapt <defines> 269264790Sbapt 270264790Sbapt #endif 271264790Sbapt- avoid writing %%xdecls skeleton section twice if -r used 272264790Sbapt- eliminated 'body_3' and 'trailer_2' skeleton segments - no need to generate 273264790Sbapt yyerror() invocation dynamically; YYERROR_CALL() is already generated 274264790Sbapt earlier, and can be used in the pareser skeleton 275264790Sbapt- if -P flag (pure_parser), add yylloc as 2nd parameter to yylex() 276264790Sbapt (declaration & call) 277264790Sbapt- change YYPOSN to YYLTYPE and yyposn to yylloc (position corresponding to 278264790Sbapt yylval) for bison compatibility; add yyloc (corresponding to yyval) 279264790Sbapt- generate yylloc parameters for yylex & yyerror if %locations present 280264790Sbapt- add location as 1st parameter to declaraion & invocation of yyerror() if 281264790Sbapt %locations present 282264790Sbapt- output backtrack parsing tables if -B flag is present 283264790Sbapt- added generation of yystos[] with output_accessing_symbols() to allow 284264790Sbapt translation from a parser internal state number to the corresponding 285264790Sbapt grammar symbol number [0 .. nsyms) of the accessing symbol of that parser 286264790Sbapt state; used in the generated code for YYDESTRUCT_CALL() & 287264790Sbapt YYSTYPE_TOSTRING() to enable the correct semantic value union tag to be 288264790Sbapt determined when executing the implementation of YYDESTRUCT_CALL() or 289264790Sbapt YYSTYPE_TOSTRING() (similar to yystos[] in bison) 290264790Sbapt- added to output_prefix(): yystos; yycindex & yyctable if compiling 291264790Sbapt backtracking; yyloc & yylloc if %locations used 292264790Sbapt- extended yyname[] to include all grammar symbols, not just the terminal 293264790Sbapt symbols: '$end', 'error', '$accept', all non-terminals, including internally 294264790Sbapt generated non-terminals for embedded actions in rules, and 'illegal-symbol' 295264790Sbapt (which bison spells '$undefined'); '$end' already defined as a symbol 0, 296264790Sbapt rathern than adding 'end-of-file' as the name of symbol 0; added 297264790Sbapt 'illegal-symbol' from byacc-20140101 (NOTE: the comment in the code that 298264790Sbapt says byacc does not predefine '$end' and '$error' is incorrect; however, 299264790Sbapt both bison and byacc spell '$error' as 'error') 300264790Sbapt- added generation of #define YYTRANSLATE() from byacc-20140101, but changed 301264790Sbapt the definition for the undefined symbol case because it is no longer in 302264790Sbapt yyname[YYMAXTOKEN+1] but rather occurs after the last non-terminal symbol; 303264790Sbapt added #define YYUNDFTOKEN to contain the index in yyname of 'illegal-symbol' 304264790Sbapt- generate YYLTYPE in output_ltype() as a struct like for bison rather than 305264790Sbapt using #define in yaccpar.skel 306264790Sbapt- added 'write_code_lineno' invocation at start of 'output_prefix' 307264790Sbapt- added 'write_code_lineno' invocation at start of 'output_pure_parser' 308264790Sbapt- added 'write_code_lineno' invocation prior to generation of #include 309264790Sbapt for externs file 310264790Sbapt- added 'write_code_lineno' invocation after 1st 'write_section(fp, xdecls)' 311264790Sbapt- added '++outline;' prior to output of '#define YYTRANSLATE' - this was 312264790Sbapt actually causing almost all of the invocations of 'write_code_lineno' to 313264790Sbapt put out the correct #line directive 314264790Sbapt- corrected 'write_code_lineno' - the line number in a #line directive is 315264790Sbapt the number of the next line, not the number of the #line line 316264790Sbapt- changed MAXSHORT to MAXYYINT; changed 'high' local static from 'int' to 317264790Sbapt 'long' so that it can get higher than 'MAXYYINT' without machine-dependent 318264790Sbapt behavior; changed related formats from '%d' to '%ld' 319264790Sbapt- generate 'YYINT' rather than 'short' for integer table types 320264790Sbapt- generate YYDESTRUCT_DECL & YYDESTRUCT_CALL macros, similar to YYERROR_DECL 321264790Sbapt and YYERROR_CALL macros, that can be redefined by user, if desired, to add 322264790Sbapt additional parameters to yydestruct() (and even change the 'yydestruct' 323264790Sbapt function name) 324264790Sbapt- if at least one %destructor present, generate yydestruct(); 1st parameter 325264790Sbapt is a string indicating the context in which yydestruct() is invoked 326264790Sbapt (e.g., discarding input token, discarding state on stack, cleanup when 327264790Sbapt aborting); 2nd parameter is the internal grammar symbol number [0..nsyms) 328264790Sbapt of the accessing symbol of the parser state on the top of the stack; 3rd 329264790Sbapt parameter is a pointer to the semantic value to be reclaimed associated 330264790Sbapt with the grammar symbol in the 2nd parameter; if %locations is defined, 331264790Sbapt the 4th parameter is a pointer to the position value to be reclaimed 332264790Sbapt associated with the grammar symbol in the 2nd parameter 333264790Sbapt 334264790Sbaptreader.c 335264790Sbapt- adopt []-actions, similar to {}-actions; {}-actions are only executed when 336264790Sbapt not in trial mode, but []-actions are executed regardless of mode 337264790Sbapt- adopt new symbol class values ACTION and ARGUMENT 338264790Sbapt- adopt inherited attributes (syntax resembles arguments to non-terminal 339264790Sbapt symbols) 340264790Sbapt- adopt keyword table lookup from btyacc, modified to handle equivalence 341264790Sbapt of '-' and '_' in spelling of keywords 342264790Sbapt- adopt refactoring of tag table creation into cache_tag() for use in 343264790Sbapt multiple locations 344264790Sbapt- added new error functions in place of btyacc's generic error() function 345264790Sbapt- changed '0' to 'NULL' for pointer initialization 346264790Sbapt- reworked for-loop at end of get_line (part of DEFERRED '%ifdef/%endif' change) 347264790Sbapt- added %locations directive for bison compatibility to enable position 348264790Sbapt processing 349264790Sbapt- added decl for 'int locations'; true if %locations present 350264790Sbapt- added decl 'int backtrack'; initialized to 0 (= false), set to 351264790Sbapt 1 (= true) if -B flag is present 352264790Sbapt- process %locations if present, set location = 1 353264790Sbapt- only process []-actions and only generate 'if (!yytrial)' prefix for 354264790Sbapt {}-actions if backtracking is enabled 355264790Sbapt- add decl for 'int destructor'; true if at least one %destructor is present 356264790Sbapt- add %destructor directive to enable semantic & position stack processing to 357264790Sbapt reclaim memory associated with discarded symbols 358264790Sbapt- process bison compatible %destructor (set destructor = 1); support @$ in 359264790Sbapt %destructor code to reference the position value if %locations is defined 360264790Sbapt- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already 361264790Sbapt used for variables/parameters that were related to variables/parameters 362264790Sbapt declared as 'short' 363264790Sbapt- if %locations present, support @N and @$ syntax as for bison to reference 364264790Sbapt the locations associated with the N-th rhs symbol and the lhs symbol, 365264790Sbapt respectively 366264790Sbapt 367264790Sbaptsymtab.c 368264790Sbapt- initialize fields added to 'struct bucket' for non-terminal symbol 369264790Sbapt inherited attributes 370264790Sbapt 371264790Sbaptverbose.c 372264790Sbapt- for parse states with conflicts, the contents of the y.output file include 373264790Sbapt the trial shift and/or trial reduce actions 374264790Sbapt- added output to the end of the verbose report showing the correspondance 375264790Sbapt between grammar symbol #, internal parser symbol #, and grammar symbol name 376264790Sbapt- changed 'short' to 'Value_t' (in some instances, 'Value_t' was already 377264790Sbapt used for variables/parameters that were related to variables/parameters 378264790Sbapt declared as 'short' 379264790Sbapt 380264790Sbaptyacc.1 381264790Sbapt- added options 'P', 'V', 'y' and '-o output_file' to the yacc command 382264790Sbapt synopsis (already covered in the description section) 383264790Sbapt- added options 'B', 'D' and 'L' to the yacc command synopsis; added text in 384264790Sbapt the description section 385264790Sbapt- added %locations description to the extensions section 386