ml-yacc/lib/base.sig

51 (* TOKEN: signature revealing the internal structure of a token. This signature
56    The representation of token was very carefully chosen here to allow the
62    type 'a token which functions to construct tokens would create.  A
63    constructor function for a integer token might be
65 	  INT: int * 'a * 'a -> 'a token.
72 	  INT: int * 'a * 'a -> (svalue,'a) token
82         datatype ('a,'b) token = TOKEN of LrTable.term * ('a * 'b * 'b)
83 	val sameToken : ('a,'b) token * ('a,'b) token -> bool
99 		     lexer : ('_b,'_c) Token.token Stream.stream,
120 			     (('_b,'_c) Token.token Stream.stream)
124    output will match.  The user is responsible for declaring type token,
127    Note that type token is abstract in the lexer.  This allows SML-Yacc to
129    treats the type token abstractly.  Lexers that are functors parametrized by
138 	        type ('a,'b) token
143          (UserDeclarations.svalue,UserDeclarations.pos) UserDeclarations.token
147    also take an argument before yielding a function from unit to a token
154 	        type ('a,'b) token
160          (UserDeclarations.svalue,UserDeclarations.pos) UserDeclarations.token
259 			 (svalue,pos) Token.token Stream.stream
266 	val parse : int * ((svalue,pos) Token.token Stream.stream) *
268 				result * (svalue,pos) Token.token Stream.stream
270 	val sameToken : (svalue,pos) Token.token * (svalue,pos) Token.token ->
291 			 (svalue,pos) Token.token Stream.stream
292 	val parse : int * ((svalue,pos) Token.token Stream.stream) *
294 				result * (svalue,pos) Token.token Stream.stream
296 	val sameToken : (svalue,pos) Token.token * (svalue,pos) Token.token ->