--- /dev/null
+CHANGES FROM 1.31
+
+This file contains the migration of PCCTS from 1.31 in the order that
+changes were made. 1.32b7 is the last beta before full 1.32.
+Terence Parr, Parr Research Corporation 1995.
+
+
+======================================================================
+1.32b1
+Added Russell Quong to banner, changed banner for output slightly
+Fixed it so that you have before / after actions for C++ in class def
+Fixed bug in optimizer that made it sometimes forget to set internal
+ token pointers. Only showed up when a {...} was in the "wrong spot".
+
+======================================================================
+1.32b2
+Added fixes by Dave Seidel for PC compilers in 32 bit mode (config.h
+and set.h).
+
+======================================================================
+1.32b3
+Fixed hideous bug in code generator for wildcard and for ~token op.
+
+from Dave Seidel
+
+ Added pcnames.bat
+ 1. in antlr/main.c: change strcasecmp() to stricmp()
+
+ 2. in dlg/output.c: use DLEXER_C instead on "DLexer.C"
+
+ 3. in h/PBlackBox.h: use <iostream.h> instead of <stream.h>
+
+======================================================================
+1.32b4
+When the -ft option was used, any path prefix screwed up
+the gate on the .h files
+
+Fixed yet another bug due to the optimizer.
+
+The exception handling thing was a bit wacko:
+
+a : ( A B )? A B
+ | A C
+ ;
+ exception ...
+
+caused an exception if "A C" was the input. In other words,
+it found that A C didn't match the (A B)? pred and caused
+an exception rather than trying the next alt. All I did
+was to change the zzmatch_wsig() macros.
+
+Fixed some problems in gen.c relating to the name of token
+class bit sets in the output.
+
+Added the tremendously cool generalized predicate. For the
+moment, I'll give this bried description.
+
+a : <<predicate>>? blah
+ | foo
+ ;
+
+This implies that (assuming blah and foo are syntactically
+ambiguous) "predicate" indicates the semantic validity of
+applying "blah". If "predicate" is false, "foo" is attempted.
+
+Previously, you had to say:
+
+a : <<LA(1)==ID ? predicate : 1>>? ID
+ | ID
+ ;
+
+Now, you can simply use "predicate" without the ?: operator
+if you turn on ANTLR command line option: "-prc on". This
+tells ANTLR to compute that all by itself. It computes n
+tokens of lookahead where LT(n) or LATEXT(n) is the farthest
+ahead you look.
+
+If you give a predicate using "-prc on" that is followed
+by a construct that can recognize more than one n-sequence,
+you will get a warning from ANTLR. For example,
+
+a : <<isTypeName(LT(1)->getText())>>? (ID|INT)
+ ;
+
+This is wrong because the predicate will be applied to INTs
+as well as ID's. You should use this syntax to make
+the predicate more specific:
+
+a : (ID)? => <<isTypeName(LT(1)->getText())>>? (ID|INT)
+ ;
+
+which says "don't apply the predicate unless ID is the
+current lookahead context".
+
+You cannot currently have anything in the "(context)? =>"
+except sequences such as:
+
+( LPAREN ID | LPAREN SCOPE )? => <<pred>>?
+
+I haven't tested this THAT much, but it does work for the
+C++ grammar.
+
+======================================================================
+1.32b5
+
+Added getLine() to the ANTLRTokenBase and DLGBasedToken classes
+left line() for backward compatibility.
+----
+Removed SORCERER_TRANSFORM from the ast.h stuff.
+-------
+Fixed bug in code gen of ANTLR such that nested syn preds work more
+efficiently now. The ANTLRTokenBuffer was getting very large
+with nested predicates.
+------
+Memory leak is now gone from ANTLRTokenBuf; all tokens are deleted.
+For backward compatibility reasons, you have to say parser->deleteTokens()
+or mytokenbuffer->deleteTokens() but later it will be the default mode.
+Say this after the parser is constructed. E.g.,
+
+ ParserBlackBox<DLGLexer, MyParser, ANTLRToken> p(stdin);
+ p.parser()->deleteTokens();
+ p.parser()->start_symbol();
+
+
+==============================
+1.32b6
+
+Changed so that deleteTokens() will do a delete ((ANTLRTokenBase *))
+on the ptr. This gets the virtual destructor.
+
+Fixed some weird things in the C++ header files (a few return types).
+
+Made the AST routines correspond to the book and SORCERER stuff.
+
+New token stuff: See testcpp/14/test.g
+
+ANTLR accepts a #pragma gc_tokens which says
+[1] Generate label = copy(LT(1)) instead of label=LT(1) for
+ all labeled token references.
+[2] User now has to define ANTLRTokenPtr (as a class or a typedef
+ to just a pointer) as well as the ANTLRToken class itself.
+ See the example.
+
+To delete tokens in token buffer, use deleteTokens() message on parser.
+
+ All tokens that fall off the ANTLRTokenBuffer get deleted
+ which is what currently happens when deleteTokens() message
+ has been sent to token buffer.
+
+We always generate ANTLRTokenPtr instead of 'ANTLRToken *' now.
+Then if no pragma set, ANTLR generates a
+
+ class ANTLRToken;
+ typedef ANTLRToken *ANTLRTokenPtr;
+
+in each file.
+
+Made a warning for x:rule_ref <<$x>>; still no warning for $i's, however.
+class BB {
+
+a : x:b y:A <<$x
+$y>>
+ ;
+
+b : B;
+
+}
+generates
+Antlr parser generator Version 1.32b6 1989-1995
+test.g, line 3: error: There are no token ptrs for rule references: '$x'
+
+===================
+1.32b7:
+
+[With respect to token object garbage collection (GC), 1.32b7
+ backtracks from 1.32b6, but results in better and less intrusive GC.
+ This is the last beta version before full 1.32.]
+
+BIGGEST CHANGES:
+
+o The "#pragma gc_tokens" is no longer used.
+
+o .C files are now .cpp files (hence, makefiles will have to
+ be changed; or you can rerun genmk). This is a good move,
+ but causes some backward incompatibility problems. You can
+ avoid this by changing CPP_FILE_SUFFIX to ".C" in pccts/h/config.h.
+
+o The token object class hierarchy has been flattened to include
+ only three classes: ANTLRAbstractToken, ANTLRCommonToken, and
+ ANTLRCommonNoRefCountToken. The common token now does garbage
+ collection via ref counting.
+
+o "Smart" pointers are now used for garbage collection. That is,
+ ANTLRTokenPtr is used instead of "ANTLRToken *".
+
+o The antlr.1 man page has been cleaned up slightly.
+
+o The SUN C++ compiler now complains less about C++ support code.
+
+o Grammars which subclass ANTLRCommonToken must wrap all token
+ pointer references in mytoken(token_ptr). This is the only
+ serious backward incompatibility. See below.
+
+
+MINOR CHANGES:
+
+--------------------------------------------------------
+1 deleteTokens()
+
+The deleteTokens() message to the parser or token buffer has been changed
+to one of:
+
+ void noGarbageCollectTokens() { inputTokens->noGarbageCollectTokens(); }
+ void garbageCollectTokens() { inputTokens->garbageCollectTokens(); }
+
+The token buffer deletes all non-referenced tokens by default now.
+
+--------------------------------------------------------
+2 makeToken()
+
+The makeToken() message returns a new type. The function should look
+like:
+
+ virtual ANTLRAbstractToken *makeToken(ANTLRTokenType tt,
+ ANTLRChar *txt,
+ int line)
+ {
+ ANTLRAbstractToken *t = new ANTLRCommonToken(tt,txt);
+ t->setLine(line);
+ return t;
+ }
+
+--------------------------------------------------------
+3 TokenType
+
+Changed TokenType-> ANTLRTokenType (often forces changes in AST defs due
+to #[] constructor called to AST(tokentype, string)).
+
+--------------------------------------------------------
+4 AST()
+
+You must define AST(ANTLRTokenPtr t) now in your AST class definition.
+You might also have to include ATokPtr.h above the definition; e.g.,
+if AST is defined in a separate file, such as AST.h, it's a good idea
+to include ATOKPTR_H (ATokPtr.h). For example,
+
+ #include ATOKPTR_H
+ class AST : public ASTBase {
+ protected:
+ ANTLRTokenPtr token;
+ public:
+ AST(ANTLRTokenPtr t) { token = t; }
+ void preorder_action() {
+ char *s = token->getText();
+ printf(" %s", s);
+ }
+ };
+
+Note the use of smart pointers rather than "ANTLRToken *".
+
+--------------------------------------------------------
+5 SUN C++
+
+From robertb@oakhill.sps.mot.com Bob Bailey. Changed ANTLR C++ output
+to avoid an error in Sun C++ 3.0.1. Made "public" return value
+structs created to hold multiple return values public.
+
+--------------------------------------------------------
+6 genmk
+
+Fixed genmk so that target List.* is not included anymore. It's
+called SList.* anyway.
+
+--------------------------------------------------------
+7 \r vs \n
+
+Scott Vorthmann <vorth@cmu.edu> fixed antlr.g in ANTLR so that \r
+is allowed as the return character as well as \n.
+
+--------------------------------------------------------
+8 Exceptions
+
+Bug in exceptions attached to labeled token/tokclass references. Didn't gen
+code for exceptions. This didn't work:
+
+a : "help" x:ID
+ ;
+ exception[x]
+ catch MismatchedToken : <<printf("eh?\n");>>
+
+Now ANTLR generates (which is kinda big, but necessary):
+
+ if ( !_match_wsig(ID) ) {
+ if ( guessing ) goto fail;
+ _signal=MismatchedToken;
+ switch ( _signal ) {
+ case MismatchedToken :
+ printf("eh?\n");
+ _signal = NoSignal;
+ break;
+ default :
+ goto _handler;
+ }
+ }
+
+which implies that you can recover and continue parsing after a missing/bad
+token reference.
+
+--------------------------------------------------------
+9 genmk
+
+genmk now correctly uses config file for CPP_FILE_SUFFIX stuff.
+
+--------------------------------------------------------
+10 general cleanup / PURIFY
+
+Anthony Green <green@vizbiz.com> suggested a bunch of good general
+clean up things for the code; he also suggested a few things to
+help out the "PURIFY" memory allocation checker.
+
+--------------------------------------------------------
+11 $-variable references.
+
+Manuel ORNATO indicated that a $-variable outside of a rule caused
+ANTLR to crash. I fixed this.
+
+12 Tom Moog suggestion
+
+Fail action of semantic predicate needs "{}" envelope. FIXED.
+
+13 references to LT(1).
+
+I have enclosed all assignments such as:
+
+ _t22 = (ANTLRTokenPtr)LT(1);
+
+in "if ( !guessing )" so that during backtracking the reference count
+for token objects is not increased.
+
+
+TOKEN OBJECT GARBAGE COLLECTION
+
+1 INTRODUCTION
+
+The class ANTLRCommonToken is now garbaged collected through a "smart"
+pointer called ANTLRTokenPtr using reference counting. Any token
+object not referenced by your grammar actions is destroyed by the
+ANTLRTokenBuffer when it must make room for more token objects.
+Referenced tokens are then destroyed in your parser when local
+ANTLRTokenPtr objects are deleted. For example,
+
+a : label:ID ;
+
+would be converted to something like:
+
+void yourclass::a(void)
+{
+ zzRULE;
+ ANTLRTokenPtr label=NULL; // used to be ANTLRToken *label;
+ zzmatch(ID);
+ label = (ANTLRTokenPtr)LT(1);
+ consume();
+ ...
+}
+
+When the "label" object is destroyed (it's just a pointer to your
+input token object LT(1)), it decrements the reference count on the
+object created for the ID. If the count goes to zero, the object
+pointed by label is deleted.
+
+To correctly manage the garbage collection, you should use
+ANTLRTokenPtr instead of "ANTLRToken *". Most ANTLR support code
+(visible to the user) has been modified to use the smart pointers.
+
+***************************************************************
+Remember that any local objects that you create are not deleted when a
+lonjmp() is executed. Unfortunately, the syntactic predicates (...)?
+use setjmp()/longjmp(). There are some situations when a few tokens
+will "leak".
+***************************************************************
+
+2 DETAILS
+
+o The default is to perform token object garbage collection.
+ You may use parser->noGarbageCollectTokens() to turn off
+ garbage collection.
+
+
+o The type ANTLRTokenPtr is always defined now (automatically).
+ If you do not wish to use smart pointers, you will have to
+ redefined ANTLRTokenPtr by subclassing, changing the header
+ file or changing ANTLR's code generation (easy enough to
+ do in gen.c).
+
+o If you don't use ParserBlackBox, the new initialization sequence is:
+
+ ANTLRTokenPtr aToken = new ANTLRToken;
+ scan.setToken(mytoken(aToken));
+
+ where mytoken(aToken) gets an ANTLRToken * from the smart pointer.
+
+o Define C++ preprocessor symbol DBG_REFCOUNTTOKEN to see a bunch of
+ debugging stuff for reference counting if you suspect something.
+
+
+3 WHY DO I HAVE TO TYPECAST ALL MY TOKEN POINTERS NOW??????
+
+If you subclass ANTLRCommonToken and then attempt to refer to one of
+your token members via a token pointer in your grammar actions, the
+C++ compiler will complain that your token object does not have that
+member. For example, if you used to do this
+
+<<
+class ANTLRToken : public ANTLRCommonToken {
+ int muck;
+ ...
+};
+>>
+
+class Foo {
+a : t:ID << t->muck = ...; >> ;
+}
+
+Now, you must do change the t->muck reference to:
+
+a : t:ID << mytoken(t)->muck = ...; >> ;
+
+in order to downcast 't' to be an "ANTLRToken *" not the
+"ANTLRAbstractToken *" resulting from ANTLRTokenPtr::operator->().
+The macro is defined as:
+
+/*
+ * Since you cannot redefine operator->() to return one of the user's
+ * token object types, we must down cast. This is a drag. Here's
+ * a macro that helps. template: "mytoken(a-smart-ptr)->myfield".
+ */
+#define mytoken(tp) ((ANTLRToken *)(tp.operator->()))
+
+You have to use macro mytoken(grammar-label) now because smart
+pointers are not specific to a parser's token objects. In other
+words, the ANTLRTokenPtr class has a pointer to a generic
+ANTLRAbstractToken not your ANTLRToken; the ANTLR support code must
+use smart pointers too, but be able to work with any kind of
+ANTLRToken. Sorry about this, but it's C++'s fault not mine. Some
+nebulous future version of the C++ compilers should obviate the need
+to downcast smart pointers with runtime type checking (and by allowing
+different return type of overridden functions).
+
+A way to have backward compatible code is to shut off the token object
+garbage collection; i.e., use parser->noGarbageCollectTokens() and
+change the definition of ANTLRTokenPtr (that's why you get source code
+<wink>).
+
+
+PARSER EXCEPTION HANDLING
+
+I've noticed some weird stuff with the exception handling. I intend
+to give this top priority for the "book release" of ANTLR.
+
+==========
+1.32 Full Release
+
+o Changed Token class hierarchy to be (Thanks to Tom Moog):
+
+ ANTLRAbstractToken
+ ANTLRRefCountToken
+ ANTLRCommonToken
+ ANTLRNoRefCountCommonToken
+
+o Added virtual panic() to ANTLRAbstractToken. Made ANTLRParser::panic()
+ virtual also.
+
+o Cleaned up the dup() stuff in AST hierarchy to use shallowCopy() to
+ make node copies. John Farr at Medtronic suggested this. I.e.,
+ if you want to use dup() with either ANTLR or SORCERER or -transform
+ mode with SORCERER, you must defined shallowCopy() as:
+
+ virtual PCCTS_AST *shallowCopy()
+ {
+ return new AST;
+ p->setDown(NULL);
+ p->setRight(NULL);
+ return p;
+ }
+
+ or
+
+ virtual PCCTS_AST *shallowCopy()
+ {
+ return new AST(*this);
+ }
+
+ if you have defined a copy constructor such as
+
+ AST(const AST &t) // shallow copy constructor
+ {
+ token = t.token;
+ iconst = t.iconst;
+ setDown(NULL);
+ setRight(NULL);
+ }
+
+o Added a warning with -CC and -gk are used together. This is broken,
+ hence a warning is appropriate.
+
+o Added warning when #-stuff is used w/o -gt option.
+
+o Updated MPW installation.
+
+o "Miller, Philip W." <MILLERPW@f1groups.fsd.jhuapl.edu> suggested
+ that genmk be use RENAME_OBJ_FLAG RENAME_EXE_FLAG instead of
+ hardcoding "-o" in genmk.c.
+
+o made all exit() calls use EXIT_SUCCESS or EXIT_FAILURE.
+
+===========================================================================
+1.33
+
+EXIT_FAILURE and EXIT_SUCCESS were not always defined. I had to modify
+a bunch of files to use PCCTS_EXIT_XXX, which forces a new version. Sorry
+about that.
+
projects / pccts / blobdiff