Re: Getting rid of backups in flex.

fjh@cs.mu.OZ.AU (Fergus Henderson)
19 Mar 2002 16:09:59 -0500

          From comp.compilers

Related articles
Getting rid of backups in flex. robertminsk@yahoo.com (2002-01-17)
Re: Getting rid of backups in flex. esmond.pitt@bigpond.com (Esmond Pitt) (2002-01-18)
Re: Getting rid of backups in flex. casse@netcourrier.com (=?ISO-8859-15?q?=22Cass=E9=.Hugues@free.fr,@free.f) (2002-02-06)
Re: Getting rid of backups in flex. dube@dino04.iro.umontreal.ca (Danny Dube) (2002-02-16)
Re: Getting rid of backups in flex. esmond.pitt@bigpond.com (Esmond Pitt) (2002-02-28)
Re: Getting rid of backups in flex. clint@0lsen.net (2002-03-17)
Re: Getting rid of backups in flex. fjh@cs.mu.OZ.AU (2002-03-19)
| List of all articles for this month |

From: fjh@cs.mu.OZ.AU (Fergus Henderson)
Newsgroups: comp.compilers
Date: 19 Mar 2002 16:09:59 -0500
Organization: Computer Science, University of Melbourne
References: 02-01-070 02-02-017 02-02-041 02-02-063 02-03-083
Keywords: lex
Posted-Date: 19 Mar 2002 16:09:58 EST

clint@0lsen.net (Clint Olsen) writes:


>In comp.compilers, you wrote:
>>
>> Apart from keeping the size of the state machine down, you get the
>> benefit of the parser's error-recovery if any, rather than just lamely
>> printing out 'illegal character %c'.
>
>This presumes that your parser allows you to return arbitrary
>"integers" back to the parser. How do you ensure that yytext[0]
>doesn't collide with an already defined token in your token
>definitions file?


A slight variant on this technique is to add a new token type
(e.g. `INVALID_TOKEN') for illegal characters.


The character value can be an attribute of this token type.


/* in parser.y */
%token INVALID_TOKEN
%union{
/* ... */
char invalid_token;
}


The lexer should return this token whenever it gets a character
which doesn't match any other rule:


/* in lexer.l */
. { yylval.invalid_token = yytext[0];
return INVALID_TOKEN; }


The parser should not contain any productions which recognize this
token type; that way, the parser's normal error handling will kick in.
--
Fergus Henderson <fjh@cs.mu.oz.au> | "I have always known that the pursuit
The University of Melbourne | of excellence is a lethal habit"
WWW: <http://www.cs.mu.oz.au/~fjh> | -- the last words of T. S. Garp.


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.