Re: LEX behaviour when given "large" automata.

Rich Salz <rsalz@BBN.COM>
18 Mar 88 16:45:26 GMT

          From comp.compilers

Related articles
LEX behaviour when given "large" automata. phs@lifia.imag.fr (1988-03-03)
Re: LEX behaviour when given "large" automata. rsalz@BBN.COM (Rich Salz) (1988-03-18)
Re: LEX behaviour when given "large" automata. chris@mimsy.UUCP (1988-03-20)
Re: LEX behaviour when given "large" automata. lbl-helios!vern@lbl-rtsg.arpa (Vern Paxson) (1988-03-18)
Re: LEX behaviour when given "large" automata. sargas.usc.edu!tli@oberon.usc.edu (1988-03-20)
| List of all articles for this month |

From: Rich Salz <rsalz@BBN.COM>
Newsgroups: comp.compilers,comp.lang.c,comp.unix.questions
Date: 18 Mar 88 16:45:26 GMT
References: <911@ima.ISC.COM>
Organization: BBN Laboratories, Cambridge MA

In comp.compilers (<911@ima.ISC.COM>), phs@lifia.imag.fr (Philippe Schnoebelen) writes:
> I'm having some problems with LEX. When my number of keywords/regexps is
>growing, the lexical analyzer begins to give strange, unexpected, (let's
>face it, wrong) results.


Lex ain't robust. As a work-around, you can get real big savings in
all of
1. creation time
2. running time
3. exectuable size
by going from one pattern/keyword, to a general pattern for identifiers,
and doing a table lookup. That is, don't do this:
for return(FOR);
if return(IF);
foo return(FOO);
[a-z]+ return(munchonit(yytext));


Do this:
table[] = { { "for", FOR }, { "if", IF }, { NULL } };


[a-z]+ {
for (p = table; p->name; p++)
if (strcmp(p->name, yytext) == 0)
return(p->value);
return(munchonit(yytext));
}


(I left out all sort of declarations and optimizations on the search
loop.)


This is a real fun thing to do: how often do you get to win on both sides of
the time-space tradeoff?
/r$
[Similar suggestion from Eddie Wyatt edw@ius1.cs.cmu.edu]
[From Rich Salz <rsalz@BBN.COM>]
--


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.