Re: is lex useful?

stefan.monnier@lia.di.epfl.ch says...
>] - No support for wide (>8 bit) character sets. Even 8-bit support is
>] fairly recent. The obvious implementation for wide characters (expand
>] tables to 16 bits) isn't practical, because you would increase the tables
>] sizes (which are already huge) 256x.
>
>The other obvious option is to treat a 16bit char as two 8bit chars.
>It might be less readable, but it works great.


This won't work if you want to use POSIX-style character classes, i.e.


[[:alpha:]]+ return(IDENTIFIER);


It would be very messy getting Lex to do this if you wanted to have, say,
Japanese as your language.


Also, what is isalpha() in one locale is not necessarily isalpha() in
another. If you change your language (i.e. LANG, LC_COLLATE, or LC_ALL
environment variables on *nix), a Lex-generated scanner won't know the
difference, because the RE's are hard-coded in tables. At best you can
make Lex work for one locale.


--
Scott Nicol
Information Advantage, Inc.
Scott.Nicol@infoadvan.com
--

Return to the comp.compilers page.
Search the comp.compilers archives again.