Re: looking for Lex/Bison unicode support

Yaron Bracha asked:
>
> Does anybody knows a flex/bison compatible parsing tools that support
> unicode and generate c++ code ?...
>
And the moderator remarked
> [Yacc and its clones parse tokens, not characters, so they shouldn't
> be a problem, give or take nits like passing through non-ASCII strings
> in C action routines correctly. Lex or flex is harder since all of
> the implementations I know of use the character codes as indexes into
> tables to implement the lex state machine. But if you do that for
> Unicode, you'll have 64K entry tables rather than 256 entry tables and
> severe program bloat. I believe that plan 9 has a Unicode lex,
> presumably with some hackery to keep the table sizes down. -John]


Plan 9 Unicode lex: no, I'm afraid, just for the reason related
to the one John mentioned: no one developed the energy to be clever about
this program. The BUGS section (even now) says
    Cannot handle UTF.
    The asteroid to kill this dinosaur is still in orbit.


Dennis

Return to the comp.compilers page.
Search the comp.compilers archives again.