Re: Commercial LEX/YACC -vs- Flex/Bison

Stephen Flanagan <Stephen_Flanagan@bsginc.com> wrote:
> While I use yacc quite a bit, I haven't used lex since
>college. Using lex to scan tokens is akin to using a robot arm and a
>nail gun to hang a picture in your living room...
>[When the robot arm costs no more than a hammer and is considerably less
>likely to land on your thumb, I think it's not a bad tool. -John]


There were several real reasons why people continued to hand-code
scanners despite the presence of lex. However, none of them is really
a *good* reason any more.


The most obvious one was that the original lex had a (well-justified)
reputation for poor performance. Van Jacobson fixed that a decade
ago, although some of the authors of lex clones may not have known
about it.


A less obvious one was that in the days of the PDP11, the size of the
lex tables was a serious problem when trying to squeeze a compiler
into a 16-bit address space. This hasn't been an issue for most
people for a long time, but hassles of this sort contributed to
persistent folklore about lex being unsuited to production
applications.


Non-Unix(TM) C compilers appeared well before non-Unix(TM) lexes, so
in the early days of writing compilers that weren't tied to Unix
licensing, there was a tendency to avoid lex because it often wasn't
available in other environments. That's been fixed by freeware
implementations, notably flex.


Finally, people have a lamentable tendency to treat lex as a panacea
rather than a tool, and then complain and abandon lex when they find
that they can't easily write a one-line pattern for a tricky case like
(the canonical bad example) C comments. A judicious leavening with
bits of C code can deal with this without throwing out the baby with
the bathwater.


A decade or so ago, when I was interested in writing a C compiler, I
used a hand-coded scanner for several reasons... but I also wrote a
lex scanner and used it for checking the correctness of the hand-coded
version, since it was a lot easier to verify the lex version against
the manuals. (In fact, I think I posted the lex version to
comp.compilers...)
--
| Henry Spencer
| henry@zoo.toronto.edu
--

Return to the comp.compilers page.
Search the comp.compilers archives again.