Re: Commercial LEX/YACC -vs- Flex/Bison

Henry Spencer <>
18 Nov 1997 12:19:19 -0500

          From comp.compilers

Related articles
Commercial LEX/YACC -vs- Flex/Bison (todd) (1997-11-13)
Re: Commercial LEX/YACC -vs- Flex/Bison (1997-11-14)
Re: Commercial LEX/YACC -vs- Flex/Bison (1997-11-16)
Re: Commercial LEX/YACC -vs- Flex/Bison (1997-11-16)
Re: Commercial LEX/YACC -vs- Flex/Bison (Henry Spencer) (1997-11-18)
| List of all articles for this month |

From: Henry Spencer <>
Newsgroups: comp.compilers
Date: 18 Nov 1997 12:19:19 -0500
Organization: SP Systems, Toronto
References: 97-11-080 97-11-092
Keywords: lex, yacc, tools, history

Stephen Flanagan <> wrote:
> While I use yacc quite a bit, I haven't used lex since
>college. Using lex to scan tokens is akin to using a robot arm and a
>nail gun to hang a picture in your living room...
>[When the robot arm costs no more than a hammer and is considerably less
>likely to land on your thumb, I think it's not a bad tool. -John]

There were several real reasons why people continued to hand-code
scanners despite the presence of lex. However, none of them is really
a *good* reason any more.

The most obvious one was that the original lex had a (well-justified)
reputation for poor performance. Van Jacobson fixed that a decade
ago, although some of the authors of lex clones may not have known
about it.

A less obvious one was that in the days of the PDP11, the size of the
lex tables was a serious problem when trying to squeeze a compiler
into a 16-bit address space. This hasn't been an issue for most
people for a long time, but hassles of this sort contributed to
persistent folklore about lex being unsuited to production

Non-Unix(TM) C compilers appeared well before non-Unix(TM) lexes, so
in the early days of writing compilers that weren't tied to Unix
licensing, there was a tendency to avoid lex because it often wasn't
available in other environments. That's been fixed by freeware
implementations, notably flex.

Finally, people have a lamentable tendency to treat lex as a panacea
rather than a tool, and then complain and abandon lex when they find
that they can't easily write a one-line pattern for a tricky case like
(the canonical bad example) C comments. A judicious leavening with
bits of C code can deal with this without throwing out the baby with
the bathwater.

A decade or so ago, when I was interested in writing a C compiler, I
used a hand-coded scanner for several reasons... but I also wrote a
lex scanner and used it for checking the correctness of the hand-coded
version, since it was a lot easier to verify the lex version against
the manuals. (In fact, I think I posted the lex version to
| Henry Spencer

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.