Re: replacements for LEX: flex, ... , Rex

Vern Paxson <lbl-pistachio!>
Fri, 01 Apr 88 19:53:59 PST

          From comp.compilers

Related articles
Re: replacements for LEX: flex, ... , Rex lbl-pistachio! (Vern Paxson) (1988-04-01)
| List of all articles for this month |

Date: Fri, 01 Apr 88 19:53:59 PST
From: Vern Paxson <lbl-pistachio!>

Josef Grosch describes Rex, a scanner generator similar to lex,
and gives some timings comparing its perfomance with lex's:

> Scanners generated by Rex are more than 4 times faster and up to 5
> times smaller than LEX-generated ones. It is possible to reach a speed
> of 180,000 to 195,000 lines per minute on a MC 68020 processor (includ-
> ing input from file)....
> .... I would like to hear some numbers from "flex".

Here they are for a C tokenizer which includes keywords.

Generation times:

lex 83.0 secs
flex 3.9
flex -cfe 7.1 # uncompressed table, equivalence classes
flex -cf 15.0 # uncompressed table, no equivalence classes

Scanner object file sizes:

lex 41.0K bytes
flex 9.4K
flex -cfe 49.6K
flex -cf 126.5K

Running times on a 28,088 line input (685K characters):

lex 29.8 secs
flex 19.3
flex -cfe 9.0
flex -cf 7.8

The timings were made on a Sun 3/60. All times are user + system CPU time,
and don't include hashing of identifiers.


        For about the same sized scanner, you get a factor of 3 in performance.
        For a 30% faster scanner, you get a scanner 1/4th the size, and it's
generated in 1/20th the time.
        For a scanner that's 3 times larger, you get a factor of 3.8 in

Rex sounds very good - taken separately, various options of flex perform in
about its league, but flex can't match its combined performance. This is
due in part to flex's compressed scanners not having been optimized for
speed nearly as much as the full table scanners.

What is Rex's availability?


Vern Paxson
Real Time Systems ucbvax!!vern
Lawrence Berkeley Laboratory (415) 486-6411

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.