Re: Speed of lex generated scanners???

Roger Trigg (rtrigg@crl.com) wrote:


: processor (preferably x86). 3) I've also heard that FLEX has a "fast"
: option; does anyone have experience with this, and know how great of an
: improvement can be expected using it?


Note that the "fast" option only implies that the generated scanner tables
are not compressed. There is an -f and a -F option and they use two
slightly(?) different table representations (they are about as fast).


To make the generated scanner really fast, you need to use -f|-F *and*
eliminate all backtracking for the set of rules you've written (a scanner
with compressed tables *always* backtracks).


I actually did this once. It's pretty messy since you must write new rules
in the right places, and the info in the manual pages is not really enough
to understand exactly how things work and why. I emailed some with Vern
Paxson (the author of flex) and he helped me to get it right and explained
some of the inner workings of the generated code (thanks Vern). He was
rather pleased that someone actually went through the trouble to eliminate
backtracking for more speed.


Anyway once you've succeeded it is VERY FAST. For the rather small set
(~85 rules) I implemented it was so fast that the disk I/O on my Sun
SPARC- station was the bottleneck.


Oh and don't use lex for anything. It generates slow scanners, cannot
handle 8-bit chars (that's why I switched to flex) and is quite buggy.


Cheers,
M
--
Michael Bergman Email: euambn@eua.ericsson.se
EUA/XO Phone: +46 8 7275709
Ellemtel Telecom Systems Labs, Armborstv 14, S-125 25 Alvsjo, Sweden
--

Return to the comp.compilers page.
Search the comp.compilers archives again.