Re: Learning only one lexer made me blind to its hidden assumptions

George Neuner <gneuner2@comcast.net> wrote:
> On Wed, 13 Jul 2022 19:52:45 -0000 (UTC), antispam@math.uni.wroc.pl
> wrote:
>
> >Hmm, from flex manual:
> >
> >: -Ce, --ecs
> >: construct equivalence classes
> >:
> >: -Cm, --meta-ecs
> >: construct meta-equivalence classes
> >
> >If you want smaller tables use options above and flex DFA will
> >work on character classes.
>
> But note that Flex /may/ run considerably slower if you make heavy use
> of equivalence classes. IIRC, that results in (moral equivalent of)
> NFA rather than DFA.


IIUC equivalence classes are reasonably cheap: single extra
array access per input character. Meta-equivalence are more
expensive.


If I wanted ultimate speed I probably would use hand-written
scanner, they tend to be significantly faster than anything
flex can generate. But in most cases flex scanner speed is
adequate.


> [On modern computers it's hard to imagine a scanner so big that the space
> savings from those two options are worth it. 64K PDP-11 and all that. -John]


Well, L1 cache on many processors is just 16K. L2 caches are
bigger, but it is not hard to imagince scanner which wastes
a lot of time due to L2 misses. If smaller tables can fit
in L2, than there may be net speed gain. Even if there is
speed loss on some machines smaller tables are likely to
lead to more predictable/uniform performance.


Also, if scanner if fast enough with smaller tables, then
using bigger tables is just waste. Of course it does not
make sense to spent a lot of effort eliminating small waste,
but with flex effort is just an extra command line switch
to flex.


--
                                                            Waldek Hebisch

Return to the comp.compilers page.
Search the comp.compilers archives again.