Re: Speeding up LEX scanning times

[ Pieter Hintjens penned : ]
[Scanning Cobol in lex is slow, how should I speed it up?]


COBOL lexical analyzers can be pretty slow because of the very complicated
regular expressions involved. In my case, I had a lot of complex classes for
example for the argument to PIC etc. These generate a sizable dfa and can slow
the scanner down a lot. Tackling the fixed format and handling identifiers
that can be in any case but mean the same can add to scanner time.


I had to strip out a lot of things from lex regular expressions and recognize
them separately. Ironically, it is these very expressions that are supposed to
be coded in lex to avoid too much coding. Also I had a preprocessor to take
care of the format, comments etc. It also used to convert all characters
outside of quotes to upper case. This solution amounts to a dual pass and the
total time may be same, but parsing alone can be faster.


--
Virendra K Mehta | IBM T J Watson Research Centre
c1veeru@watson.ibm.com | (914) 784-7774
viren@wipsys.soft.net |
--

Return to the comp.compilers page.
Search the comp.compilers archives again.