Re: Has anyone hand-written a scanner/parser module?

tuxisthebirdforme@gmail.com wrote:


> I know most people anymore use lex/yacc or some derivative of these
> tools to create scanner/parser modules for their compiler projects. I
> was wondering if anyone has developed a scanner or parser that they
> personally hand-wrote?


In my experience, most commercial compilers use hand-written scanners
and parsers. Certainly, the Embarcadero (CodeGear, ex-Borland) Delphi
and C++ compilers use hand-written scanners and parsers, and the
publicly visible C# compiler in MS shared source CLI (Rotor) is also
hand-written.


The CodeGear IDE support for editing, outline views, refactoring etc. on
the other hand, does use compiler compilers - specifically, an internal
tool that creates recursive descent LL recognizers.


> If so, I would like to know what language you
> used and what type of grammar you parsed.


The compiler I work on (Delphi, a Pascal variant) has its scanner and
parser written in C.


The main reasons for a hand-written scanner are speed and flexibility.
The scanner needs to take buffered data from multiple sources, whether
it's data from files on disk (which may need to be transcoded into
UTF-8) or data in memory from the IDE (i.e. direct from the editor's
buffers). Furthermore, the grammar has special cases where things that
would otherwise be keywords need to get parsed as identifiers, and
preprocessor expressions may recurse back into the parser to evaluate
constant expressions.


For the parser: good error handling may use a fair amount of context to
decide what the programmer's intention was. Good editor support (code
insight / intellisense) works with cooperation from the compiler, where
available symbols at the editor cursor's location in the file gets
handed back to the editor. A long history of adding new syntax
constructs while maintaining backward compatibility means the grammar is
no longer simple or clean in any real way. Furthermore, the parser
itself doesn't use a single strategy, it uses a mix of recursive descent
for declarations and operator precedence for expressions.


-- Barry


--
http://barrkel.blogspot.com/

Return to the comp.compilers page.
Search the comp.compilers archives again.