Re: Has anyone hand-written a scanner/parser module?

Raja Mukherji <rajamukherji@gmail.com>
Sun, 16 Nov 2008 14:21:07 -0800 (PST)

          From comp.compilers

Related articles
[3 earlier articles]
Re: Has anyone hand-written a scanner/parser module? mailbox@dmitry-kazakov.de (Dmitry A. Kazakov) (2008-11-16)
Re: Has anyone hand-written a scanner/parser module? jeremy.bennett@embecosm.com (Jeremy Bennett) (2008-11-16)
Re: Has anyone hand-written a scanner/parser module? barry.j.kelly@gmail.com (Barry Kelly) (2008-11-16)
Re: Has anyone hand-written a scanner/parser module? efutch@gmail.com (Egdares Futch) (2008-11-16)
Re: Has anyone hand-written a scanner/parser module? jgd@cix.compulink.co.uk (2008-11-16)
Re: Has anyone hand-written a scanner/parser module? idbaxter@semdesigns.com (Ira Baxter) (2008-11-16)
Re: Has anyone hand-written a scanner/parser module? rajamukherji@gmail.com (Raja Mukherji) (2008-11-16)
Re: Has anyone hand-written a scanner/parser module? bill@qswtools.com (Bill Cox) (2008-11-16)
Re: Has anyone hand-written a scanner/parser module? marcov@stack.nl (Marco van de Voort) (2008-11-17)
Re: Has anyone hand-written a scanner/parser module? dmaze@mit.edu (David Z Maze) (2008-11-17)
Re: Has anyone hand-written a scanner/parser module? gene.ressler@gmail.com (Gene) (2008-11-17)
Re: Has anyone hand-written a scanner/parser module? arnold@skeeve.com (2008-11-18)
Re: Has anyone hand-written a scanner/parser module? sh006d3592@blueyonder.co.uk (Stephen Horne) (2008-11-18)
[10 later articles]
| List of all articles for this month |

From: Raja Mukherji <rajamukherji@gmail.com>
Newsgroups: comp.compilers
Date: Sun, 16 Nov 2008 14:21:07 -0800 (PST)
Organization: Compilers Central
References: 08-11-061
Keywords: lex, parse, practice

On Nov 15, 5:49 pm, "tuxisthebirdfo...@gmail.com"
<tuxisthebirdfo...@gmail.com> wrote:
> I know most people anymore use lex/yacc or some derivative of these
> tools to create scanner/parser modules for their compiler projects. I
> was wondering if anyone has developed a scanner or parser that they
> personally hand-wrote? ...


I've hand written the scanner/parser for my language Wrapl for a
while. I originally wrote them in Icon, then Modula-2, Modula-3, C and
currently in C++. However they all had the same basic structure, which
you find in many scanner/parsers:


The scanner has 3 main functions:


a method/function "next" which advances the scanner to the next token


a method/function "parse" which checks that the current token is a
particular token or in a set of tokens. If the current token is
"none", call next(). If the current token matches, the current token
is set to "none". Returns true/false if the current token matched or
not. The current token values (e.g. line/column number, integer/string
value) is available from the scanner.


a method/function "accept" which behaves similarly to "parse" but
throws an error if the match fails.


The parser consists of many "parse_???" and "accept_???" functions,
built up from the other parse/accept functions. The trickiest part is
the infix operation parser, since it has to deal with different levels
of precedence. At first I had a different function for each level,
like "parse_expr_NN" where NN was the precedence level. But that
became tedious to modify, so I now use a function "parse_expr(level)"
where level is a parameter and the function contains a switch
statement that switches on the level, each case falling through to
higher precedence expressions.


If you're brave you can find the source for my scanner/parser at
http://wrapl.sf.net/download.html#source; the scanner/parser is in the
dev/src/Lib/Wrapl/Loader/scanner.c and dev/src/Lib/Wrapl/Loader/
parser.c files, but they lack commenting.


Raja



Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.