Re: Q2. Why do you split a monolitic grammar into the lexing and parsing rules?

valentin tihomirov wrote:
> So, I do not understand why do we need the artificial obstacle, the
> 2nd level?


There's nothing inherent in parsing that prevents you from writing
parsers without any distinction between lexical symbols and compound
rules. In fact I've written several parsers that doesn't enforce any
difference at all.


Some points to keep in mind:
- It's often simpler to split the two. You can write a lexer and verify
it/debug it separately, and then nicely layer the parser on top.
- It's often nicer for error reporting to separate the two - you might
want to report on the full token expected, and point the user to a
token boundary.
- When hand writing a parser it is often much easier to understand if
the higher levels deals with a stream of tokens instead of having to
take into account issues such as whitespace, comments and
disambiguation of token boundaries that can be filtered out in the
lexer.
- The split often has little practical effect, but presents a
conceptual separation that sometimes simplify understanding of the
grammar.
- It's a split that at least I believe encourages good design, in that
it makes you think consciously about the recognition of tokens and how
to reduce ambiguity in the parser and promotes context free grammars.


Vidar

Return to the comp.compilers page.
Search the comp.compilers archives again.