Re: What should be check in Lexical Analyzer along with generating tokens?

"Joachim Durchholz" <joachim_d@gmx.de>
29 Sep 2002 15:47:54 -0400

          From comp.compilers
Related articles
What should be check in Lexical Analyzer along with generating token vikramasanjeeva@hotmail.com (Vikrama Sanjeeva) (2002-09-14)
Re: What should be check in Lexical Analyzer along with generating t joachim_d@gmx.de (Joachim Durchholz) (2002-09-19)
Re: What should be check in Lexical Analyzer along with generating tok clint@0lsen.net (Clint Olsen) (2002-09-20)
Re: What should be check in Lexical Analyzer along with generating t joachim_d@gmx.de (Joachim Durchholz) (2002-09-22)
Re: What should be check in Lexical Analyzer along with generating clint@0lsen.net (Clint Olsen) (2002-09-25)
Re: What should be check in Lexical Analyzer along with generating tok joachim_d@gmx.de (Joachim Durchholz) (2002-09-29)
Re: What should be check in Lexical Analyzer along with generating tok lex@cc.gatech.edu (Lex Spoon) (2002-10-18)
| List of all articles for this month |
From: "Joachim Durchholz" <joachim_d@gmx.de>
Newsgroups: comp.compilers
Date: 29 Sep 2002 15:47:54 -0400
Organization: Compilers Central
References: 02-09-087 02-09-110 02-09-121 02-09-128 02-09-141
Keywords: lex, parse
Posted-Date: 29 Sep 2002 15:47:54 EDT

Clint Olsen wrote:
> Joachim Durchholz wrote:
>
>>comment ::= "/*" {token} "*/"
>>token ::= comment
>> | string | integer | ... (literals)
>> | "if" | "then" | ... (keywords)
>> | <error>
>>
> Your example does show how the comment would appear in your grammar in
> the first place. Comments in most free form languages can appear
> _anywhere_ except where they might split tokens (e.g. quoted strings).


That's just the grammar for the tokenizer.
The AST would be constructed on top of that, ignoring "comment" tokens.


Hope this clarifies things.


Regards,
Joachim

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.