Related articles |
---|
newbie: lexing and multi-character operators ajwitte@aol.com (2003-03-09) |
Re: newbie: lexing and multi-character operators Hans.Koerber@web.de (Hans =?ISO-8859-1?Q?K=F6rber?=) (2003-03-14) |
From: | ajwitte@aol.com (Ajwitte) |
Newsgroups: | comp.compilers |
Date: | 9 Mar 2003 17:38:21 -0500 |
Organization: | AOL http://www.aol.com |
Keywords: | lex, i18n, comment |
Posted-Date: | 09 Mar 2003 17:38:21 EST |
I'm hand-coding (for fun :) a lexer and parser for an interpreted
mathematical language. Somewhat like Mathematica, only *much*
simpler. My question is, how does one deal with multi-character
operators (for example, ++ [increment])? For example, x++1 should be
understood as [x] [plus] [unary plus] [1], while, x+++1 should be [x]
[increment] [plus] [1]. At what stage of the translation should this
distinction be made?
[The lexers I write use a simple greedy algorithm. returning the longest
possible token each time. That makes x++1 invalid without a space in the
middle, but I'm not sure that's a bad thing, since tokenizing rules that
are hard to write are usually hard for programmers to understand, and as
often as not end up "correcting" typos into valid but wrong code. -John]
Return to the
comp.compilers page.
Search the
comp.compilers archives again.