Re: is lex useful?

kanze@lts.sel.alcatel.de (James Kanze US/ESC 60/3/141 #40763)
2 Jul 1996 12:39:41 -0400

          From comp.compilers

Related articles
[20 earlier articles]
Re: is lex useful? leichter@smarts.com (1996-06-30)
Re: is lex useful? trd@murlibobo.cs.mu.OZ.AU (1996-06-30)
Re: is lex useful? WStreett@shell.monmouth.com (1996-06-30)
Re: is lex useful? dmr@bell-labs.com (1996-06-30)
Re: is lex useful? clark@quarry.zk3.dec.com (1996-07-01)
Re: is lex useful? bromage@cs.mu.OZ.AU (1996-07-02)
Re: is lex useful? kanze@lts.sel.alcatel.de (1996-07-02)
Re: is lex useful? colas@aye.inria.fr (1996-07-04)
Re: is lex useful? trd@lister.cs.mu.OZ.AU (1996-07-05)
| List of all articles for this month |

From: kanze@lts.sel.alcatel.de (James Kanze US/ESC 60/3/141 #40763)
Newsgroups: comp.compilers
Date: 2 Jul 1996 12:39:41 -0400
Organization: GABI Software, Sarl.
References: 96-06-094 96-06-129 96-06-143
Keywords: parse, Fortran, comment

Robert.Corbett@Eng.Sun.COM (Robert Corbett) writes:


        [Concerning problems in scanning Fortran...]
|> A better example is


|> DO10I = expr1, expr2


|> Since the length of expr1 is bounded only by the number of characters
|> allowed in a statement, either a multipass lexer or practically
|> unbounded lookahead are needed.


|> Because Fortran limits the maximum size of a statement, a lexer for
|> Fortran can analyze any Fortran statement in constant time.


The finite maximum length of a statement also makes rescanning a lot
easier. Since you never need unlimited backtracking, you can simply
cache the whole statement in memory, and backtrack over it there.


|> [Right, thanks for the correction. In the DO10I example, note that just
|> looking ahead for a comma isn't sufficient. You have to look for a comma
|> not enclosed in parens, which lex can't do, because REs can't count. -John]


Even counting parentheses isn't sufficient. What about commas in e.g.:
Hollerith constants? (This works both ways: a parentheses in a
Hollerith constant doesn't inhibit the interpretation of a comma
later.)


I once had a bet with a person over the question. He claimed that the
best solution was to define some sort of predicate, and scan ahead to
avoid backtracking. For every predicate he suggested, however, I was
able to come up with an example statement for which it failed. (It's
been long enough since I've done any Fortran that I wouldn't want to
make the same bet today, but the possibility of using Hollerith
constants in an expression certainly requires a fair amount of
intelligence in any look-ahead scanner.)
--
James Kanze Tel.: (+33) 88 14 49 00 email: kanze@gabi-soft.fr
GABI Software, Sarl., 8 rue des Francs-Bourgeois, F-67000 Strasbourg, France
[What I did was to buffer up the whole statement, then make a pass over it
to extract quoted and hollerith constants, then another pass to decide if
it was an assignment-ish statement or not, and then lex and parse. -John]
--


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.