Re: 2 word token as one in lex

Troy Cauble <troy@bell-labs.com>
23 Jul 2000 16:44:24 -0400

          From comp.compilers
Related articles
2 word token as one in lex makotosu@my-deja.com (2000-07-18)
Re: 2 word token as one in lex troy@bell-labs.com (Troy Cauble) (2000-07-23)
Re: 2 word token as one in lex james.d.carlson@sun.com (James Carlson) (2000-07-23)
Re: 2 word token as one in lex kszabo@nortelnetworks.com (Kevin Szabo) (2000-07-23)
| List of all articles for this month |
From: Troy Cauble <troy@bell-labs.com>
Newsgroups: comp.compilers
Date: 23 Jul 2000 16:44:24 -0400
Organization: Bell Labs, Holmdel, NJ, USA
References: 00-07-034
Keywords: SQL, parse

makotosu@my-deja.com wrote:
: I'm trying to parse SQL and I'd like to recognize UNION JOIN as one
: token in the lexer. So for example,


: if the lexer sees UNION and the next token (after any # of whitespaces,
: tabs and newlines) is JOIN it should return UNION_JOIN


What's wrong with




SP ([ ])
HTAB (\t)
WSP ({SP}|{HTAB})


%%


UNION{WSP}+JOIN { /* ACTION 1 */ }
UNION { /* ACTION 2 */ }


-troy
[Comments and newlines are also whitespace. -John]




Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.