Re: how to push token back for Bison to consider it?

Milan Durovic <mdurovic@vcisnet.net> wrote in message news:03-04-038...
> I want to do the following: when a certain grammar rule is matched I
> want to, in the action part, read all the tokens until a certain token
> is read, and then give the control back to the parser, who should
> continue with that token as the next token. In other words, I am doing
> something like:
>
>
> ruleBlah: nonterm1 nonterm2
> {
> while( yylex() != SOME_PARTICULAR_TOKEN )
> { do_something(); }
> }
>
> ruleBlah2: SOME_PARTICULAR_TOKEN nonterm3 nonterm4


A quick (and, in my option, dirty) method would be to put a wrapper
around yylex(); id est insert the following code into the top of your
parser ...


static int repeatToken = 0;


static int yylexWrapper( void ) {
        if( repeatToken != 0 ) return yylex();
        {
                int temp = repeatToken;
                repeatToken = 0;
                return temp;
} }


#define yylex yylexWrapper /* monkey with the internals of bison */


Then add 'repeatToken = SOME_PARTICULAR_TOKEN;' to the ruleBlah processor.


-----


If this is too slow you could try the following line noise :-)...


static int repeatToken = 0;
#define yylex() ((repeatToken?lex():repeatToken),repeatToken=0)


and in the scanner


#define yylex lex


-----


The trouble with both these pieces of code is they may mess up if
someone decides to alter the internals of the code BISON generates,
particularly the mess of #define statements.


A more elegant way would be to implement your own scanner which caches
the last token, then implement a 'scan_goBack()' call to inform the
scanner to use the cached version, or implement a
'scan_readUntil( int token )' call to read upto your specified token.


This is the solution I chose when faced with a similar problem
a few months ago, and can be quite effective provided your scaned
tokens are fairly simple.


C 2003/5/8


[code examples are untested, but should work (hopefully)]

Return to the comp.compilers page.
Search the comp.compilers archives again.