Fully reentrant parsers with standard flex & bison

This is just one more posting concerning the issue of reentrancy of
auto-generated parsers. I tried to create a fully reentrant parser with
*standard* flex-2.54a and bison-1.25, and everything seems to work. The
tricks are:


1) Tell flex to create a class MyFlexer, derived form FlexLexer. This is
done with the options:


%option c++
%option yyclass="MyLexer"




2) Add a private data member 'YYSTYPE semantic_value' to MyLexer. To
have YYSTYPE declared, you need to execute bison with "-d", and #include
the auto-generated header in MyLexer.h. In the lexer description, use
'semantic_value' instead of the global 'yylval'.




3) Stuff myLexer with the other lexer's execution context (e.g. symbol
table).




4) Add the following:


void MyLexer::embedded_yylex(int& result, YYSTYPE *lvalp){
    result=yylex();
    *lvalp=semantic_value;
}




5) Create a class ParserExecutionContext, containing


MyLexer& lexer_;


as a private data member. If your parser requires an extra context (e.g.
int foo), add more private data. Initialize this reference by passing a
reference to MyLexer to the ctor.




6) Add "C" accessors to this data (don't forget to make them friends to
ParserExecutionContext !):


extern "C" int* getFoo(void* parm){
    return &(((ParserExecutionContext*)parm)->foo);
}




7) Add "C" function int yylex(YYSTYPE *lvalp, void*):


extern "C" int yylex(YYSTYPE *lvalp, void* parm){
    int result;
    ((ParserExecutionContext*)parm)->lexer_->embedded_yylex(result,
lvalp);
    return result;
}




8) As the headers of these "C" functions are to be compiled both by "C"
and "C++" compilers, add the condional compilation directives like this:


#ifdef __cplusplus
extern "C"
#endif
int yylex(YYSTYPE *lvalp, void* parm)




9) Add '%pure_parser' to your grammar file.




10) Add extra parameter to your parser:


#define YYPARSE_PARAM parm
#define YYLEX_PARAM parm


There is a minor bug in bison.simple, which pops up if you add these
parameters and compile your parser on GCC. If you have an unpatched
version of bison.simple, the following workaround may help:


#ifdef __GNUC__
#undef __GNUC__
#define __GNUC_BISON_KLUDGE__
int yyparse(void*);
#endif


Of course, this is a kind of kludge. To be on the safe side, if you have
an extra "C" code following the grammar definition, add at the very
beginning of it:


#ifdef __GNUC_BISON_KLUDGE__
#define __GNUC__
#endif




11) Finally, call your parser:


/* Initialize the things needed by lexer and parser */
MyLexer lexer(/* some stuff */);
ParserExecutionContext parser_context(lexer, /* some other stuff */);
yyparse((void*)parser_context);




That's all. Maybe there are some subtle gotchas, which escaped me - if
someone see them, let me know.


Thanks,


-Pavel
--

Return to the comp.compilers page.
Search the comp.compilers archives again.