Re: Reentrant flex & bison (Paolo Bonzini)
19 Mar 2004 23:53:11 -0500

          From comp.compilers

Related articles
Reentrant flex & bison (2004-03-15)
Re: Reentrant flex & bison (Clint Olsen) (2004-03-19)
Re: Reentrant flex & bison (2004-03-19)
Re: Reentrant flex & bison (2004-03-26)
Re: Reentrant flex & bison (2004-04-03)
Re: Reentrant flex & bison (Paul Eggert) (2004-04-03)
Re: Reentrant flex & bison (2004-04-03)
Re: Reentrant flex & bison (2004-04-14)
| List of all articles for this month |

From: (Paolo Bonzini)
Newsgroups: comp.compilers
Date: 19 Mar 2004 23:53:11 -0500
References: 04-03-068
Keywords: lex
Posted-Date: 19 Mar 2004 23:53:11 EST

> I discovered that the reentrance of flex is accomplished
> by generating a c++ scanner class, while reentrant bison
> remains the old c-style function call to yylex().
> Anybody knows what the way to "glue" them (scanner & parser)
> together is. flex and bison doesn't seem to co-work well
> in this way, right?!

Much better, newer Flexes have more options that will suit your needs.
  Here are excerpts from the docs. Bracket text is mine.

`%option reentrant'

* All functions take one additional argument: `yyscanner'

* All global variables are replaced by their macro equivalents. (We
tell you this because it may be important to you during debugging.)

* `yylex_init' and `yylex_destroy' must be called before and after
`yylex', respectively.

* Accessor methods (get/set functions) provide access to common `flex'

* User-specific data can be stored in `yyextra'.

The following Functions are available in a reentrant scanner:

    char *yyget_text ( yyscan_t scanner );
    int yyget_leng ( yyscan_t scanner );
    FILE *yyget_in ( yyscan_t scanner );
    FILE *yyget_out ( yyscan_t scanner );
    int yyget_lineno ( yyscan_t scanner );
    YY_EXTRA_TYPE yyget_extra ( yyscan_t scanner );
    int yyget_debug ( yyscan_t scanner );

    void yyset_debug ( int flag, yyscan_t scanner );
    void yyset_in ( FILE * in_str , yyscan_t scanner );
    void yyset_out ( FILE * out_str , yyscan_t scanner );
    void yyset_lineno ( int line_number , yyscan_t scanner );
    void yyset_extra ( YY_EXTRA_TYPE user_defined , yyscan_t scanner );

In a reentrant C scanner, support for yylineno is always present
(i.e., you may access yylineno), but the value is never modified by
`flex' unless `%option yylineno' is enabled. This is to allow the user
to maintain the line count independently of `flex'.

The following functions and macros are made available when `%option
bison-bridge' (`--bison-bridge') is specified [together with

    YYSTYPE * yyget_lval ( yyscan_t scanner );
    void yyset_lval ( YYSTYPE * yylvalp , yyscan_t scanner );

The following functions and macros are made available when `%option
bison-locations' (`--bison-locations') is specified [together with

    YYLTYPE *yyget_lloc ( yyscan_t scanner );
    void yyset_lloc ( YYLTYPE * yyllocp , yyscan_t scanner );

[There is also yyget_column, yyset_column and yycolumn, but they are
undocumented as of Flex version 2.5.31.]

If you really want C++ or Flex 2.5.4, you can use YYPARSE_PARAM so
that a pointer to the FlexLexer object is passed to the bison scanner,
and from there to an yylex wrapper function which actually calls the
flex scanner. --bison-bridge is not supported in the C++ class.

That is, rename the FlexLexer.h that comes with Flex to
FlexLexer_base.h, and in FlexLexer.h put something like this
(untested, plus omit yylloc if you do not use locations):

/* Add a member to the Flex base class (!)
#define yyout yyout; YYSTYPE *yylval; YYLTYPE *yylloc
#include "FlexLexer_base.h"
#undef yyout

/* Define the wrapper that calls the C++ scanner. */
inline int yylex (YYSTYPE *yylval, YYLTYPE *yylloc,
                                    yyFlexLexer *yy_flex_lexer)
    yy_flex_lexer->yylval = yylval;
    yy_flex_lexer->yylloc = yylloc;
    return yy_flex_lexer->yylex ();

/* Tell bison how to call the lexer */
#ifdef YYPURE
#define YYPARSE_PARAM yyFlexLexer *yy_flex_lexer
#define YYLEX_PARAM yy_flex_lexer


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.