How to get at the entire input buffer string from inside lexer?

Eric Fowler <eric.fowler@gmail.com>
Sat, 28 Mar 2009 21:17:47 -0700

          From comp.compilers

Related articles
How to get at the entire input buffer string from inside lexer? eric.fowler@gmail.com (Eric Fowler) (2009-03-28)
Re: How to get at the entire input buffer string from inside lexer? DrDiettrich1@aol.com (Hans-Peter Diettrich) (2009-03-29)
Re: How to get at the entire input buffer string from inside lexer? m.helvensteijn@gmail.com (2009-03-29)
Re: How to get at the entire input buffer string from inside lexer? gah@ugcs.caltech.edu (glen herrmannsfeldt) (2009-03-29)
Last problem is solved eric.fowler@gmail.com (Eric Fowler) (2009-03-29)
Re: How to get at the entire input buffer string from inside lexer? eric.fowler@gmail.com (Eric Fowler) (2009-03-30)
| List of all articles for this month |

From: Eric Fowler <eric.fowler@gmail.com>
Newsgroups: comp.compilers
Date: Sat, 28 Mar 2009 21:17:47 -0700
Organization: Compilers Central
Keywords: flex
Posted-Date: 29 Mar 2009 13:44:23 EDT

I am working on a little project in which I have redefined YYINPUT to
take input from an ASCII string. I need to parse and break down
strings of the form:


$FOOBAR, some data, more data*XX<CRLF>


The *XX field is a single byte checksum calculated from the rest of
the string up to that point.


I am looking for a nice clean way of checking the sum within either
the parser or the lexer, that is, calculating the checksum from the
whole string and comparing it to the value I got embedded in the
string itself.


My approach is to calculate the sum using the whole string as soon as
I get it. Problem is, I can't figure out what part of the lexer (or
parser, I am using bison) has access to that entire buffer. I have
toyed with YY_CURRENT_BUFFER but it's not giving me what I need.


It might also help to be able to store an offset into the input when a
token is recognized ... how can I keep track of everything I have
seen, including delimiters?


By the way, I have presented a simplified version of the real problem,
so the trivial solution of calculating sum when I first see the string
will not work.


Thanks


Eric
[Assuming you're using flex, redefining YYINPUT is almost never the
right thing to do. My suggestion would be to read the string
yourself, compute the checksum, then feed the string to flex with
yy_scan_bytes or yy_scan_buffer. -John]



Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.