RE: Flex and Dynamic Data buffer: Advanced technique

Quinn Tyler Jackson <>
23 May 2002 00:45:48 -0400

          From comp.compilers

Related articles
Flex and Dynamic Data buffer: Advanced technique (2002-05-17)
RE: Flex and Dynamic Data buffer: Advanced technique (Quinn Tyler Jackson) (2002-05-23)
| List of all articles for this month |

From: Quinn Tyler Jackson <>
Newsgroups: comp.compilers
Date: 23 May 2002 00:45:48 -0400
Organization: Compilers Central
References: 02-05-088
Keywords: lex
Posted-Date: 23 May 2002 00:45:48 EDT

> Now after some time, flex receives more data. Now buffer content is "lo hi"
> Flex should restart and be able to detect the token "Hello" which is splitted
> between two buffers "Hel" and "lo hi".
> I know I can do it with making one more process/thread which waits on pipe
> or by other mechanism, unfortunately I can;t use it since this has to be
> done by a single process. This process is doing other work also so it cant
> simply be in loop to make flex waiting.

It sounds to me like an event-based mechanism might work.

1. Code calling parser registers an callback with the lexer.
2. Code calling parser -- calls parser.
3. Lexer calls back the callback when its buffer is exhausted.
4. In the callback, code does what it is it has to do by pumping an event queue.
5. When more input becomes available, callback code fills the buffer, and
returns from the callback, causing the lexer to resume where it left off.

Ugly -- but it may suit yours needs.

Quinn Tyler Jackson

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.