Re: flex code memory usage

On 1 Feb 2004 12:56:49 -0500, Stefan Frühwirth
<stefan.fruehwirth@gmx.at> wrote:


>I'm trying to use flex generated code on a embedded system. I have to
>take care about not using too much heap, because the memory of this
>system is very small.
>I've seen that flex doesn't clean up very well (in my opinion), so my
>question is: is there any way to tell flex not to waste heap?
>Maybe i should use yy_delete_buffer?


What I did was to reinitialise the dynamic memory pool every time
before using flex/bison to do some parsing. This is a very quick
operation on my system (Infineon C167 with Keil tools). Whilst it
sounds drastic, the memory allocation library routines have no
protection against the RTOS context switching, so they can only be
used by one task anyway. I have no problems with fragmentation and my
code does not have to worry about keeping track of and freeing memory.
(There is no alloca function in the RTL and the compiler's code
generation effectively makes this impossible anyway.)


When I initially had problems with memory allocation, I added
debugging code to the memory allocation functions to check what
exactly was going on. The following code to provide the lexer with a
command line works for me and results in a lexer and parser that free
all of the memory they allocate. It also avoids copying the buffer
into another buffer allocated by the flex code.


void SetInputBuffer(char* buf)
{
unsigned len = strlen(buf); // Buffer length


yy_delete_buffer(YY_CURRENT_BUFFER);
// Remove previous buffer (if any)
buf[len]= 0;
buf[len+1] = 0; // Double null-terminate buffer,
yy_scan_buffer(buf, len+2); // Specify line to scan
}


You can also reduce the memory allocation by defining manifest
constants. My system uses:


For flex:


#define YY_USE_CONST // Put constant tables into ROM
#define YY_NEVER_INTERACTIVE 1 // Lexer is not interactive
#define YY_ALWAYS_INTERACTIVE 0
#define YYLMAX 128 // Maximum line length
#define YY_READ_BUF_SIZE 128
// Read buffer size (not used)
#define YY_MAIN 0 // No main program, thank-you.
#define YY_STACK_USED 100


For bison:


#define YYINITDEPTH 20 // Initial parser stack depth
#define YYMAXDEPTH 30 // Maximum parser stack depth


--
Chris Isbell
Southampton
UK

Return to the comp.compilers page.
Search the comp.compilers archives again.