Re: Question About YACC's Memory Management

In article <2371@kiwi.mpr.ca>, andrea@mprgate.mpr.ca (Jennitta Andrea) writes:
>
> I have a parser which is intended to be called repeatedly from a driver
> program. ...
>
> I have typed the value stack to contain pointers to characters. I
> malloc the memory required for each token in lex (assigning yylval to
> point to that block of memory) before returning the token to yacc. It
> appears that this memory is not cleaned up when yacc frees the value stack.
>
> Am I required to explicitly free each token once the parser has reduced
> a rule? ...
>


Yes.


But you probably don't really want to use malloc directly for this purpose,
anyway.


One way is to write or borrow an expandable string-table package.
When you are through with the strings, free them all at one go.


But why copy the tokens at all? A technique I have used when extreme
speed was wanted is to read the entire input file into a buffer at the
start. Then a token is coded for with a structure containing a pointer
into the file-buffer, and a character count. The structures, being all
the same size, can be allocated in blocks of a thousand or so, and kept
in free-lists for re-use rather than returned with free().


Actual example follows. The pointer into the buffer and the character
count are the second and third fields respectively.


typedef struct token
{ int type; /* as per gram.y and y.tab.h */
    char* token_image; /* pointer to token within line */
    int length; /* length of token */
    char* line_image; /* \n terminated line containing token */
    int line_num; /* line number within source file */
    struct src *src_file; /* file containing the token */
    int hval; /* hash-value for identifiers... */
}
Token;
--

Return to the comp.compilers page.
Search the comp.compilers archives again.