Avoid "symbol" and "token" terms ambiguity

Vasily Karyaev <basil@ulstu.ru>
9 Jan 2005 20:55:56 -0500

          From comp.compilers

Related articles
Avoid "symbol" and "token" terms ambiguity basil@ulstu.ru (Vasily Karyaev) (2005-01-09)
Re: Avoid "symbol" and "token" terms ambiguity vbdis@aol.com (2005-01-30)
Re: Avoid "symbol" and "token" terms ambiguity vbdis@aol.com (2005-02-03)
| List of all articles for this month |

From: Vasily Karyaev <basil@ulstu.ru>
Newsgroups: comp.compilers
Date: 9 Jan 2005 20:55:56 -0500
Organization: Compilers Central
Keywords: question
Posted-Date: 09 Jan 2005 20:55:56 EST
Content-Disposition: inline

Hello folks,

I am implementing a parser generator library which will allow creating
LR parsers in runtime.

And I am having some doubts concerning naming conventions.

I need to assign appropriate class names for the two related but
different entities:

1. "A grammar symbol (token)". Grammar symbols (tokens) compound
grammar productions:

S -> A '+' C;

where S, A, C, '+' are the grammar symbols (tokens).

In essence, grammar symbols (tokens) represent classes of "parser
symbols (tokens)" (see below).

2. "A parser symbol (token)". Parser symbols (tokens) appear in the
parsing engine stack and lookahead buffer. Every parser symbol (token)
refers to a grammar symbol (token), as to its class, and also contains
some additional attributes (such as the symbol value and position in
the input stream).

The problem is: because of my lack in English, I am unable to assign
short but distinct terms for the above mentioned entities.

It appears to me that both terms "symbol" and "token" are usually used
for representing the both entities. So, I have arbitrarily chosen the
following naming convention:

1. The class name "Token" stands for representing "grammar tokens",
which compound the grammar productions.

2. The class name "Symbol" stands for representing "parser symbols",
appearing in the parser engine stack and lookahead buffer. Instances
of class "Symbol" refer to the appropriate "Token" and also contain
some additional attributes.

Unfortunately, I am not sure if this convention is consistent. I
would really appreciate if someone from this community could help me
to choose appropriate class names to distinguish grammar and parser
symbols (tokens).

Thank You.
Vasily Karyaev

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.