Re: TeX syntax?

On Sunday, April 4, 2021 at 6:12:06 PM UTC-7, Rock Brentwood wrote:
> [ This is a followup to a thread from 2007. ]
> >I've looked high and low without success. Where can i find
> >something resembling the BNF of Knuth's TeX typesetting syntax?
> It's in the file weave.web, section 14.
>
> The syntax for TeX was written as a context-sensitive translation
> grammar, suitable for a streaming translator, rather than a
> context-free grammar. It may be possible to convert it to one (either
> directly or as a context-free enveloping grammar with semantic
> constraints). That's a matter that may be worth looking into. But in
> its present form, there is no tree-building required or involved: it
> can stream. The distinction is analogous to that between SSDT's versus
> SDT's ([S]SDT = [simple-]syntax-directed translations). SSDT's can be
> streamed, SDT's require stacking or treeing values and, in effect, SDT
> = SSDT + value-stacking/treeing.


> TeX is written in Web which is essentially Pascal + hyper-linked
> comments. It is also in C-Web on the main TeX distribution site, which
> is C + hyper-linked comments. They *can* be converted directly to more
> normal programs with the comments embedded. I did so in the local
> versions of my older MiKTeX distribution, but haven't
> regression-tested it yet - since I haven't established a working
> baseline yet to work off of.


(snip)


It seems to me that macro processors were popular in the 1970's.


Some years ago, I used to work with Mortran, which is a macro processor
meant to be used to process an improved language like Fortran, which is
then converted into Fortran. It is free-form with semicolon terminated
statements, and with block structured if-then-else and for loops.
It is all done with a relatively simple macro processor written in about
as close as possible to standard Fortran 66. The only extension needed
is the ability to compare value read in as characters.


The macro format is very simple, except that macros have the ability
to define other macros. Then, as is usual with compilers, the whole
processor is written in Mortran and translated to Fortran.


So, TeX is also written as a macro processsor, with relatively few
built-in operations, and many more defined through the macro facility.


The input sequences recognized by macros likely can't be generalized.
A macro can specify just about any combination of characters to be
included between macro arguments. Not so many use that, but they can.


There are, then, three ways to name macros. Most common is the
escape character, usually \, followed by some number of letters.
They can also be the escape character folllowed by one non-letter.
The characters which are letters can be changed at any time, but
most of the time, for user-level macros, letters are letters.


The third possibility is that a macro can be any one character with
the catcode of active. No escape character is needed.


It seems that there are now some programs that accept the TeX
way of writing mathematical equations, including the standard
macros for special characters. But not the general ability to
write macros, such as defining new characters or renaming
old ones, or any of the more strange things you can do. I suppose
BNF can be written for that, but likely not for the more general
syntax.
[The m4 macro processor tokenizes its input and then checks to see
if a token is a named macro. It's still used for autoconf
configuration scripts. -John]

Return to the comp.compilers page.
Search the comp.compilers archives again.