Re: Please provide a learning path for mastering lexical analysis languages

luser droog <>
Sun, 8 May 2022 18:08:24 -0700 (PDT)

          From comp.compilers

Related articles
| List of all articles for this month |

From: luser droog <>
Newsgroups: comp.compilers
Date: Sun, 8 May 2022 18:08:24 -0700 (PDT)
Organization: Compilers Central
References: 22-05-010
Injection-Info:; posting-host=""; logging-data="66170"; mail-complaints-to=""
Keywords: lex, parse
Posted-Date: 13 May 2022 13:02:41 EDT
In-Reply-To: 22-05-010

On Friday, May 6, 2022 at 11:13:52 AM UTC-5, Roger L Costello wrote:
> Hi Folks,
> I want to master lexical analysis.
> In Chris Christopher's last post he said:
> > [Flex] is not even as powerful as some other lexical analysis languages
> > and even exploiting its power often requires things that cannot be
> > expressed in Flex alone nor can they be done in ways that are simple
> > and easy to reason about.
> I am currently in the process of mastering Flex. I feel that mastering Flex
> will give me a foundation for learning other more advanced lexical analysis
> languages. "This XYZ lexical analysis language does it this way, Flex does it
> this other way. Ah, yes, I can see how XYZ's way is simpler and more
> powerful."
> From Chris's post I see that there is much to learn beyond Flex. Thank you
> Chris.
> Can you provide a learning path, please? A learning path for mastering lexical
> analysis languages.
> After I master Flex, what lexical analysis language should I then master? And
> after mastering that, what is the next lexical analysis language that I should
> master?
> /Roger

I'd like to echo George Neuner's remark:

> it's the methods used [by] the tools that are important.

In my opinion the best way to truly "master"* such tools is to step
away from the tools per se and delve into the algorithms and data
structures involved. The best book for this (again, in my opinion) is
Marvin Minsky's "Computation: Finite and Infinite Machines". He has a
whole chapter about Regular Languages and their association with
Finite State Automata and the algorithms for translating back and
forth between a Regular Expression and its corresponding Finite State

And to really get a handle on when to use flex vs bison vs whatever
other tool, you'll also need to know something about where Regular
Languages sit in the Chomsky Hierarchy of Phrase Structured Languages.
I'm not sure what the best resource is for this part. Chomsky's
writings on linguistics are formidable for the non-specialist (even
for very motivated amateurs) so I don't really recommend going
straight to the source. Maybe the intro chapter from the Dragon Book.
Or whatever Aho or Ullman has written on it.

* of course using my own private definition of mastery.

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.