Re: Lexer capable of 16-bit characters

"Scott Stanchfield" <thetick@magelang.com>
10 Aug 1998 23:19:20 -0400

          From comp.compilers

Related articles
Lexer capable of 16-bit characters cdrom-dev@cdeurope.nl (Marcel) (1998-08-05)
Re: Lexer capable of 16-bit characters qjackson@wave.home.com (Quinn Tyler Jackson) (1998-08-10)
Re: Lexer capable of 16-bit characters gary@wheel.tiac.net (1998-08-10)
Re: Lexer capable of 16-bit characters cnwetzel@linguistik.uni-erlangen.de (Christian Wetzel) (1998-08-10)
Re: Lexer capable of 16-bit characters thetick@magelang.com (Scott Stanchfield) (1998-08-10)
Re: Lexer capable of 16-bit characters kleing@informatik.tu-muenchen.de (Gerwin Klein) (1998-08-24)
| List of all articles for this month |

From: "Scott Stanchfield" <thetick@magelang.com>
Newsgroups: comp.compilers
Date: 10 Aug 1998 23:19:20 -0400
Organization: MageLang Institute -- http://www.MageLang.com
References: 98-08-034 98-08-048
Keywords: PCCTS, i18n, lex

Nope -- 2.4 doesn't support UNICODE yet.


We've made the following changes (and a few others) in 2.4.0:


* Added the first release of ParseView, my visual debugger
    (check it out at http://java.magelang.com/~thetick/parseview)


* Set it up to support Readers instead of just InputStreams (a good step
toward UNICODE)


* Fixed the output and input (of antlr.Tool, the parser generator) to
properly handle newlines on all platforms


* added a -o option to specify an output dir




The problem left with UNICODE is that grammar analysis will blow up.
Everything is based on small bitsets to match characters. Works great for
ASCII ;)


We've come up with some very cool ways to implement "imaginary bitsets" to
deal with things like ~'a', but it's a matter of Ter getting a bit of free
time to implement them.


ANTLR (including ParseView) is totally free for whatever use anyone wants
(other than drumming up a silly lawsuit against Ter). We only ask that if
you use it you put a little ref about it in your docs somewhere.


You can get ANTLR & info at http://www.antlr.org (ANTLR 2.3.0)


NOTE: 2.4.0 is available as a "preview" (announced in
comp.compilers.tools.pccts) at
http://www.antlr.org/nirvana/antlr-2.4.0b2.zip THIS IS STILL A BETA!


--
-- Scott


Scott Stanchfield Santa Cruz, CA USA
    thetick@magelang.com http://java.magelang.com/~thetick


Gary D. Duzan wrote
> I believe the upcoming 2.4 version of ANTLR is supposed to include
>full Unicode support. Check out < http://www.antlr.org/ >.
--


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.