Re: Internal Representation of Strings

Hans-Peter Diettrich wrote:
>> in general, UTF-8 takes less space than UTF-16 (and mixes much better with
>> code designed for ASCII), but some many languages like UTF-16 more
>> potentially because it works better when being treated as an array.
>
> This IMO is a typical misconception of English-only speakers, which have
> caused a lot of trouble in the evolution of programming languages :-(


I might add: the UTF encodings were not designed with compression issues
in mind. If space is an issue, use a compression algorithm instead,
because it will be more efficient. And only use UTF-16 for backwards
compatibility (libraries and other programs you must use uses it); UTF-8
avoids the endian issue, as one nowadays mostly agrees on how to sort
out the bits of a byte. (The BOM used in some UTF-16 code to sort out
endianess is not a part of the Unicode standard.) UTF-32 might be good
in cases were variable length or perhaps speed is needed (like
internally in programs); but this requires endianess to be sorted oit
between platforms.


      Hans Aberg

Return to the comp.compilers page.
Search the comp.compilers archives again.