Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:47264
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 74.125.82.42 as permitted sender)
DomainKey-Status: bad
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        b=PI2Lr7VZMovPUhAgrqzmOxZ8LHtJhzeJLccPyJSUpDxwADvx4a31KWZzTbQ8/ztFP6
         /h7RMBcDy2HRa3cQsXpYFMV2l6kGijG4n8leq/JTMaHF6BIVQoU2giU4RYTsmFEgYXaP
         LkN25qa5f0ngDsWINY2Mb4CcG8+/ZCofLFdNI=
MIME-Version: 1.0
In-Reply-To: <13008E62F851429F84B9FE2F3F230286@pc>
References: <4B9C9007.1080802@lsces.co.uk> <4B9C91D7.2050402@rowe-clan.net>
	 <13008E62F851429F84B9FE2F3F230286@pc>
Date: Sun, 14 Mar 2010 15:00:59 +0100
Message-ID: <fe05d1541003140700w41405635iafbe5ee0b084a30d@mail.gmail.com>
To: Stan Vassilev <sv_forums@fmethod.com>
Cc: "William A. Rowe Jr." <wrowe@rowe-clan.net>, internals@lists.php.net
Content-Type: text/plain; charset=ISO-8859-1
Subject: Re: [PHP-DEV] Where are we ACTUALLY on Unicode?
From: pierre.php@gmail.com (Pierre Joye)

hi,

On Sun, Mar 14, 2010 at 12:03 PM, Stan Vassilev <sv_forums@fmethod.com> wrote:

> UTF8 is good for text that contains mostly ASCII chars and the occasional
> Unicode international chars. It's also generally ok for storing and passing
> strings between apps.

That's not completely correct. UTF-8 is used out there for almost
unicode only applications as well. I'd to say it is a matter of what
the projects are written for. See below.

>
> Still, having variable-width encoding UTF8 or UTF16 doesn't cut it for
> general use to me as in tests it shows drastic slowdown when the script
> needs to do heavy string processing. I'd rather have it take more RAM for
> Unicode strings while being fast, and use Latin-1 when what I need is
> Latin-1.

The problem I have with UTF-16 is that it does not fit well with PHP
usage. While you are right about the performence vs memory usage, it
is sadly a small part of the problem. If you take a look at the
current implementation (trunk, which uses UTF-16), we have to convert
to UTF-8 almost everywhere as long as we deal with external APIs (file
systems or other libs). The win we may have from using UTF-16 is
almost completely lost by the conversions cost.

That obviously does not apply for scripts using only core PHP features
(no file access, no extension usage, etc.), but these scripts are
barely real worlds use cases.

Please not that I'm not voting against UTF-16 or for UTF-8, but I
would like to have a real evaluation this time, unlike what has been
done for trunk a couple of years ago.

Cheers,
-- 
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org