Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:72738
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.216.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <530740B9.5000509@lsces.co.uk>
References: <CAEZPtU4zzN=Xxs03XbBGEM2UTVe3jXH5TAaXUqjsVkzm3kOOyg@mail.gmail.com>
	<53061982.2050901@googlemail.com>
	<CAEZPtU5qnhwcq1BCkqG99dXJ8F=p6K8ABALacCiAOvPBQ=huYQ@mail.gmail.com>
	<53066DE9.4090809@googlemail.com>
	<CAEZPtU6b+aLuma-nxy84BYVZdOOTr7+ZGZhOVvWMNfbZ4RdPNg@mail.gmail.com>
	<530740B9.5000509@lsces.co.uk>
Date: Fri, 21 Feb 2014 13:30:14 +0100
Message-ID: <CAEZPtU4-yG_FN2uL59XwwpDsc8BrsHdAUcfg-5iWAH1jSmJF6w@mail.gmail.com>
To: Lester Caine <lester@lsces.co.uk>
Cc: PHP internals <internals@lists.php.net>
Content-Type: text/plain; charset=UTF-8
Subject: Re: [PHP-DEV] [php6] Unicode support, options?
From: pierre.php@gmail.com (Pierre Joye)

On Fri, Feb 21, 2014 at 1:04 PM, Lester Caine <lester@lsces.co.uk> wrote:
> Pierre Joye wrote:
>>>
>>> What do you understand by "storage"?
>>
>> To have string stored as UTF-8 only, no conversion required for 99% of our
>> use.
>
>
> I think that the first thing that needs to be agreed on is if there will be
> support for UTF-8 in the core? As has already been said, in many places this
> currently just works and so blocking that may be more of a problem now? The
> question surly is "What is the 1% that needs some extra work?"

I think we pretty much agree already that we need UTF-8 as the base,
meaning are stored in UTF-8. Conversions may be needed for advanced
usages provided by ICU (or maybe not, I just do not know for sure
now).

> I light library would be most appropriate for filling the gaps currently
> created by use of UTF-8 strings in the core? It is not until one starts
> adding the mbstring level of string processing that a more powerful library
> is required. Something that simply ensures UTF-8 strings are valid and can
> carry out comparisons as required?

it is more than only comparison. If only comparison, additions and the
likes, utf8proc is enough, or librope with some additions.

> The black hole is still 'case sensitivity' and it is perhaps laying down a
> 'light' set of rules for this which would allow a path forward? As I have
> indicated, I'd prefer simply dropping case insensitivity, but a compromise
> might be to retain it where a string length does not change, and a clean
> reverse transform exists? So a library that provides that comparison as part
> of the core package?

I do not care much about languages support for UTF-8 names for
methods, functons, variables etc. My take on it is that we should
stick to ASCII for it and be done with that. But that's only my
opinion :)

We may end writing our own library for the core operations... But I
would prefer to avoid that as it is really not a trivial task.

Cheers,
-- 
Pierre

@pierrejoye | http://www.libgd.org