Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:84100
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.216.43 as permitted sender)
MIME-Version: 1.0
Sender: yohgaki@gmail.com
In-Reply-To: <CAGa2bXYdGbpohnH-zWOrBUhJuTq7+J_AFYWBfXunuLt1Bz3D+Q@mail.gmail.com>
References: <1413875212.2624.3.camel@localhost.localdomain>
 <CA+9eiLtVUsbRQA9S0eagVfOc3aovhuak=QQmnBKp_BmgN4n28w@mail.gmail.com>
 <CAGKNXQEYquY_YQvTRv7uAhaycyCNG1FRwX_GYZ1dnHfhC-biZg@mail.gmail.com>
 <54469840.3070708@sugarcrm.com> <1414051917.2624.35.camel@localhost.localdomain>
 <CA+9eiLsC1dD9vp=uWqdYTXj7hC=mBnOMSs5B6KFngOQPd1m4OA@mail.gmail.com>
 <1414060726.2624.60.camel@localhost.localdomain> <1414072403.3228.3.camel@kuechenschabe>
 <87D717D5-273B-4A32-A3E5-83EBDFD314CB@ajf.me> <1414077690.3228.12.camel@kuechenschabe>
 <54495CF6.30608@sugarcrm.com> <1414130585.2624.64.camel@localhost.localdomain>
 <CAGAGxbaKK4P2D5epQTzEzNx3jcBP7XCPexGA4gyA3hdx3p+UKg@mail.gmail.com>
 <CAL=_i_nDz0XoVA5d4gqinYg9GinJUMW55+HLQ4y9O3Nj_URoxw@mail.gmail.com> <CAGa2bXYdGbpohnH-zWOrBUhJuTq7+J_AFYWBfXunuLt1Bz3D+Q@mail.gmail.com>
Date: Sun, 1 Mar 2015 19:14:17 +0900
Message-ID: <CAGa2bXbxJruf5ZBwXJz5Sjpqw_cQMWFuhyVM04CDyzf2DOQOPg@mail.gmail.com>
To: Joe Watkins <pthreads@pthreads.org>
Cc: Chris Wright <daverandom@php.net>, Stas Malyshev <smalyshev@sugarcrm.com>, 
	=?UTF-8?Q?Johannes_Schl=C3=BCter?= <johannes@schlueters.de>, 
	Andrea Faulds <ajf@ajf.me>, Dmitry Stogov <dmitry@zend.com>, 
	Philip Hofstetter <phofstetter@sensational.ch>, PHP Internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary=001a1135a3dcad500c0510375f37
Subject: Re: [PHP-DEV] [RFC] UString
From: yohgaki@ohgaki.net (Yasuo Ohgaki)

--001a1135a3dcad500c0510375f37
Content-Type: text/plain; charset=UTF-8

Hi Joe,

On Sun, Mar 1, 2015 at 6:14 PM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote:

> On Sat, Feb 28, 2015 at 3:48 PM, Joe Watkins <pthreads@pthreads.org>
> wrote:
>
>>     This is just a quick note to announce my intention to ready this RFC
>> for voting next week.
>>
>>     I know I'm a little late maybe, I was real sick most of last week, so
>> couldn't do anything useful.
>>
>>     A couple of us intend to fix outstanding issues on github and those
>> raised here, tidy the RFC and open the vote for 7.
>>
>>    I would ask anyone interested to scan through this thread and announce
>> concerns that are not mentioned asap.
>>
>
> I appreciate your proposal!
> Rowan pointed out some important things. I don't understand details as I
> don't read your code yet. I'll try to read and comment in a few days.
>

I guess you would like to start voting today or tomorrow, so I briefly read
your code.
I think your approach is good. I like UString be UTF-8 always by default
regardless
of other settings. i.e. default_charset, internal_encoding.

I see few missing key APIs that would be critical for multibyte char
handling, like
string length, string width, normalization, string conversions like Zenkaku
to Hankaku,
encoding(codepage) converter.  However, all of these may be added later as
they
are already implemented in ICU.

I think UString may be better to use UTF-8 always to make users life a
little simpler.
Your constructor only have codepage setting that is used as UString
codepage to support
other codepage(encodings).

Rather than to have various encoding support, I think constructor needs
encoding(codepage)
conversion feature. Codepage parameter is better to be used as "from
encoding(codepage)"
parameter and convert any encoding(codepage) to UTF-8. If conversion fails,
it should raise
exception. It's better to have forgiving API for malformed strings if user
explicitly specified to do so.

Constructor may be

   public function __construct([string $string [, string $source_codepage
[, string $substitute_char] ]);

$soure_codepage is source string encoding(codepage) and $string is
converted to UTF-8 always.
If $substitute_char is omitted, raise exception for invalid $string.
If $substitute_char is specified (it can be '' empty string), convert
$string according to $source_codepage
and just remove/replace invalid byte stream in $string.

With this constructor, string stored in UString object is always valid
UTF-8. Any character encoding
(including UTF-16/32 and 200 encoding names supported by ICU) may be used
as source string.

Since there will be no variable codepage setting for UString object,
followings may be removed.

    public static function getDefaultCodepage();
    public static function setDefaultCodepage(string $codepage);

ICU uses "codepage" as "character encoding", but it may be better to use
"character
encoding" as people are not used to ICU terminology.

This is what I thought. I didn't read your code carefully, so I might be
wrong. Please
correct me if I'm mistaken.

I suppose there are other people working on Unicode string based simpler
libraries.
I would like to hear opinion from them.

BTW, we really need byte_len(). strlen() is just confusing API... It's not
a scope of
this RFC, though.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

--001a1135a3dcad500c0510375f37--