Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:73075
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain googlemail.com designates 209.85.214.43 as permitted sender)
Message-ID: <53203687.7090405@googlemail.com>
Date: Wed, 12 Mar 2014 11:27:19 +0100
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0
MIME-Version: 1.0
To: Lester Caine <lester@lsces.co.uk>, 
 PHP Developers Mailing List <internals@lists.php.net>
References: <531EE602.3090207@lsces.co.uk> <531EEE2A.2000602@googlemail.com> <531F0146.5010701@lsces.co.uk> <53202DC5.4010306@googlemail.com> <532033E1.60602@lsces.co.uk>
In-Reply-To: <532033E1.60602@lsces.co.uk>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] Unicode strings?
From: cryptocompress@googlemail.com (Crypto Compress)

Am 12.03.2014 11:16, schrieb Lester Caine:
> Crypto Compress wrote:
>>>>> The very first problem I hit is ICU's limitation to 32bit string 
>>>>> lengths. How
>>>>> does the switch to 64bit string length on 64 bit platforms impinge 
>>>>> on this.
>>>>> While I can see the advantage of this particular change, would 
>>>>> that also now
>>>>> require our own version of ICU capable of also handling longer 
>>>>> strings? This
>>>>> probably falls out in the wash of my next point ...
>>>>
>>>> Where have you found this information? Can you please provide 
>>>> source for this?
>>>
>>> This information has been published in several places on the list 
>>> and in the
>>> wiki already ...
>>> http://userguide.icu-project.org/strings/utf-8 for the ICU, and the 
>>> RFC's here
>>> for 64 bit improvements to PHP ...
>>
>> Quote #1: "You can request 64 or 32 bits with the 
>> --with-library-bits= option, ..."
>> Quote #2: "Strings are represented as UChar * as the base string type."
>>
>> http://userguide.icu-project.org/icufaq#TOC-How-do-I-get-32--or-64-bit-versions-of-the-ICU-libraries- 
>>
>>
>> String length is platform dependent.
>
> It is not only PHP that has hidden gems of information buried in the 
> documentation, but ...
> "For UTF-8 strings, ICU normally uses (const) char * pointers and 
> int32_t lengths"
>
> The question here is how UTF-8 default works in ICU as we want to 
> actually avoid using UChar altogether using UText instead - I think?
>

http://www.icu-project.org/apiref/icu4c/utext_8h.html

int64_t     utext_nativeLength (UText *ut)     Get the length of the text.

Looks like UText is utf-16.