Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:78024
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.216.176 as permitted sender)
MIME-Version: 1.0
Sender: are.you.winning@gmail.com
In-Reply-To: <543D3CA7.3060704@gmail.com>
References: <543CE705.7030203@gmail.com>
	<CAGAGxbZ2hWj-9kBd+MALRaEoyyRCkVVCEVCbO0QbQx-N6JxiFQ@mail.gmail.com>
	<543D3CA7.3060704@gmail.com>
Date: Tue, 14 Oct 2014 16:27:36 +0100
Message-ID: <CAGAGxbY0Mctk+UJtjRkaWv_azisxMKxPJ9QebQaNPcsd4q6+5Q@mail.gmail.com>
To: Aleksey Tulinov <aleksey.tulinov@gmail.com>
Cc: Chris Wright <daverandom@php.net>, PHP Internals <internals@lists.php.net>
Content-Type: text/plain; charset=UTF-8
Subject: Re: [PHP-DEV] Unicode support
From: daverandom@php.net (Chris Wright)

On 14 October 2014 16:09, Aleksey Tulinov <aleksey.tulinov@gmail.com> wrote:
> On 14/10/14 14:00, Chris Wright wrote:
>
> Chris,
>
>>> Latter is referring to difficulties like "excess memory usage" and
>>> "rewrite
>>> the language". I'm developing an open-source Unicode implementation
>>> library
>>> (nunicode), and it doesn't consume any heap at all, it also works on
>>> native
>>> binary strings, as PHP does. Hence i thinks that maybe it could help with
>>> at
>>> least these two problems.
>>
>>
>> On the face of it, this implies a rather large performance hit and a
>> tendency to overflow the stack much more readily, do you have any
>> details on these elements?
>>
>
> I can't really tell if hit is going to be large before understanding what
> final result would be, at least approximately.
>
> I can tell that internal complexity of nunicode is O(1) everywhere. I'm
> comparing performance to ICU and nunicode mostly outperforms it. I've
> compiled some numbers here:
> https://bitbucket.org/alekseyt/nunicode#markdown-header-performance-considerations

Great, thanks for this

> Regarding stack, i'm not sure if get the point. As far as i'm concerned,
> library does not have recursive calls, it does not have internal
> representation and does not allocate on stack aggressively. Everything works
> on immutable binary strings, stack will be used mostly for function calls.
>
> But honestly, i feel like i'm not answering your question at all. Could you
> possibly clarify it?

My apologies, this was a case of typing before thinking properly. I
was envisaging very large stack frames due to large char arrays being
allocated on the stack but when I actually apply my brain to what you
are doing I realise that this isn't going to be the case.

Carry on.

>>> I would appreciate if someone would point me to a good read or explain
>>> collective opinion on this topic. I'm basically interested in the
>>> following
>>> questions:
>>
>>
>> The only additional thing I can find quickly is something Pierre put
>> together earlier this year, when PHP6 (now 7) discussions were
>> started:
>> https://wiki.php.net/ideas/php6/unicode
>>
>
> Thank you, this is exactly what i was looking for.
>
> I would appreciate if someone would comment on the following:
>
>> Some of the keys point we need to take care of are:
>>
>> 1) UTF-8 storage
>> 2) UTF-8 support for almost (if not all) existing string APIs
>> 3) Performance
>>
>> As of today, I did not find any library covering at least two of these key
>> points.
>
> I think i could claim that nunicode is covering at least two key points,
> maybe all of them, but i'm not sure about point 2). API do include
> operations on strings, but this API is simply following standard string
> functions (UTF equivalents of strcoll(), strchr(), strstr(), etc). Does that
> sound good or not?