Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:78022
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.217.170 as permitted sender)
Message-ID: <543D3CA7.3060704@gmail.com>
Date: Tue, 14 Oct 2014 18:09:27 +0300
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Thunderbird/31.1.2
MIME-Version: 1.0
To: Chris Wright <daverandom@php.net>
CC: PHP Internals <internals@lists.php.net>
References: <543CE705.7030203@gmail.com> <CAGAGxbZ2hWj-9kBd+MALRaEoyyRCkVVCEVCbO0QbQx-N6JxiFQ@mail.gmail.com>
In-Reply-To: <CAGAGxbZ2hWj-9kBd+MALRaEoyyRCkVVCEVCbO0QbQx-N6JxiFQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] Unicode support
From: aleksey.tulinov@gmail.com (Aleksey Tulinov)

On 14/10/14 14:00, Chris Wright wrote:

Chris,

>> Latter is referring to difficulties like "excess memory usage" and "rewrite
>> the language". I'm developing an open-source Unicode implementation library
>> (nunicode), and it doesn't consume any heap at all, it also works on native
>> binary strings, as PHP does. Hence i thinks that maybe it could help with at
>> least these two problems.
>
> On the face of it, this implies a rather large performance hit and a
> tendency to overflow the stack much more readily, do you have any
> details on these elements?
>

I can't really tell if hit is going to be large before understanding 
what final result would be, at least approximately.

I can tell that internal complexity of nunicode is O(1) everywhere. I'm 
comparing performance to ICU and nunicode mostly outperforms it. I've 
compiled some numbers here: 
https://bitbucket.org/alekseyt/nunicode#markdown-header-performance-considerations

Regarding stack, i'm not sure if get the point. As far as i'm concerned, 
library does not have recursive calls, it does not have internal 
representation and does not allocate on stack aggressively. Everything 
works on immutable binary strings, stack will be used mostly for 
function calls.

But honestly, i feel like i'm not answering your question at all. Could 
you possibly clarify it?

>> I would appreciate if someone would point me to a good read or explain
>> collective opinion on this topic. I'm basically interested in the following
>> questions:
>
> The only additional thing I can find quickly is something Pierre put
> together earlier this year, when PHP6 (now 7) discussions were
> started:
> https://wiki.php.net/ideas/php6/unicode
>

Thank you, this is exactly what i was looking for.

I would appreciate if someone would comment on the following:

 > Some of the keys point we need to take care of are:
 >
 > 1) UTF-8 storage
 > 2) UTF-8 support for almost (if not all) existing string APIs
 > 3) Performance
 >
 > As of today, I did not find any library covering at least two of 
these key points.

I think i could claim that nunicode is covering at least two key points, 
maybe all of them, but i'm not sure about point 2). API do include 
operations on strings, but this API is simply following standard string 
functions (UTF equivalents of strcoll(), strchr(), strstr(), etc). Does 
that sound good or not?