Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:71297
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.215.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <CAGa2bXYvCkA2UxFjQUzxMgSNKgDvGoU_HSVR1HedJttgpoN0-g@mail.gmail.com>
References: <CAGa2bXauxj6Sca8qYk2cxQ=TeG4bjWgcv-Aqy3cJe35ap72Yfw@mail.gmail.com>
	<CAF+90c-ZgsCczU71TJap1yeaNhKgQeoUR5oLKwwHHaoOD8z_ig@mail.gmail.com>
	<CAGa2bXb6jLX8HMN-rL7t5j5R_0SaeRkd5r4_MrmQFfsJxxaSxQ@mail.gmail.com>
	<CAF+90c-JBP5iY_DmkSq7Ao6w4rkWoFp9xdwJ9frtQGdxyLyztQ@mail.gmail.com>
	<CAGa2bXb56tka6Ac6TkwxXfjNLUAVbuj3fDQa1SZLE6j8VwOFLw@mail.gmail.com>
	<CAEZPtU5roOCqdJPnDbPU4yU85HrnFGYXjpSBhPQ+BmYHQ7qh4Q@mail.gmail.com>
	<CAGa2bXYvCkA2UxFjQUzxMgSNKgDvGoU_HSVR1HedJttgpoN0-g@mail.gmail.com>
Date: Mon, 20 Jan 2014 07:38:12 +0100
Message-ID: <CAEZPtU6rxc8dSSVfb_8eyz5NpYm40p+hN_w+QonQ0A5NcFrJ3w@mail.gmail.com>
To: Yasuo Ohgaki <yohgaki@ohgaki.net>
Cc: PHP internals <internals@lists.php.net>, Nikita Popov <nikita.ppv@gmail.com>
Content-Type: multipart/alternative; boundary=089e0122797ac578a004f06122d6
Subject: Re: [PHP-DEV] [RFC] Multibyte char handling
From: pierre.php@gmail.com (Pierre Joye)

--089e0122797ac578a004f06122d6
Content-Type: text/plain; charset=UTF-8

On Jan 19, 2014 11:48 PM, "Yasuo Ohgaki" <yohgaki@ohgaki.net> wrote:
>
> Hi Pierre,
>
> On Mon, Jan 20, 2014 at 1:09 AM, Pierre Joye <pierre.php@gmail.com> wrote:
>>
>> On Thu, Jan 16, 2014 at 11:47 PM, Yasuo Ohgaki <yohgaki@ohgaki.net>
wrote:
>> > Hi Nikita,
>> >
>> > On Fri, Jan 17, 2014 at 7:38 AM, Nikita Popov <nikita.ppv@gmail.com>
wrote:
>> >
>> >> No, I don't want a locale-based approach. I want the string functions
to
>> >> stay as is. Multibyte variants of the functions can be added to the
>> >> multibyte extension.
>> >
>> >
>> > Creating mb_*() function would not solve security issues of
>> > multibyte char handling since multibyte aware functions are
>> > optional feature.
>>
>> We never supported nor claimed that these functions are multi bytes
>> safe. However I actually fully understand that we should solve this
>> problem, in one way or another.
>>
>> > However, it may work if PHP compiles mbstring by default and
>> > discourage use of addslashes()/var_export()/stripslashes()
>> > in favor of mb_*() variants.
>>
>> I do not think we should discourage the use of these functions but
>> clearly document to rely on mb_* APIs as long as multi bytes support
>> is required.
>>
>> I join other about not making any optional arguments in the existing
>> APIs, for a couple of reasons:
>>
>> 1. it does not solve anything as people still have to update their
>> code, and they won't unless maybe if they read the doc/changelog
>> 2. It is really not a clean solution
>> 3. we already have many duplicate functions in mb, it has worked well
>> so far and we can add the ones discussed here
>
>
> I'll leave existing ext/standard functions alone.

:)

>> The last question was about relying on locale. This is absolutely not
>> a solution. Locale has been proven to be totally unreliable, buggy and
>> unsafe. Let alone the total lack of real posix locale support on
>> Windows.
>
>
> mb_escape_shell_arg()/mb_escape_shell_cmd() need locale based
> solution, since there aren't good way to detect terminal encoding. I'll
> make mb version explicitly overrides this behavior by explicitly
specifying
> encoding.
>

Sounds good

> On UNIXes, UTF-8 encoding is popular terminal encoding, but there
> would be systems using other encoding such as EUC, or even SJIS, BIG5.

Right, and as far as I remember UTF-8 does not suffer from this problem.

> Windows uses different encoding for terminal encoding according to
locale,
> so it's much more complex.
>

Let me provide a function to detect it, but we need something to normalize
the names. Do we have such thing in mbstring?

> This is the reason why I would use locale. However, this implementation
> is debatable.
>

Yes :)

> We could say "Users should explicitly specify terminal encoding
> by themselves". In fact, I prefer this even if I am about to implement
> mb_escape_shell_*() using locale for automatic encoding detection.
>
> It may be better to raise E_NOTICE at least if encoding parameter is
> omitted for mb_escape_shell_*().

Notice sounds good too.

>
>> For anything related to locale, formats or encoding, we should rely on
>> intl (ICU) and not on systems's locale. This is the only way to be
>> portable, safe and updated.
>
>
> I agree.
> I also would like to propose
>
> https://wiki.php.net/rfc/altmbstring   - ICU version of mbstring
>

Oh, very nice.

> for future release. Most work has done by Moriyoshi. We may try to
> switch to it now, but I suppose there is not enough time for 5.6.

What's the status? We still have some time :)

Cheers,
Pierre

--089e0122797ac578a004f06122d6--