Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:73751
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.192.52 as permitted sender)
MIME-Version: 1.0
Sender: jakub.php@gmail.com
In-Reply-To: <CAEKnhAELLpTh45E8aBw_=Zq4pgXcD9vMm7jxockCHkX4wzn58A@mail.gmail.com>
References: <CA+kxMuQsoLhGgfPs5QN7tG18+HwM=6x=8W2ijyZ-O0u3BVtOMQ@mail.gmail.com>
	<CAGa2bXbkf6nwhR-umVg6AfO3boWD9Rpfp+pvYZRqjEq3ZgNwpw@mail.gmail.com>
	<CA+kxMuR-6vyMXvfnAwVwcH9+vk8TQ3w5Tb0gtQu4gNeTPdSybA@mail.gmail.com>
	<CAGa2bXaPzozXZ5dNu53EjJL+aToUR34GEnBHSBGRSzb1NXJe8Q@mail.gmail.com>
	<CA+kxMuSTATXDe800m1GQWxQZ8AA3MwEG9bgD1MB6e6cYJ=1voA@mail.gmail.com>
	<CAGa2bXb-0Bo_6E_5cSv_HvVAepeoRtdUZCy00PpROFNktHnKSA@mail.gmail.com>
	<CA+kxMuTPs053F3dz=-4Tar9tUJzW8wqspt83ccEAnZjdU9NXkA@mail.gmail.com>
	<CAGa2bXZ+6cX4vaOaEcX799VLEvs5MCgBtc5Ffz9zMLM5S12GWQ@mail.gmail.com>
	<52FF3BB7.8030408@lsces.co.uk>
	<CAGa2bXZ9dEhZxd_E+O7ofKA2rt+Qy1gRaeBrvEas4Ea_PeGTcQ@mail.gmail.com>
	<52FF465E.4040400@lsces.co.uk>
	<CAGa2bXa3Fj_2DF9CcOJRatLkE2uRFfkU+mVZ-7hq0RyO9FRYtw@mail.gmail.com>
	<CAEKnhAELLpTh45E8aBw_=Zq4pgXcD9vMm7jxockCHkX4wzn58A@mail.gmail.com>
Date: Mon, 21 Apr 2014 13:09:54 +0100
Message-ID: <CAEKnhAE+SzfxZ0P61B51tB3jt9T78hs2s0x_3F51rFXjpyzmLg@mail.gmail.com>
To: Yasuo Ohgaki <yohgaki@ohgaki.net>, Stas Malyshev <smalyshev@sugarcrm.com>
Cc: Lester Caine <lester@lsces.co.uk>, PHP internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary=001a11c3dc489a58f204f78c6046
Subject: Re: [PHP-DEV] utf-8 filenames in phar files.
From: bukka@php.net (Jakub Zelenka)

--001a11c3dc489a58f204f78c6046
Content-Type: text/plain; charset=UTF-8

On Mon, Apr 21, 2014 at 11:44 AM, Jakub Zelenka <bukka@php.net> wrote:

> On Sat, Feb 15, 2014 at 10:53 AM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote:
>
>> On Sat, Feb 15, 2014 at 7:50 PM, Lester Caine <lester@lsces.co.uk> wrote:
>>
>> > My previous post did not appear on the list ;)
>> >
>> >
>> > Yasuo Ohgaki wrote:
>> >
>> >>     A lot of the current confusion does seem to be based around the
>> >> Windows
>> >>     Wide-API as documented in 'The Problem' section of that document.
>> It
>> >> would
>> >>     seem that my 'naive' view of simply using UTF-8 strings is thwarted
>> >> by these
>> >>     problems?--
>> >>
>> >> Unicode is like one name with several encoding. We cannot get away from
>> >> conversions, normalization especially.
>> >>
>> >
>> > That is why personally I'm just looking at UTF8. Which is enough of a
>> mine
>> > field on it's own, but since a large swath of what we are working with
>> now
>> > is only UTF8 it does seem to be the right base going forward?
>> >
>>
>> I have problem, too. It seems someone is working on DKIM.
>> Anyway, there are problems, but UTF-8 is way to go. We just cannot remove
>> conversions. Normalization has number of issues, including security
>> related
>> one.
>>
>> Regards,
>>
>> --
>> Yasuo Ohgaki
>> yohgaki@ohgaki.net
>>
>
>
> Hi Stas,
>
> I just saw that the patch has been merged. There was an objection from
> Yasuo about normalization in past...
>
> What's worse the patch doesn't check UTF-8 correctly. It accepts invalid
> UTF-8. The correct spec is in
> https://tools.ietf.org/html/rfc3629#section-4 . I have got re2c
> implementation in jsond scanner :
> https://github.com/bukka/php-jsond/blob/master/jsond_scanner.re#L122-L134. As you can see it's very different from the provided impl.
>
> I think that accepting ill-formed UTF-8 would be a mistake and as such the
> patch should be reverted.
>
> Thanks
>
> Jakub
>
>
I have created a quick PR: https://github.com/php/php-src/pull/649 that is
fixing the ill-formed UTF-8 paths.

Jakub

--001a11c3dc489a58f204f78c6046--