Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:73748
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.216.43 as permitted sender)
MIME-Version: 1.0
Sender: jakub.php@gmail.com
In-Reply-To: <CAGa2bXa3Fj_2DF9CcOJRatLkE2uRFfkU+mVZ-7hq0RyO9FRYtw@mail.gmail.com>
References: <CA+kxMuQsoLhGgfPs5QN7tG18+HwM=6x=8W2ijyZ-O0u3BVtOMQ@mail.gmail.com>
	<CAGa2bXbkf6nwhR-umVg6AfO3boWD9Rpfp+pvYZRqjEq3ZgNwpw@mail.gmail.com>
	<CA+kxMuR-6vyMXvfnAwVwcH9+vk8TQ3w5Tb0gtQu4gNeTPdSybA@mail.gmail.com>
	<CAGa2bXaPzozXZ5dNu53EjJL+aToUR34GEnBHSBGRSzb1NXJe8Q@mail.gmail.com>
	<CA+kxMuSTATXDe800m1GQWxQZ8AA3MwEG9bgD1MB6e6cYJ=1voA@mail.gmail.com>
	<CAGa2bXb-0Bo_6E_5cSv_HvVAepeoRtdUZCy00PpROFNktHnKSA@mail.gmail.com>
	<CA+kxMuTPs053F3dz=-4Tar9tUJzW8wqspt83ccEAnZjdU9NXkA@mail.gmail.com>
	<CAGa2bXZ+6cX4vaOaEcX799VLEvs5MCgBtc5Ffz9zMLM5S12GWQ@mail.gmail.com>
	<52FF3BB7.8030408@lsces.co.uk>
	<CAGa2bXZ9dEhZxd_E+O7ofKA2rt+Qy1gRaeBrvEas4Ea_PeGTcQ@mail.gmail.com>
	<52FF465E.4040400@lsces.co.uk>
	<CAGa2bXa3Fj_2DF9CcOJRatLkE2uRFfkU+mVZ-7hq0RyO9FRYtw@mail.gmail.com>
Date: Mon, 21 Apr 2014 11:44:08 +0100
Message-ID: <CAEKnhAELLpTh45E8aBw_=Zq4pgXcD9vMm7jxockCHkX4wzn58A@mail.gmail.com>
To: Yasuo Ohgaki <yohgaki@ohgaki.net>, Stas Malyshev <smalyshev@sugarcrm.com>
Cc: Lester Caine <lester@lsces.co.uk>, PHP internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary=001a11c2ef6eda743c04f78b2d64
Subject: Re: [PHP-DEV] utf-8 filenames in phar files.
From: bukka@php.net (Jakub Zelenka)

--001a11c2ef6eda743c04f78b2d64
Content-Type: text/plain; charset=UTF-8

On Sat, Feb 15, 2014 at 10:53 AM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote:

> On Sat, Feb 15, 2014 at 7:50 PM, Lester Caine <lester@lsces.co.uk> wrote:
>
> > My previous post did not appear on the list ;)
> >
> >
> > Yasuo Ohgaki wrote:
> >
> >>     A lot of the current confusion does seem to be based around the
> >> Windows
> >>     Wide-API as documented in 'The Problem' section of that document. It
> >> would
> >>     seem that my 'naive' view of simply using UTF-8 strings is thwarted
> >> by these
> >>     problems?--
> >>
> >> Unicode is like one name with several encoding. We cannot get away from
> >> conversions, normalization especially.
> >>
> >
> > That is why personally I'm just looking at UTF8. Which is enough of a
> mine
> > field on it's own, but since a large swath of what we are working with
> now
> > is only UTF8 it does seem to be the right base going forward?
> >
>
> I have problem, too. It seems someone is working on DKIM.
> Anyway, there are problems, but UTF-8 is way to go. We just cannot remove
> conversions. Normalization has number of issues, including security related
> one.
>
> Regards,
>
> --
> Yasuo Ohgaki
> yohgaki@ohgaki.net
>


Hi Stas,

I just saw that the patch has been merged. There was an objection from
Yasuo about normalization in past...

What's worse the patch doesn't check UTF-8 correctly. It accepts invalid
UTF-8. The correct spec is in
https://tools.ietf.org/html/rfc3629#section-4. I have got re2c
implementation in jsond scanner :
https://github.com/bukka/php-jsond/blob/master/jsond_scanner.re#L122-L134 .
As you can see it's very different from the provided impl.

I think that accepting ill-formed UTF-8 would be a mistake and as such the
patch should be reverted.

Thanks

Jakub

--001a11c2ef6eda743c04f78b2d64--