Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:72617
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.217.181 as permitted sender)
MIME-Version: 1.0
Sender: yohgaki@gmail.com
In-Reply-To: <CA+kxMuSTATXDe800m1GQWxQZ8AA3MwEG9bgD1MB6e6cYJ=1voA@mail.gmail.com>
References: <CA+kxMuQsoLhGgfPs5QN7tG18+HwM=6x=8W2ijyZ-O0u3BVtOMQ@mail.gmail.com>
 <CAGa2bXbkf6nwhR-umVg6AfO3boWD9Rpfp+pvYZRqjEq3ZgNwpw@mail.gmail.com>
 <CA+kxMuR-6vyMXvfnAwVwcH9+vk8TQ3w5Tb0gtQu4gNeTPdSybA@mail.gmail.com>
 <CAGa2bXaPzozXZ5dNu53EjJL+aToUR34GEnBHSBGRSzb1NXJe8Q@mail.gmail.com> <CA+kxMuSTATXDe800m1GQWxQZ8AA3MwEG9bgD1MB6e6cYJ=1voA@mail.gmail.com>
Date: Sat, 15 Feb 2014 08:56:36 +0900
Message-ID: <CAGa2bXb-0Bo_6E_5cSv_HvVAepeoRtdUZCy00PpROFNktHnKSA@mail.gmail.com>
To: Dan Ackroyd <danack@basereality.com>
Cc: "internals@lists.php.net" <internals@lists.php.net>
Content-Type: multipart/alternative; boundary=001a11c36c94c9894304f2669025
Subject: Re: [PHP-DEV] utf-8 filenames in phar files.
From: yohgaki@ohgaki.net (Yasuo Ohgaki)

--001a11c36c94c9894304f2669025
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Dan,

On Sat, Feb 15, 2014 at 1:11 AM, Dan Ackroyd <danack@basereality.com> wrote=
:

> That is not an issue as:
>
> i) Phar files produced on a windows machine should be identical to
> those produced on a Linux or OSX box.
>
> ii) There is a test in the phar code, so that if you do have filenames
> that are degenerate after normalising, the extraction throws an error.
> e.g. for the files
>
>     $filename1 =3D "Am\xC3\xA9lie.txt";
>     $filename2 =3D "Am\x65\xCC\x81lie.txt";
>
> If you add both to a phar archive and then attempt to extract them
> both you get the error:
>
>     "Cannot extract "Am=C3=A9lie.txt" to "output/Am=C3=A9lie.txt", path a=
lready
> exists"
>

I suppose there is no normalization code in phar, so your system(OS / file
system) normalizes file name.

Depending on system's normalization is not good.

 - File name could be NFC or NFD
 - File names in phar may differ by system
 - Systems that do not normalize Unicode actively exist

I do see file name normalization issue on my Linux/Windows and OSX with
git. (core.precomposeunicode=3Dtrue is required for correct operation on OS=
X)
I suggest to apply NFC normalization to avoid issue, like git.

core.precomposeunicode
This option is only used by Mac OS implementation of Git. When
core.precomposeunicode=3Dtrue, Git reverts the unicode decomposition of
filenames done by Mac OS. This is useful when sharing a repository between
Mac OS and Linux or Windows. (Git for Windows 1.7.10 or higher is needed,
or Git under cygwin 1.7). When false, file names are handled fully
transparent by Git, which is backward compatible with older versions of Git=
.
http://git-scm.com/docs/git-config

As Rowan pointed out, although ICU is detected by acinclude.m4 always, #if
should be used for ICU/intl related code. (intl uses ICU, use intl =3D use
ICU. I think it's better not to rely on intl. It may be disabled or can be
DL module. There are systems without ICU also.)

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

--001a11c36c94c9894304f2669025--