Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:77334
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.216.41 as permitted sender)
MIME-Version: 1.0
Sender: are.you.winning@gmail.com
In-Reply-To: <CAEZPtU7TiLnU19UtG-Q+jUWJEPpJUyKZQoN3ZLnxO9pmUfhK5w@mail.gmail.com>
References: <CAOv1iQbHuxbHK2KVk=jooNq41ThGkKkKRsQSLznsLJc0as3Q9A@mail.gmail.com>
	<CAGAGxbbexAyomc_C+21NjTd_e4H7Rep9OrtqwocowA1fvOZ92Q@mail.gmail.com>
	<CAEZPtU7TiLnU19UtG-Q+jUWJEPpJUyKZQoN3ZLnxO9pmUfhK5w@mail.gmail.com>
Date: Fri, 19 Sep 2014 11:28:41 +0100
Message-ID: <CAGAGxbaOZVr8f7GVyGGA7CdPuuioXcSH2mBBwz=3b69ic0YOEA@mail.gmail.com>
To: Pierre Joye <pierre.php@gmail.com>
Cc: Chris Wright <cw@daverandom.com>, PHP internals <internals@lists.php.net>, 
	=?UTF-8?Q?K=C3=A9vin_Dunglas?= <dunglas@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: [PHP-DEV] Internationalized Domain Name support in FILTER_VALIDATE_URL
From: cw@daverandom.com (Chris Wright)

On 19 September 2014 10:58, Pierre Joye <pierre.php@gmail.com> wrote:
> Hi,
>
> On Sep 19, 2014 4:03 PM, "Chris Wright" <cw@daverandom.com> wrote:
>>
>> K=C3=A9vin
>>
>> On 18 September 2014 21:26, K=C3=A9vin Dunglas <dunglas@gmail.com> wrote=
:
>> > Hello,
>> >
>> > I'm working on enhancing the FILTER_VALIDATE_URL filter (
>> > https://github.com/php/php-src/pull/826).
>> > The current implementation does not support validation of
>> > internationalized
>> > domain names (i.e: http://www.acad=C3=A9mie-fran=C3=A7aise.fr/
>> > <http://www.xn--acadmie-franaise-npb1a.fr/>).
>> >
>> > Support of IDN validation can be easily added using ICU's
>> > uidna_toASCII()
>> > function.
>> >
>> > Is it acceptable to add a dependency to ICU for ext/filter?
>> > Another option is to add a HAVE_ICU constant in main/php_config.h and =
to
>> > validate IDN only if ICU is present.
>> >
>> > What strategy is preferred?
>>
>> I've done some work around this area previously, and all I will say
>> is: be careful with what you do with this from a userland PoV.
>>
>> PHP does not natively support IDN in stream open routines or SSL
>> verification routines. It will never support these things without at
>> least one of:
>> - a core dependency on ICU, libidn or similar
>> - moving streams into an extension so a dependency can be introduced
>> there (probably not sanely possible)
>> - an in-house NAMEPREP implementation (this is the hard part of IDN,
>> punycode itself is pretty trivial to implement once you have a
>> canonical set of codepoints)
>>
>> These things can be implemented with *a lot* of boilerplate in
>> userland when you have ext/intl, but it's not pretty. libcurl *can*
>> support IDN if it was built against libidn, I'm not sure if this is
>> currently the case in common distributions or not. Since one almost
>> never just validates a URL string, it's usually a precursor to
>> attempting to open it, this could lead to some pretty hefty wtfs.
>>
>> In short, while I'm generally for ext/filter being able to handle IDN,
>> I *do not* believe it should do it implicitly, it should require an
>> explicit flag, because it will break *a lot* of code if IDN is
>> suddenly treated as valid where it previously wasn't.
>
> I am really not sure about that especially the enabling by default part.
>
> The doc is pretty clear about what this filter supports and allowing idn =
may
> break a lot of codes out there.
>
> From an implementation point of view we may not need ICU to support IDN.
> Windows does not use it and there are license friendly decoder
> implementations too.

If we can agree on adding a core dependency on <some IDN support lib>,
I already have an experimental local branch that adds full IDN support
to streams. It's based on libidn but it would be easy enough to swap
it out for something else that provides the same functionality.

In my (biased) opinion, streams are a far more important element of
IDN support. Filter validation is just polish/a nicety on top.