Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:126198
Content-Type: text/plain;
	charset=utf-8
Precedence: bulk
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.300.87.4.3\))
Subject: Re: [PHP-DEV] [RFC] [Discussion] Add WHATWG compliant URL parsing API
In-Reply-To: <1BCB4144-231D-45EA-A914-98EE8F0F503A@automattic.com>
Date: Fri, 3 Jan 2025 09:18:33 +0200
Cc: Internals <internals@lists.php.net>
Content-Transfer-Encoding: quoted-printable
Message-ID: <8E614C9C-BA85-45D8-9A4E-A30D69981C5D@automattic.com>
References: <CAH5C8xUb1O20ZDrOQNC=ckFxHUUWSK7sw_njQQzFBd0qgQqoww@mail.gmail.com>
 <dd61999c-1ebd-4765-9add-cd8065968965@gmail.com>
 <CAOV5rgYZ_s1of-igLFEK7oqqh6=3HeYv0=JvvKvvjkcp0F6Q9Q@mail.gmail.com>
 <CAH5C8xV67frqOBCvLt73RM7QO86_pu40sHP5h71_kDRbKPtA8Q@mail.gmail.com>
 <1BCB4144-231D-45EA-A914-98EE8F0F503A@automattic.com>
To: =?utf-8?B?TcOhdMOpIEtvY3Npcw==?= <kocsismate90@gmail.com>
From: dennis.snell@automattic.com (Dennis Snell)

It seems that I=E2=80=99ve mucked up the mailing list again by deleting =
an old message I intended on replying to. Apologies all around for =
replying to an older message of my own.
M=C3=A1t=C3=A9, thanks for your continued work on the URL parsing RFC. =
I=E2=80=99m only no returning from a bit of an extended leave, so I =
appreciate your diligence and patience. Here are some thoughts in =
response to your message from Nov. 19, 2024.

> even though the majority does, not everyone builds a browser =
application
with PHP, especially because URIs are not necessarily accessible on the =
web

This has largely been touched on elsewhere, but I will echo the idea =
that it seems valid to have to separate parsers for the two standards, =
and truly they diverge enough that it seems like it could be only a =
superficial thing for them to share an interface.

I only harp on the WhatWG spec so much because for many people this will =
be the only one they are aware of, if they are aware of any spec at all, =
and this is a sizable vector of attack targeting servers from =
user-supplied content. I=E2=80=99m curious to hear from folks here hat =
fraction of the actual PHP code deals with RFC3986 URLs, and of those, =
if the systems using them are truly RFC3986 systems or if the =
common-enough URLs are valid in both specs.

Just to enlighten me and possibly others with less familiarity, how and =
when are RFC3986 URLs used and what are those systems supposed to do =
when an invalid URL appears, such as when dealing with percent-encodings =
as you brought up in response to Tim?

Coming from the XHTML/HTML/XML side I know that there was substantial =
effort to enforce standards on browsers and that led to decades of =
security exploits and confusion, when the =E2=80=9Cofficial=E2=80=9D =
standards never fully existed in the way people thought. I don=E2=80=99t =
mean to start any flame wars, but is the URL story at all similar here?

I=E2=80=99m mostly worried that we could accidentally encourage risky =
behavior for developers who aren=E2=80=99t familiar with the nuances of =
having to URL specifications vs. having the simplest, least-specific =
interface point them in the right direction for what they will probably =
be doing. `parse_url()` is a great example of how the thing that looks =
_right_ is actually terribly prone to failure.

> The Uri\WhatWgUri::parse() method accepts a (relative) URI parameter =
when the 2nd (base URI) parameter is provided. So essentially you need =
to use=20
this variant of the parse() method if you want to parse a WhatWg =
compliant=20
URL

If this means passing something like the following then I suppose it=E2=80=
=99s okay. It would be nice to be able to know without passing the =
second parameter, as there are multitude cases where no such base URL =
would be available, and some dummy parameter would need to be provided.

```
    $url =3D Uri\WhatWgUri::parse( $url, 'https://example.com' )
    var_dump( $url->is_relative_or_something_like_that );
```

This would be fine, knowing in hindsight that it was originally a =
relative path. Of course, this would mean that it=E2=80=99s critical =
that `https://example.com` does not replace the actual host part if one =
is provided in `$url`. For example, this code should work.

```
    $url =3D Uri\WhatWgUri::parse( 'https://wiki.php.net/rfc=E2=80=99, =
=E2=80=98https://example.com=E2=80=99 );
    $url->domain =3D=3D=3D 'wiki.php.net'
```
> The forDisplay() method also seems to be useful at the first glance, =
but since this may be a controversial optional feature, I'd defer it for =
later=E2=80=A6

Hopefully this won=E2=80=99t be too controversial, even though the =
concept was new to me when I started having to reliably work with URLs. =
I choose the example I did because of human risk factors in security =
exploits.  "xn--google.com" is not in fact a Google domain, but an IDNA =
domain decoding to "=E4=95=AE=E4=95=B5=E4=95=B6=E4=95=B1.com=E2=80=9D

This is a misleading URL to human readers, which is why the WhatWG =
indicates that =E2=80=9Cbrowsers should render a URL=E2=80=99s host by =
running domain to Unicode with the URL=E2=80=99s host and false.=E2=80=9D =
[https://url.spec.whatwg.org/#url-rendering-i18n].

The lack of a standard method here means that (a) most code won=E2=80=99t =
render the URLs the way a human would recognize them, and (b) those who =
do will run to inefficient and likely-incomplete user-space code to try =
and decode/render these hosts.

It may be something fine for a follow-up to this work, but it=E2=80=99s =
also something I personally consider essential for any native support of =
handling URLs that are destined for human review. If sending to an =
`href` attribute it should be the normalized URL; but if displayed as =
text it should be easy to prevent tricking people in this way.

In my HTML decoding RFC I tried to bake in this decision in the type of =
the function using an enum. Since I figured most people are unaware of =
the role of the context in which HTML text is decoded, I found the enum =
to be a suitable convenience as well as educational tool.

```
    $url->toString( Uri\WhatWg\RenderContext::ForHumans ); // =
=E4=95=AE=E4=95=B5=E4=95=B6=E4=95=B1.com
    $url->toString( Uri\WhatWg\RenderContext::ForMachines ); // =
xn-google.com
```

The names probably are terrible in all of my code snippets, but at this =
point I=E2=80=99m not proposing actual names, just code samples good =
enough to illustrate the point. By forcing a choice here (no default =
value) someone will see the options and probably make the right call.

----

This is all looking quite nice. I=E2=80=99m happy to see how the RFC =
continues to develop, and I=E2=80=99m eagerly looking forward to being =
able to finally rely on PHP=E2=80=99s handling of URLs.

Happy new year,
Dennis Snell=