Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:127244
Precedence: bulk
MIME-Version: 1.0
References: <CAH5C8xUb1O20ZDrOQNC=ckFxHUUWSK7sw_njQQzFBd0qgQqoww@mail.gmail.com>
 <dd61999c-1ebd-4765-9add-cd8065968965@gmail.com> <CAOV5rgYZ_s1of-igLFEK7oqqh6=3HeYv0=JvvKvvjkcp0F6Q9Q@mail.gmail.com>
 <CAH5C8xV67frqOBCvLt73RM7QO86_pu40sHP5h71_kDRbKPtA8Q@mail.gmail.com>
 <1BCB4144-231D-45EA-A914-98EE8F0F503A@automattic.com> <8E614C9C-BA85-45D8-9A4E-A30D69981C5D@automattic.com>
 <CAH5C8xV69pHnSKWJrVB8EQab7vPhcaXAwYezoG6z3Oj7ZkXBaQ@mail.gmail.com>
 <bc5472e39bc607e2c7accc9d2bd6f239@bastelstu.be> <CAH5C8xW_auVf+ig7odhTK8dVWwmXTg4ZVZN-0Fhh4SUBzrWxVw@mail.gmail.com>
 <8df04e01-deac-404b-beb7-cd982423db63@bastelstu.be> <CAH5C8xU0JcnmC2Hdd=Hj0CQUaL0-LN=1X8+5-eUbFH3ZMxCvBA@mail.gmail.com>
 <33427cd03035ef084245c44290b56a55@bastelstu.be> <CAH5C8xXwDO6Ui8pLmqoxEguURh97fAaL4CfLK-oitqC85N7+8w@mail.gmail.com>
 <0aa1eefc3941bdea0092e935074daa58@bastelstu.be> <CAH5C8xU+rnp4e5=CzqhEncRww6nhZ7x1s68NjWtu+HQMpoNZ8g@mail.gmail.com>
 <76d96ea8a78c6025128c0a4b01c94c0a@bastelstu.be> <CAH5C8xXLPoRLJ6_dtotMtOv2NWs4LFzu+Kpr5A56Xk8wmuPDEg@mail.gmail.com>
 <DBA16148-2197-4E80-B60C-EC1F02C20F13@pmjones.io> <CAOV5rgbqpU7nUxSLC50dq+vcAN2NaQxTZwiauUE_vnYugDiqZg@mail.gmail.com>
 <74F64DCB-3A10-4CF8-8DAA-6089BC34EA89@pmjones.io>
In-Reply-To: <74F64DCB-3A10-4CF8-8DAA-6089BC34EA89@pmjones.io>
Date: Tue, 29 Apr 2025 22:08:24 +0200
Message-ID: <CAOV5rgbTsUQQJfynNFbBk7JsAJwvquoRHbxH3rm6cb-CeshaFg@mail.gmail.com>
Subject: Re: [PHP-DEV] [RFC] [Discussion] Add WHATWG compliant URL parsing API
To: "Paul M. Jones" <pmjones@pmjones.io>, =?UTF-8?Q?Tim_D=C3=BCsterhus?= <tim@bastelstu.be>, 
	Internals <internals@lists.php.net>, =?UTF-8?B?TcOhdMOpIEtvY3Npcw==?= <kocsismate90@gmail.com>
Content-Type: multipart/alternative; boundary="00000000000095ac6a0633f05d54"
From: nyamsprod@gmail.com (ignace nyamagana butera)

--00000000000095ac6a0633f05d54
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Paul,

I will try to address your concerns. Keep in mind that I am not the author
of the RFC but I do like how it is currently shaped with some caveats but
those can be put under future improvements.

> So, one value added by splitting the classes is to resolve that asymmetry=
.

First, I agree with you. The method naming in the Uri\Rfc3986\Uri class
could be improved even though it does not represent a showstopper to me,
Adding the `raw` prefix or indeed flipping the raw* method and using
normalized* would perhaps make for some clarification but I will leave that
decision to M=C3=A1t=C3=A9.
Apart from that, I believe the current RFC (especially around RFC3986) does
address most if not all the issues regarding the specification. RFC3986
provides information around 3 key URI features: parsing, resolution and
equivalence. In order to offer resolution and equivalence you ought to
address normalization and thus encoding. Any userland package that does
offer those features is required to handle component encoding/normalization
first before performing the expected operation. Hence why I believe that if
the new URI class does offer equivalence by consequence it can/should be
able to expose URI component normalization out of the box. The need for a
separate class is IMHO not needed.

> For example, can you point out which projects mix "raw and
half-normalized components"?

Laminas for example or any PSR implementing class will try to encode the
input string regardless of its encoding hence the wording around not to
double encode the string you often encounter in mutator method docblock.
The Uri on the other hand only expects well formed and encoded strings
which leaves room for no wrong interpretation. This is an area that is left
to be filled by URI packages for instance.

> For Rfc3986\Uri, it looks like there are only two that are recognized:
raw and normalized. Are there other string representations you feel the Uri
class should recognize?

If there are at least two representations possible then a `__toString`
method is still a bad design because it may lead the developper to think
that this is the only one string representation which is not true. Both
representations are equivalent and represent as much the URI. And as a
bonus, not having a `__toString` method prevents accidental URI comparison
using the `=3D=3D` sign instead of using the correct `equals` method. (I kn=
ow
that because I've seen codebase where PSR-7 URI instances are compared
using the class  `__toString` method  which is just wrong).

PS1: I do appreciate the work you did put into your study around URI
packages in the PHP ecosystem but we should not restrict the new API to
only resolve or align to those used solutions instead we should try to
expose an API susceptible to allow more flexibility than what PHP currently
offers.
PS2: I do not think the new API will replace the URI packages, we will
still need them because, in the case of RFC3986 URI class, parsing is just
one aspect or URI consumption, we still need scheme specific validation
that only PHP userland package can offer.

Best regards,
Ignace Nyamagana Butera

On Tue, Apr 29, 2025 at 3:55=E2=80=AFPM Paul M. Jones <pmjones@pmjones.io> =
wrote:

> Hi Ignace & Mat=C3=A9 and all,
>
> tl;dr: I argue against Ignace's objections to splitting the URI class int=
o
> two classes (one that retains raw URI values and another that normalizes
> values as-it-goes). Jump to the very end for a discussion regarding the
> with() methods (search for the word "asymmetry" herein).
>
> * * *
>
> > On Apr 28, 2025, at 15:47, ignace nyamagana butera <nyamsprod@gmail.com=
>
> wrote:
> >
> > The current approach in userland mixes both raw and half normalized
> components as well as RFC3986 and RFC3987 specification with ambiguity
> around normalization, input, constructior, what needs to be encoded where
> and when
>
> Based on my research into existing URI projects <
> https://github.com/uri-interop/interface/blob/1.x/README-RESEARCH.md> I
> don't think that's an accurate assessment of the ecosystem.
>
> For example, can you point out which projects mix "raw and half-normalize=
d
> components"? Nette is the only one that comes to mind, in that (during
> parsing) it applies rawurldecode() to the host, user, password, and
> fragment; but that's only one of the 18 projects.
>
> Likewise, of the 15 URI-centric projects, only one of them (league/uri)
> offers both RFC3986 and 3987 parsing; the two IRI-centric projects (ml/ir=
i
> and rmccue/requests) are explicitly IRIs; and rowbot is clearly WHATWG-UR=
L
> centric.  So I don't see much ambiguity in any projects there.
>
> As far as normalization, only one project (opis) affords the ability to
> normalize at creation time, though five of them offer a normalize() metho=
d
> with various effects (<
> https://github.com/uri-interop/interface/blob/1.x/README-RESEARCH.md#norm=
alizing>).
> So, again, I don't see much ambiguity there either; they don't do
> normalizing as-you-go, it's something you have to apply explicitly.
>
> Regarding inputs, they all presume "raw" inputs. Regarding constructors,
> they mostly side with a full URI string. Regarding encoding, they mostly
> retain values in their encoded form (there are three outliers, cf. <
> https://github.com/uri-interop/interface/blob/1.x/README-RESEARCH.md#comp=
onent-encoding
> >).
>
> With all that in mind, we can see that the various authors of userland
> projects have settled on remarkably similar patterns of usage that they
> found valuable and useful for working with URIs.
>
>
> > > - fulfill existing userland expectations;
> >
> > Existing userland expectations are mostly built around `parse_url`
>
> That's kind of true; 9 of the 18 projects use parse_url(), and 7/18
> implement the RFC 3986 parsing algorithm ...
>
>
> > which is one of the reasons the RFC exists to improve the status quo an=
d
> to introduce in PHP valid parsers against recognizable URI specifications=
.
> Yes some adaptation will be needed to use them in userland but I believe
> this work is easy to do, talking from the POV of a URI package maintainer=
.
>
> ... but I don't imagine that replacing parse_url() in those projects with
> the RFC 3986 algo would cause those projects to change any of their other
> design decisions. What adaptations do you think would be needed around th=
at
> replacement?
>
>
> > > - replace the toString()/toRawString() with a single idiomatic
> __toString() in each class;
> >
> > For all the reasons explained in the RFC, adding a `__toString` method
> is a bad architectural design for an URI. There are so many ways to
> represent an URI that  having a `__toString` for string representation
> gives a false sense of "there can be only one true representation for a
> single URI" which is not true.
>
> For Rfc3986\Uri, it looks like there are only two that are recognized: ra=
w
> and normalized. Are there other string representations you feel the Uri
> class should recognize?
>
> (For Whatwg\Url, it looks like there are also only two: as-parsed, and as
> ASCII, but I'm not addressing that part of the RFC here.)
>
>
> > > - move normalization logic into the NormalizedUri class.
> >
> > The classes follow  specifications that describe how normalization
> should be. Why would you split the responsibilities in other classes ? Wh=
at
> would be the added value ?
>
> For one, unless I am missing something, there is an asymmetry between the
> get() methods and the with() methods. What I'm seeing is that (e.g.)
> Uri::withPath() expects a raw path argument, but getPath() returns the
> normalized version.  For symmetry, I would expect either:
>
> - `Uri::withPath(raw_value) : self` and `Uri::getPath() : raw_value`, or
> - `Uri::withRawPath(raw_value) : self` and `Uri::getRawPath() : raw_value=
`
>
> Thus my first intuition that the "main" values in the URI need to be the
> raw ones, and that getting the normalized ones should be the more verbose
> case (e.g. `getNormalizedPath() : normalized_value`).
>
> So, one value added by splitting the classes is to resolve that asymmetry=
.
> Consumers expecting to get back from the URI what they put into it can us=
e
> the raw Uri variation; "API clients or signers fall in this category that
> want to avoid introducing any unnecessary changes to URIs, in order to
> avoid causing subtle bugs."
>
> Other consumers, who want to do things this new and different way
> (normalized as-you-go, unlike anything currently in userland) can use the
> NormalizedUri.
>
> (Or you could flip it around and say that the normalized variation is the
> Uri class, and the raw version is RawUri.)
>
>
>
> -- pmj
>
>

--00000000000095ac6a0633f05d54
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Paul,<div><br></div><div>I will try to address your con=
cerns. Keep in mind that I am not the author of the RFC but I do like how i=
t is currently shaped with some caveats but those can be put under future=
=C2=A0improvements.</div><div><br></div><div>&gt; So, one value added by sp=
litting the classes is to resolve that asymmetry.</div><div><br></div><div>=
First, I agree with you. The method naming in the Uri\Rfc3986\Uri class cou=
ld be improved even though it does not represent a showstopper to me, Addin=
g the `raw` prefix or indeed flipping the raw* method and using normalized*=
 would perhaps make for some clarification but I will leave that decision t=
o M=C3=A1t=C3=A9.</div><div>Apart=C2=A0from that, I believe the current RFC=
 (especially around RFC3986) does address most if not all the issues regard=
ing the specification. RFC3986 provides information around 3 key URI featur=
es: parsing, resolution and equivalence. In order to offer resolution and e=
quivalence=C2=A0you ought to address normalization and thus encoding. Any u=
serland package that does offer those features is required to handle compon=
ent=C2=A0encoding/normalization first before performing the expected operat=
ion. Hence why I believe that if the new URI class does offer equivalence b=
y consequence it can/should be able to=C2=A0expose URI component normalizat=
ion out of the box. The need for a separate class is IMHO not needed.</div>=
<div><br></div><div>&gt; For example, can you point out which projects mix =
&quot;raw and half-normalized components&quot;?</div><div><br></div><div>La=
minas for example or any PSR implementing class will try to encode the inpu=
t string regardless of its encoding hence the wording around not to double =
encode=C2=A0the string you often encounter in mutator method docblock. The =
Uri on the other hand only expects well formed and encoded strings which le=
aves room for no wrong interpretation. This is an area that is left to be f=
illed by URI packages for instance.</div><div><br></div><div>&gt; For Rfc39=
86\Uri, it looks like there are only two that are recognized: raw and norma=
lized. Are there other string representations you feel the Uri class should=
 recognize?</div><br><div>If there are at least two representations possibl=
e then a `__toString` method is still a bad design because it may lead the =
developper to think that this is the only one string representation which i=
s not true. Both representations are equivalent and represent as much the U=
RI. And as a bonus, not having a `__toString` method prevents accidental UR=
I comparison using the `=3D=3D` sign instead of using the correct `equals` =
method. (I know that because I&#39;ve seen codebase where PSR-7 URI instanc=
es are compared using the class=C2=A0 `__toString` method=C2=A0 which is ju=
st wrong).</div><div><br></div><div>PS1: I do appreciate the work you did p=
ut into your study around URI packages in the PHP ecosystem but we should n=
ot restrict the new API to only resolve or align to those used solutions in=
stead we should try to expose an API susceptible to allow more flexibility=
=C2=A0than what PHP currently offers.</div><div>PS2: I do not think the new=
 API will replace the URI packages, we will still need them because, in the=
 case of RFC3986 URI class, parsing is just one aspect or URI consumption, =
we still need scheme specific validation that only PHP userland package can=
 offer.</div><div><br></div><div>Best regards,</div><div>Ignace Nyamagana B=
utera</div></div><br><div class=3D"gmail_quote gmail_quote_container"><div =
dir=3D"ltr" class=3D"gmail_attr">On Tue, Apr 29, 2025 at 3:55=E2=80=AFPM Pa=
ul M. Jones &lt;<a href=3D"mailto:pmjones@pmjones.io">pmjones@pmjones.io</a=
>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px=
 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi =
Ignace &amp; Mat=C3=A9 and all,<br>
<br>
tl;dr: I argue against Ignace&#39;s objections to splitting the URI class i=
nto two classes (one that retains raw URI values and another that normalize=
s values as-it-goes). Jump to the very end for a discussion regarding the w=
ith() methods (search for the word &quot;asymmetry&quot; herein).<br>
<br>
* * *<br>
<br>
&gt; On Apr 28, 2025, at 15:47, ignace nyamagana butera &lt;<a href=3D"mail=
to:nyamsprod@gmail.com" target=3D"_blank">nyamsprod@gmail.com</a>&gt; wrote=
:<br>
&gt; <br>
&gt; The current approach in userland mixes both raw and half normalized co=
mponents as well as RFC3986 and RFC3987 specification with ambiguity around=
 normalization, input, constructior, what needs to be encoded where and whe=
n<br>
<br>
Based on my research into existing URI projects &lt;<a href=3D"https://gith=
ub.com/uri-interop/interface/blob/1.x/README-RESEARCH.md" rel=3D"noreferrer=
" target=3D"_blank">https://github.com/uri-interop/interface/blob/1.x/READM=
E-RESEARCH.md</a>&gt; I don&#39;t think that&#39;s an accurate assessment o=
f the ecosystem.<br>
<br>
For example, can you point out which projects mix &quot;raw and half-normal=
ized components&quot;? Nette is the only one that comes to mind, in that (d=
uring parsing) it applies rawurldecode() to the host, user, password, and f=
ragment; but that&#39;s only one of the 18 projects.<br>
<br>
Likewise, of the 15 URI-centric projects, only one of them (league/uri) off=
ers both RFC3986 and 3987 parsing; the two IRI-centric projects (ml/iri and=
 rmccue/requests) are explicitly IRIs; and rowbot is clearly WHATWG-URL cen=
tric.=C2=A0 So I don&#39;t see much ambiguity in any projects there.<br>
<br>
As far as normalization, only one project (opis) affords the ability to nor=
malize at creation time, though five of them offer a normalize() method wit=
h various effects (&lt;<a href=3D"https://github.com/uri-interop/interface/=
blob/1.x/README-RESEARCH.md#normalizing" rel=3D"noreferrer" target=3D"_blan=
k">https://github.com/uri-interop/interface/blob/1.x/README-RESEARCH.md#nor=
malizing</a>&gt;). So, again, I don&#39;t see much ambiguity there either; =
they don&#39;t do normalizing as-you-go, it&#39;s something you have to app=
ly explicitly.<br>
<br>
Regarding inputs, they all presume &quot;raw&quot; inputs. Regarding constr=
uctors, they mostly side with a full URI string. Regarding encoding, they m=
ostly retain values in their encoded form (there are three outliers, cf. &l=
t;<a href=3D"https://github.com/uri-interop/interface/blob/1.x/README-RESEA=
RCH.md#component-encoding" rel=3D"noreferrer" target=3D"_blank">https://git=
hub.com/uri-interop/interface/blob/1.x/README-RESEARCH.md#component-encodin=
g</a>&gt;).<br>
<br>
With all that in mind, we can see that the various authors of userland proj=
ects have settled on remarkably similar patterns of usage that they found v=
aluable and useful for working with URIs.<br>
<br>
<br>
&gt; &gt; - fulfill existing userland expectations;<br>
&gt; <br>
&gt; Existing userland expectations are mostly built around `parse_url`<br>
<br>
That&#39;s kind of true; 9 of the 18 projects use parse_url(), and 7/18 imp=
lement the RFC 3986 parsing algorithm ...<br>
<br>
<br>
&gt; which is one of the reasons the RFC exists to improve the status quo a=
nd to introduce in PHP valid parsers against recognizable URI specification=
s. Yes some adaptation will be needed to use them in userland but I believe=
 this work is easy to do, talking from the POV of a URI package maintainer.=
<br>
<br>
... but I don&#39;t imagine that replacing parse_url() in those projects wi=
th the RFC 3986 algo would cause those projects to change any of their othe=
r design decisions. What adaptations do you think would be needed around th=
at replacement?<br>
<br>
<br>
&gt; &gt; - replace the toString()/toRawString() with a single idiomatic __=
toString() in each class;<br>
&gt; <br>
&gt; For all the reasons explained in the RFC, adding a `__toString` method=
 is a bad architectural design for an URI. There are so many ways to repres=
ent an URI that=C2=A0 having a `__toString` for string representation gives=
 a false sense of &quot;there can be only one true representation for a sin=
gle URI&quot; which is not true.<br>
<br>
For Rfc3986\Uri, it looks like there are only two that are recognized: raw =
and normalized. Are there other string representations you feel the Uri cla=
ss should recognize?<br>
<br>
(For Whatwg\Url, it looks like there are also only two: as-parsed, and as A=
SCII, but I&#39;m not addressing that part of the RFC here.)<br>
<br>
<br>
&gt; &gt; - move normalization logic into the NormalizedUri class.<br>
&gt; <br>
&gt; The classes follow=C2=A0 specifications that describe how normalizatio=
n should be. Why would you split the responsibilities in other classes ? Wh=
at would be the added value ? <br>
<br>
For one, unless I am missing something, there is an asymmetry between the g=
et() methods and the with() methods. What I&#39;m seeing is that (e.g.) Uri=
::withPath() expects a raw path argument, but getPath() returns the normali=
zed version.=C2=A0 For symmetry, I would expect either:<br>
<br>
- `Uri::withPath(raw_value) : self` and `Uri::getPath() : raw_value`, or<br=
>
- `Uri::withRawPath(raw_value) : self` and `Uri::getRawPath() : raw_value`<=
br>
<br>
Thus my first intuition that the &quot;main&quot; values in the URI need to=
 be the raw ones, and that getting the normalized ones should be the more v=
erbose case (e.g. `getNormalizedPath() : normalized_value`).<br>
<br>
So, one value added by splitting the classes is to resolve that asymmetry. =
Consumers expecting to get back from the URI what they put into it can use =
the raw Uri variation; &quot;API clients or signers fall in this category t=
hat want to avoid introducing any unnecessary changes to URIs, in order to =
avoid causing subtle bugs.&quot; <br>
<br>
Other consumers, who want to do things this new and different way (normaliz=
ed as-you-go, unlike anything currently in userland) can use the Normalized=
Uri.<br>
<br>
(Or you could flip it around and say that the normalized variation is the U=
ri class, and the raw version is RawUri.)<br>
<br>
<br>
<br>
-- pmj<br>
<br>
</blockquote></div>

--00000000000095ac6a0633f05d54--