Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:126980
Content-Type: multipart/alternative;
 boundary="------------0XuS2YN6wfArPOY8OScqrI1w"
Message-ID: <b6068753-b542-4e11-96fd-683f1614ea3b@gmail.com>
Date: Sun, 30 Mar 2025 22:53:57 +0200
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PHP-DEV] [RFC] [Discussion] Add WHATWG compliant URL parsing API
To: =?UTF-8?Q?Tim_D=C3=BCsterhus?= <tim@bastelstu.be>
Cc: =?UTF-8?B?TcOhdMOpIEtvY3Npcw==?= <kocsismate90@gmail.com>,
 PHP Internals List <internals@lists.php.net>
References: <CAH5C8xUb1O20ZDrOQNC=ckFxHUUWSK7sw_njQQzFBd0qgQqoww@mail.gmail.com>
 <dd61999c-1ebd-4765-9add-cd8065968965@gmail.com>
 <CAOV5rgYZ_s1of-igLFEK7oqqh6=3HeYv0=JvvKvvjkcp0F6Q9Q@mail.gmail.com>
 <CAH5C8xV67frqOBCvLt73RM7QO86_pu40sHP5h71_kDRbKPtA8Q@mail.gmail.com>
 <1BCB4144-231D-45EA-A914-98EE8F0F503A@automattic.com>
 <8E614C9C-BA85-45D8-9A4E-A30D69981C5D@automattic.com>
 <CAH5C8xV69pHnSKWJrVB8EQab7vPhcaXAwYezoG6z3Oj7ZkXBaQ@mail.gmail.com>
 <bc5472e39bc607e2c7accc9d2bd6f239@bastelstu.be>
 <9bf11a89-39d9-457b-b0ea-789fd07d7370@gmail.com>
 <CAH5C8xWnoytE=jm1sxr-HcSZFzmG8SrzghinmBZa-QyGqN4uUg@mail.gmail.com>
 <6430b9ed-638d-4247-9fa9-d1a9148c382b@gmail.com>
 <CAH5C8xX3tCS3B3z2YFbRudCXi=ZNHE0scrmoFO7cXxU773EfpA@mail.gmail.com>
 <2e95e8fe-7cf0-493f-bd0a-9fff0956baaa@gmail.com>
 <CAH5C8xXuSEBx6y4ds8bhReFFZe0i05DFonvEP+sz+mdkembZtQ@mail.gmail.com>
 <f5952335-898a-4707-aa6e-7668b678ff13@gmail.com>
 <7d715757cc2dfd71019d106b01c69aed@bastelstu.be>
Content-Language: fr
In-Reply-To: <7d715757cc2dfd71019d106b01c69aed@bastelstu.be>
From: nyamsprod@gmail.com (Ignace Nyamagana Butera)

This is a multi-part message in MIME format.
--------------0XuS2YN6wfArPOY8OScqrI1w
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


On 30/03/2025 14:42, Tim Düsterhus wrote:
> Hi
>
> Am 2025-03-27 23:49, schrieb Ignace Nyamagana Butera:
>> Hi Máté,
>>
>>    for RFC 3986:
>> https://datatracker.ietf.org/doc/html/rfc3986#section-5.3), and then
>>    this string is parsed and validated. Unfortunately, I recently
>>    realized that this approach may leave room for some kind of parsing
>>    confusion attack, namely when the scheme is for example "https", the
>>    authority is empty, and the path is "example.com
>> <http://example.com>". This will result in a https://example.com
>>    URI. I believe a similar bug is not possible with the rest of the
>>    components because they have their delimiters. So possibly some
>>    other solution will be needed, or maybe adding some additional
>>    validation (?).
>>
>> This is not correct according to RFC3986 
>> https://datatracker.ietf.org/doc/html/rfc3986#section-3
>>
>>
>> *When authority is present, the path must either be empty or begin 
>> with a slash ("/") character. When authority is not present, the path 
>> cannot begin with two slash characters ("//"). *
>>
>> So in your example it should throw an Uri\InvalidUriException 🙂 for 
>> RFC3986 and in case of the WhatwgUrl algorithm it should trigger a 
>> soft error and correct the behaviour for the http(s) schemes.
>> This is also one of the many reasons why at least for RFC3986 the 
>> path component can never be `null` but that's another discussion. 
>> Like I said having a `fromComponenta` named constructor would allow 
>> the "removal" of the need for a UriBuilder (in your future section) 
>> and would IMHO be useful outside of the context of the http(s) scheme 
>> but I can understand it being left out of the current implementation 
>> it might be brought back for future improvements.
>
> I just tested this with the implementation and it also appears to not 
> yet be correct:
>
>     var_dump((new Uri\Rfc3986\Uri("example.com"))->getHost()); // NULL
>     var_dump((new 
> Uri\Rfc3986\Uri("example.com"))->withScheme('https')->getHost()); // 
> string(11) "example.com"
>     var_dump((new 
> Uri\Rfc3986\Uri("example.com"))->withScheme('https')->toRawString()); 
> // string(19) "https://example.com"
>
> and
>
>     var_dump((new 
> Uri\Rfc3986\Uri("foo/bar"))->withPath('//foo/bar')->getHost()); // 
> string(3) "foo"
>
> Best regards
> Tim Düsterhus

Hi Tim and Maté upon further inspection and verification of RFC3986 I 
also see an issue with the example used for normalization in the RFC. 
According to RFC3986 
(https://www.rfc-editor.org/rfc/rfc3986.html#section-3.2.2) :

      The reg-name syntax allows percent-encoded octets in order to
       represent non-ASCII registered names in a uniform way that is
        independent of the underlying name resolution technology.  Non-ASCII
        characters must first be encoded according to UTF-8 [STD63 <https://www.rfc-editor.org/rfc/rfc3986.html#ref-STD63>], and then
        each octet of the corresponding UTF-8 sequence must be percent-
        encoded to be represented as URI characters.  URI producing
        applications must not use percent-encoding in host unless it is used
        to represent a UTF-8 character sequence.  When a non-ASCII registered
        name represents an internationalized domain name intended for
        resolution via the DNS, the name must be transformed to the IDNA
        encoding [RFC3490 <https://www.rfc-editor.org/rfc/rfc3490>] prior to name lookup.

 From this we can infer that:

- Host encoding can only happen for UTF-8 sequence but in your example 
"ex%61mple.com" is used which is not conforming to the rules (ie it 
should throw an InvalidUriException IMHO for the Uri class) I presume 
for WhatWg URL it will get correctly converted with a soft error (??).

- That when available IDNA is preferred to percent-encoded sequences

Best regards

Ignace Nyamagana Butera


--------------0XuS2YN6wfArPOY8OScqrI1w
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 30/03/2025 14:42, Tim Düsterhus
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:7d715757cc2dfd71019d106b01c69aed@bastelstu.be">Hi
      <br>
      <br>
      Am 2025-03-27 23:49, schrieb Ignace Nyamagana Butera:
      <br>
      <blockquote type="cite">Hi Máté,
        <br>
        <br>
           for RFC 3986:
        <br>
           <a class="moz-txt-link-freetext" href="https://datatracker.ietf.org/doc/html/rfc3986#section-5.3">https://datatracker.ietf.org/doc/html/rfc3986#section-5.3</a>),
        and then
        <br>
           this string is parsed and validated. Unfortunately, I
        recently
        <br>
           realized that this approach may leave room for some kind of
        parsing
        <br>
           confusion attack, namely when the scheme is for example
        "https", the
        <br>
           authority is empty, and the path is "example.com
        <br>
           <a class="moz-txt-link-rfc2396E" href="http://example.com">&lt;http://example.com&gt;</a>". This will result in a
        <a class="moz-txt-link-freetext" href="https://example.com">https://example.com</a>
        <br>
           URI. I believe a similar bug is not possible with the rest of
        the
        <br>
           components because they have their delimiters. So possibly
        some
        <br>
           other solution will be needed, or maybe adding some
        additional
        <br>
           validation (?).
        <br>
        <br>
        This is not correct according to RFC3986
        <a class="moz-txt-link-freetext" href="https://datatracker.ietf.org/doc/html/rfc3986#section-3">https://datatracker.ietf.org/doc/html/rfc3986#section-3</a>
        <br>
        <br>
        <br>
        *When authority is present, the path must either be empty or
        begin with a slash ("/") character. When authority is not
        present, the path cannot begin with two slash characters ("//").
        *
        <br>
        <br>
        So in your example it should throw an Uri\InvalidUriException 🙂
        for RFC3986 and in case of the WhatwgUrl algorithm it should
        trigger a soft error and correct the behaviour for the http(s)
        schemes.
        <br>
        This is also one of the many reasons why at least for RFC3986
        the path component can never be `null` but that's another
        discussion. Like I said having a `fromComponenta` named
        constructor would allow the "removal" of the need for a
        UriBuilder (in your future section) and would IMHO be useful
        outside of the context of the http(s) scheme but I can
        understand it being left out of the current implementation it
        might be brought back for future improvements.
        <br>
      </blockquote>
      <br>
      I just tested this with the implementation and it also appears to
      not yet be correct:
      <br>
      <br>
          var_dump((new Uri\Rfc3986\Uri("example.com"))-&gt;getHost());
      // NULL
      <br>
          var_dump((new
      Uri\Rfc3986\Uri("example.com"))-&gt;withScheme('https')-&gt;getHost());
      // string(11) "example.com"
      <br>
          var_dump((new
Uri\Rfc3986\Uri("example.com"))-&gt;withScheme('https')-&gt;toRawString());
      // string(19) <a class="moz-txt-link-rfc2396E" href="https://example.com">"https://example.com"</a>
      <br>
      <br>
      and
      <br>
      <br>
          var_dump((new
      Uri\Rfc3986\Uri("foo/bar"))-&gt;withPath('//foo/bar')-&gt;getHost());
      // string(3) "foo"
      <br>
      <br>
      Best regards
      <br>
      Tim Düsterhus
      <br>
    </blockquote>
    <p>Hi Tim and Maté upon further inspection and verification of
      RFC3986 I also see an issue with the example used for
      normalization in the RFC. According to RFC3986
      (<a class="moz-txt-link-freetext" href="https://www.rfc-editor.org/rfc/rfc3986.html#section-3.2.2">https://www.rfc-editor.org/rfc/rfc3986.html#section-3.2.2</a>) :<br>
    </p>
    <blockquote>
      <pre class="newpage"
style="font-size: 13.3333px; margin-top: 0px; margin-bottom: 0px; break-before: page; color: rgb(0, 0, 0); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"> The reg-name syntax allows percent-encoded octets in order to
  represent non-ASCII registered names in a uniform way that is
   independent of the underlying name resolution technology.  Non-ASCII
   characters must first be encoded according to UTF-8 [<a
      href="https://www.rfc-editor.org/rfc/rfc3986.html#ref-STD63"
      title="&quot;UTF-8, a transformation format of ISO 10646&quot;">STD63</a>], and then
   each octet of the corresponding UTF-8 sequence must be percent-
   encoded to be represented as URI characters.  URI producing
   applications must not use percent-encoding in host unless it is used
   to represent a UTF-8 character sequence.  When a non-ASCII registered
   name represents an internationalized domain name intended for
   resolution via the DNS, the name must be transformed to the IDNA
   encoding [<a href="https://www.rfc-editor.org/rfc/rfc3490"
title="&quot;Internationalizing Domain Names in Applications (IDNA)&quot;">RFC3490</a>] prior to name lookup.

</pre>
    </blockquote>
    <p>From this we can infer that:</p>
    <p>- Host encoding can only happen for UTF-8 sequence but in your
      example "ex%61mple.com" is used which is not conforming to the
      rules (ie it should throw an InvalidUriException IMHO for the Uri
      class) I presume for WhatWg URL it will get correctly converted
      with a soft error (??).</p>
    <p>- That when available IDNA is preferred to percent-encoded
      sequences</p>
    <p>Best regards</p>
    <p>Ignace Nyamagana Butera<br>
    </p>
    <p><br>
    </p>
  </body>
</html>

--------------0XuS2YN6wfArPOY8OScqrI1w--