Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:125192
Content-Type: text/plain;
	charset=utf-8
Precedence: bulk
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\))
Subject: Re: [PHP-DEV] [RFC] Decoding HTML and the Ambiguous Ampersand
In-Reply-To: <48fa132e-3511-4503-8523-b59972bcfd53@gmx.de>
Date: Sat, 24 Aug 2024 15:34:40 -0500
Cc: Niels Dossche <dossche.niels@gmail.com>,
 Internals <internals@lists.php.net>
Content-Transfer-Encoding: quoted-printable
Message-ID: <87F1748F-BE78-4B3E-989C-293A8F5CE2E0@automattic.com>
References: <CAA8B983-FD39-4AB1-B314-62608B0C0EF4@a8c.com>
 <76D9E1DA-57CE-45C3-8E3E-B08A0B70FB60@a8c.com>
 <efaf4c62-a552-4232-8a22-410578c13b8d@gmail.com>
 <7ED2EE07-D7C6-43A4-A4E1-E9928E8B8D31@automattic.com>
 <48fa132e-3511-4503-8523-b59972bcfd53@gmx.de>
To: "Christoph M. Becker" <cmbecker69@gmx.de>
From: dennis.snell@automattic.com (Dennis Snell)



> On Aug 24, 2024, at 7:47=E2=80=AFAM, Christoph M. Becker =
<cmbecker69@gmx.de> wrote:
>=20
> On 23.08.2024 at 01:02, Dennis Snell wrote:
>=20
>>> If we could have a single implementation, that would be great. I do =
understand of course your concern that DOM is not a required extension, =
and therefore basing the internals on Lexbor makes it tied to the DOM =
extension which may not be available. I however suspect that a large =
chunk of people needing a function like this have DOM available (as DOM =
is required by many HTML-processing-related packages). I can also look =
into it sometime soon if you want; anyway feel free to ping me.
>>=20
>> I=E2=80=99m also very open to lexbor-based approaches but I=E2=80=99ve =
so-far found it more complicated than I expected. In some part this is =
because it involves setting up the parser and state machine for the HTML =
specification and much of the actual decoding can be safely done without =
this.
>>=20
>> The other part is the extension aspect. I hear you, that you would =
expect calling code to have the DOM extensions available, but that=E2=80=99=
s simply not the case when developing a platform like WordPress, which I =
do. We don=E2=80=99t have control over the servers or environments where =
people are deploying this, and the availability of the DOM extensions is =
low enough that WordPress code simply cannot use `DOMDocument` (even =
though it shouldn=E2=80=99t because of the wild problems that has for =
attempting to parse HTML).
>>=20
>> People resort to `html_entity_decode()` because that=E2=80=99s the =
only option. In WordPress we now have a spec-compliant decoder, but as =
it=E2=80=99s in user-space PHP its performance is far below what=E2=80=99s=
 possible.
>>=20
>> I=E2=80=99d love your help in setting up lexbor=E2=80=99s state =
machine to decode text nodes. I=E2=80=99d love it even more if this =
could be part of the PHP language. It constantly surprises me that _the =
language of the web_ (PHP) doesn=E2=80=99t have the tools to speak _the =
language of the web_ (HTML). This RFC is all about taking a step towards =
ensuring that PHP developers can rely on PHP to be a reliable middle-man =
between the HTML domain and the PHP domain.
>>=20
>> In other words, requiring the DOM extension or `DOM\HtmlDocument` =
would be such a non-starter for WordPress (accounting for 43% of the web =
today) that it would completely unavailable.
>=20
> Well, I don't think it would be a big deal to move the bundled lexbor =
to
> somewhere where it is always available.  I mean, so far it's only used
> by ext/dom so it's bundled there, but if other parts of the php-src =
code
> base would use it, we could put it elsewhere.

Having a DOM parser for HTML in PHP itself without requiring an =
extension would open up many new possibilities. For example, WordPress =
test suites don=E2=80=99t have any functional =
=E2=80=9CassertEquivalentMarkup()=E2=80=9D functions because there=E2=80=99=
s no spec-compliant parser in PHP. We finally wrote our own user-space =
HTML parser, but relying on `DOM\HtmlDocument` would be much easier.

These test suites need to run on a variety of environments and PHP =
versions, so it=E2=80=99s moot thinking we could hasten the use of a =
native class to get the job done, but if it remains locked inside an =
optional extension, it may be borderline impossible to ever migrate to =
it.

>=20
> Christoph
>=20

Dennis Snell