Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:125008
Content-Type: text/plain;
	charset=utf-8
Precedence: bulk
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.8\))
Subject: Re: [PHP-DEV][Discussion] Should All String Functions Become
 Multi-Byte Safe?
In-Reply-To: <8360937cc7ca31bf3bd0f8e3050c53cb32663428.camel@ageofdream.com>
Date: Fri, 16 Aug 2024 19:32:42 -0400
Cc: internals@lists.php.net
Content-Transfer-Encoding: quoted-printable
Message-ID: <01BF288F-E0C5-4F67-A829-411FD7C1315D@newclarity.net>
References: <CAPn3h8LunRRAEfQXbfuDzMsxwGeFZfsHXs7oB=DTMhN2fgphUg@mail.gmail.com>
 <1AFE8300-D363-43D8-A989-15D001B9879C@newclarity.net>
 <dc2d9656-557a-4bf2-9971-245b30b1aa30@rwec.co.uk>
 <270D6057-626D-4720-B44A-3CB7A7B9320B@newclarity.net>
 <BC11D97A-F56E-4C22-ABCE-C4A44680BCA7@rwec.co.uk>
 <8360937cc7ca31bf3bd0f8e3050c53cb32663428.camel@ageofdream.com>
To: Nick Lockheart <lists@ageofdream.com>
From: mike@newclarity.net (Mike Schinkel)

Hi Nick,

> On Aug 16, 2024, at 6:11 PM, Nick Lockheart <lists@ageofdream.com> =
wrote:
>=20
> I wanted to reply generally to this and not to any person in
> particular, as I'm the one who started the thread.
>=20
> I used the rather broad title "Should All String Functions Become
> Multi-Byte Safe" because there are many smaller related topics, but my
> intention was to discuss multi-byte in general, and see if there was
> some consensus on action items that could have a more limited =
scope/RFC
> for that task.
>=20
> My overall intent and goal was to make PHP safer against multi-byte
> attacks by providing developers with tools that could become best
> practices for dealing with user input stings, the same way we had
> mysql_real_escape_string, and then PDO prepared statements for SQL.
>=20
> There's a lot of potential pitfalls for dealing with Unicode input, =
and
> there are some best practices per the Unicode Consortium that I'm not
> sure how to implement in PHP, and it seems that since everyone needs
> them, they might be better as a shared library in core.
>=20
> For example, there should be a function that removes unassigned code
> points.
>=20
> There should also be a function that removes "scripts" (as defined by
> Unicode).
>=20
> We should have an easy way to remove private use code points (unless
> you're running a Star Trek fan site and really do need Klingon).
>=20
> And the default replacement character for `mb_scrub` shouldn't be `?`.
>=20
> Each of these and other ideas could be part of an RFC, or we could
> brainstorm a Unicode built-in class that handles lots of the common =
use
> cases.
>=20
> Having a team-built and audited Unicode class would benefit almost
> everyone using PHP.

My suggestion =E2=80=94 take it or leave it =E2=80=94 is to create a =
GitHub repo for your own RFCs and start writing your RFC there "in the =
open." Add the code for your implementation to the repo, add a =
discussion forum to allow really interested parties to participate, and =
send an invite on this list to those who are really interested to =
discuss, comment on the RFC, and even offer PRs.=20

Then when everyone participating at your repo thinks the RFC is =
fully-baked, bring it back to the list here to discuss. =20

Doing it that way will =E2=80=94 unlike just discussing on the list =E2=80=
=94 enable comments made in the forum a place to be captured and =
converted into text and implementation visible for everyone to see, and =
really motivated people can even submit PRs to your RFC in order to =
spread the load of writing a good RFC.=20

#jmtcw #fwiw

-Mike=