Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:124854
Message-ID: <8a60a5d76bf3bbdda821160c6141b45914a33b98.camel@ageofdream.com>
Subject: [PHP-DEV][Discussion] Should All String Functions Become Multi-Byte
 Safe?
To: internals@lists.php.net
Date: Sun, 11 Aug 2024 11:50:52 -0400
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.46.4-2 
Precedence: bulk
MIME-Version: 1.0
From: lists@ageofdream.com (Nick Lockheart)


HTML 5 was adopted in 2014, over ten years ago. HTML 5 only supports
the UTF-8 multi-byte character encoding.

It seems like there's still a lot of string functions that assume that
a character is a single byte, and these may actually work as expected
when dealing with Latin characters, but may fail unexpectedly if a
sequence is more than one byte.

Are there any use cases for PHP where **single-byte** characters are
the norm?

It seems that if everything on the Internet is multi-byte encoded now,
then all of the PHP string functions should be multi-byte safe.


The WHATWG Encoding Standard:

https://encoding.spec.whatwg.org/

Also, according to Mozilla, "[The meta charset] attribute declares the
document's character encoding. If the attribute is present, its value
must be an ASCII case-insensitive match for the string "utf-8", because
UTF-8 is the only valid encoding for HTML5 documents."

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta#charset