Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:113654
To: PHP internals <internals@lists.php.net>
References: <f313c9c4-f8a2-0b39-7499-30620d80cecd@gmail.com>
 <3a4d89fc-c5f8-4720-b2e0-f6f3c28684f9@www.fastmail.com>
 <5f5fd136-e181-d5d3-fe40-1a4cc5c668f2@gmail.com>
 <CAGBsUrcsms0TfnwnrjyaugmDd549gHbTTDj+VoR1sPzm9-8TGw@mail.gmail.com>
 <CAGgaK7LDeX0nGggpQbNROD91Mgg=g0RJTvpP5d47yEYzjOR1hA@mail.gmail.com>
Message-ID: <25680b8d-af02-c1d4-e630-7bf079881f1c@gmail.com>
Date: Sun, 21 Mar 2021 21:39:14 +0000
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.8.1
MIME-Version: 1.0
In-Reply-To: <CAGgaK7LDeX0nGggpQbNROD91Mgg=g0RJTvpP5d47yEYzjOR1hA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-GB
Subject: Re: [PHP-DEV] What should we do with utf8_encode and utf8_decode?
From: rowan.collins@gmail.com (Rowan Tommins)

On 21/03/2021 21:00, Max Semenik wrote:
> Just a quick reminder that it's possible to compile PHP without 
> mbstring and intl, which means that some hosts will provide PHP 
> without these extensions, and some packagers make them available as 
> separate packages that users can't or don't know how to install. Maybe 
> we've got an opportunity to think about making these extensions mandatory?


It's somewhat relevant that until PHP 7.2, it was also possible for 
utf8_encode and utf8_decode to be missing, because they were in ext/xml, 
which is also optional.

Bundling mbstring sounds great, until you look into the details of 
what's in there and how it works. Its origin as a PHP 4 extension for 
handling Japanese-specific character encodings is visible in parts of 
its design - there's a lot of global state, and very little support for 
the nuances of Unicode.

Bundling intl would be great, but it's a wrapper around ICU, which is 
huge (because Unicode is complicated). I have read that incorporating 
that into core was one of the icebergs that sunk PHP 6. It's also 
extremely sparsely documented (if someone's looking for a project, it 
would be great to fill in all the manual stubs with a few details from 
the corresponding ICU documentation).

For what its worth, it seems these would be the relevant polyfills:

function utf8_encode(string $string) { return 
UConverter::transcode($string, 'UTF8', 'ISO-8859-1'); }
function utf8_decode(string $string) { return 
UConverter::transcode($string, 'ISO-8859-1', 'UTF8'); }


Regards,

-- 
Rowan Tommins
[IMSoP]