Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:113677
To: internals@lists.php.net
References: <f313c9c4-f8a2-0b39-7499-30620d80cecd@gmail.com>
 <CAESVnVoa2U1ZCsthWCt8Cu8Es-P8WWO=vky4TG5CmtaoumEmUA@mail.gmail.com>
 <e120bf6b-12cd-5b3e-b27c-b746d1a1661b@gmail.com>
 <CAESVnVqJh3oOM-5CkatiUctRupOdS7mce7QYzv3uxUb3QcpHWw@mail.gmail.com>
 <693767b5-a25b-b4d9-f535-6b985bf26d67@gmail.com>
 <29d5329c-bea2-7944-4820-515d4a10ae86@alec.pl>
Message-ID: <16ecfc31-33aa-4223-fb67-b5a4b5895f05@gmail.com>
Date: Mon, 22 Mar 2021 15:41:27 +0000
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.8.1
MIME-Version: 1.0
In-Reply-To: <29d5329c-bea2-7944-4820-515d4a10ae86@alec.pl>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-GB
Subject: Re: [PHP-DEV] What should we do with utf8_encode and utf8_decode?
From: rowan.collins@gmail.com (Rowan Tommins)

On 22/03/2021 15:04, Aleksander Machniak wrote:

> I'm using utf8_encode()/utf8_decode() to make input string safe to be
> stored in DB, and back. In most cases the input is utf-8, but it
> occasionally may contain "broken characters".


That is not what this function does, at all. The fact that its name 
makes you think that is exactly why I want to get rid of that name.


>      $str = "グーグル谷歌中信фδοκιμήóźdźрöß😁😃";
>
>      $this->assertSame($str, utf8_decode(utf8_encode($str)));


Let's write that out with a more descriptive function name:

$str = "グーグル谷歌中信фδοκιμήóźdźрöß😁😃";

$this->assertSame($str, utf8_to_latin1(latin1_to_utf8($str)));


Since Latin-1 does not contain any Chinese, Japanese, or Emoji 
characters, running latin1_to_uft8 on that string is clearly nonsensical.

The only reason it doesn't give you any errors is that every possible 
byte is a valid character in Latin1, and every Latin1 character has a 
Unicode code point. So the "グ" is interpreted as three Latin-1 
characters: E3, 82, and B0; those then become the corresponding Unicode 
code points U+00E3, U+00821, and U+00B0, represented in UTF-8. You then 
run utf8_to_latin1, and they get converted back.

That code will never do anything useful.

Regards,

-- 
Rowan Tommins
[IMSoP]