Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:121835
MIME-Version: 1.0
References: <CAJmy8YFy5RmW1yWuA6akPU31xRz_i1io5uWgt1GWo0n5rzY9EQ@mail.gmail.com>
 <1BA05C1A-AFAE-4E86-BAA2-420B22549519@gmail.com> <0D8856BC-DDEE-47F8-8C59-7F4DC7A64237@woofle.net>
 <CAGBsUrd=5WhKERySavP=9yQM-RSkVCwhTJ2oQ4tAQOKTNFoCew@mail.gmail.com>
 <A00495F3-EEB5-4D49-9433-84E94C6920D0@gmail.com> <CAJmy8YG91ki8M-n7id=DUDUsG+1XBvtpSNdf6sJx3Y1B1Rytsg@mail.gmail.com>
 <CAJmy8YF=cpXJAWsUKh-eYBembDAWu+X4_q24qV_xfTJLzAztiw@mail.gmail.com> <CAEPPVa2xgCh-Y2ovTL3=gX5vtTyiSAiNDnKLWPpohYCJ5Tr-cQ@mail.gmail.com>
In-Reply-To: <CAEPPVa2xgCh-Y2ovTL3=gX5vtTyiSAiNDnKLWPpohYCJ5Tr-cQ@mail.gmail.com>
Date: Wed, 29 Nov 2023 00:06:29 +0100
Message-ID: <CAJmy8YFsqvbBGTQJGUiBfKH2CoJ1sHdJYfindhtQKiUwubJwpw@mail.gmail.com>
To: youkidearitai <youkidearitai@gmail.com>
Cc: internals@lists.php.net
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Subject: Re: [PHP-DEV] Deprecate declare(encoding='...') + zend.multibyte +
 zend.script_encoding + zend.detect_unicode ?
From: divinity76@gmail.com (Hans Henrik Bergan)

@youkidearitai right now the code specifically deals with
- UTF8: removing UTF8 BOM and removing `declare(encoding=3D'UTF-8');
- UTF16LE/UTF16BE/UTF32LE/UTF32BE: converting to UTF8 removing the BOM
and removing declare(encoding=3D'...')
- ISO-8859-1: converting to UTF-8 and removing
declare(encoding=3D'ISO-8859-1'), i couldn't really find information on
a ISO-8859-1 BOM, so to the best of my knowledge it does not exist

it does not deal with any other encodings as of writing, but more can
be added if needed.

On Tue, 28 Nov 2023 at 23:58, youkidearitai <youkidearitai@gmail.com> wrote=
:
>
> 2023=E5=B9=B411=E6=9C=8829=E6=97=A5(=E6=B0=B4) 7:41 Hans Henrik Bergan <d=
ivinity76@gmail.com>:
> >
> > btw if we come to some consensus to my php2utf8.php script is actually
> > worthwhile to expand on, i can volunteer to add more encodings (SJIS,
> > BIG5, anything supported by mbstring),
> > but it wouldn't surprise me if a better approach exist and the script
> > should be rewritten entirely~
> >
> > >add that what's special about UTF-8 isn't that it's "fixed-endian".
> >
> > should've added this to the last post, but the "zend.detect_unicode"
> > ini-option is specifically to scan for BOMs, and BOMs are
> > significantly less useful in fixed-endian encodings (like UTF8) than
> > bi-endian encodings (like UTF16/UTF32) ^^
> >
> > On Tue, 28 Nov 2023 at 21:47, Hans Henrik Bergan <divinity76@gmail.com>=
 wrote:
> > >
> > > > What is the migration path for legacy code that use those directive=
s?
> > >
> > > The migration path is to convert the legacy-encoding PHP files to UTF=
-8.
> > > Luckily this can be largely automated, here is my attempt:
> > > https://github.com/divinity76/php2utf8/blob/main/src/php2utf8.php
> > > but that code definitely needs some proof-reading and additions - idk
> > > if the approach used is even a good approach, it was just the first i
> > > could think of, feel free to write one from scratch
> > >
> > >
> > > >Can you share a little more details about how this works?
> > >
> > > I hope someone else can do that, but it allows PHP to parse and
> > > execute scripts not written in UTF-8 and scripts utilizing
> > > BOM/byte-order-masks.
> > >
> > > >add that what's special about UTF-8 isn't that it's "fixed-endian".
> > >
> > > one of multiple good things about UTF-8 is that it's fixed-endian, an=
d
> > > UTF8 don't need a BOM to specify endianess (unlike UTF16 and UTF32
> > > which are bi-endian, and a BOM helps identify endianess used~)
> > >
> > > >If the solution is as easy as just converting the encoding of the
> > > source file, then why did we even need to have this setting at all?
> > > Why did PHP parser support encodings that demanded the introduction o=
f
> > >
> > > I've read your question but don't have an answer to it, hopefully
> > > someone else knows.
> > >
> > >
> > > On Tue, 28 Nov 2023 at 21:09, Claude Pache <claude.pache@gmail.com> w=
rote:
> > > >
> > > >
> > > >
> > > > > Le 28 nov. 2023 =C3=A0 20:56, Kamil Tekiela <tekiela246@gmail.com=
> a =C3=A9crit :
> > > > >
> > > > >> Convert your PHP source files to UTF-8.
> > > > >
> > > > > If the solution is as easy as just converting the encoding of the
> > > > > source file, then why did we even need to have this setting at al=
l?
> > > > > Why did PHP parser support encodings that demanded the introducti=
on of
> > > > > this declare?
> > > >
> > > > It is not necessary as simple: because your code base may contain l=
iteral strings, and changing the encoding of the source file will effective=
ly change the contents of the strings.
> > > >
> > > > =E2=80=94Claude
> > > >
> >
> > --
> > PHP Internals - PHP Runtime Development Mailing List
> > To unsubscribe, visit: https://www.php.net/unsub.php
> >
>
> Hi, Hans
>
> Is this convert PHP code from any encoding to UTF-8?
> If correct, PHP code is coded various character encoding,
> It is very difficult.
> This is because it is not necessarily implemented in UTF-8.
>
> In the world, we have many character encoding.
> PHP code will be difficult to unify.
>
> Regards
> Yuya
>
> --
> ---------------------------
> Yuya Hamada (tekimen)
> - https://tekitoh-memdhoi.info
> - https://github.com/youkidearitai
> -----------------------------
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: https://www.php.net/unsub.php
>