Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:106048
MIME-Version: 1.0
References: <8442f1fa5544b2ca03e7cebbc64e8e5c@wkhudgins.info>
 <EECABEB0-D1E5-4438-A061-477C9BEDC024@simon.geek.nz> <eab667bf34e12ee2ad8b6bedabf66cc2@wkhudgins.info>
 <B6702192-0F94-4676-9B44-0DADD0E65060@simon.geek.nz> <683c5da474e13283030cac3d0c0ec080@wkhudgins.info>
 <2c37999d1e5372ae6ab48bfce5420796@wkhudgins.info> <CAF+90c-YBYSQGkmwS0qa68fpAcQxef_7SGa6LRkavamiwLtpRA@mail.gmail.com>
 <2CF672F8-12F5-4D37-8B8C-591A6E695220@benramsey.com> <3E2100B1-7BF7-4C9F-AA77-D82924A2D5FC@gmail.com>
 <8CFCFE96-E2B7-456B-85A3-8737754C59D6@benramsey.com> <CAF+90c_yNYSr65Atf_N7Ytg1fOAYgCpM1HG3DcquhOqi3jytqw@mail.gmail.com>
In-Reply-To: <CAF+90c_yNYSr65Atf_N7Ytg1fOAYgCpM1HG3DcquhOqi3jytqw@mail.gmail.com>
Date: Mon, 24 Jun 2019 11:28:18 +0200
Message-ID: <CAF+90c88dsraskjaRZQF=rq2f2CJc1mYCzk5N8g7M4t3q30NYA@mail.gmail.com>
To: Ben Ramsey <ben@benramsey.com>
Cc: Rowan Collins <rowan.collins@gmail.com>, PHP internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary="000000000000b5aacb058c0e6f99"
Subject: Re: [PHP-DEV] [RFC] Desire to move RFC add_str_begin_and_end_functions
 to a vote
From: nikita.ppv@gmail.com (Nikita Popov)

--000000000000b5aacb058c0e6f99
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, Jun 23, 2019 at 5:46 PM Nikita Popov <nikita.ppv@gmail.com> wrote:

> On Sun, Jun 23, 2019 at 5:30 PM Ben Ramsey <ben@benramsey.com> wrote:
>
>> > On Jun 23, 2019, at 05:35, Rowan Collins <rowan.collins@gmail.com>
>> wrote:
>> >
>> > On 22 June 2019 20:56:24 BST, Ben Ramsey <ben@benramsey.com> wrote:
>> >> Perhaps it would only be an issue with the case-insensitive versions,
>> >> as Nikita points out? If so, can someone provide some example strings
>> >> where an mb_starts_with_ci() would return true, while
>> >> str_starts_with_ci() would return false?
>> >
>> >
>> > That's easy: any character that has a lower- and uppercase form, and i=
s
>> not represented as one byte in the target encoding. For that matter, any
>> such character in the non-ASCII section of a single-byte encoding, since=
 a
>> non-mbstring case insensitive flag would presumably leave everything oth=
er
>> than ASCII letters untouched.
>> >
>> > So, any non-Latin script, like Greek or Cyrillic; any accented
>> characters, unless you're lucky and they're represented by ASCII-letter
>> plus combining modifier; the Turkish "i", which if I remember rightly ha=
s
>> three forms not two; and so on.
>>
>>
>> According to Google, "=C4=B0yi ak=C5=9Famlar=E2=80=9D is the Turkish phr=
ase for =E2=80=9CGood
>> evening=E2=80=9D (Turkish speakers, please correct me, if this wrong). H=
owever,
>> using the existing mb_* functions, I can=E2=80=99t get mb_stripos() to r=
eturn 0
>> when trying to see if the string =E2=80=9C=C4=B0YI AK=C5=9EAMLAR=E2=80=
=9D begins with =E2=80=9Ci=CC=87yi.=E2=80=9D
>>
>> I=E2=80=99m just using UTF-8, so maybe there=E2=80=99s an encoding issue=
 here?
>>
>> $string =3D '=C4=B0yi ak=C5=9Famlar';
>> $upper =3D mb_strtoupper($string);
>> $lowerChars =3D mb_strtolower(mb_substr($string, 0, 3));
>>
>> var_dump($string, $upper, $lowerChars);
>> var_dump(mb_stripos($upper, $lowerChars));
>>
>
> The reason why this doesn't work is that mb_stripos internally performs a
> simple case fold, while a full case fold would be needed in this case
> (Turkish i is hard). It's a bit tricky due to the need to remap character
> offsets.
>

I've implemented use of full case folding in
https://github.com/php/php-src/pull/4303. While doing that I kind of
convinced myself that we probably shouldn't actually do this, because it
breaks simple mb_stripos loops in a subtle way. It probably makes more
sense for people to explicitly call mb_convert_case($string, MB_CASE_FOLD)
and then operate on the resulting strings. Both much more efficient, and
avoids offset remapping issues.

Nikita

--000000000000b5aacb058c0e6f99--