Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:122562
Precedence: bulk
MIME-Version: 1.0
References: <CAEPPVa2JLAbXh0hVY72srY2FAVywr=s7baafHCF_ggqmyVgAyA@mail.gmail.com>
 <f94eb6cc-007a-4578-83cf-a4dccbec40a4@gmail.com> <CAEPPVa2SNQS_Mggq50eLUSrNaabTnQTgHQ3Y0bjqCa3FgAP4Ew@mail.gmail.com>
In-Reply-To: <CAEPPVa2SNQS_Mggq50eLUSrNaabTnQTgHQ3Y0bjqCa3FgAP4Ew@mail.gmail.com>
Date: Tue, 5 Mar 2024 17:25:11 +0900
Message-ID: <CAEPPVa3Dg=N7JNhWmvDp4tq4eiHkD84e16mSNtRBiGCWBHsYOw@mail.gmail.com>
Subject: Re: [PHP-DEV] [Discussion] grapheme cluster for str_split function
To: internals@lists.php.net
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
From: youkidearitai@gmail.com (youkidearitai)

>
> Hi, Niels
>
> Thank you for your comment.
> Indeed, returns false is make sense.
>
> Therefore, I changed to returns false when invalid UTF-8 strings.
>
> Regards
> Yuya
>
> --
> ---------------------------
> Yuya Hamada (tekimen)
> - https://tekitoh-memdhoi.info
> - https://github.com/youkidearitai
> -----------------------------

Sorry, again.
I checked behavior of mb_str_split function. So Illegal byte sequences
are returned as is.

```
sapi/cli/php -r 'var_dump(mb_str_split("=E3=81=82\xc2\xf4\x80=E3=81=82"));'
array(4) {
  [0]=3D>
  string(3) "=E3=81=82"
  [1]=3D>
  string(2) "=EF=BF=BD=EF=BF=BD"
  [2]=3D>
  string(1) "=EF=BF=BD"
  [3]=3D>
  string(3) "=E3=81=82"
}
```

And, I reading ICU document about utext_openUTF8 (below is link):
https://unicode-org.github.io/icu-docs/apidoc/dev/icu4c/utext_8h.html#a130e=
7cba201c4b38799b432eb269f6d5

> Any invalid UTF-8 in the input will be handled in this way: a sequence of=
 bytes that has the form of a truncated, but otherwise valid, UTF-8 sequenc=
e will be replaced by a single unicode replacement character, \uFFFD. Any o=
ther illegal bytes will each be replaced by a \uFFFD.

Therefore, I think encoding check is not need.
Returns only arrays together with mb_str_split.

Regards
Yuya


--=20
---------------------------
Yuya Hamada (tekimen)
- https://tekitoh-memdhoi.info
- https://github.com/youkidearitai
-----------------------------