Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:122781
Feedback-ID: id5114917:Fastmail
Message-ID: <4952eb36-1c45-4b6f-ae62-fa641e391246@rwec.co.uk>
Date: Wed, 27 Mar 2024 23:18:21 +0000
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PHP-DEV][RFC] grapheme cluster for str_split, grapheme_str_split
 function
To: internals@lists.php.net
References: <CAEPPVa0RKdhCcogf+z77rgLK5xw55R-1VBo-z8pFPKNBH3u34A@mail.gmail.com>
 <CAEPPVa2SW5uDZCS_oESk1k-RMyjuYNmLa+=xFYHb4QXhOQGc=A@mail.gmail.com>
 <CANLbj-q10+Fiz7zvnHpUgL49KNA7DCYxq21vK3gPbSpDq-BHag@mail.gmail.com>
 <CA+XhMqzgd0vyC_qB0-4an4kTNuH9ayFqshs6PdQMweyOxBw1Fg@mail.gmail.com>
 <CAEPPVa0O9qZY88wry7Ox_hRq44L6xnMboTGvPCv_GC2ZfHu5EA@mail.gmail.com>
 <141e31f3-b7cf-4bd1-9bac-c9ec078767ed@app.fastmail.com>
 <E82F0279-47D8-4BC1-9281-45799B7C018C@php.net>
 <216d4a09-7921-48f6-b892-8d9605d367ab@app.fastmail.com>
Content-Language: en-GB
In-Reply-To: <216d4a09-7921-48f6-b892-8d9605d367ab@app.fastmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
From: imsop.php@rwec.co.uk ("Rowan Tommins [IMSoP]")

On 26/03/2024 21:14, Casper Langemeijer wrote:
> If you need someone to help for the grapheme_ marketing team, let me know.

I think a big part of the problem is that very few people dig into the 
complexities of text encoding, and so don't know that a "grapheme" is 
what they're looking for.

Unicode documentation is, generally, very careful with its terminology - 
distinguishing between "code points", "code units" "graphemes" , 
"grapheme clusters", "glyphs", etc. Pretty much everyone else just says 
"character", and assumes that everyone knows what they mean.


As a case in point, looking at the PHP manual pages for strlen, 
mb_strlen, and grapheme_strlen:

Short summary:

- strlen — Get string length
- mb_strlen — Get string length
- grapheme_strlen — Get string length in grapheme units

Description:

- Returns the length of the given string.
- Gets the length of a string.
- Get string length in grapheme units (not bytes or characters)


The first two don't actually say what units they're measuring in. Maybe 
it's millimetres? ;)

The last one uses the term "grapheme" without explaining what it means, 
and makes a contrast with "characters", which is confusing, as one of 
the definitions in the Unicode glossary 
[https://unicode.org/glossary/#grapheme] is:

 > What a user thinks of as a character.


The mb_strlen documentation has a bit more explanation in its Return 
Values section:

 > Returns the number of characters in string string having character 
encoding encoding. A multi-byte character is counted as 1.

For Unicode in particular, this is a poor description; it is completely 
missing the term "code point", which is what it actually counts.

That's probably because ext/mbstring wasn't written with Unicode in 
mind, it was "developed to handle Japanese characters", back in 2001; 
and it still does support several pre-Unicode "multi-byte encodings". 
For a bit of nostalgia: 
http://web.archive.org/web/20010605075550/http://www.php.net/manual/en/ref.mbstring.php


So... if you want to help make people more aware of the grapheme_* 
functions, one place to start would be editing the documentation for the 
various string, mbstring, and grapheme functions to use consistent 
terminology, and sign-post each other more clearly. 
http://doc.php.net/tutorial/


Regards,

-- 
Rowan Tommins
[IMSoP]