# [DISCUSSION] Multibyte for levenshtein function

- Thread: https://externals.io/message/125674
- Original archive: https://news-web.php.net/php.internals/125674
- Started by youkidearitai on 2024-09-25T07:21:56+00:00
- Messages: 4
- RFC: Add mb_levenshtein function (Declined) — https://externals.io/rfc/mb_levenshtein.md
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · RFCs at https://externals.io/rfc.md · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #125674 — youkidearitai — 2024-09-25T07:21:56+00:00

URL: https://externals.io/message/125674#125674

Hello Internals.

I tried implement mb_levenshtein function and create an RFC.
https://wiki.php.net/rfc/mb_levenshtein
https://github.com/php/php-src/pull/16043

I would like discussion, feel free to comment.

Thank you.
Yuya.

-- 
---------------------------
Yuya Hamada (tekimen)
- https://tekitoh-memdhoi.info
- https://github.com/youkidearitai

---

## #125751 — tim@bastelstu.be — 2024-10-04T16:20:52+00:00

URL: https://externals.io/message/125674#125751 · In reply to: #125674

Hi

Am 2024-09-25 09:21, schrieb youkidearitai:
> I tried implement mb_levenshtein function and create an RFC.
> https://wiki.php.net/rfc/mb_levenshtein
> https://github.com/php/php-src/pull/16043
> 
> I would like discussion, feel free to comment.

Thank you for your RFC. I share the concern raised by cmb in the PR 
discussion: 
https://github.com/php/php-src/pull/16043#issuecomment-2374574538

Generally working with codepoints is going to be confusing for a user, 
but sometimes it is necessary when dealing with external systems that 
themselves work with codepoints (MySQL comes to my mind). However 
calculating the Levenshtein distance is most certainly something that 
purely is "user-facing" and not constrained by external systems. 
Calculating the distance of codepoints is going to be extremely 
confusing when dealing with things like Emoji. It would probably best to 
either only offer a `grapheme_*` function here or to leave this fully to 
userland.

Best regards
Tim Düsterhus

---

## #125755 — youkidearitai — 2024-10-06T05:45:07+00:00

URL: https://externals.io/message/125674#125755 · In reply to: #125751

2024年10月5日(土) 1:20 Tim Düsterhus <tim@bastelstu.be>:
>
> Hi
>
> Am 2024-09-25 09:21, schrieb youkidearitai:
> > […]
>
> Thank you for your RFC. I share the concern raised by cmb in the PR
> discussion:
> https://github.com/php/php-src/pull/16043#issuecomment-2374574538
>
> Generally working with codepoints is going to be confusing for a user,
> but sometimes it is necessary when dealing with external systems that
> themselves work with codepoints (MySQL comes to my mind). However
> calculating the Levenshtein distance is most certainly something that
> purely is "user-facing" and not constrained by external systems.
> Calculating the distance of codepoints is going to be extremely
> confusing when dealing with things like Emoji. It would probably best to
> either only offer a `grapheme_*` function here or to leave this fully to
> userland.
>
> Best regards
> Tim Düsterhus

Hi, Tim

Thank you for response.
I thinking about wants users what is levenshtein distance.
Surely, I think Levenshtein distance should be measured in terms of
grapheme clusters.

In most userland codes that based on UTF-8. So seems move to grapheme
function is make sense.
I more thinking usecase of levenshtein. Probably I'm going to grapheme function.

Thanks
Yuya

-- 
---------------------------
Yuya Hamada (tekimen)
- https://tekitoh-memdhoi.info
- https://github.com/youkidearitai

---

## #125797 — youkidearitai — 2024-10-14T07:37:53+00:00

URL: https://externals.io/message/125674#125797 · In reply to: #125755

2024年10月6日(日) 14:45 youkidearitai <youkidearitai@gmail.com>:
>
> 2024年10月5日(土) 1:20 Tim Düsterhus <tim@bastelstu.be>:
> > […]
>
> Hi, Tim
>
> Thank you for response.
> I thinking about wants users what is levenshtein distance.
> Surely, I think Levenshtein distance should be measured in terms of
> grapheme clusters.
>
> In most userland codes that based on UTF-8. So seems move to grapheme
> function is make sense.
> I more thinking usecase of levenshtein. Probably I'm going to grapheme function.
>
> Thanks
> Yuya
>
> --
> ---------------------------
> Yuya Hamada (tekimen)
> - https://tekitoh-memdhoi.info
> - https://github.com/youkidearitai
> -----------------------------

Hi, internals
I'm thinking more about use case of mb_levenshtein.
I added test case of mb_levenshtein that compare emoji per code point.
https://github.com/php/php-src/pull/16043/files#diff-d6aca000d2b0ac5982f9f9a0fe0425246cfd8411fdfb8645cdfe6f786d526597R86

It means make sense to compare Unicode codepoint.
I think need mb_levenshtein, and also needs grapheme_levenshtein.

What do you think?

Regards
Yuya

-- 
---------------------------
Yuya Hamada (tekimen)
- https://tekitoh-memdhoi.info
- https://github.com/youkidearitai