Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:121205
MIME-Version: 1.0
References: <CAEPPVa3d6okq-wmeFSjjcuix_TyiU7pK45TrquvgMjKY1eSX3A@mail.gmail.com>
 <B5AAA3E4-525B-4D01-8873-051EFE599BBB@sakiot.com> <CAEPPVa12Rv6bwG8rzSA8R9kdmXP+CatvMiQNr279-jVzez9AOg@mail.gmail.com>
 <CAJmy8YEe3EOyEgH1hnCnctqYOPGhzy0HTpQ=LX77vHC9Cb40+Q@mail.gmail.com> <CAJmy8YHAvjMWLc1Tbb4qNj1650orvAExdO_Ef8t2CdNriPutdw@mail.gmail.com>
In-Reply-To: <CAJmy8YHAvjMWLc1Tbb4qNj1650orvAExdO_Ef8t2CdNriPutdw@mail.gmail.com>
Date: Mon, 2 Oct 2023 22:36:05 +0900
Message-ID: <CAEPPVa3EMrJFnCnir9jSVA_UiLWM4EoCea7MriTU3WyCOdxR9A@mail.gmail.com>
To: internals@lists.php.net
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Subject: Re: [PHP-DEV] trim support for multibyte spaces
From: youkidearitai@gmail.com (youkidearitai)

Hi, Hans.

Thank you very much for your code. And sorry for late.
I confirmed works fine.
https://gist.github.com/youkidearitai/0018dee27353c00aebaff3bf57c5b8c6

However, this code is C++17, php-src is C code.
If you would like contribute, I would like written to C code.

Regards
Yuya

2023=E5=B9=B410=E6=9C=881=E6=97=A5(=E6=97=A5) 19:46 Hans Henrik Bergan <div=
inity76@gmail.com>:
>
> .. probably a bunch of stuff that *could* be optimized or done better,
> but one i saw just now is that instead of 2x nested loops and goto,
> the outer loop and labels could be removed and the `goto
> remove_from_start_continue_2;` could be replaced with `i=3D-1;` eg
>
>     size_t local_strlen =3D *strlen;
>     char *local_str =3D *str;
>     for (size_t i =3D 0; i < trim_lengths_num; ++i)
>     {
>         if (local_strlen >=3D trim_lengths[i] && memcmp(local_str,
> trim_chars[i], trim_lengths[i]) =3D=3D 0)
>         {
>             local_strlen -=3D trim_lengths[i];
>             local_str +=3D trim_lengths[i];
>             i =3D -1;
>         }
>     }
>
> 2x nested loops reduced to 1 loop, and goto removed~
>
> On Sun, 1 Oct 2023 at 10:43, Hans Henrik Bergan <divinity76@gmail.com> wr=
ote:
> >
> > > If have any idea, feel free to comment to me.
> >
> > i think the C code would look something like
> >
> >
> > void mb_trim(size_t *strlen, char **str, const size_t
> > trim_lengths_num, const size_t *trim_lengths, const char **trim_chars)
> > {
> >     size_t local_strlen =3D *strlen;
> >     char *local_str =3D *str;
> >     for (;;)
> >     {
> >         for (size_t i =3D 0; i < trim_lengths_num; ++i)
> >         {
> >             if (local_strlen >=3D trim_lengths[i] && memcmp(local_str,
> > trim_chars[i], trim_lengths[i]) =3D=3D 0)
> >             {
> >                 local_strlen -=3D trim_lengths[i];
> >                 local_str +=3D trim_lengths[i];
> >                 goto remove_from_start_continue_2;
> >             }
> >         }
> >         break;
> >     remove_from_start_continue_2:;
> >     }
> >     for (;;)
> >     {
> >         for (size_t i =3D 0; i < trim_lengths_num; ++i)
> >         {
> >             if (local_strlen >=3D trim_lengths[i] && memcmp(((local_str
> > + local_strlen) - trim_lengths[i]), trim_chars[i], trim_lengths[i]) =3D=
=3D
> > 0)
> >             {
> >                 local_strlen -=3D trim_lengths[i];
> >                 goto remove_from_end_continue_2;
> >             }
> >         }
> >         break;
> >     remove_from_end_continue_2:;
> >     }
> >     memmove(*str, local_str, local_strlen);
> >     char *newstr =3D (char *)realloc(*str, local_strlen);
> >     if (newstr !=3D nullptr)
> >     {
> >         *strlen =3D local_strlen;
> >         *str =3D newstr;
> >     }
> >     else
> >     {
> >         // some error handling
> >     }
> > }
> >
> > with my simple testcode looking like
> >
> > int main()
> > {
> >     const char *trim_chars[] =3D {
> >         " ",
> >         "!",
> >         // utf8 whitespace:
> >         "\xE2\x80\x80", // EN QUAD
> >         "\xE2\x80\x81", // EM QUAD
> >         "\xE2\x80\x82", // EN SPACE
> >         "\xE2\x80\x83", // EM SPACE
> >         "\xE2\x80\x84", // THREE-PER-EM SPACE
> >         "\xE2\x80\x85", // FOUR-PER-EM SPACE
> >         "\xE2\x80\x86", // SIX-PER-EM SPACE
> >     };
> >     size_t trim_lengths[] =3D {
> >         strlen(trim_chars[0]),
> >         strlen(trim_chars[1]),
> >         strlen(trim_chars[2]),
> >         strlen(trim_chars[3]),
> >         strlen(trim_chars[4]),
> >         strlen(trim_chars[5]),
> >         strlen(trim_chars[6]),
> >         strlen(trim_chars[7]),
> >         strlen(trim_chars[8]),
> >     };
> >     size_t trim_lengths_num =3D sizeof(trim_lengths) / sizeof(trim_leng=
ths[0]);
> >     char *teststr =3D strdup("  !  \xE2\x80\x80\xE2\x80\x81\xE2\x80\x82
> >  Hello World !  \xE2\x80\x83\xE2\x80\x84\xE2\x80\x85\xE2\x80\x86  !
> > ");
> >     // char *teststr =3D strdup("      !    Hello World !        !  ");
> >     size_t teststrlen =3D strlen(teststr);
> >     std::cout << teststrlen << ": \"" << std::string_view(teststr,
> > teststrlen) << "\"" << std::endl;
> >     mb_trim(&teststrlen, &teststr, trim_lengths_num, trim_lengths, trim=
_chars);
> >     std::cout << teststrlen << ": \"" << std::string_view(teststr,
> > teststrlen) << "\"" << std::endl;
> >     return 0;
> > }
> >
> >
> > On Sat, 30 Sept 2023 at 13:16, youkidearitai <youkidearitai@gmail.com> =
wrote:
> > >
> > > 2023=E5=B9=B49=E6=9C=8830=E6=97=A5(=E5=9C=9F) 17:42 Saki Takamachi <s=
aki@sakiot.com>:
> > > >
> > > > > I also want to trim function of multibyte trim functions.
> > > >
> > > > > I think that in addition to mb_trim,
> > > > mb_ltrim and mb_rtrim are also necessary.
> > > >
> > > > Hi.
> > > >
> > > > Having a new option besides regex sounds like a good idea for me, a=
s a user of a language that benefits from `mb_trim()`.
> > > >
> > > > Perhaps users are more intuitive about the usefulness of those func=
tions who have spent more time cleaning "multibyte spaces".
> > >
> > > Hi
> > >
> > > Thanks for reply.
> > > I'm trying implements to trim for multibyte function.
> > > Please give me some time.
> > >
> > > If have any idea, feel free to comment to me.
> > >
> > > Regards
> > > Yuya
> > >
> > > --
> > > ---------------------------
> > > Yuya Hamada (tekimen)
> > > - https://tekitoh-memdhoi.info
> > > - https://github.com/youkidearitai
> > > -----------------------------
> > >
> > > --
> > > PHP Internals - PHP Runtime Development Mailing List
> > > To unsubscribe, visit: https://www.php.net/unsub.php
> > >



--=20
---------------------------
Yuya Hamada (tekimen)
- https://tekitoh-memdhoi.info
- https://github.com/youkidearitai
-----------------------------