Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:128273
Precedence: bulk
MIME-Version: 1.0
References: <7f892c24-8eef-4238-844a-5dde7f5a536d@gmail.com>
In-Reply-To: <7f892c24-8eef-4238-844a-5dde7f5a536d@gmail.com>
Date: Mon, 28 Jul 2025 18:26:25 +0100
Message-ID: <CA+XhMqwi8P_-T8XKpX8wBQ8dbdy2Zvw=+TfgUocKQuRA0J-YNg@mail.gmail.com>
Subject: Re: [PHP-DEV] pcre extended character class support
To: Niels Dossche <dossche.niels@gmail.com>, PHP internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary="000000000000042b9a063b009829"
From: devnexen@gmail.com (David CARLIER)

--000000000000042b9a063b009829
Content-Type: text/plain; charset="UTF-8"

+1000 for me.

Cheers.

On Fri, 25 Jul 2025 at 23:20, Niels Dossche <dossche.niels@gmail.com> wrote:

> Hi internals
>
> On PHP 8.5-dev, we ship with pcre2lib 10.45.
>
> This includes a new opt-in feature called "PCRE2_ALT_EXTENDED_CLASS".
> It enables the use of complex character set operations in accordance to
> UTS#18 (Unicode Technical Standard 18).
> This means it becomes possible to nest character sets, perform set
> operations on them, etc.
> One example of such a set operation is a set subtraction, e.g. the regex
> "[\ep{L}--[QW]]" means "Unicode letters other than Q and W".
> Or a more realistic example (inspired from [1]): the regex
> "[\p{Lu}--[0-9]]" matches all non-ASCII unicode numbers.
> You can also do ORs, ANDs, etc.
>
> The reason this is opt-in in pcre2lib, is because the interpretation of
> existing regexes may change.
> This standard is being adopted in other languages too, also opt-in, for
> example in JavaScript [1].
> To expose this functionality in PHP, we also have to make it opt-in via a
> modifier.
>
> In JavaScript, this is enabled via the /v modifier at the end of the regex
> [1].
> This does the same thing as the /u modifier, but extends it with this
> UTS#18 standard.
> We also already have /u in PHP that enables UTF-8 unicode mode. So we
> could do the same as JavaScript and add a /v modifier that extends /u and
> also enables PCRE2_ALT_EXTENDED_CLASS. Technically, you don't need unicode
> processing for enabling PCRE2_ALT_EXTENDED_CLASS, but as it comes from a
> unicode standard (and that at least JavaScript does this too), it may make
> sense to enable them both.
>
> The actual patch is trivial:
> ```diff
> diff --git a/ext/pcre/php_pcre.c b/ext/pcre/php_pcre.c
> index 8e0fb2cce5f..4a4727545ad 100644
> --- a/ext/pcre/php_pcre.c
> +++ b/ext/pcre/php_pcre.c
> @@ -718,6 +718,9 @@ PHPAPI pcre_cache_entry*
> pcre_get_compiled_regex_cache_ex(zend_string *regex, bo
>                         case 'S':       /* Pass. */
>              break;
>                         case 'X':       /* Pass. */
>              break;
>                         case 'U':       coptions |= PCRE2_UNGREEDY;
>      break;
> +#ifdef PCRE2_ALT_EXTENDED_CLASS
> +                       case 'v':       coptions |=
> PCRE2_ALT_EXTENDED_CLASS; ZEND_FALLTHROUGH;
> +#endif
>                         case 'u':       coptions |= PCRE2_UTF;
>         /* In  PCRE,  by  default, \d, \D, \s, \S, \w, and \W recognize
> only ASCII
>            characters, even in UTF-8 mode. However, this can be changed by
> setting
>
> ```
>
> What do we think?
>
> [1] https://github.com/tc39/proposal-regexp-v-flag
>
> Kind regards
> Niels
>

--000000000000042b9a063b009829
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">+1000 for me.<br><div><br></div><div>Cheers.</div></div><b=
r><div class=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=
=3D"gmail_attr">On Fri, 25 Jul 2025 at 23:20, Niels Dossche &lt;<a href=3D"=
mailto:dossche.niels@gmail.com">dossche.niels@gmail.com</a>&gt; wrote:<br><=
/div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo=
rder-left:1px solid rgb(204,204,204);padding-left:1ex">Hi internals<br>
<br>
On PHP 8.5-dev, we ship with pcre2lib 10.45.<br>
<br>
This includes a new opt-in feature called &quot;PCRE2_ALT_EXTENDED_CLASS&qu=
ot;.<br>
It enables the use of complex character set operations in accordance to UTS=
#18 (Unicode Technical Standard 18).<br>
This means it becomes possible to nest character sets, perform set operatio=
ns on them, etc.<br>
One example of such a set operation is a set subtraction, e.g. the regex &q=
uot;[\ep{L}--[QW]]&quot; means &quot;Unicode letters other than Q and W&quo=
t;.<br>
Or a more realistic example (inspired from [1]): the regex &quot;[\p{Lu}--[=
0-9]]&quot; matches all non-ASCII unicode numbers.<br>
You can also do ORs, ANDs, etc.<br>
<br>
The reason this is opt-in in pcre2lib, is because the interpretation of exi=
sting regexes may change.<br>
This standard is being adopted in other languages too, also opt-in, for exa=
mple in JavaScript [1].<br>
To expose this functionality in PHP, we also have to make it opt-in via a m=
odifier.<br>
<br>
In JavaScript, this is enabled via the /v modifier at the end of the regex =
[1].<br>
This does the same thing as the /u modifier, but extends it with this UTS#1=
8 standard.<br>
We also already have /u in PHP that enables UTF-8 unicode mode. So we could=
 do the same as JavaScript and add a /v modifier that extends /u and also e=
nables PCRE2_ALT_EXTENDED_CLASS. Technically, you don&#39;t need unicode pr=
ocessing for enabling PCRE2_ALT_EXTENDED_CLASS, but as it comes from a unic=
ode standard (and that at least JavaScript does this too), it may make sens=
e to enable them both.<br>
<br>
The actual patch is trivial:<br>
```diff<br>
diff --git a/ext/pcre/php_pcre.c b/ext/pcre/php_pcre.c<br>
index 8e0fb2cce5f..4a4727545ad 100644<br>
--- a/ext/pcre/php_pcre.c<br>
+++ b/ext/pcre/php_pcre.c<br>
@@ -718,6 +718,9 @@ PHPAPI pcre_cache_entry* pcre_get_compiled_regex_cache_=
ex(zend_string *regex, bo<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 case &#39;S&#39;:=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Pass. */=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0break;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 case &#39;X&#39;:=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Pass. */=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0break;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 case &#39;U&#39;:=C2=A0 =C2=A0 =C2=A0 =C2=A0coptions |=3D PCRE2_=
UNGREEDY;=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0break;<br>
+#ifdef PCRE2_ALT_EXTENDED_CLASS<br>
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0case &#39;v&#39;:=C2=A0 =C2=A0 =C2=A0 =C2=A0coptions |=3D PCRE2_A=
LT_EXTENDED_CLASS; ZEND_FALLTHROUGH;<br>
+#endif<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 case &#39;u&#39;:=C2=A0 =C2=A0 =C2=A0 =C2=A0coptions |=3D PCRE2_=
UTF;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 /* In=C2=A0 PCRE,=C2=A0 by=C2=A0 default, \d, \=
D, \s, \S, \w, and \W recognize only ASCII<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0characters, even in UTF-8 mode. Ho=
wever, this can be changed by setting<br>
<br>
```<br>
<br>
What do we think?<br>
<br>
[1] <a href=3D"https://github.com/tc39/proposal-regexp-v-flag" rel=3D"noref=
errer" target=3D"_blank">https://github.com/tc39/proposal-regexp-v-flag</a>=
<br>
<br>
Kind regards<br>
Niels<br>
</blockquote></div>

--000000000000042b9a063b009829--