Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:124651
Feedback-ID: ifab94697:Fastmail
User-Agent: Cyrus-JMAP/3.11.0-alpha0-582-g5a02f8850-fm-20240719.002-g5a02f885
Precedence: bulk
MIME-Version: 1.0
Message-ID: <c6478a39-8415-407a-912d-65b6e8345073@app.fastmail.com>
In-Reply-To: <5155581a-3531-4af7-98d3-e637accac16e@gmx.de>
References: <e352423f-b740-07c9-2c4a-996112e17bbe@cubiclesoft.com>
 <92c4514f-70e3-75c9-7084-9e29641e25e7@gmail.com>
 <7e86a2d2-b971-592c-64e3-e86c13b5be80@cubiclesoft.com>
 <E963ED74-0404-4A5D-9811-8D1E662F764A@gmail.com>
 <5155581a-3531-4af7-98d3-e637accac16e@gmx.de>
Date: Sat, 27 Jul 2024 16:31:09 +0200
To: "Christoph M. Becker" <cmbecker69@gmx.de>,
 "Rowan Tommins" <rowan.collins@gmail.com>, internals@lists.php.net,
 "Thomas Hruska" <thruska@cubiclesoft.com>
Subject: Re: [PHP-DEV] [RFC] Working With Substrings
Content-Type: multipart/alternative;
 boundary=d558cd7714c84ed49a020f51ad73180f
From: rob@bottled.codes ("Rob Landers")

--d558cd7714c84ed49a020f51ad73180f
Content-Type: text/plain;charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Sat, Jul 27, 2024, at 15:26, Christoph M. Becker wrote:
> On 15.02.2023 at 06:18, Rowan Tommins wrote:
>=20
> > On 15 February 2023 02:35:42 GMT, Thomas Hruska <thruska@cubiclesoft=
.com> wrote:
> >
> >> On 2/14/2023 2:02 PM, Rowan Tommins wrote:
> >>
> >> I thought about that but didn't know how well it would be received =
nor, perhaps more importantly, the direction it should take (i.e. a form=
al Zend type in the engine, extending the existing zend_string type, a c=
lass, some combination, or something else entirely).  All of the more ad=
vanced options I came up with would have required some code changes to t=
he PHP source itself with a new data type being the most involved and pr=
obably the most controversial.
> >
> > My instinct was that it could just be a built-in class, with an inte=
rnal pointer to a zend_string that's completely invisible to userland. S=
omething like how the SimpleXML and DOM objects just point into a libxml=
 parse result.
> >
> > Then to add to existing functions requires changing an argument type=
 from string to string|Buffer, rather than adding new arguments.
> >
> > No change to the type system needed, internally or externally, just =
some code to unwrap the pointer. But perhaps I'm being naive and oversim=
plifying, as I don't have a deep understanding of the engine.
> >
> >> I'm not entirely sure what the next step here should be.  Should I =
go research the above, or go back and develop/test and then propose some=
thing concrete in an OO direction and gather feedback at that point, or =
should we hash it out a bit more here on the list to get a more specific=
 direction to go in?
> >
> > Well, those were just my thoughts; maybe someone else will come alon=
g shortly with a very different take.
>=20
> I'm very late on this discussion, but I think it is an interesting
> topic, and maybe <https://github.com/cmb69/php-stringbuilder>, which I
> had written long ago just to check some assumptions, can serve as POC.
> It is certainly possible to have such a string buffer class without
> having to patch the engine; it could even be made available as PECL
> extension (first).
>=20
> Note that this StringBuilder uses `smart_str`s[1] what might be a good
> idea or not.  But certainly you could use some other internal handling;
> interoperability with `zend_string`s[2] requires to copy the char arra=
ys
> in most cases anyway, since these have a fixed length, and if these
> copies are reduced to a minimum (i.e. the new class has enough
> flexibility to work without casting to and from string), that should be
> bearable.
>=20
> Not sure if that would work for the "gd imageexportpixels() and
> imageimportpixels()" RFC[3], but it might be worth investigating.
>=20
> [1]
> <https://www.phpinternalsbook.com/php7/internal_types/strings/smart_st=
r.html>
> [2]
> <https://www.phpinternalsbook.com/php7/internal_types/strings/zend_str=
ings.html>
> [3] <https://wiki.php.net/rfc/gd_image_export_import_pixels>
>=20
> Cheers,
> Christoph
>=20

Huh, I am also very late and somewhat poignant, last weekend, I managed =
to refactor all zend_strings to contain a char* instead of char[1] and t=
he char* pointed to the memory just after the pointer. It increased zend=
_string by a few bytes on a 64bit machine, but would allow for some nice=
 optimizations, such as zend_strings sharing memory (effectively removin=
g the need for the current interned strings implementation). I ended up =
ditching it because it would break literally every extension that does i=
ts own allocations instead of calling zend_string_alloc|init() and it wa=
s also hard to manage when copying strings, which also some core extensi=
ons do instead of calling core zend_string_* functions. Needless to say,=
 "vanilla php" worked fine and all tests passed.

I did submit a small part of my refactoring here: https://github.com/php=
/php-src/pull/15054 but even something that simple didn't seem well rece=
ived. So, I won't continue this approach.

But, fwiw, I wouldn't advise changing zend_strings too much, many extens=
ions appear to do one of two things: their own allocations and/or their =
own copying and/or their own freeing.

=E2=80=94 Rob
--d558cd7714c84ed49a020f51ad73180f
Content-Type: text/html;charset=utf-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso=
Normal,p.MsoNoSpacing{margin:0}</style></head><body><div>On Sat, Jul 27,=
 2024, at 15:26, Christoph M. Becker wrote:<br></div><blockquote type=3D=
"cite" id=3D"qt" style=3D""><div>On 15.02.2023 at 06:18, Rowan Tommins w=
rote:<br></div><div><br></div><div>&gt; On 15 February 2023 02:35:42 GMT=
, Thomas Hruska &lt;<a href=3D"mailto:thruska@cubiclesoft.com">thruska@c=
ubiclesoft.com</a>&gt; wrote:<br></div><div>&gt;<br></div><div>&gt;&gt; =
On 2/14/2023 2:02 PM, Rowan Tommins wrote:<br></div><div>&gt;&gt;<br></d=
iv><div>&gt;&gt; I thought about that but didn't know how well it would =
be received nor, perhaps more importantly, the direction it should take =
(i.e. a formal Zend type in the engine, extending the existing zend_stri=
ng type, a class, some combination, or something else entirely).&nbsp; A=
ll of the more advanced options I came up with would have required some =
code changes to the PHP source itself with a new data type being the mos=
t involved and probably the most controversial.<br></div><div>&gt;<br></=
div><div>&gt; My instinct was that it could just be a built-in class, wi=
th an internal pointer to a zend_string that's completely invisible to u=
serland. Something like how the SimpleXML and DOM objects just point int=
o a libxml parse result.<br></div><div>&gt;<br></div><div>&gt; Then to a=
dd to existing functions requires changing an argument type from string =
to string|Buffer, rather than adding new arguments.<br></div><div>&gt;<b=
r></div><div>&gt; No change to the type system needed, internally or ext=
ernally, just some code to unwrap the pointer. But perhaps I'm being nai=
ve and oversimplifying, as I don't have a deep understanding of the engi=
ne.<br></div><div>&gt;<br></div><div>&gt;&gt; I'm not entirely sure what=
 the next step here should be.&nbsp; Should I go research the above, or =
go back and develop/test and then propose something concrete in an OO di=
rection and gather feedback at that point, or should we hash it out a bi=
t more here on the list to get a more specific direction to go in?<br></=
div><div>&gt;<br></div><div>&gt; Well, those were just my thoughts; mayb=
e someone else will come along shortly with a very different take.<br></=
div><div><br></div><div>I'm very late on this discussion, but I think it=
 is an interesting<br></div><div>topic, and maybe &lt;<a href=3D"https:/=
/github.com/cmb69/php-stringbuilder">https://github.com/cmb69/php-string=
builder</a>&gt;, which I<br></div><div>had written long ago just to chec=
k some assumptions, can serve as POC.<br></div><div>It is certainly poss=
ible to have such a string buffer class without<br></div><div>having to =
patch the engine; it could even be made available as PECL<br></div><div>=
extension (first).<br></div><div><br></div><div>Note that this StringBui=
lder uses `smart_str`s[1] what might be a good<br></div><div>idea or not=
.&nbsp; But certainly you could use some other internal handling;<br></d=
iv><div>interoperability with `zend_string`s[2] requires to copy the cha=
r arrays<br></div><div>in most cases anyway, since these have a fixed le=
ngth, and if these<br></div><div>copies are reduced to a minimum (i.e. t=
he new class has enough<br></div><div>flexibility to work without castin=
g to and from string), that should be<br></div><div>bearable.<br></div><=
div><br></div><div>Not sure if that would work for the "gd imageexportpi=
xels() and<br></div><div>imageimportpixels()" RFC[3], but it might be wo=
rth investigating.<br></div><div><br></div><div>[1]<br></div><div>&lt;<a=
 href=3D"https://www.phpinternalsbook.com/php7/internal_types/strings/sm=
art_str.html">https://www.phpinternalsbook.com/php7/internal_types/strin=
gs/smart_str.html</a>&gt;<br></div><div>[2]<br></div><div>&lt;<a href=3D=
"https://www.phpinternalsbook.com/php7/internal_types/strings/zend_strin=
gs.html">https://www.phpinternalsbook.com/php7/internal_types/strings/ze=
nd_strings.html</a>&gt;<br></div><div>[3] &lt;<a href=3D"https://wiki.ph=
p.net/rfc/gd_image_export_import_pixels">https://wiki.php.net/rfc/gd_ima=
ge_export_import_pixels</a>&gt;<br></div><div><br></div><div>Cheers,<br>=
</div><div>Christoph<br></div><div><br></div></blockquote><div><br></div=
><div>Huh, I am also very late and somewhat poignant, last weekend, I ma=
naged to refactor all zend_strings to contain a char* instead of char[1]=
 and the char* pointed to the memory just after the pointer. It increase=
d zend_string by a few bytes on a 64bit machine, but would allow for som=
e nice optimizations, such as zend_strings sharing memory (effectively r=
emoving the need for the current interned strings implementation). I end=
ed up ditching it because it would break literally every extension that =
does its own allocations instead of calling zend_string_alloc|init() and=
 it was also hard to manage when copying strings, which also some core e=
xtensions do instead of calling core zend_string_* functions. Needless t=
o say, "vanilla php" worked fine and all tests passed.<br></div><div><br=
></div><div>I did submit a small part of my refactoring here:&nbsp;<a hr=
ef=3D"https://github.com/php/php-src/pull/15054">https://github.com/php/=
php-src/pull/15054</a>&nbsp;but even something that simple didn't seem w=
ell received. So, I won't continue this approach.<br></div><div><br></di=
v><div>But, fwiw, I wouldn't advise changing zend_strings too much, many=
 extensions appear to do one of two things: their own allocations and/or=
 their own copying and/or their own freeing.</div><div><br></div><div id=
=3D"sig121229152">=E2=80=94 Rob<br></div></body></html>
--d558cd7714c84ed49a020f51ad73180f--