# Exposing object handles to userland - Thread: https://externals.io/message/87437 - Original archive: https://news-web.php.net/php.internals/87437 - Started by Julien Pauli on 2015-07-31T14:23:40+00:00 - Messages: 35 - Page 1 of 2: messages 1 to 25 (25 per page, add `?page=N` to the URL) - Next page: https://externals.io/message/87437.md?page=2 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #87437 — Julien Pauli — 2015-07-31T14:23:40+00:00 URL: https://externals.io/message/87437#87437 Hi people. I've been pinged many times to add a new `spl_object_id()` function to PHP, that would return the internal object handle of an object. Today, `spl_object_hash()` partially allows that, but adds many randomness to the result, which is not very cool to use later (why does it even add randomness ?). There has been topics about this subject. For example, at http://marc.info/?l=php-internals&m=141814350920452&w=2 Beeing able to get the object handle back in PHP userland would ease many tools, mainly debug-oriented tools. I know PHPUnit, Symfony and many big projects today make use of `spl_object_hash()` to identify objects. I also know people that print_r($an_object) and parse the output just to extract the object handle from there... Crazy isn't it ? Why couldn't we help those people by simply adding a new function that does the job ? Thoughts ? Julien.Pauli --- ## #87440 — Anatol Belski — 2015-07-31T14:44:11+00:00 URL: https://externals.io/message/87437#87440 · In reply to: #87437 Hi, > -----Original Message----- > From: julienpauli@gmail.com [mailto:julienpauli@gmail.com] On Behalf Of > Julien Pauli > Sent: Friday, July 31, 2015 4:24 PM > To: PHP Internals > Subject: [PHP-DEV] Exposing object handles to userland > > Hi people. > > I've been pinged many times to add a new `spl_object_id()` function to PHP, that > would return the internal object handle of an object. > > Today, `spl_object_hash()` partially allows that, but adds many randomness to the > result, which is not very cool to use later (why does it even add randomness ?). > > There has been topics about this subject. > For example, at http://marc.info/?l=php-internals&m=141814350920452&w=2 > > > Beeing able to get the object handle back in PHP userland would ease many > tools, mainly debug-oriented tools. > I know PHPUnit, Symfony and many big projects today make use of > `spl_object_hash()` to identify objects. > > I also know people that print_r($an_object) and parse the output just to extract > the object handle from there... Crazy isn't it ? > Why couldn't we help those people by simply adding a new function that does > the job ? > Regarding the ID keywords, that reminds on https://docs.python.org/2/library/functions.html#id . While not everything is object in PHP, fe it could be interesting at least for the strings since we have interned strings. Maybe there were some wider range of useful information to provide in user land, not only related to objects. Regards Anatol --- ## #87442 — Nicolas Grekas — 2015-07-31T14:53:05+00:00 URL: https://externals.io/message/87437#87442 · In reply to: #87437 > I also know people that print_r($an_object) and parse the output just to > extract the object handle from there... Crazy isn't it ? > I plead guilty for doing this, but php let me no better choice for now ;) The attached patch removes the XOR hashing for the object handle (it's useless, the "secret" is trivially guessed after parsing the output of var_dump). It would be awesome if this patch could be applied for php 7.0! Cheers, Nicolas &q=spl_object_hash https://github.com/symfony/symfony/blob/master/src/Symfony/Component/VarDumper/Cloner/VarCloner.php https://github.com/horde/horde/blob/master/imp/lib/Factory/MimeViewer.php https://github.com/horde/horde/blob/master/framework/Support/lib/Horde/Support/Randomid.php https://github.com/WordPress/WordPress/blob/master/wp-includes/plugin.php Obviously there will be more. Seems some stuff even depends on the exact output. So it would be most likely a BC break. Now, I’d see this more like a feature request, as Julien was mentioning another function at the start. But IMHO opinions from people specializing on test/debug user land tools would give a better sight, so CC’ing. Regards Anatol From: nicolas.grekas@gmail.com [mailto:nicolas.grekas@gmail.com] On Behalf Of Nicolas Grekas Sent: Friday, July 31, 2015 4:53 PM To: Julien Pauli Cc: PHP Internals Subject: Re: [PHP-DEV] Exposing object handles to userland I also know people that print_r($an_object) and parse the output just to extract the object handle from there... Crazy isn't it ? I plead guilty for doing this, but php let me no better choice for now ;) The attached patch removes the XOR hashing for the object handle (it's useless, the "secret" is trivially guessed after parsing the output of var_dump). It would be awesome if this patch could be applied for php 7.0! Cheers, Nicolas --- ## #87444 — Nicolas Grekas — 2015-07-31T16:11:03+00:00 URL: https://externals.io/message/87437#87444 · In reply to: #87443 > > Have you checked the impact of changing the existing function? > Yes I did, and this breaks absolutely nothing: the spl_object_hash output has exactly the same format (otherwise it's a bug). > > > > > https://github.com/symfony/symfony/blob/master/src/Symfony/Component/VarDumper/Cloner/VarCloner.php > That's my code and the very place where I'd use this feature first :) --- ## #87445 — Anthony Ferrara — 2015-07-31T16:35:45+00:00 URL: https://externals.io/message/87437#87445 · In reply to: #87437 Julien, > Hi people. > > I've been pinged many times to add a new `spl_object_id()` function to PHP, > that would return the internal object handle of an object. > > Today, `spl_object_hash()` partially allows that, but adds many randomness to > the result, which is not very cool to use later (why does it even add > randomness ?). > > There has been topics about this subject. > For example, at http://marc.info/?l=php-internals&m=141814350920452&w=2 > > > Beeing able to get the object handle back in PHP userland would ease many > tools, mainly debug-oriented tools. > I know PHPUnit, Symfony and many big projects today make use of > `spl_object_hash()` to identify objects. > > I also know people that print_r($an_object) and parse the output just to > extract the object handle from there... Crazy isn't it ? > Why couldn't we help those people by simply adding a new function that does > the job ? > > Thoughts ? > I'm not sure about the randomness to the handle, but the hash also includes the object handler pointer. So without the random, we'd be leaking information about the memory layout of the application. And for the record, I am cool with simply exposing the handle. Anthony --- ## #87447 — Anatol Belski — 2015-07-31T17:43:23+00:00 URL: https://externals.io/message/87437#87447 · In reply to: #87444 > -----Original Message----- > From: nicolas.grekas@gmail.com [mailto:nicolas.grekas@gmail.com] On Behalf > Of Nicolas Grekas > Sent: Friday, July 31, 2015 6:11 PM > To: Anatol Belski > Cc: Julien Pauli ; PHP Internals ; > Sebastian Bergmann ; Ivan Enderlin@Hoa > ; contact@jubianchi.fr > Subject: Re: [PHP-DEV] Exposing object handles to userland > > > […] > > Yes I did, and this breaks absolutely nothing: the spl_object_hash output has > exactly the same format (otherwise it's a bug). > > > > […] > > […] > https://github.com/symfony/symfony/blob/master/src/Symfony/Component/V > > […] > > That's my code and the very place where I'd use this feature first :) Yeah, saw your name in the commits :) Anthony's argument about exposing the mem layout is crucial, though. Regards anatol --- ## #87448 — Nicolas Grekas — 2015-07-31T18:24:14+00:00 URL: https://externals.io/message/87437#87448 · In reply to: #87447 > Anthony's argument about exposing the mem layout is crucial, though. > Yes it is! The patch I attached un-xors only the part for the object's handle. The memory pointer is kept xored. --- ## #87449 — Anthony Ferrara — 2015-07-31T19:19:56+00:00 URL: https://externals.io/message/87437#87449 · In reply to: #87448 Nicolas, On Fri, Jul 31, 2015 at 2:24 PM, Nicolas Grekas wrote: > > […] > > Yes it is! > > The patch I attached un-xors only the part for the object's handle. > The memory pointer is kept xored. Just checked the patch, perfect :-) You have a +1 from me, for what it's worth. Anthony --- ## #87454 — Stephen Coakley — 2015-07-31T19:54:50+00:00 URL: https://externals.io/message/87437#87454 · In reply to: #87437 > Hi people. > > I've been pinged many times to add a new `spl_object_id()` function to PHP, > that would return the internal object handle of an object. > > Today, `spl_object_hash()` partially allows that, but adds many randomness to > the result, which is not very cool to use later (why does it even add > randomness ?). > > There has been topics about this subject. > For example, at http://marc.info/?l=php-internals&m=141814350920452&w=2 > > > Beeing able to get the object handle back in PHP userland would ease many > tools, mainly debug-oriented tools. > I know PHPUnit, Symfony and many big projects today make use of > `spl_object_hash()` to identify objects. > > I also know people that print_r($an_object) and parse the output just to > extract the object handle from there... Crazy isn't it ? > Why couldn't we help those people by simply adding a new function that does > the job ? > > Thoughts ? > > > Julien.Pauli > I can think of several use cases why this might be useful, and not just for debugging-related code. It could be used for indexing some sort of complex object storage data structure (if you can't/won't use SplObjectStorage). I can think of a few libraries that us `spl_object_hash()` to do the same thing but doesn't work well in conjunction with forking (probably due to the randomness factor). +1 -- Stephen Coakley --- ## #87456 — Nikita Popov — 2015-07-31T20:08:56+00:00 URL: https://externals.io/message/87437#87456 · In reply to: #87442 On Fri, Jul 31, 2015 at 4:53 PM, Nicolas Grekas < nicolas.grekas+php@gmail.com> wrote: > > I also know people that print_r($an_object) and parse the output just to > > […] > > I plead guilty for doing this, but php let me no better choice for now ;) > > The attached patch removes the XOR hashing for the object handle (it's > useless, the "secret" is trivially guessed after parsing the output of > var_dump). > > It would be awesome if this patch could be applied for php 7.0! > > Cheers, > Nicolas > > > > -- > > I'd prefer to add a separate function spl_object_id, which directly returns the handle. This should supersede spl_object_hash in the long run. spl_object_hash does a bunch of pointless things that serve no purpose other than making the function slower and making the result more bulky. Nikita --- ## #87490 — Stanislav Malyshev — 2015-08-02T06:41:57+00:00 URL: https://externals.io/message/87437#87490 · In reply to: #87437 Hi! > I've been pinged many times to add a new `spl_object_id()` function to PHP, > that would return the internal object handle of an object. What would be the use of this? > Today, `spl_object_hash()` partially allows that, but adds many randomness to > the result, which is not very cool to use later (why does it even add > randomness ?). So that people would be less inclined to use these hashes as if they mean something. And to not reveal internal memory details if these IDs are leaked externally for some reason (which they shouldn't be but sigh...) > There has been topics about this subject. > For example, at http://marc.info/?l=php-internals&m=141814350920452&w=2 > > > Beeing able to get the object handle back in PHP userland would ease many > tools, mainly debug-oriented tools. I would say debug tools probably need extensions which can access internal data structures. Exposing internal data creates bad dependencies, then we can't fix bugs or refactor stuff because somebody used internal API in wrong way and now depends on it (like object unserialization problem where people started using unserialization for purposes it was never intended to). Let's say in 8.x we remove handles altogether and replace them with pointers or some other magic? What these tools would be doing then? By creating this function, we put values of object IDs (not even their existence, but also their values!) into public API, which means not we take BC obligations on them. I don't think it is a good thing. > I also know people that print_r($an_object) and parse the output just to > extract the object handle from there... Crazy isn't it ? Yes, they should not be using object handles for any practical purposes, it's an implementation detail depending on which is a bad idea. > Why couldn't we help those people by simply adding a new function that does > the job ? I'm not sure that is the job that should be done. -- Stas Malyshev smalyshev@gmail.com --- ## #87491 — Stanislav Malyshev — 2015-08-02T06:52:21+00:00 URL: https://externals.io/message/87437#87491 · In reply to: #87443 Hi! Some suspicious use of `spl_object_hash()` out there... > https://github.com/symfony/symfony/blob/master/src/Symfony/Component/VarDumper/Cloner/VarCloner.php Not sure what this one does... but calculations with `spl_object_hash()` look very suspicious. > > https://github.com/horde/horde/blob/master/imp/lib/Factory/MimeViewer.php > This one _might_ be doing it right, but not sure as basing caching instances on hash of a (mutable) object may produce weird results. > https://github.com/horde/horde/blob/master/framework/Support/lib/Horde/Support/Randomid.php Oh wow, what's going on there? That's obviously not a proper use of `spl_object_hash()`. > > https://github.com/WordPress/WordPress/blob/master/wp-includes/plugin.php I guess this one is wrong too, as it mentions storage, and storing object ID is pointless. Maybe I am misunderstanding what "storage" means there. -- Stas Malyshev smalyshev@gmail.com --- ## #87492 — Stanislav Malyshev — 2015-08-02T07:07:02+00:00 URL: https://externals.io/message/87437#87492 · In reply to: #87456 Hi! > I'd prefer to add a separate function spl_object_id, which directly returns > the handle. This should supersede spl_object_hash in the long run. Since in PHP 7 there's no longer a possibility of having different object stores, as far as I can see, spl_object_id can identify an object (it wasn't possible before 7). However, committing to having specific object ID (e.g. as integer) as part of public API is pretty big commitment which I personally think creates unnecessary coupling between PHP internals and user code and is a bad design decision which may bite us later. > spl_object_hash does a bunch of pointless things that serve no purpose > other than making the function slower and making the result more bulky. Before PHP 7, it created proper object identification, as objects could be stored in different stores. As PHP 7 removed that possibility, indeed adding Z_OBJ_HT_P is no longer serving any purpose. Performance though has very little to do with it - it's just two XORs and one printf. -- Stas Malyshev smalyshev@gmail.com --- ## #87493 — Stanislav Malyshev — 2015-08-02T07:12:23+00:00 URL: https://externals.io/message/87437#87493 · In reply to: #87442 Hi! > It would be awesome if this patch could be applied for php 7.0! Please don't. This changes the whole meaning of spl_object_hash, and serves no useful purpose. If you need the id so much, it should be a separate function. I am still against exposing raw IDs, but if the consensus is we need them then it should be done cleanly not by just taking existing function and repurposing half of its output. That's not the right design by any measure. And rushing with it into 7.0 does not give any time to properly discuss it. -- Stas Malyshev smalyshev@gmail.com --- ## #87495 — Nicolas Grekas — 2015-08-02T08:58:36+00:00 URL: https://externals.io/message/87437#87495 · In reply to: #87492 Hi Stas, thanks for hooking into this thread However, committing to having specific object ID (e.g. as integer) as > part of public API is pretty big commitment This is already part of the public API: var_dump and debug_zval_dump do expose object IDs to userland. It's just inefficient to get for more advanced use cases. Even if its internal implementation is totally different from PHP's, HHVM adds an id to every object, just for this requirement: matching what PHP exposes to userland. I'd more than happy to have an `spl_object_id()` function btw! Nicolas --- ## #87496 — Stanislav Malyshev — 2015-08-02T09:05:49+00:00 URL: https://externals.io/message/87437#87496 · In reply to: #87495 Hi! > This is already part of the public API: var_dump and debug_zval_dump do > expose object IDs to userland. It's just inefficient to get for more Debug functions are not the part of public API. If you rely on anything var_dump is producing for your code to work, you risk your code being broken any time, without warning or any BC consideration. > Even if its internal implementation is totally different from PHP's, > HHVM adds an id to every object, just for this requirement: matching > what PHP exposes to userland. That's HHVM's decision, however the fact that they need to fake IDs is not a good thing. Proper code should not depend on such details of engine implementation. > I'd more than happy to have an `spl_object_id()` function btw! I wouldn't be very happy, because it is wrong design to base userland PHP code on a detail of an implementation of the engine. That's why there was a hash and not direct ID - so that the code would not depend on implementation detail and anything that produces different hash for different objects would work. The alternative (stuffing the ID into spl_object_hash) is worse however. -- Stas Malyshev smalyshev@gmail.com --- ## #87521 — Derick Rethans — 2015-08-02T17:09:16+00:00 URL: https://externals.io/message/87437#87521 · In reply to: #87437 > Hi people. > > I've been pinged many times to add a new `spl_object_id()` function to PHP, > that would return the internal object handle of an object. > > Today, `spl_object_hash()` partially allows that, but adds many randomness to > the result, which is not very cool to use later (why does it even add > randomness ?). > > There has been topics about this subject. > For example, at http://marc.info/?l=php-internals&m=141814350920452&w=2 > > > Beeing able to get the object handle back in PHP userland would ease many > tools, mainly debug-oriented tools. > I know PHPUnit, Symfony and many big projects today make use of > `spl_object_hash()` to identify objects. > > I also know people that print_r($an_object) and parse the output just to > extract the object handle from there... Crazy isn't it ? > Why couldn't we help those people by simply adding a new function that does > the job ? You realize that these object handles aren't particularly stable? The same object ID can be reused: derick@whisky:/tmp $ cat objid.php Am 02.08.2015 um 08:52 schrieb Stanislav Malyshev : > > Hi! > > Some suspicious use of `spl_object_hash()` out there... > > > […] > > Not sure what this one does... but calculations with `spl_object_hash()` > look very suspicious. Actually, it's doing the right thing… calculating the value the object id is xor'ed with (as we know that consecutively defined objects have consecutive ids). It's relying on the implementation of `spl_object_hash()` and will even continue to work when we remove that part of randomness as that value it's xor'ed with is then nothing else than 0. That's why we should expose `spl_object_id()`… so that such hacks are unnecessary. > > […] > > This one _might_ be doing it right, but not sure as basing caching > instances on hash of a (mutable) object may produce weird results. It works and I'm using it that way too… that's currently what spl_object_hash is good for. The only issue is when it overflows after 2^32 objects. Though it isn't an issue on 64 bit systems… > > […] > > Oh wow, what's going on there? That's obviously not a proper use of > `spl_object_hash()`. Yup, that one is misuse as source of entropy… while it doesn't really provide much more entropy as it's internally anyway xor'ed with `mt_rand()`. > > […] > > I guess this one is wrong too, as it mentions storage, and storing > object ID is pointless. Maybe I am misunderstanding what "storage" means > there. The storage is only used in case `spl_object_hash()` does not exist for trying to get an unique identifier for that object (Wordpress has 5.2 compatibility, yeah…). The object id / hash is basically just an identifier for an object in PHP, just like we sometimes put pointers in e.g. a binary tree to easily map two pointers together in C. > -- > Stas Malyshev > smalyshev@gmail.com Bob --- ## #87523 — Bob Weinand — 2015-08-02T17:46:12+00:00 URL: https://externals.io/message/87437#87523 · In reply to: #87521 > Am 02.08.2015 um 19:09 schrieb Derick Rethans : > > > > > […] > > You realize that these object handles aren't particularly stable? The > same object ID can be reused: > > derick@whisky:/tmp $ cat objid.php > class Foo {} > $a = new Foo; > var_dump($a); > $a = new Foo; > var_dump($a); > $a = new Foo; > var_dump($a); > > derick@whisky:/tmp $ php objid.php > class Foo#1 (0) { > } > class Foo#2 (0) { > } > class Foo#1 (0) { > } > > You can't deterministically reference an object by it's class and > handle... so I also think this implementation detail should not be > shared through an API. > > cheers, > Derick Ehm, you realize that object id is only reset because the old object is freed? As long as the target object is referenced, nothing will have the same object id. Which is why this isn't an issue. Is there any point in having an unique id? It's only 32 bit on some systems. When it once overflows, it'd be reused anyway. What's important is having an exact 1:1 mapping of id and [existing] object. Which is all we need. Bob --- ## #87525 — Rowan Collins — 2015-08-02T18:03:53+00:00 URL: https://externals.io/message/87437#87525 · In reply to: #87522 > > […] > Actually, it's doing the right thing… calculating the value the object id is xor'ed with (as we know that consecutively defined objects have consecutive ids). > It's relying on the implementation of `spl_object_hash()` and will even continue to work when we remove that part of randomness as that value it's xor'ed with is then nothing else than 0. The right thing for what purpose? Why does it need that ID, rather than the value that `spl_object_hash()` gave in the first place? Just to be prettier to the user? Regards, -- Rowan Collins [IMSoP] --- ## #87557 — Nicolas Grekas — 2015-08-03T07:20:18+00:00 URL: https://externals.io/message/87437#87557 · In reply to: #87525 2015-08-02 20:03 GMT+02:00 Rowan Collins : > > > > […] > > The right thing for what purpose? Why does it need that ID, rather than > the value that `spl_object_hash()` gave in the first place? Just to be > prettier to the user? > For the purpose of providing an id that humans can read and compare, to easily spot if two objects are identical or not. Try comparing two spl_object_hashes and you'll understand the need... The id as displayed by e.g. var_dump is not an implementation detail that leaks through it. It's really an important feature of the output. I learned that while implementing VarDumper (see link above), by user requests. Regards, Nicolas --- ## #87566 — Rowan Collins — 2015-08-03T10:27:28+00:00 URL: https://externals.io/message/87437#87566 · In reply to: #87557 >2015-08-02 20:03 GMT+02:00 Rowan Collins : > > > […] >https://github.com/symfony/symfony/blob/master/src/Symfony/Component/VarDumper/Cloner/VarCloner.php > > […] >spl_object_hash() > > […] >object id > > […] >even > > […] >value it's > > […] >than > > […] > > >For the purpose of providing an id that humans can read and compare, So, yes, just to make it more human friendly. In which case, the actual value doesn't matter, and rather than reverse-engineering the hash, you could just make up your own ID: function my_object_id($obj) { static $hash_to_id=[]; $hash = spl_object_hash($obj); if ( ! array_key_exists($hash, $hash_to_id) ) { $hash_to_id[$hash] = count( $hash_to_id); } return $hash_to_id[$hash]; } (Untested, but pretty much all you need.) Regards, -- Rowan Collins [IMSoP] --- ## #87568 — Nicolas Grekas — 2015-08-03T10:52:06+00:00 URL: https://externals.io/message/87437#87568 · In reply to: #87566 > So, yes, just to make it more human friendly. In which case, the actual > value doesn't matter, and rather than reverse-engineering the hash, you > could just make up your own ID: > > function my_object_id($obj) { > static $hash_to_id=[]; > $hash = spl_object_hash($obj); > if ( ! array_key_exists($hash, $hash_to_id) ) { > $hash_to_id[$hash] = count( $hash_to_id); > } > return $hash_to_id[$hash]; > } > Yep, that would work, but with the following drawbacks: - it would create a memory leak/overhead - more importantly for users, these ids couldn't be compared to var_dumps'. --- ## #87569 — Rowan Collins — 2015-08-03T11:16:08+00:00 URL: https://externals.io/message/87437#87569 · In reply to: #87568 > > […] >actual > > […] >you > > […] > > >Yep, that would work, but with the following drawbacks: >- it would create a memory leak/overhead If you're dumping so many objects that the array would grow to noticeable size, you're likely to have bigger problems actually using the output. There's probably more data in an optimised autoloader classmap than most people will generate with a function like this. >- more importantly for users, these ids couldn't be compared to >var_dumps'. Why do you need to compare the output of two dump functions? Just add a note to the documentation that these are not the same IDs. Regards, -- Rowan Collins [IMSoP] --- Next page: https://externals.io/message/87437.md?page=2