PHP Object-Caching and Reference Counting

18 years ago by Lars Schultz — view source

unread

Hello there...

I am making my debut appearance here hoping that I am not giving offense
to anyones well honed php-instincts or programming practices with my
feature-request. This is what it comes down to, I guess. A Feature
Request for some meta-info on the instantiated PHP-Objects. Specifically
I want to know how many references are currently pointing to any one
object. Since PHP's Memory Management is based on Reference-Counting
this information should already be available "somewhere". Not knowing
any php-source-code internals, I can't even guess at the problems which
prohibit the implementation of such a feature. Nevertheless I'd like to
point out an advantage to the community if the reference-count were
available.

I have been working on a large project for 6 years now and I got to a
point where I'd like to cache certain objects singleton-like. To better
explain my problem I provide this small example which surely is
error-prone but should get the basic idea across.

<?
class MyObject {
private static $instances = array();

    protected $id;
   
    private function __constructor($id){
        $this->id = $id;
    }
   
    /**
     * Returns an instance of the class.
     * If no object was instantiated with the given $id before,
     * it will instantiate a new object which will be associated

with the given $id.
*
* It will also check the current memory-usage and if its used
up for more than
* 80% it will call the collect() method.
*
* @param int $id
* @return MyObject
*/
public static function getInstance($id){
if ( !isset(self::$instances[$id]) ) {
//check the used up memory wether to collect unused
objects or not
if ( memory_get_usage() > ini_get('memory limit')*0.8 )
self::collect();

            self::$instances[$id] = new MyObject($id);
        }
       
        return self::$instances[$id];
    }
   
    /**
     * Removes any object from the cache-list which is not

referenced anymore.
* (safe for the list and the local-reference to it)
*/
public static function collect(){
foreach( self::$instances as $id => $instance ) {
if ( reference_count($instance) < 3 )
unset(self::$instances[$id]);
}
}
}
?>

As you can see I boldly used the function reference_count(Object) even
though it doesn't exist. This is exactly what I'd need to allow my
concept to work. I keep thinking of any other solutions, but up until
now, I didn't find any better solution for this problem. Of course I can
always ignore the memory usage and hope that it wont hit the limit...but
hoping is a rather ethereal and mystic idea which doesn't go well with
my preferred down-to-earth 1 plus 1 equals 2 programming.

I know that a lot of you may say: Why do need to do that?...and I'd have
to think for 3 days and then write for 4 days and would give up. It
seems like a nice solution to me and if any of you have an idea as to
how I could implement this differently, please feel free to point it out
to me.

Thanks for reading this far.
Lars

18 years ago by Andreas Korthaus — view source

unread

Hi!

Lars Schultz wrote:

As you can see I boldly used the function reference_count(Object) even
though it doesn't exist. This is exactly what I'd need to allow my
concept to work.

Some time ago I created a patch to implement a ref_count() function
which works the way you suggested. But I'm not sure if it's
OK/complete... and I'm not sure if something like that is really needed
in the core (and I'm definetly not the person to decide on that ;-)).

Anyway, I attatched a simple patch against current 5_2 branch to this
mail, perhaps it helps.

Best regards
Andreas

18 years ago by Stanislav Malyshev — view source

unread

Some time ago I created a patch to implement a ref_count() function
which works the way you suggested. But I'm not sure if it's
OK/complete... and I'm not sure if something like that is really needed
in the core (and I'm definetly not the person to decide on that ;-)).

If you talk about objects, your patch exposes wrong refcount. Your patch
exposes zval refcount, however there can be a number of zvals referring
to the same object, and the actual object storage keeps its own
refcount, see zend_objects_API.c (provided your object is a standard
Zend object).
Actually, since each of these zvals can hold own refcount and we can not
know what variables refer to the given object, I do not see how one
could really calculate the number of different places that the object is
referred to, unless you know your application really well so you could
know how many places refer to the object and what exactly they are.

--
Stanislav Malyshev, Zend Products Engineer
stas@zend.com http://www.zend.com/

18 years ago by Lars Schultz — view source

unread

Andreas Korthaus wrote:

Some time ago I created a patch to implement a ref_count() function
which works the way you suggested. But I'm not sure if it's
OK/complete... and I'm not sure if something like that is really
needed in the core (and I'm definetly not the person to decide on that
;-)).

Wow! This is great news...though I have no idea what to do with the
patch and how to apply it to the php-source-code. The trouble is that I
have to use the default php-engine because it should be runnable on any
host...with some extensions. I don't guess it'd be possible to pack the
functionality into en extension which could then be loaded more easily
into the engine?

Anyway, I attatched a simple patch against current 5_2 branch to this
mail, perhaps it helps.

Thanks anyway! It was an interesting read though my c and c++ knowledge
is far from complete so i don't understand the workings, really...Maybe,
if I feel restless, I'll have a look at the sources, though I wouldn't
know where to start.

18 years ago by Stanislav Malyshev — view source

unread

As you can see I boldly used the function reference_count(Object) even
though it doesn't exist. This is exactly what I'd need to allow my
concept to work. I keep thinking of any other solutions, but up until
now, I didn't find any better solution for this problem. Of course I can

Note that the engine does not guarantee you any particular value of the
refcount. Passing parameters around, accessing variables, etc. may
change the refcount. One thing the engine guarantees you is that once no
links to the variable exist it would be destroyed (which may not happen
immediately after you do unset, for example, but probably would happen
"soon"). So actually figuring out if the object is referenced by any
other place could prove non-trivial, depending on the application
structure.

--
Stanislav Malyshev, Zend Products Engineer
stas@zend.com http://www.zend.com/

18 years ago by pstradomski@gmail.com — view source

unread

Lars Schultz wrote:

I know that a lot of you may say: Why do need to do that?...and I'd have
to think for 3 days and then write for 4 days and would give up. It
seems like a nice solution to me and if any of you have an idea as to
how I could implement this differently, please feel free to point it out
to me.

Perhaps you could use your own reference counting?
Keep your "real" objects in private array and instead of returning them return
a simple object which would forward all methods and properties by
__get/__set/__call. The constructor would increase your internal refcount by
one, destructor would decrease it.

--
Paweł Stradomski

18 years ago by Lars Schultz — view source

unread

Perhaps you could use your own reference counting?
Keep your "real" objects in private array and instead of returning them return
a simple object which would forward all methods and properties by
__get/__set/__call. The constructor would increase your internal refcount by
one, destructor would decrease it.

I use those objects quite often as they are used to access the database
to manipulate or search for related data. It wouldn't perform well and
is actually not at all practicable because there are so many classes for
which I'd would have to copy the interface.

18 years ago by pstradomski@gmail.com — view source

unread

On 21 jan 2007 Lars Schultz wrote:

Perhaps you could use your own reference counting?
Keep your "real" objects in private array and instead of returning them
return a simple object which would forward all methods and properties by
__get/__set/__call. The constructor would increase your internal refcount
by one, destructor would decrease it.

I use those objects quite often as they are used to access the database
to manipulate or search for related data. It wouldn't perform well and
is actually not at all practicable because there are so many classes for
which I'd would have to copy the interface.

I don't see any place where you'd have to change the interface - except for
the class which contains that private array; you can use the name of your
current class (MyObject)as the name of this new "forwarding" class and make a
new name for the class which would be a copy of your current class (eg.
MyObject_private). Therefore in all function definitions required class
specification doesn't need to change.

Well, perhaps you would have to change some "factory" class if you use one,
but that should be a one place - if you kept your code clean.

I think this solution is better, as you don't have to rely on internal
refcounts, which can be implemented in another way (or dropped altogether) in
future releases.

--
Paweł Stradomski

18 years ago by Lars Schultz — view source

unread

That's correct. However, it is not guaranteed that your unset would
delete last reference - there can be references on stack, in temp
variables in current expressions being calculated (function call could
happen in mid-expression), etc. Of course, it might be that your code
is very controlled so you know it never happens, but as general case
you can't rely on it.

Ok. I got your point. You are right that it's hard to predict where
references could be found.

Also, if you check only zval refcount, you might unset objects that
actually are referenced from other places but by different zvals,
since one object could be referenced by a number of zvals.

I am not familiar with the term: "zval"...My understanding of php comes
from extensive usage...Can you give me a specific example?

$foo = new MyObject(); //ref count to Object #1 is 1
$bar = $foo; //ref count to Object #1 is 2
if ( ($zapp = $bar)->doIt() ) {...} //ref count is 3 or 4? but it
doesn't matter really...

I also believe that my reasons for this feature only require that there
are not any less references than are returned by ref_count()...I am
aware that the usage is for quite an advanced concept...but why
shouldn't PHP also be used for this.

The one argument against this feature that carries the most weight, for
me, is that PHP might change the behaviour of its memory management and
then keeping count of references would be an unecessary overhead, which
might lead to my code not working anymore if the feature were to be
dropped. Are there likely to be any changes in the future?...and is
there an other solution to this concept? I find it rather neat...;)
Especially if you couple it with the magic function __wake() where the
objects re-register themselves in the cache-list...

<?
class MyObject {
...
public function __wake(){
self::$instances[$this->id] = $this;
}
...
}
?>

In this case all objects not referenced by any variable in the $_SESSION
array (or it's descendants), would be destructed at the end of the
script. This way the objects remain unique even across sessions and some
kind of cleaning up is done by the serialization process. The trouble is
that this only works at the end of the script where of course even the
current stack is not used anymore. So I can't use this principle in a
function-call. Or does anyone of you have a different solution?;)

18 years ago by Arnold Daniels — view source

unread

Hi,

To get an understanding of zval's and php internal basics, I recommend
you read the following tutorials:

http://devzone.zend.com/node/view/id/1021 - Part I: Introduction to PHP
and Zend
http://devzone.zend.com/node/view/id/1022 - Part II: Parameters, Arrays,
and ZVALs
http://devzone.zend.com/node/view/id/1023 - Part II: Parameters, Arrays,
and ZVALs (continued)
http://devzone.zend.com/node/view/id/1024 - Part III: Resources

(Hmm this is the 3rd time I advertise these tutorials, I should get paid
for it :p )

Best regards,
Arnold

Lars Schultz schreef:

That's correct. However, it is not guaranteed that your unset would
delete last reference - there can be references on stack, in temp
variables in current expressions being calculated (function call
could happen in mid-expression), etc. Of course, it might be that
your code is very controlled so you know it never happens, but as
general case you can't rely on it.

Ok. I got your point. You are right that it's hard to predict where
references could be found.

Also, if you check only zval refcount, you might unset objects that
actually are referenced from other places but by different zvals,
since one object could be referenced by a number of zvals.

I am not familiar with the term: "zval"...My understanding of php
comes from extensive usage...Can you give me a specific example?

$foo = new MyObject(); //ref count to Object #1 is 1
$bar = $foo; //ref count to Object #1 is 2
if ( ($zapp = $bar)->doIt() ) {...} //ref count is 3 or 4? but it
doesn't matter really...

I also believe that my reasons for this feature only require that
there are not any less references than are returned by ref_count()...I
am aware that the usage is for quite an advanced concept...but why
shouldn't PHP also be used for this.

The one argument against this feature that carries the most weight,
for me, is that PHP might change the behaviour of its memory
management and then keeping count of references would be an unecessary
overhead, which might lead to my code not working anymore if the
feature were to be dropped. Are there likely to be any changes in the
future?...and is there an other solution to this concept? I find it
rather neat...;) Especially if you couple it with the magic function
__wake() where the objects re-register themselves in the cache-list...

<?
class MyObject {
...
public function __wake(){
self::$instances[$this->id] = $this;
}
...
}
?>

In this case all objects not referenced by any variable in the
$_SESSION array (or it's descendants), would be destructed at the end
of the script. This way the objects remain unique even across sessions
and some kind of cleaning up is done by the serialization process. The
trouble is that this only works at the end of the script where of
course even the current stack is not used anymore. So I can't use this
principle in a function-call. Or does anyone of you have a different
solution?;)

18 years ago by Stanislav Malyshev — view source

unread

I am not familiar with the term: "zval"...My understanding of php comes
from extensive usage...Can you give me a specific example?

OK, sorry, got a bit into engine-speak, basically zval is the internal
representation of a variable, more or less, though $foo and $bar can be
linked to the same zval, and may be linked to different ones, depending
on the context. The objects have its won refcounts internally, which
means two unrelated variables can point to the same object. For these
variables, both refcounts would be 1, but if you destroy any variable,
object won't be destroyed since there's another variable pointing to it
and the object knows that. So checking just refcount of the variable is
not enough, since low refcount doesn't mean there's no variables
referring to the object, only there's no "related" variables. If you
want to dive into this in detail, I think the links recommended by
Arnold Daniels would be a good start, especially in part talking about
references and copies.

I also believe that my reasons for this feature only require that there
are not any less references than are returned by ref_count()...I am

ref_count() can return small number and there can still be references to
this object - just from another variable. So to be sure you would have
basically to inspect both refcounts - variable's and object's.

Stanislav Malyshev, Zend Products Engineer
stas@zend.com http://www.zend.com/

PHP Object-Caching and Reference Counting

ref_count() can return small number and there can still be references to this object - just from another variable. So to be sure you would have basically to inspect both refcounts - variable's and object's.

ref_count() can return small number and there can still be references to
this object - just from another variable. So to be sure you would have
basically to inspect both refcounts - variable's and object's.