To spare the "5.4 features vote" thread from the off-topic discussion,
we can continue here.
Hannes Landeholm brought this idea up, see
http://news.php.net/php.internals/53956 and
http://news.php.net/php.internals/53959
I would suggest that Hannes create an rfc, and add some examples and
introduce the weak references in general and how this can be useful,
what problems do they solve (it was discussed in the bugreport, but it
wasn't clear as far as I can tell from the comments).
if I understand this right, Hannes proposed adding a new Spl
class(interface?), which can store and retrive a collection of
variables without incrementing the refcounts, so they can be freed if
they aren't used other than in this collection.
Hannes please correct me if I'm wrong somewhere.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
Hi,
To spare the "5.4 features vote" thread from the off-topic discussion,
we can continue here.
Hannes Landeholm brought this idea up, see
http://news.php.net/php.internals/53956 and
http://news.php.net/php.internals/53959
I would suggest that Hannes create an rfc, and add some examples and
introduce the weak references in general and how this can be useful,
what problems do they solve (it was discussed in the bugreport, but it
wasn't clear as far as I can tell from the comments).
if I understand this right, Hannes proposed adding a new Spl
class(interface?), which can store and retrive a collection of
variables without incrementing the refcounts, so they can be freed if
they aren't used other than in this collection.Hannes please correct me if I'm wrong somewhere.
Some draft here: https://wiki.php.net/rfc/weakreferences
With a patch implementing some magic for a POC here
http://patches.colder.ch/php-src/weakref-trunk.patch?markup. It still
requires some work though.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu--
--
Etienne Kneuss
http://www.colder.ch
Hi,
Ferenc: Good idea to move the discussion to a separate thread.
Etienne: Awesome. I wish I had time to write the patch myself. I could
probably put some hours into adding more details to the RFC though.
The only problem as I see it by making it a weak reference class instead of
a weak reference collection is that you would end up with a bunch of
unnecessary indexes in the array that points to invalid references. The
simplest solution to this problem if you still want to implement it as you
have begun to is to allow catching the cleanup event in a callback so you
can manually unset the index. Should I add these concerns to the RFC?
Thanks a bunch,
Hannes
Hello,
I just realized you could actually utilize the __destruct method to catch
the event I described in my last post. The SplWeakRef class as currently
defined in the RFC would therefore suffice to build a userland class that
automatically garbage collects indexes.
~Hannes
I was going to add some content to
https://wiki.php.net/rfc/weakreferencesnow but the wiki tells me that
the page is read only for me. I couldn't find
any details on who to contact to change this.
~Hannes
I was going to add some content to
https://wiki.php.net/rfc/weakreferencesnow but the wiki tells me that
the page is read only for me. I couldn't find
any details on who to contact to change this.~Hannes
I already asked bjori (Hannes Magnusson) to give you karma to the rfc
namespace in the wiki.
Ferenc Kovács
@Tyr43l - http://tyrael.hu
I was going to add some content to
https://wiki.php.net/rfc/weakreferencesnow but the wiki tells me that
the page is read only for me. I couldn't find
any details on who to contact to change this.~Hannes
Hannes, now you should have karma for the rfc namespace, so you can
now extend the article with your suggestions.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
To spare the "5.4 features vote" thread from the off-topic discussion,
we can continue here.
Hannes Landeholm brought this idea up, see
http://news.php.net/php.internals/53956 and
http://news.php.net/php.internals/53959
I would suggest that Hannes create an rfc, and add some examples and
introduce the weak references in general and how this can be useful,
what problems do they solve (it was discussed in the bugreport, but it
wasn't clear as far as I can tell from the comments).
Not being an expert here, so the questions may have obvious answers.
How often does the gc cycle run during a script?
Can preemptive cleanup actually cause more of a headache?
In reading about weak references on
http://en.wikipedia.org/wiki/Weak_reference, weak references look like
a perfect use case for memory limited caches, but I wonder if there
would be a way of synching a TTL to the weak referenced objects, so
that they essentially automatically expire (or can be expired and
expunged). A system controlled TTL for weak referenced objects.
So,
void SplWeakRef::__construct(object $ref, [int $ttl = -1])
$ttl = -1 would allow the object to be gc'd in the normal way, but any
other value would be a hint to how long the object would be expected
to last for. If gc happens frequently, then $ttl acts as a limiter and
I think this could be very useful.
Richard Quadling
Twitter : EE : Zend : PHPDoc
@RQuadling : e-e.com/M_248814.html : bit.ly/9O8vFY : bit.ly/lFnVea
To spare the "5.4 features vote" thread from the off-topic discussion,
we can continue here.
Hannes Landeholm brought this idea up, see
http://news.php.net/php.internals/53956 and
http://news.php.net/php.internals/53959
I would suggest that Hannes create an rfc, and add some examples and
introduce the weak references in general and how this can be useful,
what problems do they solve (it was discussed in the bugreport, but it
wasn't clear as far as I can tell from the comments).Not being an expert here, so the questions may have obvious answers.
How often does the gc cycle run during a script?
"First of all, the whole reason for implementing the garbage
collection mechanism is to reduce memory usage by cleaning up
circular-referenced variables as soon as the prerequisites are
fulfilled. In PHP's implementation, this happens as soon as the
root-buffer is full, or when the function gc_collect_cycles() is
called. In the graph below, we display the memory usage of the script
below, in both PHP 5.2 and PHP 5.3, excluding the base memory that PHP
itself uses when starting up."
http://www.php.net/manual/en/features.gc.performance-considerations.php#features.gc.performance-considerations.reduced-mem
Can preemptive cleanup actually cause more of a headache?
In reading about weak references on
http://en.wikipedia.org/wiki/Weak_reference, weak references look like
a perfect use case for memory limited caches, but I wonder if there
would be a way of synching a TTL to the weak referenced objects, so
that they essentially automatically expire (or can be expired and
expunged). A system controlled TTL for weak referenced objects.So,
void SplWeakRef::__construct(object $ref, [int $ttl = -1])
$ttl = -1 would allow the object to be gc'd in the normal way, but any
other value would be a hint to how long the object would be expected
to last for. If gc happens frequently, then $ttl acts as a limiter and
I think this could be very useful.
the current implementation doesn't change the way that gc works.
it only stores the reference in such a way, than the gc can free the
object, if nothing else references it.
of course we could improve the gc mechanism, as you suggested, but I
think that would be a more complex and risky, so maybe that would be a
feature-creep for now.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
Since Weak references is already an established concept in many OOP
languages a lot can be learned from their implementation. A lot of the
potential related problems and solutions has already been studied.
Anyone who wish to contribute to this feature should start by reading
this excellent article by Ethan Nicholas:
http://weblogs.java.net/blog/2006/05/04/understanding-weak-references
For example I previously stated that I wanted a callback to clean up
the weak reference object and indexes in the collection - then came to
the conclusion that __destruct method would suffice for this. Then I
read this part:
The ReferenceQueue class makes it easy to keep track of dead
references. If you pass a ReferenceQueue into a weak reference's
constructor, the reference object will be automatically inserted into
the reference queue when the object to which it pointed becomes
garbage. You can then, at some regular interval, process the
ReferenceQueue and perform whatever cleanup is needed for dead
references.
</quote>
Java utilizes a "ReferenceQueue" that tracks dead references so you
can process and clean up dead references which is an excellent design
solution. The Queue part is not that important, the references might
as well be organized in a LIFO than a FIFO but a FIFO conceptually
more fitting. Imaging dead references standing in a queue, awaiting
cleanup.
The solution is also superior to the __destruct solution since you
would have to extend or modify the class and override the __destruct
method. It makes it impossible to separate the "how the instances is
externally stored"-design from the actual class itself, ergo: you
can't cache classes in external libraries without hacking the
libraries.
I therefore propose changing the constructor (and implementation) to:
void SplWeakRef::__construct(object ref, SplQueue ref_queue = null)
If the SplWeakRef is given a ref_queue, it will call
ref_queue->push($this) whenever it becomes invalid. If additional
cleanup is required (if you want to catch the event and do cleanup
immediately) one could simply extend SplQueue and overload
SplQueue::push().
Note that resurrection must be avoided so the SplWeakRef does not flip
between valid -> invalid and then turns valid again (this would be a
confusing behavior which can lead to unintentional behavior = bugs).
The implementation should therefore make sure the SplWeakRef does not
turn invalid before __destruct has been run for the object since
__destruct can resurrect an object.
~Hannes
Since Weak references is already an established concept in many OOP
languages a lot can be learned from their implementation. A lot of the
potential related problems and solutions has already been studied.
Anyone who wish to contribute to this feature should start by reading
this excellent article by Ethan Nicholas:http://weblogs.java.net/blog/2006/05/04/understanding-weak-references
For example I previously stated that I wanted a callback to clean up
<quote> Reference queue Once a WeakReference starts returning null, the object it pointed to has become garbage and the WeakReference object is pretty much useless. This generally means that some sort of cleanup is required; WeakHashMap, for example, has to remove such defunct entries to avoid holding onto an ever-increasing number of dead WeakReferences.
the weak reference object and indexes in the collection - then came to
the conclusion that __destruct method would suffice for this. Then I
read this part:The ReferenceQueue class makes it easy to keep track of dead
references. If you pass a ReferenceQueue into a weak reference's
constructor, the reference object will be automatically inserted into
the reference queue when the object to which it pointed becomes
garbage. You can then, at some regular interval, process the
ReferenceQueue and perform whatever cleanup is needed for dead
references.
</quote>Java utilizes a "ReferenceQueue" that tracks dead references so you
can process and clean up dead references which is an excellent design
solution. The Queue part is not that important, the references might
as well be organized in a LIFO than a FIFO but a FIFO conceptually
more fitting. Imaging dead references standing in a queue, awaiting
cleanup.The solution is also superior to the __destruct solution since you
would have to extend or modify the class and override the __destruct
method. It makes it impossible to separate the "how the instances is
externally stored"-design from the actual class itself, ergo: you
can't cache classes in external libraries without hacking the
libraries.I therefore propose changing the constructor (and implementation) to:
void SplWeakRef::__construct(object ref, SplQueue ref_queue = null)
If the SplWeakRef is given a ref_queue, it will call
ref_queue->push($this) whenever it becomes invalid. If additional
cleanup is required (if you want to catch the event and do cleanup
immediately) one could simply extend SplQueue and overload
SplQueue::push().Note that resurrection must be avoided so the SplWeakRef does not flip
between valid -> invalid and then turns valid again (this would be a
confusing behavior which can lead to unintentional behavior = bugs).
The implementation should therefore make sure the SplWeakRef does not
turn invalid before __destruct has been run for the object since
__destruct can resurrect an object.
Etienne Kneuss has more knowledge about the implementation, but AFAIK
as we didn't modified the garbage collector we didn't get notified if
a weakreferenced object gets gced, so we can't notify the SplQueue
either.
please be aware that the bigger the change that we introduce it is
less likely that we can make this happen in 5.4, or at all.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
I'm not familiar with the internal implementation of GC in PHP but if
the current garbage collection implementation makes catching the gc
event non-trivial I'd agree and vote for postponing my "SplQueue
ref_queue" suggestion until the future. It could always be added later
and there is the temporary __destruct workaround for the meantime.
~Hannes
Etienne Kneuss has more knowledge about the implementation, but AFAIK
as we didn't modified the garbage collector we didn't get notified if
a weakreferenced object gets gced, so we can't notify the SplQueue
either.
please be aware that the bigger the change that we introduce it is
less likely that we can make this happen in 5.4, or at all.--
Ferenc Kovács
@Tyr43l - http://tyrael.hu
Am 16.07.2011 13:23, schrieb Ferenc Kovacs:
we have debug_zval_dump but it is hard to use correctly
Well, it's not a simple problem then is it? It wouldn't be too hard if
you understood the source of the problem...
with weak references, you wouldn't need to worry about keeping a count
of the refcount only check the valid() method.
yeah...a magic bullet is cool, but you could, to quote a contributor to
this list, shoot yourself in the foot;)
for your last paragraph: that's totally unrelated to weak references,
but you could achive what you want throughmemory_get_usage()+
You're right. It's related to one of the usecases of weak-references
though...and your proposition wouldn't works ince I'd have to check my
memory-usage all the time...that's what events are for.
btw. if I remember correctly, somebody requested a non-fatal callback
when memory limit is reached callback functionality, I have to look
that up.
By then it's probably too late since I don't have any memory left to run
my cleanup code.
I too would welcome a solution to this problem, I've run into it several
times already and always had to use a semi-satisfactory solution. I
hadn't heard about weak-references before, and I generally like the
concept. What I am not so sure is wether this makes everything alot more
complicated for users who do not know/care about the problem or the
solution. Of course the impact on those users depends on the actual
solution.
I came at the problem from a different angle and came to a different
solution: If I could get the refcount on a certain object, and kept the
object stored centrally in one list, I'd know that if the refcount is
down to 1 (or some other constant) that the object ist not in use
anymore. I believe that this would be quite a simple PHP addition...a
simple function called: get_ref_count() or something and as I remember,
that value is always stored in the zval...
The only other thing that would help would be a callback (or callable)
which is triggered by PHP reaching a certain memory-limit, absolute or
relative. Then I could clean up memory according to my own business-logic...
This way, PHP would not feature something totally new, but one could
still solve the problem with some work.
Do I make any sense at all? Am I missing the point? Anyway, this is an
interesting problem.
Cheers.
Lars
I too would welcome a solution to this problem, I've run into it several
times already and always had to use a semi-satisfactory solution. I hadn't
heard about weak-references before, and I generally like the concept. What I
am not so sure is wether this makes everything alot more complicated for
users who do not know/care about the problem or the solution. Of course the
impact on those users depends on the actual solution.I came at the problem from a different angle and came to a different
solution: If I could get the refcount on a certain object, and kept the
object stored centrally in one list, I'd know that if the refcount is down
to 1 (or some other constant) that the object ist not in use anymore. I
believe that this would be quite a simple PHP addition...a simple function
called: get_ref_count() or something and as I remember, that value is always
stored in the zval...The only other thing that would help would be a callback (or callable) which
is triggered by PHP reaching a certain memory-limit, absolute or relative.
Then I could clean up memory according to my own business-logic...This way, PHP would not feature something totally new, but one could still
solve the problem with some work.Do I make any sense at all? Am I missing the point? Anyway, this is an
interesting problem.Cheers.
Lars--
we have debug_zval_dump but it is hard to use correctly (see the
notes) and xdebug has xdebug_debug_zval which is slightly better, but
it is not part of the language.
with weak references, you wouldn't need to worry about keeping a count
of the refcount only check the valid() method.
for your last paragraph: that's totally unrelated to weak references,
but you could achive what you want through memory_get_usage() +
unsetting your unnedid variables and maybe calling
gc_collect_cycles().
btw. if I remember correctly, somebody requested a non-fatal callback
when memory limit is reached callback functionality, I have to look
that up.
--
Ferenc Kovács
@Tyr43l - http://tyrael.hu