Hi,
This is another shot at a patch to manage refcount and is_ref of zvals
with macros. The macros are implemented with inline functions that are
forcibly inlined (so they will behave the same as macros). This will
make it possible to put multiple statements in each macro later on for
garbage collection purposes.
One thing to note is that I removed the existing ZVAL_ADDREF and
ZVAL_DELREF in favor of Z_ADDREF_P and Z_DELREF_P that do the same
thing. The original duo acted on zval pointers contrary to standard
form for macro names.
The patch is probably too large to fit as an attachment, so here's a URL:
http://zoo.cs.yale.edu/~yw226/macros.diff.txt
Comments?
W liście David Wang z dnia wtorek, 4 września 2007 16:24:
http://zoo.cs.yale.edu/~yw226/macros.diff.txt
Comments?
+#define always_inline inlinue
Shouldn't that be
+#define always_inline inline
?
(in zend.h)
Paweł Stradomski
Shouldn't that be
+#define always_inline inline
?
Yes, sorry, I just fixed that a moment ago (and the file the link is
pointing to should be correct).
I think this is a pretty good idea. It can also have some use if we
decide to investigate the usage of an off-the-shelf (conservative)
garbage collector such as Boehm's (maybe in next year's SoC).
About the patch, the zval_*_p() functions don't really need to exist..
they can be macros as well. (and the online keyword is also handled by
the Zend engine)
Nuno
Hi,
This is another shot at a patch to manage refcount and is_ref of zvals
with macros. The macros are implemented with inline functions that are
forcibly inlined (so they will behave the same as macros). This will
make it possible to put multiple statements in each macro later on for
garbage collection purposes.One thing to note is that I removed the existing ZVAL_ADDREF and
ZVAL_DELREF in favor of Z_ADDREF_P and Z_DELREF_P that do the same
thing. The original duo acted on zval pointers contrary to standard
form for macro names.The patch is probably too large to fit as an attachment, so here's a URL:
http://zoo.cs.yale.edu/~yw226/macros.diff.txt
Comments?
About the patch, the zval_*_p() functions don't really need to exist..
they can be macros as well.
As I said, if those were macros, it would be very difficult to squeeze
more than one statement into them.
Consider the usage in Zend/zend_execute.c:
if (!Z_DELREF_P(z)) {
Z_SET_REFCOUNT_P(z, 1);
Suppose I wanted to do the following in Z_DELREF_P(z):
"some_function(z); z->refcount--;" Defining the macro as that would
break that section of the code. Also, suppose I have code like this:
if(some_condition) {
Z_DELREF_P(z);
Having multiple statements in Z_DELREF_P will also break that code. It
seems to me inline functions are the best solution (the second best
solution would be to put braces around the statements in the define,
but then no return value would be possible).
(and the online keyword is also handled by
the Zend engine)
Do you mean ZEND_VM_ALWAYS_INLINE? There is a ZEND_VM_ALWAYS_INLINE
#define that does something similar to my "always_inline". However, it
doesn't work for Microsoft compilers, and since it doesn't include the
"inline" keyword within its definition, I'd have to change the meaning
of it in order to fix that. As far as I can tell, it's not referenced
by any code at all, but that still might be a bad idea.
About the patch, the zval_*_p() functions don't really need to exist..
they can be macros as well.As I said, if those were macros, it would be very difficult to squeeze
more than one statement into them.Having multiple statements in Z_DELREF_P will also break that code. It
seems to me inline functions are the best solution (the second best
solution would be to put braces around the statements in the define,
but then no return value would be possible).
currently you only have one statement in each of those functions..
anyway it's not critical to have static inline functions.
(and the online keyword is also handled by
the Zend engine)Do you mean ZEND_VM_ALWAYS_INLINE? There is a ZEND_VM_ALWAYS_INLINE
#define that does something similar to my "always_inline". However, it
doesn't work for Microsoft compilers, and since it doesn't include the
"inline" keyword within its definition, I'd have to change the meaning
of it in order to fix that. As far as I can tell, it's not referenced
by any code at all, but that still might be a bad idea.
nops. The Zend engine redefines the inline keyword itself somewhere.
Nuno
currently you only have one statement in each of those functions..
anyway it's not critical to have static inline functions.
I will need to have them (multiple statements) for the cycle
collection code, and I don't think there are many situations involving
changing the way reference counting is done where only one statement
is sufficient. In any case, it's good future-proofing.
nops. The Zend engine redefines the inline keyword itself somewhere.
Hmm. I did not know that. Where? A quick grep doesn't seem to show
anything significant. And I do know (experimentally) that a vast
number of supposed "inline" functions aren't actually inlined
currently because the compiler decides it doesn't want to do it. It
gave me a lot of problems when I was trying to optimize my gc code.
It can also have some use if we
decide to investigate the usage of an off-the-shelf (conservative)
garbage collector such as Boehm's (maybe in next year's SoC).
As an aside, I was also thinking about this throughout the course of
the project. It's said that reference counting is the slowest form of
garbage collection since the reference counts must constantly be
maintained. Changing to a tracing garbage collector won't require
these macros, because reference counts would be eliminated altogether.
However, it would be sort of a big pain to implement
Off the shelf garbage collectors such as BDW would be inappropriate
because we use some weird kinds of "pointers" (such as object handles)
stored in weird kinds of ways (such as a zend_hash object). I think it
would be pretty inefficient, since those implementations just scan the
stack, registers and heap and we're trying to do GC not for the PHP
interpreter, but for the code the PHP interpreter is running.
I have a certain suspicion that a traditional mark-and-sweep collector
might be faster if just on the virtue of eliminating the refcount
field and getting rid of tons of cache misses that way. For just
displaying page, there wouldn't be much memory used and that's all
freed at the end of a request anyway: all of that reference counting
overhead would just disappear. For larger scripts that use a lot of
memory, the only problem would be pause times but in most real life
cases, it seems the total time would be shorter than reference
counting. However, I'm not sure if that would be the case in PHP:
rummaging through objects scattered all over memory would result in a
lot of cache misses. The question is whether that is greater than all
the misses we're currently having just managing the refcount.
However, answering that question would require implementing the thing,
and that honestly seems like it would be a bit of a nightmare. Roots
would include zvals linked to PHP variables, the stack of the running
PHP code, and the stack and heap of the PHP interpreter itself. It
would've been far easier if PHP had been designed from the ground up
to use some sane way of managing memory, but with the current
situation, with extensions depending on reference counting, it's
pretty difficult.
If ever a version of this patch is committed, you'll be able to see
that the cycle collector touches the whole reference counting mess
extremely minimally, which is why it was relatively safe to implement.
Off the shelf garbage collectors such as BDW would be inappropriate
because we use some weird kinds of "pointers" (such as object handles)
stored in weird kinds of ways (such as a zend_hash object). I think it
would be pretty inefficient, since those implementations just scan the
stack, registers and heap and we're trying to do GC not for the PHP
interpreter, but for the code the PHP interpreter is running.
surely it wouldn't be the top performance GC, but I think it worths a
try. And it doesn't seem difficult to use that GC (after your patch
specially).
For just
displaying page, there wouldn't be much memory used and that's all
freed at the end of a request anyway: all of that reference counting
overhead would just disappear. For larger scripts that use a lot of
memory, the only problem would be pause times but in most real life
cases, it seems the total time would be shorter than reference
counting. However, I'm not sure if that would be the case in PHP:
rummaging through objects scattered all over memory would result in a
lot of cache misses. The question is whether that is greater than all
the misses we're currently having just managing the refcount.
exactly. for most PHP requests the GC wouldn't even run. The garbage
would be collected after the request, thus reducing the latency of the
request (for Gopal pleasure :)
However, answering that question would require implementing the thing,
and that honestly seems like it would be a bit of a nightmare. Roots
would include zvals linked to PHP variables, the stack of the running
PHP code, and the stack and heap of the PHP interpreter itself. It
would've been far easier if PHP had been designed from the ground up
to use some sane way of managing memory, but with the current
situation, with extensions depending on reference counting, it's
pretty difficult.
implementing a GC from scratch is a difficult job, yes and hence my
idea to try an existenting GC. But if we look to the GCs used by e.g.
the Common Lisp implementations, we see that they have highly-tuned
and highly-performant GC implementations that take advantage of how
the internal structures are implemented. And I suspect Java does that,
too.
Well, maybe we can find some crazy student next year to do it hint
:). Or maybe I get crazy too one of these days ;) (well I still have
to pick something to do for the master thesis..)
Nuno
See below:
-----Original Message-----
From: Nuno Lopes [mailto:nlopess@php.net]
Sent: Tuesday, September 04, 2007 11:35 AM
To: David Wang
Cc: internals@lists.php.net; andi@php.net; dmitry@php.net
Subject: Re: Re: [PHP-DEV] Patch for macros for tracking refcount
andis_refOff the shelf garbage collectors such as BDW would be inappropriate
because we use some weird kinds of "pointers" (such as object
handles)
stored in weird kinds of ways (such as a zend_hash object). I think
it
would be pretty inefficient, since those implementations just scan
the
stack, registers and heap and we're trying to do GC not for the PHP
interpreter, but for the code the PHP interpreter is running.surely it wouldn't be the top performance GC, but I think it worths a
try. And it doesn't seem difficult to use that GC (after your patch
specially).
Figuring out the roots is very hard with PHP because of all the
extensions managing their own zvals. I don't think it's feasible nor
will it be very beneficial.
For just
displaying page, there wouldn't be much memory used and that's all
freed at the end of a request anyway: all of that reference counting
overhead would just disappear. For larger scripts that use a lot of
memory, the only problem would be pause times but in most real life
cases, it seems the total time would be shorter than reference
counting. However, I'm not sure if that would be the case in PHP:
rummaging through objects scattered all over memory would result in
a
lot of cache misses. The question is whether that is greater than
all
the misses we're currently having just managing the refcount.exactly. for most PHP requests the GC wouldn't even run. The garbage
would be collected after the request, thus reducing the latency of the
request (for Gopal pleasure :)
Probably not. Executing PHP scripts is very heap intensive and if you
didn't run the GC during execution you would be eating up lots of
memory even for relatively short requests. This would have a significant
impact on the # of Apache processes you can run on a given box. So I
think it actually wouldn't work well for us.
However, answering that question would require implementing the
thing,
and that honestly seems like it would be a bit of a nightmare. Roots
would include zvals linked to PHP variables, the stack of the
running
PHP code, and the stack and heap of the PHP interpreter itself. It
would've been far easier if PHP had been designed from the ground up
to use some sane way of managing memory, but with the current
situation, with extensions depending on reference counting, it's
pretty difficult.implementing a GC from scratch is a difficult job, yes and hence my
idea to try an existenting GC. But if we look to the GCs used by e.g.
the Common Lisp implementations, we see that they have highly-tuned
and highly-performant GC implementations that take advantage of how
the internal structures are implemented. And I suspect Java does that,
too.Well, maybe we can find some crazy student next year to do it hint
:). Or maybe I get crazy too one of these days ;) (well I still have
to pick something to do for the master thesis..)
Not that I think these kind of GCs are always a bad idea but they don't
come without their own baggage and set of problems.
Andi
Yes I agree that a more traditional mark-and-sweep collector would not
be suitable for PHP, and as many have experienced with Java it comes
with its own set of baggage. Actually I think the deterministic nature
of refcounting is more suitable to Web requests esp. for sites who want
to consistently serve below a certain threshold (150ms and so). Python
takes the same approach btw and also has a hybrid of reference counting
and then detecting cycles.
Btw, I also prefer using inline functions over pure macros as it's
easier to debug. The patch looks OK but I'd like to wait another 1-2
days to give others who got back from the long weekend here to review
and comment on it.
Andi
-----Original Message-----
From: David Wang [mailto:planetbeing@gmail.com]
Sent: Tuesday, September 04, 2007 9:38 AM
To: Nuno Lopes
Cc: internals@lists.php.net; andi@php.net; dmitry@php.net
Subject: Re: [PHP-DEV] Patch for macros for tracking refcount and
is_refIt can also have some use if we
decide to investigate the usage of an off-the-shelf (conservative)
garbage collector such as Boehm's (maybe in next year's SoC).As an aside, I was also thinking about this throughout the course of
the project. It's said that reference counting is the slowest form of
garbage collection since the reference counts must constantly be
maintained. Changing to a tracing garbage collector won't require
these macros, because reference counts would be eliminated altogether.
However, it would be sort of a big pain to implementOff the shelf garbage collectors such as BDW would be inappropriate
because we use some weird kinds of "pointers" (such as object handles)
stored in weird kinds of ways (such as a zend_hash object). I think it
would be pretty inefficient, since those implementations just scan the
stack, registers and heap and we're trying to do GC not for the PHP
interpreter, but for the code the PHP interpreter is running.I have a certain suspicion that a traditional mark-and-sweep collector
might be faster if just on the virtue of eliminating the refcount
field and getting rid of tons of cache misses that way. For just
displaying page, there wouldn't be much memory used and that's all
freed at the end of a request anyway: all of that reference counting
overhead would just disappear. For larger scripts that use a lot of
memory, the only problem would be pause times but in most real life
cases, it seems the total time would be shorter than reference
counting. However, I'm not sure if that would be the case in PHP:
rummaging through objects scattered all over memory would result in a
lot of cache misses. The question is whether that is greater than all
the misses we're currently having just managing the refcount.However, answering that question would require implementing the thing,
and that honestly seems like it would be a bit of a nightmare. Roots
would include zvals linked to PHP variables, the stack of the running
PHP code, and the stack and heap of the PHP interpreter itself. It
would've been far easier if PHP had been designed from the ground up
to use some sane way of managing memory, but with the current
situation, with extensions depending on reference counting, it's
pretty difficult.If ever a version of this patch is committed, you'll be able to see
that the cycle collector touches the whole reference counting mess
extremely minimally, which is why it was relatively safe to implement.
Btw, when can you make a PHP 5.2.x version of the patch available so
that we can start playing around with it and test it?
-----Original Message-----
From: David Wang [mailto:planetbeing@gmail.com]
Sent: Tuesday, September 04, 2007 9:38 AM
To: Nuno Lopes
Cc: internals@lists.php.net; andi@php.net; dmitry@php.net
Subject: Re: [PHP-DEV] Patch for macros for tracking refcount and
is_refIt can also have some use if we
decide to investigate the usage of an off-the-shelf (conservative)
garbage collector such as Boehm's (maybe in next year's SoC).As an aside, I was also thinking about this throughout the course of
the project. It's said that reference counting is the slowest form of
garbage collection since the reference counts must constantly be
maintained. Changing to a tracing garbage collector won't require
these macros, because reference counts would be eliminated altogether.
However, it would be sort of a big pain to implementOff the shelf garbage collectors such as BDW would be inappropriate
because we use some weird kinds of "pointers" (such as object handles)
stored in weird kinds of ways (such as a zend_hash object). I think it
would be pretty inefficient, since those implementations just scan the
stack, registers and heap and we're trying to do GC not for the PHP
interpreter, but for the code the PHP interpreter is running.I have a certain suspicion that a traditional mark-and-sweep collector
might be faster if just on the virtue of eliminating the refcount
field and getting rid of tons of cache misses that way. For just
displaying page, there wouldn't be much memory used and that's all
freed at the end of a request anyway: all of that reference counting
overhead would just disappear. For larger scripts that use a lot of
memory, the only problem would be pause times but in most real life
cases, it seems the total time would be shorter than reference
counting. However, I'm not sure if that would be the case in PHP:
rummaging through objects scattered all over memory would result in a
lot of cache misses. The question is whether that is greater than all
the misses we're currently having just managing the refcount.However, answering that question would require implementing the thing,
and that honestly seems like it would be a bit of a nightmare. Roots
would include zvals linked to PHP variables, the stack of the running
PHP code, and the stack and heap of the PHP interpreter itself. It
would've been far easier if PHP had been designed from the ground up
to use some sane way of managing memory, but with the current
situation, with extensions depending on reference counting, it's
pretty difficult.If ever a version of this patch is committed, you'll be able to see
that the cycle collector touches the whole reference counting mess
extremely minimally, which is why it was relatively safe to implement.
Btw, when can you make a PHP 5.2.x version of the patch available so
that we can start playing around with it and test it?
THis is the 5.2 version... it still needs to be ported to HEAD.
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
-----Original Message-----
From: Derick Rethans [mailto:derick@php.net]
Sent: Tuesday, September 04, 2007 11:57 AM
To: Andi Gutmans
Cc: David Wang; Nuno Lopes; internals@lists.php.net; andi@php.net;
dmitry@php.net
Subject: RE: [PHP-DEV] Patch for macros for tracking refcount and
is_refBtw, when can you make a PHP 5.2.x version of the patch available so
that we can start playing around with it and test it?THis is the 5.2 version... it still needs to be ported to HEAD.
Sorry I meant of the whole GC patch. This one is just the piece which adds macros. Or do you prefer to send that out once we finalize the macros patch so that it's a cleaner patch?
Andi
Btw, when can you make a PHP 5.2.x version of the patch available so
that we can start playing around with it and test it?THis is the 5.2 version... it still needs to be ported to HEAD.
Sorry I meant of the whole GC patch. This one is just the piece which
adds macros. Or do you prefer to send that out once we finalize the
macros patch so that it's a cleaner patch?
Yeah, that's the idea :)
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
Hello Andi,
Tuesday, September 4, 2007, 9:46:22 PM, you wrote:
-----Original Message-----
From: Derick Rethans [mailto:derick@php.net]
Sent: Tuesday, September 04, 2007 11:57 AM
To: Andi Gutmans
Cc: David Wang; Nuno Lopes; internals@lists.php.net; andi@php.net;
dmitry@php.net
Subject: RE: [PHP-DEV] Patch for macros for tracking refcount and
is_refBtw, when can you make a PHP 5.2.x version of the patch available so
that we can start playing around with it and test it?THis is the 5.2 version... it still needs to be ported to HEAD.
Sorry I meant of the whole GC patch. This one is just the piece which
adds macros. Or do you prefer to send that out once we finalize the macros patch so that it's a cleaner patch?
I think this is a good approach and cleaning those macros doesn't hurt. It
might actually make our code base cleaner whether we are going the full way
or not.
Best regards,
Marcus
-----Original Message-----
From: Marcus Boerger [mailto:helly@php.net]
Sent: Tuesday, September 04, 2007 2:12 PM
To: Andi Gutmans
Cc: Derick Rethans; David Wang; Nuno Lopes; internals@lists.php.net;
andi@php.net; dmitry@php.net
Subject: Re: [PHP-DEV] Patch for macros for tracking refcount and
is_refHello Andi,
Tuesday, September 4, 2007, 9:46:22 PM, you wrote:
I think this is a good approach and cleaning those macros doesn't
hurt.
It
might actually make our code base cleaner whether we are going the
full
way
or not.
Yes I agree. I just want to give another day chance for others to review
it.
Btw, we may also want to think of a way for maintenance mode to not
build if the macros aren't being used...
Andi
Hello David,
Tuesday, September 4, 2007, 6:38:17 PM, you wrote:
It can also have some use if we
decide to investigate the usage of an off-the-shelf (conservative)
garbage collector such as Boehm's (maybe in next year's SoC).
As an aside, I was also thinking about this throughout the course of
the project. It's said that reference counting is the slowest form of
garbage collection since the reference counts must constantly be
maintained. Changing to a tracing garbage collector won't require
these macros, because reference counts would be eliminated altogether.
However, it would be sort of a big pain to implement
What is the problem with those objects? Basically there are at least three
seperated memory areas involved. First the zend_object container, the real
object and one or several zvals. The gc would simply have to decrease
refcount on zend_objects if their zval gets down to zero and then leave
zend_object gc'ing to the object storage.
Off the shelf garbage collectors such as BDW would be inappropriate
because we use some weird kinds of "pointers" (such as object handles)
stored in weird kinds of ways (such as a zend_hash object). I think it
would be pretty inefficient, since those implementations just scan the
stack, registers and heap and we're trying to do GC not for the PHP
interpreter, but for the code the PHP interpreter is running.
I have a certain suspicion that a traditional mark-and-sweep collector
might be faster if just on the virtue of eliminating the refcount
field and getting rid of tons of cache misses that way. For just
displaying page, there wouldn't be much memory used and that's all
freed at the end of a request anyway: all of that reference counting
overhead would just disappear. For larger scripts that use a lot of
memory, the only problem would be pause times but in most real life
cases, it seems the total time would be shorter than reference
counting. However, I'm not sure if that would be the case in PHP:
rummaging through objects scattered all over memory would result in a
lot of cache misses. The question is whether that is greater than all
the misses we're currently having just managing the refcount.
However, answering that question would require implementing the thing,
and that honestly seems like it would be a bit of a nightmare. Roots
would include zvals linked to PHP variables, the stack of the running
PHP code, and the stack and heap of the PHP interpreter itself. It
would've been far easier if PHP had been designed from the ground up
to use some sane way of managing memory, but with the current
situation, with extensions depending on reference counting, it's
pretty difficult.
If ever a version of this patch is committed, you'll be able to see
that the cycle collector touches the whole reference counting mess
extremely minimally, which is why it was relatively safe to implement.
Question for development, how do we ensure that starting from a specific
point in time we enforce usage of those macros? The one thing that comes
into my mind is that we could have the members [is_ref,refcount] prefixed
with something different when running in debug mode, or insert some random
prefix there....(?)
Best regards,
Marcus
Tuesday, September 4, 2007, 6:38:17 PM, you wrote:
If ever a version of this patch is committed, you'll be able to see
that the cycle collector touches the whole reference counting mess
extremely minimally, which is why it was relatively safe to implement.Question for development, how do we ensure that starting from a specific
point in time we enforce usage of those macros? The one thing that comes
into my mind is that we could have the members [is_ref,refcount] prefixed
with something different when running in debug mode, or insert some random
prefix there....(?)
Yeah, that's what a previous patch did - I assume something like this is
part of the patch that comes after this macrofication patch.
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
What is the problem with those objects? Basically there are at least three
seperated memory areas involved. First the zend_object container, the real
object and one or several zvals. The gc would simply have to decrease
refcount on zend_objects if their zval gets down to zero and then leave
zend_object gc'ing to the object storage.
Hmm, maybe we have a misunderstanding here because I don't quite
understand what you mean. :) In that paragraph, I was saying how
implementing a traditional tracing garbage collector would mean
refcounts are no longer necessary. These macros track refcounts, so
they would also be no longer necessary.
Question for development, how do we ensure that starting from a specific
point in time we enforce usage of those macros? The one thing that comes
into my mind is that we could have the members [is_ref,refcount] prefixed
with something different when running in debug mode, or insert some random
prefix there....(?)
In my own code, I have a "__gc" on refcount and is_ref so I get thrown
an error if there's a place I failed to macroize. I removed that for
this patch, but that's a very good point. If there are no objections,
that or another prefix or suffix can be put back into the patch.
David
In my own code, I have a "__gc" on refcount and is_ref so I get thrown
an error if there's a place I failed to macroize. I removed that for
this patch, but that's a very good point. If there are no objections,
that or another prefix or suffix can be put back into the patch.
I've posted another version of the patch with the __gc suffix at
http://zoo.cs.yale.edu/~yw226/macros2.diff.txt
The GC patch is also ready. As soon as the macros patch has been
finalized and committed, I will send that out to the mailing list.
David
Hi David,
In general the patch is fine.
The only thing which I'd like to change is to make sure the __gc naming
is in an #if ZEND_GC for now otherwise we'll be breaking lots of third
party libraries and PECL extensions.
Andi
-----Original Message-----
From: David Wang [mailto:planetbeing@gmail.com]
Sent: Wednesday, September 05, 2007 10:47 PM
To: Marcus Boerger
Cc: Nuno Lopes; internals@lists.php.net; andi@php.net; dmitry@php.net
Subject: Re: [PHP-DEV] Patch for macros for tracking refcount and
is_refIn my own code, I have a "__gc" on refcount and is_ref so I get
thrown
an error if there's a place I failed to macroize. I removed that for
this patch, but that's a very good point. If there are no
objections,
that or another prefix or suffix can be put back into the patch.I've posted another version of the patch with the __gc suffix at
http://zoo.cs.yale.edu/~yw226/macros2.diff.txtThe GC patch is also ready. As soon as the macros patch has been
finalized and committed, I will send that out to the mailing list.David
The only thing which I'd like to change is to make sure the __gc naming
is in an #if ZEND_GC for now otherwise we'll be breaking lots of third
party libraries and PECL extensions.
Here's an updated version: http://zoo.cs.yale.edu/~yw226/macros.diff.txt
David
In general the patch is fine.
The only thing which I'd like to change is to make sure the __gc naming
is in an #if ZEND_GC for now otherwise we'll be breaking lots of third
party libraries and PECL extensions.
But you might want to wonder if that's not a good thing? Without the
prefix there will be no indication for third party extensions that they
might be doing something that's not going to work nicely with the new GC
anymore. I'd prefer it to have the __gc always there.
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
But you might want to wonder if that's not a good thing? Without the
prefix there will be no indication for third party extensions that they
might be doing something that's not going to work nicely with the new GC
anymore. I'd prefer it to have the __gc always there.
And extensions will break anyway with or without the #if ZEND_GC
ps: remember to change the Zend/PHP API number as well ;)
ps: remember to change the Zend/PHP API number as well ;)
Done.
I personally think having a ZEND_GC switch sort of defeats the purpose
of forcing everyone to start using the macros (whether or not we want
the GC), which is something we want to do anyway. However, here are
the two versions of the patch, for everyone's consideration:
With ZEND_GC: http://zoo.cs.yale.edu/~yw226/macros.diff.txt
Without: http://zoo.cs.yale.edu/~yw226/macros2.diff.txt
David
We are in an intermediate step right now where you are commiting these
macros to make sure that review of GC is easier. There'll be a period of
testing, benchmarks and stabilizing which we need to go through. Only
after that will we figure out what we'd like to do. Do we want to always
enable it, make it a config parameter or a compile-time option. All
options are open the way I see it so let's make the right steps
accordingly. Just randomly breaking source code compatibility at this
point for no reason doesn't make sense.
Btw, I agree that we will want to make sure that the macros are used and
we will find a way to do that.
Andi
-----Original Message-----
From: David Wang [mailto:planetbeing@gmail.com]
Sent: Friday, September 07, 2007 8:38 AM
To: Cristian Rodriguez
Cc: internals@lists.php.net
Subject: Re: [PHP-DEV] Patch for macros for tracking refcount and
is_refps: remember to change the Zend/PHP API number as well ;)
Done.
I personally think having a ZEND_GC switch sort of defeats the purpose
of forcing everyone to start using the macros (whether or not we want
the GC), which is something we want to do anyway. However, here are
the two versions of the patch, for everyone's consideration:With ZEND_GC: http://zoo.cs.yale.edu/~yw226/macros.diff.txt
Without: http://zoo.cs.yale.edu/~yw226/macros2.diff.txtDavid
Here's my analysis of what the options are:
In my mind, the macros patch is actually unrelated to the GC patch. It
is a Good Idea (tm) to use macros to manipulate refcounts no matter if
the GC is enabled, disabled, present in the source, or not added at
all. The only question is whether to break source compatibility just
for the sake of the aesthetics and cleanness of using macros to
manipulate refcounts.
If one accepts that PHP should and will use macros to manipulate
refcounts in the future no matter what happens, then it's only a
question of when to break source compatibility. The choice is either
right now, or (if the GC is added) when the GC is added, or (if the GC
is not added) at some undefined point in the future.
Breaking it now instead of later might save some more work in the
future, because new refcount manipulations might be added that will
also have to be changed later. However, breaking it now without adding
a new feature (GC) might seem rather arbitrary to developers of PECL
extensions. Breaking it when GC is added might be an easier pill to
swallow.
It's probably also true that the work saved if we break source
compatibility early is not extremely significant: Replacing two
"z->refcount++"s in your code is about as much work as just replacing
one. It may also be true that the addition of GC might require the
source of some extensions that do heavy refcount manipulation to be
looked at at that point anyway (don't worry! I think this is actually
VERY unlikely, but nevertheless possible) so it might be more
efficient to break source compatibility at the point where the GC is
officially added.
In consideration of those points, and in the interest of moving things
along I am in favor of going with the version with ZEND_GC, but
removing that option when the GC is integrated in some form or
another. Deferring the source compatibility break is not very
expensive in any case.
David
As far as I'm concerned you can commit it with ZEND_GC. If you don't
have karma let me know and we'll commit it.
Thanks!
Andi
-----Original Message-----
From: David Wang [mailto:planetbeing@gmail.com]
Sent: Friday, September 07, 2007 12:07 PM
To: Andi Gutmans
Cc: Cristian Rodriguez; internals@lists.php.net
Subject: Re: [PHP-DEV] Patch for macros for tracking refcount and
is_refHere's my analysis of what the options are:
In my mind, the macros patch is actually unrelated to the GC patch. It
is a Good Idea (tm) to use macros to manipulate refcounts no matter if
the GC is enabled, disabled, present in the source, or not added at
all. The only question is whether to break source compatibility just
for the sake of the aesthetics and cleanness of using macros to
manipulate refcounts.If one accepts that PHP should and will use macros to manipulate
refcounts in the future no matter what happens, then it's only a
question of when to break source compatibility. The choice is either
right now, or (if the GC is added) when the GC is added, or (if the GC
is not added) at some undefined point in the future.Breaking it now instead of later might save some more work in the
future, because new refcount manipulations might be added that will
also have to be changed later. However, breaking it now without adding
a new feature (GC) might seem rather arbitrary to developers of PECL
extensions. Breaking it when GC is added might be an easier pill to
swallow.It's probably also true that the work saved if we break source
compatibility early is not extremely significant: Replacing two
"z->refcount++"s in your code is about as much work as just replacing
one. It may also be true that the addition of GC might require the
source of some extensions that do heavy refcount manipulation to be
looked at at that point anyway (don't worry! I think this is actually
VERY unlikely, but nevertheless possible) so it might be more
efficient to break source compatibility at the point where the GC is
officially added.In consideration of those points, and in the interest of moving things
along I am in favor of going with the version with ZEND_GC, but
removing that option when the GC is integrated in some form or
another. Deferring the source compatibility break is not very
expensive in any case.David
As far as I'm concerned you can commit it with ZEND_GC. If you don't
have karma let me know and we'll commit it.
I don't even have a CVS account yet so I can't commit this patch. I
haven't needed it so I never requested it. However, I will soon in
order to quickly respond to GC-related bugs that might crop up. Could
one be granted to me with the appropriate karma?
David
I don't even have a CVS account yet so I can't commit this patch. I
haven't needed it so I never requested it. However, I will soon in
order to quickly respond to GC-related bugs that might crop up. Could
one be granted to me with the appropriate karma?
http://www.php.net/cvs-php.php
-adam
--
adam@trachtenberg.com | http://www.trachtenberg.com
author of o'reilly's "upgrading to php 5" and "php cookbook"
avoid the holiday rush, buy your copies today!
Not source compatibility.
-----Original Message-----
From: Cristian Rodriguez [mailto:judas.iscariote@gmail.com]
Sent: Friday, September 07, 2007 1:06 AM
To: internals@lists.php.net
Subject: Re: [PHP-DEV] Patch for macros for tracking refcount and
is_refBut you might want to wonder if that's not a good thing? Without the
prefix there will be no indication for third party extensions that
they
might be doing something that's not going to work nicely with the new
GC
anymore. I'd prefer it to have the __gc always there.And extensions will break anyway with or without the #if ZEND_GC
ps: remember to change the Zend/PHP API number as well ;)
-----Original Message-----
From: Derick Rethans [mailto:derick@php.net]
Sent: Friday, September 07, 2007 12:09 AM
To: Andi Gutmans
Cc: David Wang; Marcus Boerger; Nuno Lopes; internals@lists.php.net;
andi@php.net; dmitry@php.net
Subject: RE: [PHP-DEV] Patch for macros for tracking refcount and
is_refIn general the patch is fine.
The only thing which I'd like to change is to make sure the __gc
naming
is in an #if ZEND_GC for now otherwise we'll be breaking lots of
third
party libraries and PECL extensions.But you might want to wonder if that's not a good thing? Without the
prefix there will be no indication for third party extensions that they
might be doing something that's not going to work nicely with the new
GC
anymore. I'd prefer it to have the __gc always there.
GC will probably not always be enabled; at least not to begin with (in fact many apps won't even need it). Let's not run before we walk. There's still testing and review and some stabilizing period we should be doing (as we discussed previously).
For now, we shouldn't with PHP 5.3 just break any extension (source compatibility) out there. If GC is not being used it should compile and work just fine.
Andi
Let's not run before we walk.
The GC already "walks."
There's still testing and review and some stabilizing period we
should be doing
if it is disabled by default it is unlikely that people will test it
with real life code even more with in you place a #if ZEND_GC around
the functionality...
Hello Andi, Cristian, David,
agreeed. We should put this into 5.3 where we'll break BC anyway. Then for
5.3 and HEAD people will have enough time to adapt (given current release
plans. The only thing left to think of is whether it makes sense to use '__'
prefix for internal stuff just like we do for php?
And please no more magic switches that lead to broken code. In fact hasn't
it always been Zend Engine development police to rather break to see the
issues then trying to avoid them?
marcus
Saturday, September 8, 2007, 8:54:07 AM, you wrote:
Let's not run before we walk.
The GC already "walks."
There's still testing and review and some stabilizing period we
should be doing
if it is disabled by default it is unlikely that people will test it
with real life code even more with in you place a #if ZEND_GC around
the functionality...
Best regards,
Marcus
And please no more magic switches that lead to broken code.
that's what worries me, I dont get why people wants to introduce yet
another switch to disable to garbage collector ( that will lead to
different/broken behaviuor) I think there is enough non-sense already
with the unicode.semantics switch thingy.
that's what worries me, I dont get why people wants to introduce yet
another switch to disable to garbage collector ( that will lead to
different/broken behaviuor) I think there is enough non-sense already
with the unicode.semantics switch thingy.
This switch (currently) has nothing to do with the garbage collector.
If on, it prevents legacy code that do not use the refcount and is_ref
manipulation macros from compiling. That's all it does. In fact,
naming it ZEND_GC is misleading.
David
Hello David,
Wednesday, September 5, 2007, 4:19:05 AM, you wrote:
What is the problem with those objects? Basically there are at least three
seperated memory areas involved. First the zend_object container, the real
object and one or several zvals. The gc would simply have to decrease
refcount on zend_objects if their zval gets down to zero and then leave
zend_object gc'ing to the object storage.
Hmm, maybe we have a misunderstanding here because I don't quite
understand what you mean. :) In that paragraph, I was saying how
implementing a traditional tracing garbage collector would mean
refcounts are no longer necessary. These macros track refcounts, so
they would also be no longer necessary.
All true, yet at soume point your gc will free a zval - and a zval only -
unless you also use the gc'ing for every object storage in use. Either way
zvals and their objects are seperate things and multiple different zvals
can point to the same object in an object storage.
Question for development, how do we ensure that starting from a specific
point in time we enforce usage of those macros? The one thing that comes
into my mind is that we could have the members [is_ref,refcount] prefixed
with something different when running in debug mode, or insert some random
prefix there....(?)
In my own code, I have a "__gc" on refcount and is_ref so I get thrown
an error if there's a place I failed to macroize. I removed that for
this patch, but that's a very good point. If there are no objections,
that or another prefix or suffix can be put back into the patch.
David
Best regards,
Marcus
why not keep ZVAL_ADDREF ZVAL_DELREF for 3rd party source level
compatibility reason and deprecate it?
e.g.:
pecl/event/event.c: 790
ZVAL_ADDREF(ev->php_cb_arg);
One thing to note is that I removed the existing ZVAL_ADDREF and
ZVAL_DELREF in favor of Z_ADDREF_P and Z_DELREF_P that do the same
thing. The original duo acted on zval pointers contrary to standard
form for macro names.