Hello,
okay, mistakes happen everyday but it really sucks that PHP.net
continues trying to hide mistakes.
- PHP 5.1.4 was released with a nonsense announcement claiming that
there was only a problem with POST arrays or POST fileuploads.
-> In reality a paid Zend developer had destroyed the handling of
arrays in any kind of user input in PHP 5.1.3 completely. Ironically
after that incident another Zend man came forward and dares to say "I
don't trust our core testers anymore" - PHP 5.1.4 was lacking the PEAR installer which resulted in make
install downloading the file from the web.
a) this part should be removed from the make file completlely
because 'make install' is usually executed as root and under no
circumstances should download a file from an insecure HTTP source.
b) this fact was again hidden by silently replacing the PHP 5.1.4
tarball with a new one, after the other one was out for more than a week.
PHP.net is more and more turning into Microsoft (more than 3 months to
resolve critical security problems). I guess that comes with the
involvement of Enterprise companies.
Yours,
Stefan Esser
Stefan,
Ironically
after that incident another Zend man came forward and dares to say "I
don't trust our core testers anymore"
He dared to say it because there's a QA mechanism in place that isn't
working - AKA a bunch of application developers testing Release Candidates
on their real-world applications rather than testing only against PHP's own
test suite. If any of them had done so, they would have picked up the
problem with user input arrays straight away. They evidently didn't, and
didn't.
I don't see what's so 'ironic' about mentioning that, and I certainly don't
think it's any kind of bad thing that Tony went on to request extra time
prior to 5.1.4's being tagged so that he could download a bunch of PHP
applications out there and run his own real-world tests.
- Steph
Hello,
okay, mistakes happen everyday but it really sucks that PHP.net
continues trying to hide mistakes.
- PHP 5.1.4 was released with a nonsense announcement claiming that
there was only a problem with POST arrays or POST fileuploads.
-> In reality a paid Zend developer had destroyed the handling of
arrays in any kind of user input in PHP 5.1.3 completely. Ironically
after that incident another Zend man came forward and dares to say "I
don't trust our core testers anymore"
That is what the bug reports that appeared in relation to the problem
were complaining about, the announcement was tailored to address
issues raised by those people.
- PHP 5.1.4 was lacking the PEAR installer which resulted in make
install downloading the file from the web.
a) this part should be removed from the make file completlely
because 'make install' is usually executed as root and under no
circumstances should download a file from an insecure HTTP source.
We didn't have an automated process for including PEAR's phar file
in, it is now part of the release generation script. As far as
automated download, for full releases this will not be necessary in
the future as it will be included by default. As far as the phar
download, I think it either needs to require a confirmation prompt or
simply not done automatically, either way is fine by me.
b) this fact was again hidden by silently replacing the PHP 5.1.4tarball with a new one, after the other one was out for more than a
week.
The code in the release did not change on bit, the only change was
the inclusion of the missing phar file, this hardly warrants 5.1.5 or
even 5.1.4pl1. This will have no impact of people who have already
downloaded and installed PHP, nor will this impact people who have
yet to download PHP.
PHP.net is more and more turning into Microsoft (more than 3 months to
resolve critical security problems). I guess that comes with the
involvement of Enterprise companies.
The patches for security holes are usually in within a week, if you
want to fetch them you can do so either in a form of a PHP snapshot
of a specific patch from CVS. To make releases every-time we get
security fault is impractical. Like most other projects a number of
security fixes is combined into group and when a group is
sufficiently large a release is made. None of the bugs resolved were
super-critical issues such as remote code execution, which would
indeed warrant something like an emergency release just for that
particular patch.
Ilia
Hey,
The code in the release did not change on bit, the only change was the
inclusion of the missing phar file, this hardly warrants 5.1.5 or even
5.1.4pl1. This will have no impact of people who have already
downloaded and installed PHP, nor will this impact people who have yet
to download PHP.
It will have an effect on everyone using f.e. gentoo linux or BSD port
system, because mysteriously the hash of the tarball changed and people
will get warnings about modified tarballs. It also has the effect that I
am getting emails from people asking me if PHP.net was backdoored,
because the MD5 hash changed....
And if you want to change tarballs and don't change the version number
(which is considered very bad by many people) then atleast WARN people
about the modified tarball. A simple message: tarball was missing PEAR
and was therefore rerolled is not so bad...
The patches for security holes are usually in within a week, if you
want to fetch them you can do so either in a form of a PHP snapshot of
a specific patch from CVS. To make releases every-time we get security
fault is impractical.
First the zend_hash_del() bug caused remote code execution in a bunch of
popular PHP scripts. Secondly most open source projects release security
bugfix releases. PHP.net on the other hand doesn't do this anymore.
There are no security only fixes anymore. Instead we release not
properly tested new versions of PHP that break tons of servers. (fastcgi
....)
And well... I still see no PHP 4.4.3 on PHP.net... However we still
offer the PHP 4.4.2 tarball (knowing that it has critical security
holes). So either we release a security FIX release or we kick the
tarball and declare PHP4 unsupported from now on.
Yours,
Stefan Esser
Hey,
The code in the release did not change on bit, the only change was
the
inclusion of the missing phar file, this hardly warrants 5.1.5 or
even
5.1.4pl1. This will have no impact of people who have already
downloaded and installed PHP, nor will this impact people who have
yet
to download PHP.It will have an effect on everyone using f.e. gentoo linux or BSD port
system, because mysteriously the hash of the tarball changed and
people
will get warnings about modified tarballs. It also has the effect
that I
am getting emails from people asking me if PHP.net was backdoored,
because the MD5 hash changed....
And if you want to change tarballs and don't change the version number
(which is considered very bad by many people) then atleast WARN people
about the modified tarball. A simple message: tarball was missing PEAR
and was therefore rerolled is not so bad...
I'll add that to the 5.1.4 release message on the front page, that
was an oversight on my part.
The patches for security holes are usually in within a week, if you
want to fetch them you can do so either in a form of a PHP
snapshot of
a specific patch from CVS. To make releases every-time we get
security
fault is impractical.
First the zend_hash_del() bug caused remote code execution in a
bunch of
popular PHP scripts. Secondly most open source projects release
security
bugfix releases. PHP.net on the other hand doesn't do this anymore.
There are no security only fixes anymore. Instead we release not
properly tested new versions of PHP that break tons of servers.
(fastcgi
....)
"Tons" is a very quantitate number ;-), while fastcgi is definitely a
used SAPI, it is no where near the usage of the Apache sapi or even
plain cgi. According to a basic Google Search mod_php is about 8
times more popular then CGI/FastCGI and of the 24,000 found phpinfo()
for the latter I'd wager no more then 1/2 actually use FastCGI.
And well... I still see no PHP 4.4.3 on PHP.net... However we still
offer the PHP 4.4.2 tarball (knowing that it has critical security
holes). So either we release a security FIX release or we kick the
tarball and declare PHP4 unsupported from now on.
PHP 4 is still supported, no one is suggesting that we discontinue
it. Derick can better comment on when he plans on making the release,
but it will definitely happen.
Ilia
Ilia Alshanetsky wrote:
The code in the release did not change on bit, the only change was the
inclusion of the missing phar file, this hardly warrants 5.1.5 or even
5.1.4pl1. This will have no impact of people who have already downloaded
and installed PHP, nor will this impact people who have yet to download
PHP.
The checksum(s) changed, which is a problem for Gentoo Linux, for
instance.
--
Sebastian Bergmann http://www.sebastian-bergmann.de/
GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69
Stefan,
I don't see why this attack is directed at Zend people working on
PHP, where the release process is completely a community driven
effort (and last time I checked, no enterprise was involved in that
process either).
I agree the release process isn't perfect yet, and it becomes
increasingly hard as PHP grows, but your points would be better made
if they were not directed against individual contributors but as an
email to raise general awareness and discussion.
Ilia for one works hard and does his best (probably better than any
release manager before him) to juggle between the various issues and
priorities of each release.
Andi
At 06:39 AM 5/15/2006, Stefan Esser wrote:
Hello,
okay, mistakes happen everyday but it really sucks that PHP.net
continues trying to hide mistakes.
- PHP 5.1.4 was released with a nonsense announcement claiming that
there was only a problem with POST arrays or POST fileuploads.
-> In reality a paid Zend developer had destroyed the handling of
arrays in any kind of user input in PHP 5.1.3 completely. Ironically
after that incident another Zend man came forward and dares to say "I
don't trust our core testers anymore"- PHP 5.1.4 was lacking the PEAR installer which resulted in make
install downloading the file from the web.
a) this part should be removed from the make file completlely
because 'make install' is usually executed as root and under no
circumstances should download a file from an insecure HTTP source.
b) this fact was again hidden by silently replacing the PHP 5.1.4
tarball with a new one, after the other one was out for more than a week.PHP.net is more and more turning into Microsoft (more than 3 months to
resolve critical security problems). I guess that comes with the
involvement of Enterprise companies.Yours,
Stefan Esser
Hello Andi,
I don't see why this attack is directed at Zend people working on PHP,
where the release process is completely a community driven effort (and
last time I checked, no enterprise was involved in that process either).
Well I don't see why Zend people commit code that obviously broke a key
functionality without testing it. And then Zend people come out under
the trees and speak up that our testers are doing bad jobs. It is true
that obviously NOONE has tested the code, because otherwise it would
have been seen instantly.
Ilia for one works hard and does his best (probably better than any
release manager before him) to juggle between the various issues and
priorities of each release.
Andi, I was not speaking against Ilia. I know that he does a job that
actually noone really likes todo. The problems with 5.1.x releases have
to be blamed on other parties. Maybe you can say it was his fault that
this stupid .phar was not in the archive, but the bigger fault is the
lack of a sense of basic security principles. From my point of view it
is unbelievable how someone can add a 'wget pear.php.net/bla...' to the
'make install' process. Of course this is not triggered if the .phar
file is packaged, but this fallback is insane.
And Ilia has also nothing todo with the lack of a PHP 4.4.3 release.
Yours,
Stefan Esser
At 09:32 AM 5/15/2006, Stefan Esser wrote:
Hello Andi,
I don't see why this attack is directed at Zend people working on PHP,
where the release process is completely a community driven effort (and
last time I checked, no enterprise was involved in that process either).
Well I don't see why Zend people commit code that obviously broke a key
functionality without testing it. And then Zend people come out under
the trees and speak up that our testers are doing bad jobs. It is true
that obviously NOONE has tested the code, because otherwise it would
have been seen instantly.
My point was that this has nothing to do with Zend or not Zend. It's
happened to all of us in the past (except for maybe you) where we
have commited code that had problems. It's counter-productive to
attack individuals and/or entities about this especially when
everyone's working towards the same goal of contributing to PHP.
So people screw up. I prefer having the occasional screw up then less
people helping out.
Andi
Hey Andi
My point was that this has nothing to do with Zend or not Zend.
My point is not that someone from Zend broke it, but that someone from
Zend blamed the community that THEY failed to find the problem. I
thought Zend is enough into PHP to test their own products against RC's,
too. It makes me angry that in the last months again and again people
who earn their money with PHP come and insult those who work on PHP for
free and do all the hard work.
It's happened to all of us in the past (except for maybe you) where we
have commited code that had problems.
Hehe... I also commited code that cause problems in a release... Like a
very slowunserialize()in 4.3.??. And yes I tested it back then but
unfortunately that bug only was causing problems were large quantities
of data had to be unserialized.
So people screw up. I prefer having the occasional screw up then less
people helping out.
Yeah. But maybe we should setup some rules what exactly we should do
when there is a broken tarball or missing files. While it seems nice to
just repackage and silently change it (or change it with a warning), it
is considered bad practice and is especially bad for people using
distributions like Gentoo or the BSD ports system. Even if we don't
change the version number we could rename the tarball to
php-5.1.4-repack-1.tar.bz2 or something alike.
Hey Andi
My point was that this has nothing to do with Zend or not Zend.
My point is not that someone from Zend broke it, but that someone from
Zend blamed the community that THEY failed to find the problem. I
thought Zend is enough into PHP to test their own products against RC's,
too. It makes me angry that in the last months again and again people
who earn their money with PHP come and insult those who work on PHP for
free and do all the hard work.
Oh, no.. You said it.
I get NO MONEY for testing PHP, fixing bugs, writing tests, helping users to solve their own problems @ bugs.php.net, maintaining several my own PECL modules etc. etc.
This is all my own "just for fun" job that I do in my spare time FOR FREE.
So it makes me a bit angry when someone who did nothing (except for a couple of mails to internals@) for PHP since December starts treating me and Dmitry (who's one of the most active PHP contributors) like a millionaires who earned their millions from poor PHP community.
Whether you want it or not, but we are the part of this community and yes, it was partly MY own fault that the bug has slipped in.
So please stop your trolling session and start writing new tests or at least doing something more productive than you're doing now.
Thank you.
--
Wbr,
Antony Dovgal
Antony Dovgal wrote:
So it makes me a bit angry when someone who did nothing (except for a
couple of mails to internals@) for PHP since December starts treating me
and Dmitry (who's one of the most active PHP contributors) like a
millionaires who earned their millions from poor PHP community.
Now this is an unfair argument as Stefan cannot (for whatever reasons)
commit his work to cvs.php.net.
--
Sebastian Bergmann http://www.sebastian-bergmann.de/
GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69
Sebastian Bergmann wrote:
Now this is an unfair argument as Stefan cannot (for whatever reasons)
commit his work to cvs.php.net.
Strike that, I was educated about this on IRC just now. My initial
point is still valid to some degree, IMHO: just because Stefan's work
does not go into cvs.php.net does not mean that he is not helping to
make PHP better (more secure in his case).
--
Sebastian Bergmann http://www.sebastian-bergmann.de/
GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69
Sebastian Bergmann wrote:
Now this is an unfair argument as Stefan cannot (for whatever reasons)
commit his work to cvs.php.net.Strike that, I was educated about this on IRC just now. My initial
point is still valid to some degree, IMHO: just because Stefan's work
does not go into cvs.php.net does not mean that he is not helping to
make PHP better (more secure in his case).
True, but it'd be far more useful to have Stefan's feedback at RC stage
rather than at post-release stage. We all know he's good at what he does.
- Steph
So it makes me a bit angry when someone who did nothing (except for a
couple of mails to internals@) for PHP since December starts treating
me and Dmitry (who's one of the most active PHP contributors) like a
millionaires who earned their millions from poor PHP community.
Tony, you have obviously no clue about my contributions to the PHP
community. Do you have any clue how many security holes I have reported
in the last months? Do you have any clue how many mails I had to write
to angry security researchers that got ignored by the rest of the PHP
Security Response Team.
Yours,
Stefan Esser
Stefan,
If anything, that was a good drill on why it's not a good idea to
write sarcastic, negative emails to people. Unless of course, your
aim is to annoy them into starting a heated threads. You've raised
some good points in your original email, and it's a shame you diluted
your message by implying all sorts of things about Dmitry and
Tony. Instead of discussing the points, we're discussing these
pointless topics of who contributes more. I suggest we stop here, I
think it's absurd to question the level of contribution from any of you three.
Zeev
At 21:08 15/05/2006, Stefan Esser wrote:
So it makes me a bit angry when someone who did nothing (except for a
couple of mails to internals@) for PHP since December starts treating
me and Dmitry (who's one of the most active PHP contributors) like a
millionaires who earned their millions from poor PHP community.
Tony, you have obviously no clue about my contributions to the PHP
community. Do you have any clue how many security holes I have reported
in the last months? Do you have any clue how many mails I had to write
to angry security researchers that got ignored by the rest of the PHP
Security Response Team.Yours,
Stefan Esser
Zeev,
Instead of discussing the points, we're discussing these pointless
topics of who contributes more. I suggest we stop here, I think it's
absurd to question the level of contribution from any of you three.
You are right. The discussion can stop here. Antony once again proved
every single word I wrote about him with his reaction.
Stefan
At 21:29 15/05/2006, Stefan Esser wrote:
Zeev,
Instead of discussing the points, we're discussing these pointless
topics of who contributes more. I suggest we stop here, I think it's
absurd to question the level of contribution from any of you three.
You are right. The discussion can stop here. Antony once again proved
every single word I wrote about him with his reaction.
Stefan,
You've got to be kidding. Just stop.
Zeev
Hello Stefan,
Monday, May 15, 2006, 6:32:25 PM, you wrote:
Hello Andi,
I don't see why this attack is directed at Zend people working on PHP,
where the release process is completely a community driven effort (and
last time I checked, no enterprise was involved in that process either).
Well I don't see why Zend people commit code that obviously broke a key
functionality without testing it. And then Zend people come out under
the trees and speak up that our testers are doing bad jobs. It is true
that obviously NOONE has tested the code, because otherwise it would
have been seen instantly.
To rephrase Andi "So people screw up. I prefer having the occasional
screw up then less people helping out." We are a community [...]
What we need is more helping hands and less comlaining notes. Actually
we are constantly working on increasing or QA efforts. And from my
point Ilia so far has done a very good job. A few minor mistakes so
far maybe.
marcus
Marcus Boerger wrote:
To rephrase Andi "So people screw up. I prefer having the occasional
screw up then less people helping out." We are a community [...]
What we need is more helping hands and less comlaining notes. Actually
we are constantly working on increasing or QA efforts. And from my
point Ilia so far has done a very good job. A few minor mistakes so
far maybe.
That being said .. if there are any additions people have to give better
guidelines for RM's please drop a comment at the following address:
http://oss.backendmedia.com/ReleaseChecklist
regards,
Lukas
That being said .. if there are any additions people have to give better
guidelines for RM's please drop a comment at the following address:
http://oss.backendmedia.com/ReleaseChecklist
I've just added the following comment, which addresses part of the problem:
Somewhere between the RC and the release, the release manager should
receive email confirmation from a number of volunteers that the RC has
been tested not only with 'make test', but on real world applications such
as the volunteers' own code and high profile applications such as
phpMyAdmin, SquirrelMail and so on.
The RC should not become a release until such confirmation is received.
I hereby volunteer to do my code, phpMyAdmin and SquirrelMail for php5 RCs.
Clearly, for this to have maximum impact we need to cover as many
applications, platforms and SAPIs as possible.
Phil Driscoll
-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160
Phil Driscoll wrote:
That being said .. if there are any additions people have to give better
guidelines for RM's please drop a comment at the following address:
http://oss.backendmedia.com/ReleaseChecklistI've just added the following comment, which addresses part of the problem:
Somewhere between the RC and the release, the release manager should
receive email confirmation from a number of volunteers that the RC has
been tested not only with 'make test', but on real world applications such
as the volunteers' own code and high profile applications such as
phpMyAdmin, SquirrelMail and so on.The RC should not become a release until such confirmation is received.
I hereby volunteer to do my code, phpMyAdmin and SquirrelMail for php5 RCs.
Clearly, for this to have maximum impact we need to cover as many
applications, platforms and SAPIs as possible.
I like this idea. I can volunteer to do reasonably thorough testing
(both a 'make test' and real-world app testing) of 5.x releases on a
very large OO application running on Linux / Apache 2.2. It's not an
open-source app, but that doesn't mean I can't help with testing, does it?
Jasper Bryant-Greene
General Manager
Album Limited
http://www.album.co.nz/ 0800 4 ALBUM
jasper@album.co.nz 021 708 334
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (MingW32)
iD8DBQFEaXtmFfAeHhDzT4gRA55iAKCVTuNMLR4l+aCmp1D86FoNQX12qwCffAgW
HaAcyfx6XSZlISShJVveg8o=
=9zMN
-----END PGP SIGNATURE
I like this idea. I can volunteer to do reasonably thorough testing
(both a 'make test' and real-world app testing) of 5.x releases on a
very large OO application running on Linux / Apache 2.2. It's not an
open-source app, but that doesn't mean I can't help with testing, does
it?
Certainly, running the tests and reporting any problems is VERY VERY
GOOD.
But...
Now don't take this wrong...
But I really don't think it would be fair/right of PHP Devs to commit
to NOT releasing a version until after a closed-source application
"works" with it.
If the problems you have are wide-spread and likely to affect many
users, you're all set anyway, right?
But if it's some weird thing that only affects your application...
Well, what should happen?
The rest of the world waits for your problem to get resolved?
Just don't sit right in my gut, personally.
I trust you see my viewpoint and don't take this as an affront to the
offer to test -- which IS needed and will benefit you.
--
Like Music?
http://l-i-e.com/artists.htm
-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160
Richard Lynch wrote:
I like this idea. I can volunteer to do reasonably thorough testing
(both a 'make test' and real-world app testing) of 5.x releases on a
very large OO application running on Linux / Apache 2.2. It's not an
open-source app, but that doesn't mean I can't help with testing, does
it?
[snip]
But I really don't think it would be fair/right of PHP Devs to commit
to NOT releasing a version until after a closed-source application
"works" with it.If the problems you have are wide-spread and likely to affect many
users, you're all set anyway, right?But if it's some weird thing that only affects your application...
Well, what should happen?
Well, in that situation, if the problem lies with PHP, then probably
something should be done about it...
The point is that the more applications PHP is being tested against, the
higher the chance of catching (possibly obscure) bugs/regressions.
I'm not necessarily suggesting that a release be delayed because my
application doesn't work, I'm just offering to test RCs against it and
report any bugs that it turns up. If the bug is found to be with PHP and
is sufficiently serious, then perhaps the release will be delayed. That
would be up to the devs.
I would also suggest that our application is a good candidate for RC
testing as uses most of PHP5's OO features, is E_ALL|E_STRICT clean (so
it's easier to spot problems in the error log :), and has virtually 100%
test-case coverage (so we can test quickly and efficiently).
Jasper Bryant-Greene
General Manager
Album Limited
http://www.album.co.nz/ 0800 4 ALBUM
jasper@album.co.nz 021 708 334
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (MingW32)
iD8DBQFEak75FfAeHhDzT4gRA8qUAJsFsZLPbAU+VH7HaEO6/BnXdnF4ZgCglxhl
Bu+EJ0kTs6+H36Y1GMKPBJs=
=JfcT
-----END PGP SIGNATURE
Jasper Bryant-Greene wrote:
I'm not necessarily suggesting that a release be delayed because my
application doesn't work, I'm just offering to test RCs against it and
report any bugs that it turns up. If the bug is found to be with PHP and
is sufficiently serious, then perhaps the release will be delayed. That
would be up to the devs.I would also suggest that our application is a good candidate for RC
testing as uses most of PHP5's OO features, is E_ALL|E_STRICT clean (so
it's easier to spot problems in the error log :), and has virtually 100%
test-case coverage (so we can test quickly and efficiently).
Maybe some background as to why this primary testers idea was brought
up. As Marcus points out we still needs tons more tests. However even
then we are probably never able to test every possible permutation of
code and data flowing through it so thats where this primary testers
thing came up.
We actually went out and mailed tons of projects and got feedback from a
few. These have already helped in a few cases, but not as often as we
would have hoped in spotting show stopper bugs in a release.
Here are the current lists:
http://oss.backendmedia.com/PhP4yz
http://oss.backendmedia.com/PhP5yz
http://oss.backendmedia.com/PhP6yz
This will now obviously be moved to the moderated read only primary
testers mailing list which Wez just setup:
primary-qa-tester@lists.php.net
Note: someone still needs to figure out how to set the reply-to so it
points to php-qa@lists.php.net
So the conclusion:
-
Yes, it would be very important that as many of the big time projects
try out the RC's and report back both success/failure. In the best case
along with an analysis of what PHP feature broke. We do not want to food
the RM with application specific bugs. -
We still need to continuously expand the number of tests we have. You
can see the numbers of on the gcov.php.net site. This might be an
opportunity for bored students to step up. Along with that it would be
nice to increase the number of hardware/OS platforms we test on. This
might be an opportunity for some of the bigger corporate users to step up ;)
regards,
Lukas
Hello Phil,
Tuesday, May 16, 2006, 9:07:13 AM, you wrote:
That being said .. if there are any additions people have to give better
guidelines for RM's please drop a comment at the following address:
http://oss.backendmedia.com/ReleaseChecklist
I've just added the following comment, which addresses part of the problem:
Somewhere between the RC and the release, the release manager should
receive email confirmation from a number of volunteers that the RC has
been tested not only with 'make test', but on real world applications such
as the volunteers' own code and high profile applications such as
phpMyAdmin, SquirrelMail and so on.
The RC should not become a release until such confirmation is received.
Maybe we should even just repack the RC with a single change allowed, the
vrsion bump.
Best regards,
Marcus
For every RC we already send an e-mail to about 12 projects asking
them to test their code against the release and let us know of any
issues that come up. If you want to be part of this group ask Lukas
to add you to the wiki page that can be found here: http://
oss.backendmedia.com/PhP5yz
That being said .. if there are any additions people have to give
better
guidelines for RM's please drop a comment at the following address:
http://oss.backendmedia.com/ReleaseChecklistI've just added the following comment, which addresses part of the
problem:Somewhere between the RC and the release, the release manager should
receive email confirmation from a number of volunteers that the RC has
been tested not only with 'make test', but on real world
applications such
as the volunteers' own code and high profile applications such as
phpMyAdmin, SquirrelMail and so on.The RC should not become a release until such confirmation is
received.I hereby volunteer to do my code, phpMyAdmin and SquirrelMail for
php5 RCs.Clearly, for this to have maximum impact we need to cover as many
applications, platforms and SAPIs as possible.Phil Driscoll
--
Ilia Alshanetsky
Advanced Internet Designs Inc.
ilia@prohost.org
For every RC we already send an e-mail to about 12 projects asking
them to test their code against the release and let us know of any
issues that come up. If you want to be part of this group ask Lukas
to add you to the wiki page that can be found here: http://
oss.backendmedia.com/PhP5yz
I already get these via the php-qa list, however the improvement I want to
make to the process is to stop an RC becoming a release until positive
responses have been received from at least a reasonable proportion of a list
of people who have committed to do some testing.
--
Phil Driscoll
Ilia Alshanetsky wrote:
For every RC we already send an e-mail to about 12 projects asking them
to test their code against the release and let us know of any issues
that come up.
Sadly, this does not seem to work. Maybe because nobody from the
projects feels responsible?
--
Sebastian Bergmann http://www.sebastian-bergmann.de/
GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69
Ilia Alshanetsky wrote:
For every RC we already send an e-mail to about 12 projects asking
them
to test their code against the release and let us know of any issues
that come up.Sadly, this does not seem to work. Maybe because nobody from the
projects feels responsible?
I would submit that:
They assume they only need to tell you if something is WRONG because
that is what they are told to do. (See first sentence above)
That's quite different from:
We're waiting for a "green light" from your project so we know it's a
good solid release.
If you only ask to hear about problems, you don't hear a "200 OK" when
you need it. :-)
It's also possible that the PHP RC release cycle is too quick for them
to test, depending on their current availability, so it never gets
done on time...
Though obviously it's gotta be pretty high priority for them to be
sure it's all good.
Development of automated testing for these projects (by the projects
themselves, not PHP Devs) would go a long way towards the ideal of
catching bugs before release...
But it seems like PHP itself could use some work in this area to lead
by example. :-)
Just some random thoughts when I should be working...
--
Like Music?
http://l-i-e.com/artists.htm
Sebastian Bergmann wrote:
Ilia Alshanetsky wrote:
For every RC we already send an e-mail to about 12 projects asking them
to test their code against the release and let us know of any issues
that come up.Sadly, this does not seem to work. Maybe because nobody from the
projects feels responsible?
Geez tensions are running high.
I am the tester for PEAR, and I tested the PEAR Installer (I do not test
the gazillion packages that I don't maintain) and it worked great with
PHP 5.1.4. PEAR doesn't use $_POST and I did not detect any errors.
In the past I have found at least 4 PHP bugs when testing PEAR with PHP.
The fact is, most of the bugs I find are found the instant they are
introduced as I use the CVS of PHP_5_X for my testing. In fact the
system is working quite well from my end, but I can't vouch for other
projects.
Greg
Bonjour,
Greg Beaver wrote:
Sebastian Bergmann wrote:
Ilia Alshanetsky wrote:
For every RC we already send an e-mail to about 12 projects asking them
to test their code against the release and let us know of any issues
that come up.Sadly, this does not seem to work. Maybe because nobody from the
projects feels responsible?Geez tensions are running high.
I am the tester for PEAR, and I tested the PEAR Installer (I do not test
the gazillion packages that I don't maintain) and it worked great with
PHP 5.1.4. PEAR doesn't use $_POST and I did not detect any errors.In the past I have found at least 4 PHP bugs when testing PEAR with PHP.
The fact is, most of the bugs I find are found the instant they are
introduced as I use the CVS of PHP_5_X for my testing. In fact the
system is working quite well from my end, but I can't vouch for other
projects.Greg
I've also been working in the same area.
I would like to add that:
PEAR code is often not php5 E_STRICT compatible for the obvious reason
that we belongs to the real user area , meaning not running only on some
dream installations but real production machines with all the no-choices
that induces and furnishing some layers to practical cases.
But that is not the point here.
Even if we use some kind of fork of run-tests , we have a bunch of tests
(.phpt and others).
So by release process, it could be a good idea to run them as a whole on
different installations.
Yes, we would have to shut off E_STRICT for some of them :)
Outer the fact that some packages need some external means , I believe
it would not be so hard to implement.
History shows we would have trap some bloody stuff if not bugs.
No, no , Greg , it's cool discussion :)
toggg
Ilia Alshanetsky wrote:
For every RC we already send an e-mail to about 12 projects asking them
to test their code against the release and let us know of any issues
that come up. If you want to be part of this group ask Lukas to add you
to the wiki page that can be found here: http://oss.backendmedia.com/PhP5yz
Could someone add php-testing@phorum.org to that list? We will do our
best to test Phorum against any releases.
--
Brian Moon
http://dealnews.com/
Its good to be cheap =)
That being said .. if there are any additions people have to give
better
guidelines for RM's please drop a comment at the following address:
http://oss.backendmedia.com/ReleaseChecklistI've just added the following comment, which addresses part of the
problem:Somewhere between the RC and the release, the release manager should
receive email confirmation from a number of volunteers that the RC has
been tested not only with 'make test', but on real world applications
such
as the volunteers' own code and high profile applications such as
phpMyAdmin, SquirrelMail and so on.The RC should not become a release until such confirmation is
received.I hereby volunteer to do my code, phpMyAdmin and SquirrelMail for php5
RCs.Clearly, for this to have maximum impact we need to cover as many
applications, platforms and SAPIs as possible.
Would it be worth having a checklist of "must-have" software?
It's easy to see something big getting missed just because "Joe" is on
vacation or too busy to test, and everybody knows "Joe" is gonna test
it...
--
Like Music?
http://l-i-e.com/artists.htm
Would it be worth having a checklist of "must-have" software?
It's easy to see something big getting missed just because "Joe" is on
vacation or too busy to test, and everybody knows "Joe" is gonna test
it...
Yes, so we need a list plus at least two volunteers for everything on the list
for php4 and php5 testing.
So, to start the list (with apologies to authors of important software I've
missed out!):
make test
phpMyAdmin
Phorum
phpBB
SquirrelMail
FUDForum
I guess as well as a software list, we need an important platform list so we
know that an RC has been exercised on at least:
Linux X86 Apache module
Linux X86_64 Apache module
BSD Apache module
Solaris Apache module
Windows Apache CGI
Windows IIS CGI (and perhaps ISAPI - is that OK these days?)
Something with Fast CGI
Please add and/subtract to/from the above lists. We don't want either list to
get too big, otherwise we'll never get enough volunteers to cover the
spectrum.
--
Phil Driscoll
Phil Driscoll wrote:
Please add and/subtract to/from the above lists.
MediaWiki
Mambo/Joombla/Nuke
Those are the big in their space. We use MediaWiki daily in our company.
--
Brian Moon
http://dealnews.com/
Its good to be cheap =)
Brian Moon wrote:
Phil Driscoll wrote:
Please add and/subtract to/from the above lists.
MediaWiki
[snip]Those are the big in their space. We use MediaWiki daily in our company.
We're already testers.
Note that MediaWiki uses few (if any) $_POST arrays, so we encountered no
problems with 5.1.3 release candidates.
-- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Phil Driscoll wrote:
Would it be worth having a checklist of "must-have" software?
It's easy to see something big getting missed just because "Joe" is on
vacation or too busy to test, and everybody knows "Joe" is gonna test
it...Yes, so we need a list plus at least two volunteers for everything on the list
for php4 and php5 testing.So, to start the list (with apologies to authors of important software I've
missed out!):make test
phpMyAdmin
Phorum
phpBB
SquirrelMail
FUDForum
Is there any software out there which uses PHPUnit(2)? So those
automatic suites could be used as well.
-
- Markus
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
- Markus
iD8DBQFEauE/1nS0RcInK9ARAnjBAJ9A0J0HtnNO+40TjRxjmyQn1o0iXACfc8eo
DIcgr2idhDImje+GpOa0o0Y=
=aAti
-----END PGP SIGNATURE
Markus Fischer wrote:
Is there any software out there which uses PHPUnit(2)? So those
automatic suites could be used as well.
I have brought this up before. Sebastian Nohn has taken up this idea and
set up a box that checks out PHP from CVS, builds it, and then runs
"make test" as well as the unit tests of projects like PHPUnit and the
Zend Framework.
--
Sebastian Bergmann http://www.sebastian-bergmann.de/
GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69
I have brought this up before. Sebastian Nohn has taken up this idea and
set up a box that checks out PHP from CVS, builds it, and then runs
"make test" as well as the unit tests of projects like PHPUnit and the
Zend Framework.
The big whohoo would be a complete PHPUnit test framework for PHP with
code coverage that replaces the old test framework.
--
regards,
Robin
Robin Ericsson wrote:
The big whohoo would be a complete PHPUnit test framework for PHP with
code coverage that replaces the old test framework.
As long as PHPUnit cannot run each test in a new PHP interpreter
process this would make no sense. And even then, why would we want to
replace run-tests.php with PHPUnit? Also note that PHPUnit's code
coverage functionality can, of course, not generate code coverage
information for the PHP interpreter itself, as that is written in C :-)
If you are looking for code coverage information for the PHP interpreter
have a look at http://gcov.php.net/.
--
Sebastian Bergmann http://www.sebastian-bergmann.de/
GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69
I guess as well as a software list, we need an important platform list so we
know that an RC has been exercised on at least:
Linux X86 Apache module
Linux X86_64 Apache module
BSD Apache module
Solaris Apache module
Windows Apache CGI
Windows IIS CGI (and perhaps ISAPI - is that OK these days?)
Something with Fast CGI
OS X, preferably both on the Intel- and the PowerPC-platform.
- Martin
OS X, preferably both on the Intel- and the PowerPC-platform.
Oh yes. I need to be able to develop on my MacBook Pro.
--
Brian Moon
http://dealnews.com/
Its good to be cheap =)
OS X, preferably both on the Intel- and the PowerPC-platform.
added to
http://www.dialsolutions.com/phil/php/primarytesters.html
:)
--
Phil Driscoll
On Wed, 17 May 2006 17:19:06 +0100
phil@dialsolutions.co.uk (Phil Driscoll) wrote:
OS X, preferably both on the Intel- and the PowerPC-platform.
added to
http://www.dialsolutions.com/phil/php/primarytesters.html
:)
Why don't you ask for an access to the wiki and update this list there?
-- Pierre
Bonsoir
Pierre wrote:
Why don't you ask for an access to the wiki and update this list there?
-- Pierre
You need access to that ?
toggg
Let's keep this number of lists to the minimum. Thanks to Lukas, we
already have a wiki with all this information, lets keep using it,
the release process is complex enough already without having to
scramble around looking for information on dozens of websites.
OS X, preferably both on the Intel- and the PowerPC-platform.
added to
http://www.dialsolutions.com/phil/php/primarytesters.html
:)
--
Phil Driscoll--
Ilia Alshanetsky
Advanced Internet Designs Inc.
ilia@prohost.org
Ilia Alshanetsky wrote:
Let's keep this number of lists to the minimum. Thanks to Lukas, we
already have a wiki with all this information, lets keep using it, the
release process is complex enough already without having to scramble
around looking for information on dozens of websites.
Speaking of keeping the number of lists to a minimum, can I suggest that
we move this discussion to the QA mailing list?
I think the internals list (while a reasonable place to start) has had
more than enough chance to give input, and something like this really
does belong on the QA list (and internals-only subscribers can subscribe
to the QA list).
S
Let's keep this number of lists to the minimum. Thanks to Lukas, we
already have a wiki with all this information, lets keep using it,
the release process is complex enough already without having to
scramble around looking for information on dozens of websites.
Sure - I said that is would be sensible for the list to be on Lukas's wiki
when I first suggested it, but it's hard to demonstrate an idea in a location
you don't have write access to :)
Ilia, if you and Derick are happy to work with a primary testers list in the
way I suggested, let me know and I'll liaise with Lukas to get everything
needed up on the Wiki, and try and recruit testers for the various
applications and platforms.
Phil Driscoll
Let's keep this number of lists to the minimum. Thanks to Lukas, we
already have a wiki with all this information, lets keep using it,
the release process is complex enough already without having to
scramble around looking for information on dozens of websites.
Sure - I said that is would be sensible for the list to be on Lukas's wiki
when I first suggested it, but it's hard to demonstrate an idea in a location
you don't have write access to :)Ilia, if you and Derick are happy to work with a primary testers list in the
way I suggested, let me know and I'll liaise with Lukas to get everything
needed up on the Wiki, and try and recruit testers for the various
applications and platforms.
I'm fine with it... as long as I know where to find the list :)
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
Phil,
Based on past experience in dealing with primary testers we usually
only get responses when things go bad, as such waiting for positive
responses until rolling the final is not practical in my opinion. I
think the best I can propose is that we give testers at least one
week to try the final RC and if we don't hear any complaints assume
all is well.
Let's keep this number of lists to the minimum. Thanks to Lukas, we
already have a wiki with all this information, lets keep using it,
the release process is complex enough already without having to
scramble around looking for information on dozens of websites.
Sure - I said that is would be sensible for the list to be on
Lukas's wiki
when I first suggested it, but it's hard to demonstrate an idea in
a location
you don't have write access to :)Ilia, if you and Derick are happy to work with a primary testers
list in the
way I suggested, let me know and I'll liaise with Lukas to get
everything
needed up on the Wiki, and try and recruit testers for the various
applications and platforms.Phil Driscoll
--
Ilia Alshanetsky
Advanced Internet Designs Inc.
ilia@prohost.org
Ilia Alshanetsky wrote:
Phil,
Based on past experience in dealing with primary testers we usually
only get responses when things go bad, as such waiting for positive
responses until rolling the final is not practical in my opinion. I
think the best I can propose is that we give testers at least one week
to try the final RC and if we don't hear any complaints assume all is
well.
Sounds like a good plan. I don't think that waiting for positive answers
is practical either.
Edin
Lukas
To move this process forward, is there any chance that you could make a few
changes to http://oss.backendmedia.com/ReleaseChecklist?
First add an extra item to 'Getting the Release Candidate out there' saying:
Email primary-qa-tester@lists.php.net to notify the primary testers to get
testing. An example email is here:
Hello!
You are receiving this email because you have volunteered to be a php primary
tester. Can you please test the latest PHP release candidate on the platform
and with the applications you have volunteered to be responsible for (and
also if possible do a make test and allow it to post the results to the
php-qa list). For a reminder of what you volunteered to test, check the table
at http://www.dialsolutions.com/phil/php/primarytesters.html. Please report
back by <some date - maybe a week in the future> to
whoever@is.realease.manager and cc internals@lists.php.net, letting us
know what applications you tested, on what platforms and informing us of
success or failure.
Until we have had sufficient responses from our primary testers to be sure
that this RC performs well on a range of applications and platforms, we will
not go ahead with the release. Consequently, your input to this process is
very important. If you are unable for any reason to test this RC, please
notify whoever@is.realease.manager as soon as possible so that we can make
alternative arrangements this time.
If you do not want to receive these heads-up emails, please reply to me
personally and I will remove you from the list; but, we hope that you
want to actively help us making PHP a better and more stable tool.
The first release candidate of PHP N can be found at
http://downloads.php.net/somewhere/ .
If you know anyone else who would like to be a PHP primary tester, please let
me know.
regards,
<Whoever> PHP <N> Release Manager -----------------------Then add a new item between 'Getting the Release Candidate out there' and
'Rolling a release' entitled 'Making sure that the Release Candidate is fit to
become a release':
-
Await responses from the primary testers, and issue a reminder email to
primary-qa-tester@lists.php.net if responses are slow to come in. -
Keep an eye on the make test results for the RC posted to the php-qa list
looking out for any significant problems. -
Check that problem-free responses have been received in respect of all the
critical applications and platforms listed (with their volunteers) at
http://www.dialsolutions.com/phil/php/primarytesters.html (which I am happy
to look after, but another wiki page may be more sensible), and if necessary
chase up individual volunteers for their results.
Please feel free to add/delete/amend as appropriate. Suggestions and
volunteers welcome :)
--
Phil Driscoll
Phil Driscoll wrote:
To move this process forward, is there any chance that you could make a few
changes to http://oss.backendmedia.com/ReleaseChecklist?
I have added a link to your post as a comment in the page. Since I am
not an RM I will leave it to Ilia and Derick to incorporate these ideas
into the document.
Aside from that this document could probably be moved to php-qa website
once its mature enough (and therefore less volatile).
I feel the todo lists are so dynamic and have a tendency to require some
simple markup that its better to leave them in a wiki. Obviously there
is still history maintained there. The only drawback is that for people
to get access I need to create a new local user instead of being able to
reuse the cvs.php.net authentication scheme (which as an outsider I
should of course not be allowed to interface with).
regards,
Lukas