Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:23534
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Message-ID: <446B1502.7080808@php.net>
Date: Wed, 17 May 2006 14:20:18 +0200
User-Agent: Thunderbird 1.5.0.2 (Windows/20060308)
MIME-Version: 1.0
To: Jasper Bryant-Greene <jasper@album.co.nz>
Cc: ceo@l-i-e.com, internals@lists.php.net
References: <4468848D.5020602@php.net>	<1645696871.20060515204232@marcus-boerger.de>	<446916F5.9050804@php.net>	<200605160807.13383.phil@dialsolutions.co.uk>	<44697B66.1000505@album.co.nz> <60843.67.108.68.36.1147812094.squirrel@www.l-i-e.com> <446A4EF9.50605@album.co.nz>
In-Reply-To: <446A4EF9.50605@album.co.nz>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] PHP Release Process Sucks
From: lsmith@php.net (Lukas Smith)

Jasper Bryant-Greene wrote:

> I'm not necessarily suggesting that a release be delayed because *my*
> application doesn't work, I'm just offering to test RCs against it and
> report any bugs that it turns up. If the bug is found to be with PHP and
> is sufficiently serious, then perhaps the release will be delayed. That
> would be up to the devs.
> 
> I would also suggest that our application is a good candidate for RC
> testing as uses most of PHP5's OO features, is E_ALL|E_STRICT clean (so
> it's easier to spot problems in the error log :), and has virtually 100%
> test-case coverage (so we can test quickly and efficiently).

Maybe some background as to why this primary testers idea was brought 
up. As Marcus points out we still needs tons more tests. However even 
then we are probably never able to test every possible permutation of 
code and data flowing through it so thats where this primary testers 
thing came up.

We actually went out and mailed tons of projects and got feedback from a 
few. These have already helped in a few cases, but not as often as we 
would have hoped in spotting show stopper bugs in a release.

Here are the current lists:
http://oss.backendmedia.com/PhP4yz
http://oss.backendmedia.com/PhP5yz
http://oss.backendmedia.com/PhP6yz

This will now obviously be moved to the moderated read only primary 
testers mailing list which Wez just setup:
primary-qa-tester@lists.php.net

Note: someone still needs to figure out how to set the reply-to so it 
points to php-qa@lists.php.net

So the conclusion:
1) Yes, it would be very important that as many of the big time projects 
try out the RC's and report back both success/failure. In the best case 
along with an analysis of what PHP feature broke. We do not want to food 
the RM with application specific bugs.

2) We still need to continuously expand the number of tests we have. You 
can see the numbers of on the gcov.php.net site. This might be an 
opportunity for bored students to step up. Along with that it would be 
nice to increase the number of hardware/OS platforms we test on. This 
might be an opportunity for some of the bigger corporate users to step up ;)

regards,
Lukas