Hi,
As per the release check list that Derick and Ilia wrote up [1] we now
have something called "primary testers". This is a list of projects per
php branch that will get directly notified of php RC's. We have one for
PHP 4.x [2], 5.x [3] and 6.x [4].
Now as far as I know the level of success has not been all to great, but
atleast it should push some responsibility to the given projects if we
happen to have a release snafu. Maybe we can make this more high
profile, by having some buttons "official php primary tester", you know
for the warm fuzzy feeling ;-)
Anyways I remember some discussions about this on internals (or maybe it
was a PEAR meeting) about what platforms to support. Obviously this is a
question on what the developers focus and also on what we test on.
So there are a number of different operating systems to consider, a
number of processor types to consider (or even just 32bit versus 64bit).
As for testing this is an area where some of our high profile newly one
corporate friends might be able to step up?
Especially with the emergence of the gcov setup [5][6] we also have a
nice setup to make the testing automated with pretty reports and even
visual gadgets.
So I guess I am soliciting feedback on how to make the "primary tester"
idea more successful, how to get testing done on as many platforms as
possible (which would probably implicitly define our "target" platforms)
and finally highliht the cool work that was done in the entire gcov
department.
regards,
Lukas
[1] http://oss.backendmedia.com/ReleaseChecklist
[2] http://oss.backendmedia.com/PhP4yz
[3] http://oss.backendmedia.com/PhP5yz
[4] http://oss.backendmedia.com/PhP6yz
[5] http://gcov.php.net/
[6] http://gcov.php.net/PHP_5_1/run-tests.log.php
Lukas Smith wrote:
As per the release check list that Derick and Ilia wrote up [1] we now
have something called "primary testers". This is a list of projects per
php branch that will get directly notified of php RC's. We have one for
PHP 4.x [2], 5.x [3] and 6.x [4].
I disagree with you in that respect, I received a fair amount of input
from our test projects. One of the responses lead to a resolution of a
nasty bug affecting smarty templating system and through it the project
being tested. If we get A LOT of feedback it means our release is badly
broken ;-).
Now as far as I know the level of success has not been all to great, but
atleast it should push some responsibility to the given projects if we
happen to have a release snafu. Maybe we can make this more high
profile, by having some buttons "official php primary tester", you know
for the warm fuzzy feeling ;-)
I think it is fine as is.
Anyways I remember some discussions about this on internals (or maybe it
was a PEAR meeting) about what platforms to support. Obviously this is a
question on what the developers focus and also on what we test on.
I see no reason to add limits on the supported platforms, the bottom
line if you can compile PHP on your platform of choice it is technically
supported.
Especially with the emergence of the gcov setup [5][6] we also have a
nice setup to make the testing automated with pretty reports and even
visual gadgets.
gcov shows code coverage it does not tell if you the code is working or
not. If anything it is a tool to help devise tests that would examine as
much code behavior as possible.
Ilia
Ilia Alshanetsky wrote:
Lukas Smith wrote:
As per the release check list that Derick and Ilia wrote up [1] we now
have something called "primary testers". This is a list of projects per
php branch that will get directly notified of php RC's. We have one for
PHP 4.x [2], 5.x [3] and 6.x [4].I disagree with you in that respect, I received a fair amount of input
from our test projects. One of the responses lead to a resolution of a
nasty bug affecting smarty templating system and through it the project
being tested. If we get A LOT of feedback it means our release is badly
broken ;-).
Ah great! Last time I checked back with Derick the feedback was minimal.
Anyways I remember some discussions about this on internals (or maybe it
was a PEAR meeting) about what platforms to support. Obviously this is a
question on what the developers focus and also on what we test on.I see no reason to add limits on the supported platforms, the bottom
line if you can compile PHP on your platform of choice it is technically
supported.
Ok, I probably did not word things optimally. I guess the focus was more
on getting more platforms in the test setup.
gcov shows code coverage it does not tell if you the code is working or
not. If anything it is a tool to help devise tests that would examine as
much code behavior as possible.
But the way I read the reports we also get the test results as a "side
benefit" including valgrind.
regards,
Lukas
Especially with the emergence of the gcov setup [5][6] we also have a
nice setup to make the testing automated with pretty reports and even
visual gadgets.
very nice!
Are there instructions anywhere for creating this report locally? Such
as
http://qa.php.net/running-tests.php
What is the process for submitting new tests to the test suite?
Best Regards,
Jeff Moore
Especially with the emergence of the gcov setup [5][6] we also have a
nice setup to make the testing automated with pretty reports and even
visual gadgets.very nice!
Are there instructions anywhere for creating this report locally? Such as
http://qa.php.net/running-tests.php
What is the process for submitting new tests to the test suite?
http://qa.php.net/write-test.php
You can send the .phpt files to php-qa@lists.php.net.
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org