Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:52334
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain roshambo.org from 209.85.214.170 cause and error)
Mime-Version: 1.0 (Apple Message framework v1084)
Content-Type: text/plain; charset=us-ascii
In-Reply-To: <4DCB05F9.2000406@lerdorf.com>
Date: Wed, 11 May 2011 17:28:35 -0600
Cc: PHP Internals <internals@lists.php.net>
Content-Transfer-Encoding: quoted-printable
Message-ID: <3FCE63CB-3198-4253-9DB6-486AA9F3833D@roshambo.org>
References: <4DC729EE.9090600@sugarcrm.com> <8757232E56758B42B2EE4F9D2CA019C901401CB0@US-EX2.zend.net> <1305138029.2232.9.camel@guybrush> <691BF1CD-66E1-4123-AA98-A205A5E67F78@roshambo.org> <4DCADA70.7080505@sugarcrm.com> <41C30BEF-CC30-4A32-A8FB-2B8A6132804C@roshambo.org> <4DCB05F9.2000406@lerdorf.com>
To: Rasmus Lerdorf <rasmus@lerdorf.com>
Subject: Re: [PHP-DEV] 5.4 again
From: philip@roshambo.org (Philip Olson)


On May 11, 2011, at 3:56 PM, Rasmus Lerdorf wrote:

> On 05/11/2011 02:10 PM, Philip Olson wrote:
>=20
>> So, that's the concern there. But if the alpha is simply a trick to =
convince people to test out a specific PHP 5.4 snapshot, and feel 5.4 is =
real, then do it. ;)
>=20
> There are still quite a few test failures in trunk. Some of them are =
also in the 5_3 branch. In some cases the tests are simply bad. In a few =
the test case contains binary data that got mangled in the move to =
Subversion. It would be nice if just 1 in 10 people reading the list =
here would grab both trunk and 5_3 and run "make test" in each tree and =
then fix at least 1 test each. We would have no test failures by the end =
of the day other than a few tricky ones. If an alpha release will =
encourage this, great. If we could get people to just do it on their own =
without the alpha, even better.
>=20
> And yes, I know the tests take forever to run. Get yourself a fast =
machine with an SSD, and remember you can run partial tests using:
>=20
>  make test TESTS=3Dext/hash
>=20
> for example to just run the tests for the hash extension.
>=20
> Also, when a test fails, cd into the ext/hash/tests directory and you =
will see .out, .exp, .diff and .php files for the failed test. That is, =
the output, the expected output, the diff between them and the php =
script itself extracted from the .phpt file containing the failed test =
case.
>=20
> And if you can't figure out how to fix a test, post the details here. =
I'd love to point some of the obvious talents and energy of this list =
towards the code. If you don't have an svn account for committing your =
fixed test, go to http://www.php.net/svn-php.php and fill in the little =
form at the bottom there and put in the test that you fixed and a =
1-liner about how you fixed it and I will set you up with an account =
right away. Info on how to check out the code from svn is here:
> https://wiki.php.net/vcs/svnfaq

This is a great challenge. Long ago I decided that failed tests are =
expected in PHP, because out of the ~100 or so times I've run it these =
past n years, it has always (afair) failed. Changing this fact by having =
it pass would change peoples (at least my) perception that running 'make =
test' is a waste of time, as is (was) looking into why they failed. For =
example, just now trunk gives me 55 failed tests and 17 expected fails =
with a vanilla build.

We once had a matrix showing test results per setup (OS, phpversion, per =
configure switches) but it was someones pet project and the code has =
long since been lost (he looked years ago). Maybe such a beast would be =
useful.

Also, how about we change[1] it so php.qa.reports[2] stops receiving 5_2 =
results, and always receives snapshot results? For example, snapshot =
test results (5_3 and trunk) are configured to _not_ go there today.

Regards,
Philip

[1] http://svn.php.net/viewvc/web/qa/trunk/include/release-qa.php
[2] http://news.php.net/php.qa.reports