Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:47176
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain iki.fi from 204.13.248.72 cause and error)
Message-ID: <4B9A865A.3000106@iki.fi>
Date: Fri, 12 Mar 2010 20:22:18 +0200
Reply-To: jani.taskinen@iki.fi
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100301 Fedora/3.0.3-1.fc11 Thunderbird/3.0.3
MIME-Version: 1.0
To: Ulf Wendel <Ulf.Wendel@Sun.COM>
CC: internals@lists.php.net
References: <4B9926E8.4080202@lerdorf.com> <4B995F83.1000605@prohost.org> <4B99639B.9030405@oracle.com> <4B996FC6.7040102@zend.com> <4B997381.1050300@oracle.com> <4B9A14FF.6000703@iki.fi> <4B9A2A20.8030405@sun.com> <4B9A4085.8020607@iki.fi> <4B9A4C8B.70608@sun.com>
In-Reply-To: <4B9A4C8B.70608@sun.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] Tests repository
From: jani.taskinen@iki.fi (Jani Taskinen)

On 03/12/2010 04:15 PM, Ulf Wendel wrote:
> For a transition period there's likely to be more work and the number of
> test failures is likely to go up. That is nothing to really worry about
> as long as you manage to educate users that it is not a quality defect
> of PHP as such but a temporary matter of an different and improved
> testing approach.

I'd say it's more work since some tests do not exists in some branch. As 
we might soon have quite closely related 3 branches, such differences 
are quite small. As trunk would have exactly same tests as PHP_5_3 has, 
the real differences would be only with PHP_5_2. So if we want such 
common tests, I think the time to do it is about now.

> Let's assume the worst case status quo of different versions of the same
> test in each branch. The one in an old branch has been written with the
> versions of the external library in mind that has been current when the
> branch was current. The one in a current branch has been written against
> the latest version of the external library.
>
> I continue to assume that users of the old PHP branch run rather old
> systems whereas users of a current PHP branch run on current systems.
>
> Therefore only few people may have tried to run the old test against the
> latest library or the other way around. It is just a feeling that your
> proposal will implicitly cause some combinations of PHP version x test x
> external library version to be tested that have not been checked before.

So you actually fear that you'd have to fix some things in older 
branches too? Well, I think (hope) such cases are rare and such problems 
actually already exist too. Of course we can always stay with the status 
quo, but is that really good for PHP in general?

> Your "one test for all" proposal is likely to unveil some "different
> versions of the same test" and "external library is not BC" issues.
> That's good. But its gonna be work...

IMO, such work is never a waste.

--Jani