Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:47182
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 74.125.82.42 as permitted sender)
DomainKey-Status: bad
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        b=ZvuwjbkOLNtuvLicWb2u3jvFYFShrRhSK0JHihSW/iikeBPnITw2/DmNV2Jckv5Ypa
         0MgWgOTavX5jxilvT4hJBZerfaITy77K4Dou2UryDgNp+KQH458GsoL2ptZDRIGR9wYj
         537hFGxwtSKhs/osfsqahsmShXkFXVy/tZ7nQ=
MIME-Version: 1.0
In-Reply-To: <4B9A848C.3080701@iki.fi>
References: <4B9926E8.4080202@lerdorf.com> <4B995F83.1000605@prohost.org>
	 <4B99639B.9030405@oracle.com> <4B996FC6.7040102@zend.com>
	 <4B997381.1050300@oracle.com> <4B9A14FF.6000703@iki.fi>
	 <4B9A833A.8040402@zend.com> <4B9A848C.3080701@iki.fi>
Date: Fri, 12 Mar 2010 20:06:10 +0100
Message-ID: <fe05d1541003121106g1e5f0251hbfccedf0e4ffac1f@mail.gmail.com>
To: jani.taskinen@iki.fi
Cc: Stanislav Malyshev <stas@zend.com>, internals@lists.php.net
Content-Type: text/plain; charset=ISO-8859-1
Subject: Re: [PHP-DEV] Tests repository
From: pierre.php@gmail.com (Pierre Joye)

On Fri, Mar 12, 2010 at 7:14 PM, Jani Taskinen <jani.taskinen@iki.fi> wrote:
> On 03/12/2010 08:08 PM, Stanislav Malyshev wrote:
>>
>> Hi!
>>
>>> Having tests in multiple branches is PITA. Hasn't anyone considered that
>>> the best way would be to move all tests into their own repository
>>> (directory..whatever :) in SVN..? Considering they are supposed to be
>>
>> Yes, but: some tests are version-dependant, some are not. And since we
>
> That's why we'd need to add some section to select the minimum version
> required to run the test.
>
>> have this "output match" testing paradigm, it would probably keep being
>> this way. If you're going to have branches in that SVN, how it would be
>> different from what we have now?
>
> Considering what I said above about version check..why would you need
> branches in it? And what/how does it matter what the tests output if it
> needs to be same in any PHP version anyway..?

What you describe may work for PHP core features without external
dependencies but will fail for anything with external dependencies.

Many tests fail because they are written for a given platform, or even
worst (from a portability point of view), only for a given
configuration (library version, system version,etc.). And that's not
about windows vs other, tests can work on a Debian/ubuntu version and
fail on another.

However the idea of shared tests is what I've been adovocating for
some time already, to improve portability and BC checks. It requires
to write true unit tests, and huge test including dozen of cases being
tested in one test. It will also require to rewite many tests to
output something when it fails instead of testing standard outpout
with system dependent messages (think about localization too, which
can affect behaviors as well).

All in all I lilke the idea and will support it as much as I can. We
have to keep in mind that it will need a couple of testfests to
realize it(read: huge work, more like doing it from scratch for 40-60%
of the existing test, rough estimation).

Cheers,
-- 
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org