Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:47367
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain googlemail.com designates 72.14.220.152 as permitted sender)
DomainKey-Status: bad
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=cc:message-id:from:to:in-reply-to:content-type
         :content-transfer-encoding:mime-version:subject:date:references
         :x-mailer;
        b=v7A1jbV+o39lkxWAp9Mu+AJBhnQg5j2o9mRBKfXlFR60F4B40QeLXCnRsIkWzk1Bjj
         bPEksDgpAyUsNxQt2JTR9W8iFFddUZe43gVOwZPQIQPe52PKGl03Lliavp0SZti1PGSY
         6UeTzVZFKNQYJBDIxxShuxdiCY5tcYmHRURuI=
Cc: Stanislav Malyshev <stas@zend.com>,
 internals@lists.php.net
Message-ID: <733369BC-C7BB-4E24-A26C-4ADB62A5063B@googlemail.com>
To: jani.taskinen@iki.fi
In-Reply-To: <4B9AC13B.401@sci.fi>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v936)
Date: Tue, 16 Mar 2010 23:27:52 +0100
References: <4B9926E8.4080202@lerdorf.com> <4B997381.1050300@oracle.com>	 <4B9A14FF.6000703@iki.fi> <4B9A833A.8040402@zend.com>	 <4B9A848C.3080701@iki.fi>	 <fe05d1541003121106g1e5f0251hbfccedf0e4ffac1f@mail.gmail.com>	 <F920F09D-893F-43E0-A2D7-49D12410DBEF@gmail.com>	 <4B9A93EE.7080403@zend.com>	 <1418253B-84A3-4CA6-965B-A2501A5D432F@gmail.com>	 <4B9AB840.1010209@zend.com> <ce6b011c1003121413p4b9fbff9qa79af5ee0db0778@mail.gmail.com> <4B9ABDCE.2070904@zend.com> <4B9AC13B.401@sci.fi>
Subject: Re: [PHP-DEV] Tests repository
From: foolistbar@googlemail.com (Geoffrey Sneddon)


On 12 Mar 2010, at 23:33, Jani Taskinen wrote:

> 13.3.2010 0:18, Stanislav Malyshev wrote:
>> Hi!
>>
>>> There are going to be some technical challenges. Some (maybe a  
>>> lot) of
>>> test
>>> will need updates or rewriting. run-tests.php may need more  
>>> improvements
>>> than what is already planned. Knowing this, I would still rather  
>>> update
>>> run-tests.php and fix the tests, then continue to applying tests to
>>> different branches of the code base.
>>
>> I still have yet to hear *how* these tests are supposed to be  
>> updated,
>> to work everywhere. It's very easy to say "oh, we'll just fix them" -
>> but how exactly you're going to fix them if the test is supposed to  
>> do
>> one thing in 5.2 and another in 5.3? Are you going to have 2  
>> versions of
>
> What tests are you really talking about here? I thought we have  
> regression tests in there which test that stuff does not change  
> between versions. Such test AFAICT help us to keep stuff to work  
> like it worked before and after some change somewhere in the related  
> code. So there should not be any need for any updates given the  
> tests aren't for some reason different between branches in which  
> case they aren't really the same test anymore.
>
> Short version: if test works in 5.2 it also has to (!) work in 5.3.  
> Otherwise the test is pointless.
>
> Can you define the case you're referring to here or are we actually  
> talking about totally different thing?

I recently wrote a patch for DOMDocument::saveHTML which made a test  
from 5.3 start failing: it expected there was no first argument,  
triggering an error when one was provided. I, in changing the function  
to mirror DOMDocument::saveXML, made this fail by adding a node  
argument: if we cannot even add functionality to where we currently  
trigger errors, then we can do very little to the language.

For the subset of tests which are /not/ testing errors are thrown, I  
entirely agree, we should just change behaviour of code that does not  
currently throw errors. But we have tests for more than just that. We  
have tests that check errors are thrown in cases, and those tests we  
should be able to change at will.


--
Geoffrey Sneddon
<http://gsnedders.com/>