Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:47193
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.223.177 as permitted sender)
DomainKey-Status: bad
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=YBbXWwDW1p6FXJJGHOQFQX5ioMHCR4buh7t8pjP5rcBgeT2JV/1wDpY92MGqPYghGl
         iWCBHK2xuBKWnG57AWT42RVEa0aIbVaRY21zOXY5Wf0UFe/K/A31Y6vS3C6coNgycGOC
         m3R1XlL9XANsZkplRH0ZQcK6igzKFOujC9jkU=
MIME-Version: 1.0
In-Reply-To: <4B9AB840.1010209@zend.com>
References: <4B9926E8.4080202@lerdorf.com> <4B997381.1050300@oracle.com>
	 <4B9A14FF.6000703@iki.fi> <4B9A833A.8040402@zend.com>
	 <4B9A848C.3080701@iki.fi>
	 <fe05d1541003121106g1e5f0251hbfccedf0e4ffac1f@mail.gmail.com>
	 <F920F09D-893F-43E0-A2D7-49D12410DBEF@gmail.com>
	 <4B9A93EE.7080403@zend.com>
	 <1418253B-84A3-4CA6-965B-A2501A5D432F@gmail.com>
	 <4B9AB840.1010209@zend.com>
Date: Fri, 12 Mar 2010 17:13:41 -0500
Message-ID: <ce6b011c1003121413p4b9fbff9qa79af5ee0db0778@mail.gmail.com>
To: PHP Internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary=001636c933da147bd80481a1d800
Subject: Re: [PHP-DEV] Tests repository
From: ericleestewart@gmail.com (Eric Stewart)

--001636c933da147bd80481a1d800
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Mar 12, 2010 at 4:55 PM, Stanislav Malyshev <stas@zend.com> wrote:

> Hi!
>
>
>  That's easy to say - but how exactly you're going to test
>>> functionality of, say. ext/intl without testing the underlying ICU
>>> library?
>>>
>>
>> Well, here's the way I see it:
>>
>> Extensions (including ext/intl) declare their API and tests are made
>> againt this API. No more and no less.
>>
>
> It's all sounds very nice in theory, except that in practice it
> doesn't work this way. Different versions of the library return
> different data, different errors, have different conditions, etc. You
> can just declare half of the tests to be "not API", but in fact
> it is, that's what people use and that's what they want to be sure that
> it works - just different versions of it act differently.
>
>
>  If, for example, some error-text comes directly from underlying
>> library and extension doesn't have control over it it shouldn't be
>> part of API. In this case we can test that "some error-text was
>> returned" but we shouldn't test for exact text-match
>>
>
> How do you know that "some text" is the error and that it's the correct
> one and not just generic "wrong library call" one which is produced on
> any bad call?
> And you can't just declare that number formatting routine returns "some
> text" - it would be useless. You want very specific text. But what if some
> detail of that text changes for some of the test cases?
>
> --
> Stanislav Malyshev, Zend Software Architect
> stas@zend.com   http://www.zend.com/
> (408)253-8829   MSN: stas@zend.com
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
There are going to be some technical challenges. Some (maybe a lot) of test
will need updates or rewriting. run-tests.php may need more improvements
than what is already planned. Knowing this, I would still rather update
run-tests.php and fix the tests, then continue to applying tests to
different branches of the code base.

+1 to centralize tests.

Eric Lee Stewart

--001636c933da147bd80481a1d800--