Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:47197
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.223.177 as permitted sender)
DomainKey-Status: bad
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=Vt/RvGiV7cS7sRWNTQkiCJMzm7wVKBJ+urKRtQjdz/8w6A7M2bbDvuvUrXUbzbxn0u
         tcUOWGRCjgT/WYmuRLDYCSnTsgoBT1zPeLeetHFw9MhUrkJkprqrChv2iVV35wOo5ZF6
         LiWKtttx5++pikmMp23Ugmxxjn2x1DhtQooN4=
MIME-Version: 1.0
In-Reply-To: <fe05d1541003121439y60838573j5206d41c238d0431@mail.gmail.com>
References: <4B9926E8.4080202@lerdorf.com>
	 <fe05d1541003121106g1e5f0251hbfccedf0e4ffac1f@mail.gmail.com>
	 <F920F09D-893F-43E0-A2D7-49D12410DBEF@gmail.com>
	 <4B9A93EE.7080403@zend.com>
	 <1418253B-84A3-4CA6-965B-A2501A5D432F@gmail.com>
	 <4B9AB840.1010209@zend.com>
	 <ce6b011c1003121413p4b9fbff9qa79af5ee0db0778@mail.gmail.com>
	 <4B9ABDCE.2070904@zend.com> <4B9AC13B.401@sci.fi>
	 <fe05d1541003121439y60838573j5206d41c238d0431@mail.gmail.com>
Date: Fri, 12 Mar 2010 18:04:40 -0500
Message-ID: <ce6b011c1003121504g7eb006dt4bca9716e7c6a848@mail.gmail.com>
To: internals@lists.php.net
Content-Type: multipart/alternative; boundary=001636c933da62b7810481a28ea4
Subject: Re: [PHP-DEV] Tests repository
From: ericleestewart@gmail.com (Eric Stewart)

--001636c933da62b7810481a28ea4
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Mar 12, 2010 at 5:39 PM, Pierre Joye <pierre.php@gmail.com> wrote:

> On Fri, Mar 12, 2010 at 11:33 PM, Jani Taskinen <jani.taskinen@sci.fi>
> wrote:
> > 13.3.2010 0:18, Stanislav Malyshev wrote:
> >>
> >> Hi!
> >>
> >>> There are going to be some technical challenges. Some (maybe a lot) of
> >>> test
> >>> will need updates or rewriting. run-tests.php may need more
> improvements
> >>> than what is already planned. Knowing this, I would still rather update
> >>> run-tests.php and fix the tests, then continue to applying tests to
> >>> different branches of the code base.
> >>
> >> I still have yet to hear *how* these tests are supposed to be updated,
> >> to work everywhere. It's very easy to say "oh, we'll just fix them" -
> >> but how exactly you're going to fix them if the test is supposed to do
> >> one thing in 5.2 and another in 5.3? Are you going to have 2 versions of
> >
> > What tests are you really talking about here? I thought we have
> regression
> > tests in there which test that stuff does not change between versions.
> Such
> > test AFAICT help us to keep stuff to work like it worked before and after
> > some change somewhere in the related code. So there should not be any
> need
> > for any updates given the tests aren't for some reason different between
> > branches in which case they aren't really the same test anymore.
> >
> > Short version: if test works in 5.2 it also has to (!) work in 5.3.
> > Otherwise the test is pointless.
> >
> > Can you define the case you're referring to here or are we actually
> talking
> > about totally different thing?
>
> Did you read our replies? I cited two kind of tests failing even on
> the same version. There are also many new errors or behaviors changes
> between 5.2 and 5.3, some errors are based on system error messages or
> using the underlying libraries errors.
>
> As I said already, it is possible but the way we test things have to
> be rewamped. Especially the "variationxyz" kind of tests with dozen of
> cases in one single phpt, including all kind of errors, system, php,
> etc. That's not a small move, but a huge task which will take some
> months to be done.
>
>
> Cheers,
> --
> Pierre
>
> @pierrejoye | http://blog.thepimp.net | http://www.libgd.org
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
I think specific examples (ie: a link to a 5.2 test and the corresponding
5.3 test) which are currently impossible to merge would be most beneficial
at this point.

I'll search for some this evening. If anyone knows of one or two off the top
of their head, please provide links.

Eric Lee Stewart

--001636c933da62b7810481a28ea4--