Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:73830
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.192.43 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1398495101.28614.9.camel@localhost.localdomain>
References: <535A13E1.3050106@sugarcrm.com>
	<CAH-PCH7ZM-RLyYUkCz6OeWMS74zrMiFj4eu8RBLcjTjWuQ8Stg@mail.gmail.com>
	<535AAC49.9050004@sugarcrm.com>
	<1398495101.28614.9.camel@localhost.localdomain>
Date: Mon, 28 Apr 2014 14:47:27 +0200
Message-ID: <CAH-PCH43jZXAmQi=7qq+hYwUH_vv3S+FCSXkxpj9wsSr=GWzLw@mail.gmail.com>
To: Joe Watkins <pthreads@pthreads.org>
Cc: Stas Malyshev <smalyshev@sugarcrm.com>, PHP Internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary=001a1133bbd0be912704f819b7b1
Subject: Re: [PHP-DEV] [VOTE] CI tests RFC
From: tyra3l@gmail.com (Ferenc Kovacs)

--001a1133bbd0be912704f819b7b1
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Sat, Apr 26, 2014 at 8:51 AM, Joe Watkins <pthreads@pthreads.org> wrote:

> On Fri, 2014-04-25 at 11:41 -0700, Stas Malyshev wrote:
> > Hi!
> >
> > > Would be nice if we could also create some policy regarding the usage
> of
> > > XFAILs (when should be a failing test marked as XFAIL, should we add
> the
> > > tests from the open bugreports as XFAIL by default, whose
> responsibility
> > > is to make sure that it will be fixed eventually, etc.).
> >
> > I agree. I didn't get into this but it definitely makes sense to have
> > some rules there.
> >
> > > I would also like to extend the current travis config a bit (we could
> > > have more exts, more axes for stuff like ts/nts builds, enable debug
> > > builds, so memory leaks are also triggering the test failures, etc.),
> >
> > You're more than welcome :) I've planned to get to some of it next -
> > i.e. going through the list of exts and see which ones we can support o=
n
> > Travis - but any help would be great.
> >
> >
> > --
> > Stanislav Malyshev, Software Architect
> > SugarCRM: http://www.sugarcrm.com/
> > (408)454-6900 ext. 227
> >
>
> Morning Stas,
>
>         I got time to read through this morning properly, obviously, yes
> ...
> but I'm not sure about the options, maybe I missed some relevant
> conversation around it, not sure ... a few questions quick before I
> finish voting if you don't mind ...
>
>         Update test seems like questionable action, but see that you and
> mike
> voted for it, can I hear reasoning for that ?
>
>         I don't see when there would be a good reason to retain a broken
> commit
> for a week, isn't that going to just render all of this pointless if
> tests can fail for a week at a time ?
>
>         Wouldn't it be better if all changes were done on branches, so
> they can
> be reviewed and integrated before being merged at all ?
>
>         Maybe problematic for minor fixes but are they really the kind of
> thing
> that cause CI to bork ?
>
> Cheers
> Joe
>
>
I also agree that update test is a questionable option there.
If we are talking about fixing a bad test, then it is obvious, and
shouldn't be even put up for a vote, as naturally everybody including the
RMs are free to fix the broken tests.
So the only other reason listing that option is when somebody intentionally
or unintentionally changes the behavior of some code without updating the
tests.
If that happens without proper discussion or RFC, I would say that it would
be mandatory to revert such changes until consensus/approval of that can be
made.

tl;dr: fixing the wrong test is always an option, but to fix up the test to
cover a bad code change shouldn't be an option.

--=20
Ferenc Kov=C3=A1cs
@Tyr43l - http://tyrael.hu

--001a1133bbd0be912704f819b7b1--