Newsgroups: php.internals,php.qa
Path: news.php.net
Xref: news.php.net php.internals:56057 php.qa:65954
Return-Path: <tyra3l@gmail.com>
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Delivered-To: mailing list internals@lists.php.net
Received: (qmail 40337 invoked from network); 3 Nov 2011 22:47:12 -0000
Received: from unknown (HELO lists.php.net) (127.0.0.1)
  by localhost with SMTP; 3 Nov 2011 22:47:12 -0000
Authentication-Results: pb1.pair.com smtp.mail=tyra3l@gmail.com; spf=pass; sender-id=pass
Authentication-Results: pb1.pair.com header.from=tyra3l@gmail.com; sender-id=pass
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.160.170 as permitted sender)
X-PHP-List-Original-Sender: tyra3l@gmail.com
X-Host-Fingerprint: 209.85.160.170 mail-gy0-f170.google.com  
Received: from [209.85.160.170] ([209.85.160.170:41574] helo=mail-gy0-f170.google.com)
	by pb1.pair.com (ecelerity 2.1.1.9-wez r(12769M)) with ESMTP
	id 7F/8F-50864-EE913BE4 for <internals@lists.php.net>; Thu, 03 Nov 2011 17:47:11 -0500
Received: by gyg13 with SMTP id 13so2018380gyg.29
        for <multiple recipients>; Thu, 03 Nov 2011 15:47:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        bh=Z2WEANz95usXtcAJghy0JvpTysAjnK8YALspGgLvZiI=;
        b=t0F625xxpb2+URdVkc7ns1uvW8OZD9TWts40hg7LuHDo/r5OjwwsIe46LA09M2R+yT
         ta0FWvAnF+35DP5H7w1rzBcqmcieq2BYtczK4rQIMX363ULS2h15AibJyI28SFu7cahv
         1YpkfuqMEwkKxYRK0zfcOpGEUWJUIUMiLXJRw=
MIME-Version: 1.0
Received: by 10.236.161.4 with SMTP id v4mr14956759yhk.89.1320360427417; Thu,
 03 Nov 2011 15:47:07 -0700 (PDT)
Received: by 10.146.71.16 with HTTP; Thu, 3 Nov 2011 15:47:07 -0700 (PDT)
In-Reply-To: <4EB316C2.3050000@lerdorf.com>
References: <4E66C906.7060402@sugarcrm.com>
	<CAH-PCH7fTzkrLWEqvJ++0C_KuFADqNaF3CK9oJh-J8AjL68A3w@mail.gmail.com>
	<CADNQb0XM1+Xn9peyv7uG3wZUz7WBNqOhcyyVmZKTADNXucfodw@mail.gmail.com>
	<CAH-PCH7_SKFckOYQt7vSJr063M9RA_zJGTGVc1Q+xjgW=hXu2Q@mail.gmail.com>
	<CAH-PCH5XdejMmO1d=LzupudsEQxT5rTw3ewPAKvcg9Pu7J9gGA@mail.gmail.com>
	<CAFst3WqTzSExTUK-KUoPgvL7djfr3A4_sd4GgmLJvakcpWy+_w@mail.gmail.com>
	<CAH-PCH5fT4nknjyjqN4=Shp0dPiadZGg0mafOaX93qeDTgiSGQ@mail.gmail.com>
	<F59ECA58-A69F-4DEE-8A47-41E3E71D2B9C@stefan-marr.de>
	<CAH-PCH4EU5V02kwQ4O_Yrd5qhudjqaztKxjM=m7GnrCyh89wAA@mail.gmail.com>
	<02FCF0AC-207E-4783-B590-E1F8150BD0E6@stefan-marr.de>
	<CAFst3WpbPKufEtJ60oCanz4dyCvG0HS-Fkie3E8ofv+=L4QDrw@mail.gmail.com>
	<CAH-PCH5J=m-9d-JiT3khrWQJTiRyYg-NSuzfUBSbBO=M2uZutg@mail.gmail.com>
	<CA+5g0SKjzshJdrKhcLkyYc3=wE830aPoSkHZWXTjvuU6rvfftA@mail.gmail.com>
	<CAH-PCH5d-2Dy03LC0aRy0dv6W2VWTxj7+ME9wqSko3qpYG1k1Q@mail.gmail.com>
	<4EB316C2.3050000@lerdorf.com>
Date: Thu, 3 Nov 2011 23:47:07 +0100
Message-ID: <CAH-PCH4sWeHvWKiuEVjrCChgKFt6gHVxgPzcEwfbRY03TAuxnA@mail.gmail.com>
To: Rasmus Lerdorf <rasmus@lerdorf.com>
Cc: Felipe Pena <felipensp@gmail.com>, Klaus Silveira <contato@klaussilveira.com>, 
	Stefan Marr <php@stefan-marr.de>, PHP Internals <internals@lists.php.net>, 
	PHP QA <php-qa@lists.php.net>
Content-Type: multipart/alternative; boundary=20cf303f67b8406f9104b0dc5e1a
Subject: Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] CI for 5.4
From: tyra3l@gmail.com (Ferenc Kovacs)

--20cf303f67b8406f9104b0dc5e1a
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Thu, Nov 3, 2011 at 11:33 PM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:

> On 11/03/2011 02:26 PM, Ferenc Kovacs wrote:
> > We could set up the email notification any time, we would just have to
> > agree where to send (internals, php-qa, creating a dedicated mailing
> list?)
> > and when to send (as I added to the RFC, having failing tests can be a
> > problem from the notification POV).
> > We have to find a good middle way, else we will either miss the
> > notifications, or we will be overwhelmed, so people will just ignore it=
.
>
> Ideally they should go to internals if you can get them to be non-noisy.
>

true


>
> As long as it only reports a test status change from the previous run,
> hopefully that wouldn't create much noise.


yeah, and that is a little bit of a problem, because as I mentioned by
default the jenkins way is that your tests passes:
http://jenkins.361315.n4.nabble.com/Test-failure-baseline-td386476.html
so we have three option as far as I can see:
A, it will report every test failure, but we fix all of our tests so we are
cool. (my preference)
B, same as A, but until we fix all of the tests, we mark the failing test
as XFAIL, which won't be counted as failed test (I'm against it, as time
taught us that we are lazy, and XFAILs will be kept that way. :/)
C, Somebody(me) do some scripting, and our currently failing tests won't
fail the build, only make it unstable, but any other test failure will fail
the build.

I think I didn't explained in the RFC, because currently we don't use it,
but in jenkins there is a third result type for a build apart success and
failure: unstable.
Most plugins support this, as does the email notification plugin also, so
this should also work.

--=20
Ferenc Kov=C3=A1cs
@Tyr43l - http://tyrael.hu

--20cf303f67b8406f9104b0dc5e1a--