Newsgroups: php.internals,php.qa
Path: news.php.net
Xref: news.php.net php.internals:56155 php.qa:65976
Return-Path: <tyra3l@gmail.com>
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Delivered-To: mailing list internals@lists.php.net
Received: (qmail 36164 invoked from network); 8 Nov 2011 10:15:27 -0000
Received: from unknown (HELO lists.php.net) (127.0.0.1)
  by localhost with SMTP; 8 Nov 2011 10:15:27 -0000
Authentication-Results: pb1.pair.com header.from=tyra3l@gmail.com; sender-id=pass
Authentication-Results: pb1.pair.com smtp.mail=tyra3l@gmail.com; spf=pass; sender-id=pass
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.213.42 as permitted sender)
X-PHP-List-Original-Sender: tyra3l@gmail.com
X-Host-Fingerprint: 209.85.213.42 mail-yw0-f42.google.com  
Received: from [209.85.213.42] ([209.85.213.42:34940] helo=mail-yw0-f42.google.com)
	by pb1.pair.com (ecelerity 2.1.1.9-wez r(12769M)) with ESMTP
	id 78/40-34278-73109BE4 for <internals@lists.php.net>; Tue, 08 Nov 2011 05:15:24 -0500
Received: by ywb26 with SMTP id 26so399415ywb.29
        for <multiple recipients>; Tue, 08 Nov 2011 02:15:17 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        bh=JnbzglHwwnOVB6QHZ2rQudTUwQHd5P1RiuZOGsxgs3Q=;
        b=pmnVJ0PuI2+onDYWrhJcwyXYJeSLxG1UTl1xDG+3XwafmikSSXT+3VYI6Eye/OSdnP
         Q8kIC5TFhY3+Ecl1dethi1Qx/B4s4cSnZzsSVn/DJZG4p4dVAzsdpkW1KRUctzZKy1O1
         p0E//o/LC1gI+wcFiI7OaIOayroIQDbwkVxwQ=
MIME-Version: 1.0
Received: by 10.146.173.19 with SMTP id v19mr3039449yae.4.1320747317020; Tue,
 08 Nov 2011 02:15:17 -0800 (PST)
Received: by 10.146.71.16 with HTTP; Tue, 8 Nov 2011 02:15:16 -0800 (PST)
In-Reply-To: <CAH-PCH5pz8cJ7aXmVzbEUUUkoTGOvz3w+K2gYLngEcz5-af7MA@mail.gmail.com>
References: <4E66C906.7060402@sugarcrm.com>
	<CAH-PCH7fTzkrLWEqvJ++0C_KuFADqNaF3CK9oJh-J8AjL68A3w@mail.gmail.com>
	<CADNQb0XM1+Xn9peyv7uG3wZUz7WBNqOhcyyVmZKTADNXucfodw@mail.gmail.com>
	<CAH-PCH7_SKFckOYQt7vSJr063M9RA_zJGTGVc1Q+xjgW=hXu2Q@mail.gmail.com>
	<CAH-PCH5XdejMmO1d=LzupudsEQxT5rTw3ewPAKvcg9Pu7J9gGA@mail.gmail.com>
	<CAFst3WqTzSExTUK-KUoPgvL7djfr3A4_sd4GgmLJvakcpWy+_w@mail.gmail.com>
	<CAH-PCH5fT4nknjyjqN4=Shp0dPiadZGg0mafOaX93qeDTgiSGQ@mail.gmail.com>
	<F59ECA58-A69F-4DEE-8A47-41E3E71D2B9C@stefan-marr.de>
	<CAH-PCH4EU5V02kwQ4O_Yrd5qhudjqaztKxjM=m7GnrCyh89wAA@mail.gmail.com>
	<CAH-PCH4aRFZ5ggLyBqDR_aOn-GmoocjxH8C_GcSC6okb0_xopA@mail.gmail.com>
	<CAH-PCH5pz8cJ7aXmVzbEUUUkoTGOvz3w+K2gYLngEcz5-af7MA@mail.gmail.com>
Date: Tue, 8 Nov 2011 11:15:16 +0100
Message-ID: <CAH-PCH6bq50mawrwXdkZK=h29qJ2ZQf7JOonm1UD3sK=Qt8muA@mail.gmail.com>
To: PHP Internals <internals@lists.php.net>
Cc: PHP QA <php-qa@lists.php.net>
Content-Type: multipart/alternative; boundary=000e0cd34e26ab470004b136727f
Subject: Re: [PHP-DEV] CI for 5.4
From: tyra3l@gmail.com (Ferenc Kovacs)

--000e0cd34e26ab470004b136727f
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Mon, Nov 7, 2011 at 4:42 PM, Ferenc Kovacs <tyra3l@gmail.com> wrote:

>
>
> On Mon, Nov 7, 2011 at 11:14 AM, Ferenc Kovacs <tyra3l@gmail.com> wrote:
>
>>
>>
>> On Thu, Nov 3, 2011 at 8:15 PM, Ferenc Kovacs <tyra3l@gmail.com> wrote:
>>
>>>
>>>
>>> On Thu, Nov 3, 2011 at 7:43 PM, Stefan Marr <php@stefan-marr.de> wrote:
>>>
>>>> Hi Ferenc:
>>>>
>>>> On 03 Nov 2011, at 19:01, Ferenc Kovacs wrote:
>>>>
>>>> > Of course there are ways to improve the current setup, I listed thos=
e
>>>> ideas
>>>> > at https://wiki.php.net/rfc/jenkins#future_plans
>>>>
>>>> Very nice.
>>>> I don't know Jenkins, but would it be possible to mail the author of a
>>>> change in case it breaks something? I just had such a case, where my l=
ocal
>>>> setup was insufficient to catch it.
>>>>
>>>> Thanks a lot
>>>> Stefan
>>>>
>>>>
>>> It is, and as I mentioned in the RFC the email notification can be
>>> enabled when needed, I didn't wanted to do that before introducing the =
RFC.
>>> There is a slight problem however: if we don't build each and every
>>> revision (which can happen, if two commit happens before the build star=
ts,
>>> which can happen more easily if another build is in progress for that j=
ob,
>>> because then jenkins will just queue the build, not execute it right aw=
ay
>>> for obvious reasons), so there is a slight chance for "blaming" the wro=
ng
>>> person.
>>> The usual setup is that you have a mailing list/alias which always gets
>>> a mail about the build results (which can also be customized in detail,
>>> when to mail, etc.) and optionally you can notify the person whose comm=
it
>>> break the build.
>>>
>>> As Klaus pointed out, there are other possibilities for notification,
>>> currently we were using the irc plugin, so if you hang around on #php.q=
aon efnet, you could see the build results or ask phpcibot about the status
>>> of the builds, However I just noticed that there is no way to restrict =
who
>>> can start builds through the irc bot(see
>>> https://issues.jenkins-ci.org/browse/JENKINS-5931), so I turned it off.
>>>
>>> 20:13 -!- phpcibot [~PircBotX@80.82.121.132] has quit [Remote host
>>> closed the connection]
>>>
>>> :(
>>>
>>> --
>>> Ferenc Kov=C3=A1cs
>>> @Tyr43l - http://tyrael.hu
>>>
>>
>> I reported https://issues.jenkins-ci.org/browse/JENKINS-11606 on the
>> jenkins issue tracker, and they fixed it super fast, so phpcibot is back
>> again. \o/
>>
>>
>>
>> --
>> Ferenc Kov=C3=A1cs
>> @Tyr43l - http://tyrael.hu
>>
>
> Hi.
>
> I would like to set up a windows slave also, as I mentioned in the RFC.
> I was informed that oti1 is mostly idle, so if somebody could give me
> access there, I would set up a jenkins slave, and the neccessary
> dependencies there.
> Thanks!
>
> --
> Ferenc Kov=C3=A1cs
> @Tyr43l - http://tyrael.hu
>

Yesterday we were talking about enabling more extensions and adding pecl
into the mix:
http://www.mail-archive.com/internals@lists.php.net/msg54220.html
I think it would be a good idea to try to fix the currently failing tests.
http://ci.qa.php.net/job/php-src-5.3-matrix-tests/35/testReport/?
http://ci.qa.php.net/job/php-src-5.4-matrix-tests/88/testReport/?
http://ci.qa.php.net/job/php-src-trunk-matrix-tests/52/testReport/?

There is only three failing tests left on the debian slaves:

For 5.3
ext/phar/tests/phar_oo_005.phpt.Phar and RecursiveDirectoryIterator 0.01 20
ext/spl/tests/bug60082.phpt.Bug #60082 (100% CPU / when using references
with ArrayObject(&$ref))

For 5.4 and trunk
ext/standard/tests/general_functions/bug39322.phpt.Bug #39322
(proc_terminate() loosing process resource)

For the freebsd failures, some of them can be tackled from the test (the
locale related tests), but I see some things that need fixing in the code,
rather than in the tests:
https://bugs.php.net/bug.php?id=3D60186
https://bugs.php.net/bug.php?id=3D60222

I think I also come up with a solution for the xfail issue (where do we
count those):
I would add support to run-tests to be able only run the XFAILS, or skip
them.
This way I could run the testsuite twice(either adding another axis to the
matrix config, or duplicating the php-src-$version-matrix-tests jobs in
two), and we could get a distinct report about the failing tests and the
xfails.
Having an xfail would mean a test success(in the junit.xml), having an
xfail test pass would be a fail((in the junit.xml)) but wouldn't fail the
build, only make it unstable.
This would mean that if we have failing tests ( not xfail tests, but tests
which are expected to pass failing instead)
our php-src-$version-matrix-tests build would result in a failure (red
ball).
Having only passing xfails, and no other failure would make our
php-src-$version-matrix-tests would result in an unstable build (yellow
ball).
Having no xfail tests present, and everything else passing would result
in successful and stable build (blue ball).

--=20
Ferenc Kov=C3=A1cs
@Tyr43l - http://tyrael.hu

--000e0cd34e26ab470004b136727f--