Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:44514
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain macvicar.net from 97.107.131.220 cause and error)
Cc: Hannes Magnusson <hannes.magnusson@gmail.com>,
 PHP Development <internals@lists.php.net>,
 PHP QA <php-qa@lists.php.net>
Message-ID: <19A8B858-410D-497B-A040-72B71FB7BEB0@macvicar.net>
To: Mark Karpeles <mark@hell.ne.jp>
In-Reply-To: <1246069850.23605.12.camel@localhost>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Apple Message framework v935.3)
Date: Sat, 27 Jun 2009 04:02:16 +0100
References: <7f3ed2c30906261529w5103b4f6leb8dd9860aeca741@mail.gmail.com> <F7FFC613-AF9F-42E5-8094-A55EA22A81F4@macvicar.net> <1246069850.23605.12.camel@localhost>
Subject: Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] Do you need different OS/platform to test on? Let "me" know
From: scott@macvicar.net (Scott MacVicar)

On 27 Jun 2009, at 03:30, Mark Karpeles wrote:

> Hi,
>
> Le samedi 27 juin 2009 =E0 02:24 +0100, Scott MacVicar a =E9crit :
>> I think we should look at getting buildbot setup and deployed to as
>> many machines as humanly possible. It would at least give us a heads
>> up when we've written something that doesn't work.
>>
>> You can see the ones for Google Chromium at =
http://build.chromium.org/buildbot/waterfall/waterfall
>>
>> It just needs someone to spend the time writing the initial slave
>> deployment script, any volunteers? :)
>
> This looks like something that matches my subset of things I can do.
>
> I already had some ideas about this, and already done an interface =20
> that
> can display various test information on a same page.
>
> http://php.magicaltux.net/browse/ (lots of memleaks reported on 5.3 &
> HEAD, seems to be glibc-related and not real memleaks, and this is not
> up-to-date as I didn't run the tests for a while and didn't automate =20=

> it
> because it takes a lot of cpu)
>
> Of course we can also make something that looks like Google's =20
> waterfall,
> but I don't find it to be really easy to read.
>
> The good thing with the interface I made is the fact you can see the
> status for all modules directly, eg:
> http://php.magicaltux.net/browse/ext/
>
> I could, instead of providing branch columns provide instead arch
> columns, and let the user select what he want to see before accessing
> the interface (ie. compare all arch on branch PHP_5_3).
>
> Anyway that is just an idea, the good part is that most of the code =20=

> has
> been written, missing parts would be ability to remotely provide =20
> compile
> results (ie. run tests from a different arch) and make code looks =20
> better
> (I wrote this under influence of beer, the code is missing comments).
>
> Would also need a deployment script as suggested by Scott that will
> initialize a slave directly, saving the time required to install =20
> this on
> xxxxxxx comptuers.
>
>

I don't think I explained Buildbot properly, its more than just a test =20=

runner, its kind of glue to join together an applications various =20
building and testing results and capture it all and report back to the =20=

build master.

We'd just need to write a few dozen lines of configuration script that =20=

tells the slaves what to do.

They aren't that hard, it just requires someone to sit and write the =20
first version.

You write a bootstrapping master.cfg in python and it can then spawn =20
whatever other language it wants, in our case probably the freshly =20
built PHP ;-)

 =46rom there it can run tests and do whatever we wish with the results, =
=20
like grab all test failures and upload it to your test result viewer. =20=

If a build fails we can have it use twitter / irc to inform people =20
that a slave failed to build PHP for whatever reason.

It's also got various scheduling parts built in, so we can do =20
continuous regular builds to make sure its still compiling and daily =20
memory builds to check for new memory leaks.

Other cool features are notifying people when it broke via IRC, =20
twitter or mailing list. And optionally packaging the freshly built =20
tests and uploading them. We could hypothetically get our snapshots =20
built this way so we never release a broken snapshot?

The project URL is http://buildbot.net/trac and the vast documentation =20=

is at http://djmitche.github.com/buildbot/docs/0.7.10/

I might be able to sit down and give this a go on the plane next week.

Scott=