Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:77207
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.216.175 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <541029AF.6080000@php.net>
References: <CALJZF3hXcsFiShTeOj5LjGEeUwO2TgiAiGT_OPJW3owjhQ-K5w@mail.gmail.com>
	<CAESVnVqwL-3-n-ZMxnmCJ=u9z0CO1Lm9ZmSuV62OTy_kajSJ3Q@mail.gmail.com>
	<05091D9021CD48208FCE220DE1CCC3CE@pc07654>
	<53BD9F71.1090401@sugarcrm.com>
	<541029AF.6080000@php.net>
Date: Sun, 14 Sep 2014 20:59:16 -0700
Message-ID: <CAMbn9ZHwvK2mo9ubnAF+TNiv9Oto_PpFDN9_E87Rs4wi3kXJow@mail.gmail.com>
To: Michael Wallner <mike@php.net>
Cc: Stas Malyshev <smalyshev@sugarcrm.com>, Nuno Lopes <nlopess@php.net>, 
	"internals@lists.php.net" <internals@lists.php.net>, Sara Golemon <pollita@php.net>
Content-Type: multipart/alternative; boundary=089e0149d2e89be05e050312a80c
Subject: Re: [PHP-DEV] Concurrency support for run-tests
From: themattficken@gmail.com (Matt Ficken)

--089e0149d2e89be05e050312a80c
Content-Type: text/plain; charset=UTF-8

When building PFTT, the PHP Full Test Tool, one goal was maximizing speed
so that we would be more likely to frequently Fully test PHP (Full-coverage
of many scenarios), and to do that I built it to run tests concurrently...

Concurrent/parallel execution can cause breaks that otherwise work with
normal serial execution, so definitely don't enable concurrency in
run-tests by default.

run-tests.php's two main benefits are that it supports running the huge
library of 15k+pecl PHPTs (compatiblity) and its a lightweight tool so that
anyone who just built PHP can run `make test` to test their build (whereas
PFTT Is a heavyweight tool).

In making/merging major changes (like concurrency) the most important thing
is to not break that compatibility and dependability:

Your patch will fail without the /proc filesystem(mac osx?). On Windows,
you can check the %NUMBER_OF_PROCESSORS% environment variable. For unices
that don't have /proc there may not be a single simple way to check.


For better performance, PFTT runs a pool of 3 to 4 processes per CPU, when
the processes finish quickly, it runs fewer and when they slow down (due to
pending IO ops), it runs more processes.  See:
http://git.php.net/?p=pftt2.git;a=blob;f=src/com/mostc/pftt/scenario/CLIScenario.java;h=30c81aabcb56698eebe4884d915298c92332b50d;hb=refs/heads/master#l61
and for other SAPI scenarios like Apache it runs 2 or 3 processes, see:
http://git.php.net/?p=pftt2.git;a=blob;f=src/com/mostc/pftt/scenario/ProductionWebServerScenario.java;h=d509bddda2b3756f05321ddc273600aeb38372be;hb=refs/heads/master


For more speed, rather than grouping all, fe open_basedir tests into a
single `Concurrency Group` its better to have them all operate in separate
temporary directories and database tables (then you could run 32 or 64
openbase dir tests at once).

The ext/standard/tests/file and ext/mysql* PHPTs take the longest to run
and mostly do IO operations so they benefit the most from maximizing
concurrency.

For databases, PFTT will create a new database (and for phpunit tests
populate the database) for each test runner thread, See:
http://git.php.net/?p=pftt2.git;a=blob;f=src/com/mostc/pftt/scenario/MySQLScenario.java;h=f768f58c4494cb2705a571ece22d533bfdc15192;hb=refs/heads/master#l164




On Wed, Sep 10, 2014 at 3:36 AM, Michael Wallner <mike@php.net> wrote:

> On 09/07/14 22:00, Stas Malyshev wrote:
> > Hi!
> >
> >> Having thought a bit about the subject, the heuristic I came up with
> was to
> >> 1) do not run tests from the same extension in parallel, and 2) do not
> run
> >> tests from certain extensions in parallel (e.g., mysql and pdo_mysql).
> >
> > How about making it a whitelist? For tests/extensions we know it's easy
> > to paralellize (like ones not having external dependencies/servers/etc)
> > simple script could whitelist a lot of them. For those more tricky, we
> > could whitelist them gradually.
> >
>
> So how about adding the concurrency switch, but leaving it disabled by
> default?
>
> --
> Regards,
> Mike
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

--089e0149d2e89be05e050312a80c--