Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:75351
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 74.125.82.47 as permitted sender)
Sender: Michael Wallner <mike.php.net@gmail.com>
Message-ID: <53BE5824.2020209@php.net>
Date: Thu, 10 Jul 2014 11:08:52 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0
MIME-Version: 1.0
To: Stas Malyshev <smalyshev@sugarcrm.com>
CC: Sara Golemon <pollita@php.net>, Nuno Lopes <nlopess@php.net>, 
 "internals@lists.php.net" <internals@lists.php.net>
References: <CALJZF3hXcsFiShTeOj5LjGEeUwO2TgiAiGT_OPJW3owjhQ-K5w@mail.gmail.com> <CAESVnVqwL-3-n-ZMxnmCJ=u9z0CO1Lm9ZmSuV62OTy_kajSJ3Q@mail.gmail.com> <05091D9021CD48208FCE220DE1CCC3CE@pc07654> <53BD9F71.1090401@sugarcrm.com>
In-Reply-To: <53BD9F71.1090401@sugarcrm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] Concurrency support for run-tests
From: mike@php.net (Michael Wallner)

On Wed 09 Jul 2014 22:00:49 CEST, Stas Malyshev wrote:
> Hi!
>
>> Having thought a bit about the subject, the heuristic I came up with was to
>> 1) do not run tests from the same extension in parallel, and 2) do not run
>> tests from certain extensions in parallel (e.g., mysql and pdo_mysql).

Heuristics are not reliable enough.

> How about making it a whitelist? For tests/extensions we know it's easy
> to paralellize (like ones not having external dependencies/servers/etc)
> simple script could whitelist a lot of them. For those more tricky, we
> could whitelist them gradually.
>

Whitelists are very limitating by nature, nobody will go and add tests 
to this whitelist, I think. I don't think a script is a viable way to 
whitelist tests (think about all those tests with with includes or 
config files, etc). Tests that fail due to concurrency will be easier 
noticed and added to a concurrency group or even better be fixed.

I added a CONCURRENCY_GROUP section, see
https://github.com/m6w6/php-src/compare/parallel-run-tests?expand=1

If the next test to execute is in the same CONCURRENCY_GROUP as one 
which is already/still running, it will be pushed back.

So tests with f.e. listening servers could use "port:9999" as 
CONCURRENCY_GROUP or mysql tests could use "mysql:dbname", etc. As 
example, I added the open_basedir tests to one group because they all  
tamper around with the same subdirectories.

I also added a shuffle($test_files) if $concurrency is greater than 2 
because that helps running the test suite concurrently actually. 
Anybody got an idea why the tests are sorted at all?

--
Regards,
Mike