Newsgroups: php.internals,php.qa
Path: news.php.net
Xref: news.php.net php.internals:60619 php.qa:66590
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: unknown (pb1.pair.com: domain php.net does not designate 212.55.154.23 as permitted sender)
Message-ID: <9AF025709ED6452F9E5A91834F82F788@PC07655>
To: "zoe slattery" <aparachic@gmail.com>,
	<internals@lists.php.net>,
	"PHP QA" <php-qa@lists.php.net>
References: <4FB4E844.2070305@gmail.com>
In-Reply-To: <4FB4E844.2070305@gmail.com>
Date: Mon, 21 May 2012 06:45:28 +0100
MIME-Version: 1.0
Content-Type: text/plain;
	format=flowed;
	charset="iso-8859-1";
	reply-type=response
Content-Transfer-Encoding: 7bit
Importance: Normal
Subject: Re: [PHP-QA] Parallel run-tests
From: nlopess@php.net ("Nuno Lopes")

Hi Zoe,

Thanks for undertaking this project!

I just have a few concerns about this:
- The speedup seems a bit low to me. Maybe with a higher number of 
extensions enabled it will improve?..
- Is there any developer documentation available?  If I wanted to do a 
change or a bug fix today, how could I do it?
- Can it be packaged as a single drop-in file replacement for run-tests.php? 
The deployment is very important to me.  IMHO, the optimal solution would be 
to have a drop-in replacement for the current script, so that most 
developers wouldn't even notice the difference.
- From previous emails exchanged in this thread, it seems that this new 
version requires a few extensions to run (gzip, soap??). This is not 
acceptable. The script must be able to run with --disable-all. Of course in 
that case the parallel version will fail.

Nuno


-----Original Message----- 
From: zoe slattery
Sent: Thursday, May 17, 2012 1:00 PM
To: internals@lists.php.net ; PHP QA
Subject: [PHP-QA] Parallel run-tests

Hi

Over the past couple of weeks I have updated the parallel run-tests
(fixed a couple of minor bugs in the PHP code and the build.xml), it's
now almost at the point where I could go ahead and implement the last
pieces.  Here is a summary and a few questions:

1. In rebasing the code the the dev't stream I found a number of tests
with non-standard sections. My code checks test case structure and
objects to anything non-standard, the current run-tests.php mainly
ignores this kind of thing. I fixed up about 15 of these tests (see
#62022) already - I'll fix the rest if there are no objections - I will
open another bug report first.

2. If there is agreement to use this code it would make sense to
replace  the existing run-tests code with it, or rather,  it would make
no sense to try and maintain both versions. The new code is OO PHP, it's
in http://git.php.net/repository/phpruntests.git, is there any problem
with it staying there long term and maybe copying a run-tests.phar into
the PHP source directory? I have no idea what the right answer is,
suggestions welcome.

3. I ran a couple of small tests on my dual core Mac yesterday. For a
standard set of tests, the parallel code ran in 207 seconds, sequential
in 293 seconds and the standard run-tests.php took 298 seconds. This is
an improvement but I suspect we could still do better by looking at the
scheduling algorithm.
At the moment it's very simple, we just assemble a list of directories
with tests in and hand them out to processors till everything is done.
Being able to handle tests that must be run in sequence (mysql, mysqli)
will mean making some changes to this. So, perhaps we give an explicit
list to p1 and let the scheduler distribute the rest of the tests? Or
maybe we should have a 'process map' for all tests for extensions that
are build by default? Again, suggestions welcome.

4. REDIRECTTEST still needs to be implemented, I understand how it works
and this isn't (afaict) a major issue.

5. Testing. I'm able to do basic testing on Mac OSX  and Linux. I really
need access to an 8 way Linux system, or someone who has access and
would be interested in looking at performance? Any volunteers? This is
probably the most interesting part of the project :-)

6. Windows. I'm not in a position to do anything much with Windows
except some very basic checks to make sure that the sequential version
runs. The parallel code won't work because we used pcntl(), however I
know that Stefan and George were keen to design the code so that a
Windows solution could be implemented if anyone thought of one. If
anyone wants to pick up this aspect I'd be happy to get them started.

Zoe