Newsgroups: php.internals,php.qa
Path: news.php.net
Xref: news.php.net php.internals:60620 php.qa:66591
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.214.42 as permitted sender)
Message-ID: <4FB9EE1E.3050701@gmail.com>
Date: Mon, 21 May 2012 08:26:22 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:12.0) Gecko/20120428 Thunderbird/12.0.1
MIME-Version: 1.0
To: Nuno Lopes <nlopess@php.net>
CC: internals@lists.php.net, PHP QA <php-qa@lists.php.net>
References: <4FB4E844.2070305@gmail.com> <9AF025709ED6452F9E5A91834F82F788@PC07655>
In-Reply-To: <9AF025709ED6452F9E5A91834F82F788@PC07655>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-QA] Parallel run-tests
From: aparachic@gmail.com (zoe slattery)

On 21/05/2012 06:45, Nuno Lopes wrote:
> Hi Zoe,
>
> Thanks for undertaking this project!
>
> I just have a few concerns about this:
> - The speedup seems a bit low to me. Maybe with a higher number of 
> extensions enabled it will improve?..
I don't think it will improve with a higher number of extensions. It 
_may_ improve as a result of someone doing some work on the tests are 
scheduled - but I wouldn't count on it. Looking at improving performance 
is important and I wanted to get some confirmation that it is faster, I 
think we have that level of confirmation now. However, if I'm the only 
person working on this, further performance work will  come after 
complete implementation + debugging.
> - Is there any developer documentation available?  If I wanted to do a 
> change or a bug fix today, how could I do it?
Lots - https://wiki.php.net/qa/runtests. I have just updated the 
"Development" section which has instructions on how to build and test 
the code. The 'Documentation' section has not been updated recently but 
I think it is still valid.

> - Can it be packaged as a single drop-in file replacement for 
> run-tests.php? The deployment is very important to me.  IMHO, the 
> optimal solution would be to have a drop-in replacement for the 
> current script, so that most developers wouldn't even notice the 
> difference.
Yes - totally agree. I think I experimented with packaging it as a .phar 
(so it would be called run-tests.phar, not run-tests.php), but it's so 
long ago that I can't remember. I have added it to the to-do list 
(https://wiki.php.net/qa/runtests/todos).
> - From previous emails exchanged in this thread, it seems that this 
> new version requires a few extensions to run (gzip, soap??). This is 
> not acceptable. The script must be able to run with --disable-all. Of 
> course in that case the parallel version will fail.
No, it doesn't. That was a stupid bug and I fixed it last night :-)

Zoe
>
> Nuno
>
>
> -----Original Message----- From: zoe slattery
> Sent: Thursday, May 17, 2012 1:00 PM
> To: internals@lists.php.net ; PHP QA
> Subject: [PHP-QA] Parallel run-tests
>
> Hi
>
> Over the past couple of weeks I have updated the parallel run-tests
> (fixed a couple of minor bugs in the PHP code and the build.xml), it's
> now almost at the point where I could go ahead and implement the last
> pieces.  Here is a summary and a few questions:
>
> 1. In rebasing the code the the dev't stream I found a number of tests
> with non-standard sections. My code checks test case structure and
> objects to anything non-standard, the current run-tests.php mainly
> ignores this kind of thing. I fixed up about 15 of these tests (see
> #62022) already - I'll fix the rest if there are no objections - I will
> open another bug report first.
>
> 2. If there is agreement to use this code it would make sense to
> replace  the existing run-tests code with it, or rather,  it would make
> no sense to try and maintain both versions. The new code is OO PHP, it's
> in http://git.php.net/repository/phpruntests.git, is there any problem
> with it staying there long term and maybe copying a run-tests.phar into
> the PHP source directory? I have no idea what the right answer is,
> suggestions welcome.
>
> 3. I ran a couple of small tests on my dual core Mac yesterday. For a
> standard set of tests, the parallel code ran in 207 seconds, sequential
> in 293 seconds and the standard run-tests.php took 298 seconds. This is
> an improvement but I suspect we could still do better by looking at the
> scheduling algorithm.
> At the moment it's very simple, we just assemble a list of directories
> with tests in and hand them out to processors till everything is done.
> Being able to handle tests that must be run in sequence (mysql, mysqli)
> will mean making some changes to this. So, perhaps we give an explicit
> list to p1 and let the scheduler distribute the rest of the tests? Or
> maybe we should have a 'process map' for all tests for extensions that
> are build by default? Again, suggestions welcome.
>
> 4. REDIRECTTEST still needs to be implemented, I understand how it works
> and this isn't (afaict) a major issue.
>
> 5. Testing. I'm able to do basic testing on Mac OSX  and Linux. I really
> need access to an 8 way Linux system, or someone who has access and
> would be interested in looking at performance? Any volunteers? This is
> probably the most interesting part of the project :-)
>
> 6. Windows. I'm not in a position to do anything much with Windows
> except some very basic checks to make sure that the sequential version
> runs. The parallel code won't work because we used pcntl(), however I
> know that Stefan and George were keen to design the code so that a
> Windows solution could be implemented if anyone thought of one. If
> anyone wants to pick up this aspect I'd be happy to get them started.
>
> Zoe