# multi-jobs run-tests.php - Thread: https://externals.io/message/47014 - Original archive: https://news-web.php.net/php.internals/47014 - Started by Raphael Geissert on 2010-02-22T01:11:46+00:00 - Messages: 6 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #47014 — Raphael Geissert — 2010-02-22T01:11:46+00:00 URL: https://externals.io/message/47014#47014 Hi, I've spent the last hours working on making run-tests.php be able to run tests in parallel. The main reason being the time it takes to run the whole testsuite even on multicore CPUs. Attached is the first set of changes needed. It uses pcntl's fork and sysvmsg to send some of the results back to the parent process. Main functionalities still work, but the summary doesn't include failed tests. I plan to move away from sysvmsg because with the current implementation killing the parent process can leave message queues open (nothing that can't be fixed by adding a proper exit handler) and sysvmsg is not enabled by default. Probably to something php://memory-based. To support all the features under multi-jobs mode I think the proper fix is to cleanup the code so that run_test() stores the state information via an interface. This would make the code cleaner and would let multi-jobs mode just change the state-storing interface. How should I handle this? just by committing the changes? first sending them here (to internals)? by contacting somebody else? Cheers, -- Raphael Geissert --- ## #47016 — Pierre Joye — 2010-02-22T08:20:27+00:00 URL: https://externals.io/message/47014#47016 · In reply to: #47014 hi, There is an on-going work on run-test (which began as part of last year GSoC). I would begin there instead of working on the current run-test.php. Parallel testing is part of the new features as well, in a portable way (like not only where pcntl is available). Thanks for your work, Cheers, -- Pierre > Hi, > > I've spent the last hours working on making run-tests.php be able to run > tests in parallel. The main reason being the time it takes to run the whole > testsuite even on multicore CPUs. > > Attached is the first set of changes needed. It uses pcntl's fork and > sysvmsg to send some of the results back to the parent process. > > Main functionalities still work, but the summary doesn't include failed > tests. > > I plan to move away from sysvmsg because with the current implementation > killing the parent process can leave message queues open (nothing that can't > be fixed by adding a proper exit handler) and sysvmsg is not enabled by > default. Probably to something php://memory-based. > > To support all the features under multi-jobs mode I think the proper fix is > to cleanup the code so that run_test() stores the state information via an > interface. This would make the code cleaner and would let multi-jobs mode > just change the state-storing interface. > > How should I handle this? just by committing the changes? first sending them > here (to internals)? by contacting somebody else? > > Cheers, > -- > Raphael Geissert > -- > > > -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #47017 — Pierre Joye — 2010-02-22T08:21:50+00:00 URL: https://externals.io/message/47014#47017 · In reply to: #47016 forgot to add the link to the repo: http://svn.php.net/viewvc/php/phpruntests/ > hi, > > There is an on-going work on run-test (which began as part of last > year GSoC). I would begin there instead of working on the current > run-test.php. Parallel testing is part of the new features as well, in > a portable way (like not only where pcntl is available). > > Thanks for your work, > > Cheers, > -- > Pierre > > > > […] > > > > -- > Pierre > > @pierrejoye | http://blog.thepimp.net | http://www.libgd.org > -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #47025 — Raphael Geissert — 2010-02-22T16:50:22+00:00 URL: https://externals.io/message/47014#47025 · In reply to: #47016 Hi Pierre, Pierre Joye wrote: > There is an on-going work on run-test (which began as part of last > year GSoC). I would begin there instead of working on the current > run-test.php. Parallel testing is part of the new features as well, in > a portable way (like not only where pcntl is available). > Ah, I see, great. From a quick look at the code it looks like it also uses pcntl. Is there any plan to switch to that other implementation? No work has been done in something like the threads extension, has there? Cheers, -- Raphael Geissert --- ## #47026 — Jess Portnoy — 2010-02-22T16:53:38+00:00 URL: https://externals.io/message/47014#47026 · In reply to: #47025 Hello all, While on the subject of run-tests.php, any plans for it to support web server SAPIs and not just CLI/CGI? I've looked at http://qa.php.net/projects.php and saw no such bullet but I thought asking anyhow won't hurt. Thanks, May the source be with you, Best regards, Jess Portnoy Raphael Geissert wrote: > Hi Pierre, > > Pierre Joye wrote: > > > […] > > Ah, I see, great. > From a quick look at the code it looks like it also uses pcntl. > > Is there any plan to switch to that other implementation? > > No work has been done in something like the threads extension, has there? > > Cheers, --- ## #47027 — johannes@php.net — 2010-02-22T17:12:07+00:00 URL: https://externals.io/message/47014#47027 · In reply to: #47025 > Ah, I see, great. > > […] > > Is there any plan to switch to that other implementation? > > No work has been done in something like the threads extension, has there? Why would you need threads? - Won't some async process control be enough? - The test manager doesn't have to be parallelized, it should just run multiple tests at once. And yes, I'm very interested I have a 64 way SPARC machine which currently needs 7 days for a gcov run which, in theory, might be reduced to 7/64 days ... Another interesting thing might be directory-wide skipif so we don't have to run two PHP interpreters per test. But when working on all kinds of things there one should look on the rewrite by Zoe (and others) which should be nicer for modification than the grown code of the existing run-tests.php johannes