# multi-threaded run-tests.php - Thread: https://externals.io/message/31244 - Original archive: https://news-web.php.net/php.internals/31244 - Started by Nuno Lopes on 2007-07-24T22:24:16+00:00 - Messages: 16 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #31244 — Nuno Lopes — 2007-07-24T22:24:16+00:00 URL: https://externals.io/message/31244#31244 Hi, Recently we have moved http://gcov.php.net to a new (and much faster) server. This allowed us to reduce the time to build all branches to "just" 2 days (compared with almost one week that would take previously). However I'm still not happy :P The server has 2x2 CPU cores and I would like to use them :) Currently I can speed-up the make process, because it can be parallelized (with -j xx). But the major time is still spent with run-tests.php. So, what I would like to propose is a multi-threaded version of run-tests.php. We could spawn x processes (configurable) and one of them would test y extensions (I think it's better to have each extension's tests tested serially or they may break, because they may use the same resources: BDs, files, etc..). Any comments? Thanks, Nuno --- ## #31255 — Richard Lynch — 2007-07-25T01:45:55+00:00 URL: https://externals.io/message/31244#31255 · In reply to: #31244 > Recently we have moved http://gcov.php.net to a new (and much faster) > server. This allowed us to reduce the time to build all branches to > "just" 2 > days (compared with almost one week that would take previously). > However I'm still not happy :P The server has 2x2 CPU cores and I > would like > to use them :) Currently I can speed-up the make process, because it > can be > parallelized (with -j xx). But the major time is still spent with > run-tests.php. > > So, what I would like to propose is a multi-threaded version of > run-tests.php. We could spawn x processes (configurable) and one of > them > would test y extensions (I think it's better to have each extension's > tests > tested serially or they may break, because they may use the same > resources: > BDs, files, etc..). Sounds good to this naive reader. Set X high enough, and you might even find an multi-extension thread-safe issue we've been missing. :-v If I'm reading it right, for sure the pgsql extension tests are serial within the extension, as it re-uses the DB connection and presumes test db setup in early tests, so you're 100% right on the inability to parallelize that. -- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? --- ## #31257 — David Wang — 2007-07-25T02:37:13+00:00 URL: https://externals.io/message/31244#31257 · In reply to: #31255 Please. This would be a big help to me, since I'm running these tests many, many times a day as I develop the reference cycle collector. > > > […] > > Sounds good to this naive reader. > > Set X high enough, and you might even find an multi-extension > thread-safe issue we've been missing. :-v > > If I'm reading it right, for sure the pgsql extension tests are serial > within the extension, as it re-uses the DB connection and presumes > test db setup in early tests, so you're 100% right on the inability to > parallelize that. > > -- > Some people have a "gift" link here. > Know what I want? > I want you to buy a CD from some indie artist. > http://cdbaby.com/browse/from/lynch > Yeah, I get a buck. So? --- ## #31259 — Sebastian Bergmann — 2007-07-25T07:33:04+00:00 URL: https://externals.io/message/31244#31259 · In reply to: #31244 Nuno Lopes wrote: > Any comments? It's on the TODO for the run-tests.php rewrite. I think I/we should start a wiki for this somewhere. Lukas? -- Sebastian Bergmann http://sebastian-bergmann.de/ GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69 --- ## #31261 — Antony Dovgal — 2007-07-25T08:19:27+00:00 URL: https://externals.io/message/31244#31261 · In reply to: #31259 > Nuno Lopes wrote: > > […] > > It's on the TODO for the run-tests.php rewrite. I think I/we should > start a wiki for this somewhere. Lukas? Good idea, let's gather all the idea there. -- Wbr, Antony Dovgal --- ## #31265 — Nuno Lopes — 2007-07-25T15:37:25+00:00 URL: https://externals.io/message/31244#31265 · In reply to: #31244 So here it is my first attempt: http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt WARNING: The patch is ugly and the output of the script isn't the prettiest. Anyway, for automated testing seems to be good enough. Nuno P.S.: after this little hack, I also start to feel that run-tests.php should be rewritten :P ""Nuno Lopes"" wrote: > Hi, > > Recently we have moved http://gcov.php.net to a new (and much faster) > server. This allowed us to reduce the time to build all branches to "just" > 2 days (compared with almost one week that would take previously). > However I'm still not happy :P The server has 2x2 CPU cores and I would > like to use them :) Currently I can speed-up the make process, because it > can be parallelized (with -j xx). But the major time is still spent with > run-tests.php. > > So, what I would like to propose is a multi-threaded version of > run-tests.php. We could spawn x processes (configurable) and one of them > would test y extensions (I think it's better to have each extension's > tests tested serially or they may break, because they may use the same > resources: BDs, files, etc..). > > Any comments? > > Thanks, > Nuno --- ## #31266 — Richard Quadling — 2007-07-25T15:39:52+00:00 URL: https://externals.io/message/31244#31266 · In reply to: #31265 > So here it is my first attempt: > http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt > WARNING: The patch is ugly and the output of the script isn't the prettiest. > > Anyway, for automated testing seems to be good enough. > > Nuno > > P.S.: after this little hack, I also start to feel that run-tests.php should > be rewritten :P > > > ""Nuno Lopes"" wrote: > > […] Hey Nuno, you should stop looking at things. PhD and now run-tests. Soon you'll be taking a look at the php-src and thinking, "pah, look what those idiots have done, time for a rewrite!". :-) -- ----- Richard Quadling Zend Certified Engineer : http://zend.com/zce.php?c=ZEND002498&r=213474731 "Standing on the shoulders of some very clever giants!" --- ## #31268 — Antony Dovgal — 2007-07-25T15:48:16+00:00 URL: https://externals.io/message/31244#31268 · In reply to: #31265 > So here it is my first attempt: > http://gcov.php.net/~nlopess/multi_threaded_run_tests.txt > WARNING: The patch is ugly and the output of the script isn't the prettiest. > > Anyway, for automated testing seems to be good enough. > > Nuno > > P.S.: after this little hack, I also start to feel that run-tests.php should > be rewritten :P Unfortunately, this is 100% truth, Sebastian have had some ideas about it. -- Wbr, Antony Dovgal --- ## #31270 — Andrey Hristov — 2007-07-25T17:38:02+00:00 URL: https://externals.io/message/31244#31270 · In reply to: #31268 Hi Tony, Antony Dovgal wrote: > > > […] > > Unfortunately, this is 100% truth, Sebastian have had some ideas about it. > I also have a hack, that makes it possible to re-run a test few times. I call it iteration. For every iteration there is $_ENV['TEST_ITERATION']. Based on which different consts are defined by the pre-include script. I am using it for testing mysqlnd against a bunch of MySQL Server, the minimalistic configuration is doing 3 iterations against 4.1, 5.0 and 5.1 Server. Not sure that other extensions could benefit from it, but at least the DB connectivity ones..like testing OCI8.... Regards, Andrey --- ## #31271 — David Coallier — 2007-07-25T17:59:32+00:00 URL: https://externals.io/message/31244#31271 · In reply to: #31270 > Hi Tony, > Antony Dovgal wrote: > > […] > > I also have a hack, that makes it possible to re-run a test few times. I > call it iteration. For every iteration there is $_ENV['TEST_ITERATION']. > Based on which different consts are defined by the pre-include script. I > am using it for testing mysqlnd against a bunch of MySQL Server, the > minimalistic configuration is doing 3 iterations against 4.1, 5.0 and > 5.1 Server. Not sure that other extensions could benefit from it, but at > least the DB connectivity ones..like testing OCI8.... > Where ? :) I am interested in seeing it. > Regards, > Andrey > > -- > > > > -- David Coallier, Founder & Software Architect, Agora Production (http://agoraproduction.com) 51.42.06.70.18 --- ## #31288 — Richard Lynch — 2007-07-27T03:54:15+00:00 URL: https://externals.io/message/31244#31288 · In reply to: #31266 > Hey Nuno, you should stop looking at things. PhD and now run-tests. > Soon you'll be taking a look at the php-src and thinking, "pah, look > what those idiots have done, time for a rewrite!". :-) Deja-vu: Isn't that exactly what Ze'ev and Andi did for PHP4? :-) Only you have to convince one of your professors that re-writing it instead of doing all your homework is worth a full course credit like they did. :-) Apologies in advance to Ze'ev and Andi for slightly distorting history, I think. (But not much, I hope) -- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? --- ## #31289 — Richard Lynch — 2007-07-27T03:56:10+00:00 URL: https://externals.io/message/31244#31289 · In reply to: #31270 > Hi Tony, > Antony Dovgal wrote: > > […] > > I also have a hack, that makes it possible to re-run a test few times. > I > call it iteration. For every iteration there is > $_ENV['TEST_ITERATION']. > Based on which different consts are defined by the pre-include script. > I > am using it for testing mysqlnd against a bunch of MySQL Server, the > minimalistic configuration is doing 3 iterations against 4.1, 5.0 and > 5.1 Server. Not sure that other extensions could benefit from it, but > at > least the DB connectivity ones..like testing OCI8.... FWIW: I spent about 10 to 20 minutes trying to figure out if it was possible to easily run just a subset of tests and failed. I dunno if that indicates that it's a feature to be added, one to be better documented, or that I'm just being stupid (again), but there it is. :-) -- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? --- ## #31291 — Rasmus Lerdorf — 2007-07-27T03:59:43+00:00 URL: https://externals.io/message/31244#31291 · In reply to: #31289 Richard Lynch wrote: > > > […] > > FWIW: > > I spent about 10 to 20 minutes trying to figure out if it was possible > to easily run just a subset of tests and failed. > > I dunno if that indicates that it's a feature to be added, one to be > better documented, or that I'm just being stupid (again), but there it > is. :-) make test TESTS=ext/foo Or, add this little shell script which I call phpt: #!/bin/bash TEST_PHP_EXECUTABLE=/usr/local/bin/php php /usr/local/lib/php/build/run-tests.php $1 Then cd into whatever directory has your tests and type: phpt . -Rasmus -Rasmus --- ## #31293 — Sebastian Bergmann — 2007-07-27T07:46:39+00:00 URL: https://externals.io/message/31244#31293 · In reply to: #31291 Rasmus Lerdorf wrote: > make test TESTS=ext/foo > > Or, add this little shell script which I call phpt: > > #!/bin/bash > TEST_PHP_EXECUTABLE=/usr/local/bin/php > php /usr/local/lib/php/build/run-tests.php $1 > > Then cd into whatever directory has your tests and type: > > phpt . Yeah, letting make take care of this sounds like the right thing to do. It should also solve Ulf's problem of not running the tests for the various MySQL extensions at the same time somehow. -- Sebastian Bergmann http://sebastian-bergmann.de/ GnuPG Key: 0xB85B5D69 / 27A7 2B14 09E4 98CD 6277 0E5B 6867 C514 B85B 5D69 --- ## #31326 — Marcus Boerger — 2007-07-30T18:40:30+00:00 URL: https://externals.io/message/31244#31326 · In reply to: #31289 Hello Richard, erm, php run-tests.php -h or --help was to hard? Friday, July 27, 2007, 5:56:10 AM, you wrote: > > > […] > FWIW: > I spent about 10 to 20 minutes trying to figure out if it was possible > to easily run just a subset of tests and failed. > I dunno if that indicates that it's a feature to be added, one to be > better documented, or that I'm just being stupid (again), but there it > is. > -- > Some people have a "gift" link here. > Know what I want? > I want you to buy a CD from some indie artist. > http://cdbaby.com/browse/from/lynch > Yeah, I get a buck. So? Best regards, Marcus --- ## #31339 — Marcus Boerger — 2007-07-31T14:50:23+00:00 URL: https://externals.io/message/31244#31339 Hello Richard, that -h issue is actually understandable. If run-tests-pgp was exectuable you would more likely be tempted to try -h. Maybe that means we should make it so by adding a hash bang line or by adding a short shell script that will do so call run-tests.pgp with the correct php version. marcus Tuesday, July 31, 2007, 5:40:53 AM, you wrote: > I did try variations on "make test --help" but that just got me make > help... > I would have expected php xxx.php -h to simply print the help for PHP > and exit and didn't think to try that. > Sorry! > > > […] -- Best regards, Marcus mailto:mail@marcus-boerger.de