# PHP 5.4 Benchmarks - Thread: https://externals.io/message/57601 - Original archive: https://news-web.php.net/php.internals/57601 - Started by Andi Gutmans on 2012-02-01T18:08:54+00:00 - Messages: 16 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #57601 — Andi Gutmans — 2012-02-01T18:08:54+00:00 URL: https://externals.io/message/57601#57601 Hi, As many of you know we made huge improvements in PHP 5.4 memory usage (enabling higher concurrency) and runtime performance. We've done our own benchmarks but it'd be great to hear from others re: what you're seeing with your real-world apps. If you have any experiences to share please do. Benchmarking is not easy and can be misleading (config not being the same, compiler options, different version of byte code caches, etc..) so the more data points we get the better the info re: performance gains. Thanks, Andi --- ## #57602 — Pierre Joye — 2012-02-01T18:18:42+00:00 URL: https://externals.io/message/57601#57602 · In reply to: #57601 hi Andi, We do automatic perf tests as part of the CI. Some of the results can be seen here for example here: http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-20120119-5.3.9-5.4.0RC6.html for a 5.3.9 vs 5.4.0RC6 comparison. There are other in the same location. We are going to do add some more perf related tests later. Cheers, > Hi, > > As many of you know we made huge improvements in PHP 5.4 memory usage (enabling higher concurrency) and runtime performance. > We've done our own benchmarks but it'd be great to hear from others re: what you're seeing with your real-world apps. > > If you have any experiences to share please do. Benchmarking is not easy and can be misleading (config not being the same, compiler options, different version of byte code caches, etc..) so the more data points we get the better the info re: performance gains. > > Thanks, > Andi -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #57603 — Andi Gutmans — 2012-02-01T18:24:10+00:00 URL: https://externals.io/message/57601#57603 · In reply to: #57602 Cache results are of course important. Memory usage would be great too. Thanks!! Andi >-----Original Message----- >From: Pierre Joye [mailto:pierre.php@gmail.com] >Sent: Wednesday, February 01, 2012 10:19 AM >To: Andi Gutmans >Cc: internals@lists.php.net >Subject: Re: [PHP-DEV] PHP 5.4 Benchmarks > >hi Andi, > >We do automatic perf tests as part of the CI. Some of the results can be seen >here for example here: > >http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-20120119- >5.3.9-5.4.0RC6.html > >for a 5.3.9 vs 5.4.0RC6 comparison. There are other in the same location. > >We are going to do add some more perf related tests later. > >Cheers, > > > > […] >usage (enabling higher concurrency) and runtime performance. > > […] >what you're seeing with your real-world apps. > > […] >and can be misleading (config not being the same, compiler options, >different version of byte code caches, etc..) so the more data points we get >the better the info re: performance gains. > > […] > > > >-- >Pierre > >@pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #57604 — Pierre Joye — 2012-02-01T18:37:24+00:00 URL: https://externals.io/message/57601#57604 · In reply to: #57603 hi! I could add zend optimizer to the stack but we do not have a license (or is it free for the optimizer only?). APC should join the party soon, not sure yet if it is ready yet. We will run compat tests first. About the memory usage, yes, this is on my todos. I have a tool for that (on win) which has less than 1% impact on perf so we can compare all in one run. But that will have to wait post final. Cheers, > Cache results are of course important. Memory usage would be great too. > > Thanks!! > Andi > > > […] -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #57605 — Andi Gutmans — 2012-02-01T18:42:53+00:00 URL: https://externals.io/message/57601#57605 · In reply to: #57604 Optimizer+ is free and packaged in Zend Server Community Edition. Happy to get you and anyone else who wants to run benchmarks a copy out of cycle. We haven't put a preview up yet but have it working. Re: memory usage, I've seen some great results so definitely will be nice to see what others are seeing. Thanks, Andi >-----Original Message----- >From: Pierre Joye [mailto:pierre.php@gmail.com] >Sent: Wednesday, February 01, 2012 10:37 AM >To: Andi Gutmans >Cc: internals@lists.php.net >Subject: Re: [PHP-DEV] PHP 5.4 Benchmarks > >hi! > >I could add zend optimizer to the stack but we do not have a license (or is it >free for the optimizer only?). > >APC should join the party soon, not sure yet if it is ready yet. We will run >compat tests first. > >About the memory usage, yes, this is on my todos. I have a tool for that (on >win) which has less than 1% impact on perf so we can compare all in one run. >But that will have to wait post final. > >Cheers, > > > > […] >20120119 > > […] > > > >-- >Pierre > >@pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #57606 — johannes@schlueters.de — 2012-02-01T18:49:20+00:00 URL: https://externals.io/message/57601#57606 · In reply to: #57601 > We've done our own benchmarks but it'd be great to hear from others Can we also hear from you? :-) johannes who doesn't have proper benchmarks himself, but is interested in seeing them ;-) --- ## #57607 — Sean Coates — 2012-02-01T19:19:49+00:00 URL: https://externals.io/message/57601#57607 · In reply to: #57604 > APC should join the party soon, not sure yet if it is ready yet. We > will run compat tests first. I ran into a crashing blocker with APC on 5.4: https://bugs.php.net/bug.php?id=60845 If anyone has the bandwidth to look into that, I'd love to help out in any way I can. (Sorry for the thread hijack…) S --- ## #57608 — Andi Gutmans — 2012-02-01T19:32:16+00:00 URL: https://externals.io/message/57601#57608 · In reply to: #57606 Absolutely. Will gather those and send them over... Andi >-----Original Message----- >From: Johannes Schlüter [mailto:johannes@schlueters.de] >Sent: Wednesday, February 01, 2012 10:49 AM >To: Andi Gutmans >Cc: internals@lists.php.net >Subject: Re: [PHP-DEV] PHP 5.4 Benchmarks > > > > […] > >Can we also hear from you? :-) > >johannes > who doesn't have proper benchmarks himself, but is interested in > seeing them ;-) --- ## #57740 — Timm Friebe — 2012-02-05T18:26:19+00:00 URL: https://externals.io/message/57601#57740 · In reply to: #57601 Hi, > As many of you know we made huge improvements in PHP 5.4 memory usage > (enabling higher concurrency) and runtime performance. Here are the unit test results for https://github.com/xp-framework/xp-framework and https://github.com/xp-framework/xp-language after applying workarounds for two bugs in current SVN (see https://gist.github.com/1746945). I think one can most definitely say there are some great improvements to be seen. PHP 5.3.10 (Windows, Non-Thread-Safe) OK: 4593/4893 run (300 skipped), 4593 succeeded, 0 failed Memory used: 54871.11 kB (60676.78 kB peak) Time taken: 16.188 seconds OK: 1328/1338 run (10 skipped), 1328 succeeded, 0 failed Memory used: 25259.16 kB (25562.36 kB peak) Time taken: 4.571 seconds PHP 5.4.0RC8-DEV (Windows, Non-Thread-Safe) OK: 4594/4893 run (299 skipped), 4594 succeeded, 0 failed Memory used: 39910.35 kB (47000.82 kB peak) Time taken: 15.674 seconds OK: 1328/1338 run (10 skipped), 1328 succeeded, 0 failed Memory used: 16318.44 kB (16931.49 kB peak) Time taken: 3.612 seconds The memory usage is measured by `memory_get_usage()` / `memory_get_peak_usage()` - Timm --- ## #57744 — Jaroslav Hanslik — 2012-02-05T20:34:56+00:00 URL: https://externals.io/message/57601#57744 · In reply to: #57601 > > As many of you know we made huge improvements in PHP 5.4 memory usage (enabling higher concurrency) and runtime performance. > We've done our own benchmarks but it'd be great to hear from others re: what you're seeing with your real-world apps. > Zend Framework API documentation generated by ApiGen, http://apigen.org PHP 5.3.10 (VC9 x86 Thread Safe) - Total time: 777 seconds, used: 316 MB RAM PHP 5.4.0RC7 (VC9 x86 Thread Safe) - Total time: 452 seconds, used: 156 MB RAM Jaroslav Hanslik --- ## #57747 — Sebastian Bergmann — 2012-02-05T21:07:23+00:00 URL: https://externals.io/message/57601#57747 · In reply to: #57744 Am 05.02.2012 21:34, schrieb Jaroslav Hanslik: > PHP 5.3.10 - Total time: 777 seconds, used: 316 MB RAM > PHP 5.4.0RC7 - Total time: 452 seconds, used: 156 MB RAM phpDox, initial run for Zend Framework 2 PHP 5.4: Time: 54 seconds, Memory: 30.50Mb PHP 5.3: Time: 01:09, Memory: 42.25Mb phpDox, cached run for Zend Framework 2 PHP 5.4: Time: 14 seconds, Memory: 4.75Mb PHP 5.3: Time: 19 seconds, Memory: 7.50Mb -- Sebastian Bergmann Co-Founder and Principal Consultant http://sebastian-bergmann.de/ http://thePHP.cc/ --- ## #57752 — Benjamin Eberlei — 2012-02-05T22:38:55+00:00 URL: https://externals.io/message/57601#57752 · In reply to: #57747 You cannot use Travis CI to compare for runtime of php5.4 and 5.3 since the used boxes are propably either different and also under high load due to virtualization, but you can check for memory differences. And they are huge: Doctrine 2: 5.3: http://travis-ci.org/#!/doctrine/doctrine2/jobs/593850 85.5 MB 5.4: http://travis-ci.org/#!/doctrine/doctrine2/jobs/593851 50 MB Symfony 2: 5.3: http://travis-ci.org/#!/symfony/symfony/jobs/624578 143 MB 5.4: http://travis-ci.org/#!/symfony/symfony/jobs/624579 91 MB On Sun, Feb 5, 2012 at 10:07 PM, Sebastian Bergmann wrote: > Am 05.02.2012 21:34, schrieb Jaroslav Hanslik: > > […] > > phpDox, initial run for Zend Framework 2 > > PHP 5.4: Time: 54 seconds, Memory: 30.50Mb > PHP 5.3: Time: 01:09, Memory: 42.25Mb > > phpDox, cached run for Zend Framework 2 > > PHP 5.4: Time: 14 seconds, Memory: 4.75Mb > PHP 5.3: Time: 19 seconds, Memory: 7.50Mb > > -- > Sebastian Bergmann Co-Founder and Principal Consultant > http://sebastian-bergmann.de/ http://thePHP.cc/ --- ## #57754 — Christopher Jones — 2012-02-06T00:03:44+00:00 URL: https://externals.io/message/57601#57754 · In reply to: #57740 > Hi, > > > […] > > Here are the unit test results for https://github.com/xp-framework/xp-framework and https://github.com/xp-framework/xp-language after applying workarounds for two bugs in current SVN (see https://gist.github.com/1746945). I think one can most definitely say there are some great improvements to be seen. > > PHP 5.3.10 (Windows, Non-Thread-Safe) > > OK: 4593/4893 run (300 skipped), 4593 succeeded, 0 failed > Memory used: 54871.11 kB (60676.78 kB peak) > Time taken: 16.188 seconds > > OK: 1328/1338 run (10 skipped), 1328 succeeded, 0 failed > Memory used: 25259.16 kB (25562.36 kB peak) > Time taken: 4.571 seconds > > > PHP 5.4.0RC8-DEV (Windows, Non-Thread-Safe) > > OK: 4594/4893 run (299 skipped), 4594 succeeded, 0 failed > Memory used: 39910.35 kB (47000.82 kB peak) > Time taken: 15.674 seconds > > OK: 1328/1338 run (10 skipped), 1328 succeeded, 0 failed > Memory used: 16318.44 kB (16931.49 kB peak) > Time taken: 3.612 seconds > > > The memory usage is measured by `memory_get_usage()` / `memory_get_peak_usage()` > > - Timm Windows PHP 5.4 is built using profile-guided optimizations, but PHP 5.3 isn't. My reading of Pierre's own benchmarks are that this seems to be contributing significantly to the 5.4 gains on Windows. It makes Windows a poor platform to evaluate generic PHP 5.4 improvements. Chris -- Email: christopher.jones@oracle.com Tel: +1 650 506 8630 Blog: http://blogs.oracle.com/opal/ --- ## #57759 — Pierre Joye — 2012-02-06T07:12:29+00:00 URL: https://externals.io/message/57601#57759 · In reply to: #57754 On Mon, Feb 6, 2012 at 1:03 AM, Christopher Jones wrote: > Windows PHP 5.4 is built using profile-guided optimizations, but > PHP 5.3 isn't.  My reading of Pierre's own benchmarks are that this > seems to be contributing significantly to the 5.4 gains on Windows. > It makes Windows a poor platform to evaluate generic PHP 5.4 improvements. Not for memory usage improvements. Cheers, -- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org --- ## #57760 — Dmitry Stogov — 2012-02-06T07:52:51+00:00 URL: https://externals.io/message/57601#57760 · In reply to: #57759 Hi, I've just rerun some synthetic and real-life benchmarks. All the test were run on the same box (Linux, Core2 Duo 3GHz, 4GB RAM). 5.3 and 5.4 where configured and build with the same options and ran as FastCGI server with the same number of processes and set of extensions. PHP performance [sec] --------------------- 5.3 5.4 speedup bench.php 3.21 2.56 20% micro_bench.php 23.15 11.23 50% PHP + Zend OptimizerPlus performance [req/sec] ---------------------------------------------- 5.3 5.4 speedup blog 62.8 71.0 13% drupal 1074.3 1146.7 7% zend framework 102.9 124.6 21% hello 5955.4 7826.5 31% qdig 267.6 280.4 5% typo3 357.0 405.6 14% wordpress 119.8 130.8 9% xoops 78.4 93.0 19% scrum 96.3 117.4 22% PHP/O+ memory usage (memory_get_peak_usage) [KB] ------------------------------------------------ 5.3 5.4 improvement blog 1280 768 40% drupal 512 256 50% zend framework 7680 5888 23% hello 512 256 50% qdig 768 512 33% typo3 1536 1024 33% xoops 1536 1280 17% scrum 1792 1280 19% Thanks. Dmitry. --- ## #57780 — Larry Garfield — 2012-02-06T23:42:19+00:00 URL: https://externals.io/message/57601#57780 · In reply to: #57760 Yowza. If I were seeing that kind of memory change in just one project I'd say there was a bug and the code was just dying. :-) What sort of configuration were you using in each case? A 512 KB Drupal process seems, um, low, especially compared to the other projects listed there. (Drupal 7 is normally rather memory hungry.) --Larry Garfield > Hi, > > I've just rerun some synthetic and real-life benchmarks. > All the test were run on the same box (Linux, Core2 Duo 3GHz, 4GB RAM). > 5.3 and 5.4 where configured and build with the same options and ran as > FastCGI server with the same number of processes and set of extensions. > > PHP performance [sec] > --------------------- > 5.3 5.4 speedup > bench.php 3.21 2.56 20% > micro_bench.php 23.15 11.23 50% > > PHP + Zend OptimizerPlus performance [req/sec] > ---------------------------------------------- > 5.3 5.4 speedup > blog 62.8 71.0 13% > drupal 1074.3 1146.7 7% > zend framework 102.9 124.6 21% > hello 5955.4 7826.5 31% > qdig 267.6 280.4 5% > typo3 357.0 405.6 14% > wordpress 119.8 130.8 9% > xoops 78.4 93.0 19% > scrum 96.3 117.4 22% > > PHP/O+ memory usage (memory_get_peak_usage) [KB] > ------------------------------------------------ > 5.3 5.4 improvement > blog 1280 768 40% > drupal 512 256 50% > zend framework 7680 5888 23% > hello 512 256 50% > qdig 768 512 33% > typo3 1536 1024 33% > xoops 1536 1280 17% > scrum 1792 1280 19% > > Thanks. Dmitry.