# Benchmarking ... - Thread: https://externals.io/message/55872 - Original archive: https://news-web.php.net/php.internals/55872 - Started by Lester Caine on 2011-10-19T10:35:59+00:00 - Messages: 15 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #55872 — Lester Caine — 2011-10-19T10:35:59+00:00 URL: https://externals.io/message/55872#55872 OK ... I've pulled out all the old benchmarking stuff, but being several years old, they seem to be a little 'light' in what they are doing. I'm getting 34mS test times on things that were probably a second or more when I first used them. I've got two Linux machines running public accessible websites, and three Windows boxes which I can test internally, with various versions of 32 and 64 bit windows so I want a benchmark that will do a reasonable job of testing PHP5.3+ installations and which other people can compare against. So what are people currently using for benchmarking performance? -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php --- ## #55874 — Bostjan Skufca — 2011-10-20T04:52:30+00:00 URL: https://externals.io/message/55872#55874 · In reply to: #55872 Are you talking about general server benchmarks, HTTP server benchmarks or PHP-only specific benchmarks (make test?)? b. > OK ... I've pulled out all the old benchmarking stuff, but being several > years old, they seem to be a little 'light' in what they are doing. I'm > getting 34mS test times on things that were probably a second or more when I > first used them. > > I've got two Linux machines running public accessible websites, and three > Windows boxes which I can test internally, with various versions of 32 and > 64 bit windows so I want a benchmark that will do a reasonable job of > testing PHP5.3+ installations and which other people can compare against. > > So what are people currently using for benchmarking performance? > > -- > Lester Caine - G8HFL > ----------------------------- > Contact - http://lsces.co.uk/wiki/?page=**contact > L.S.Caine Electronic Services - http://lsces.co.uk > EnquirySolve - http://enquirysolve.com/ > Model Engineers Digital Workshop - http://medw.co.uk// > Firebird - http://www.firebirdsql.org/**index.php Are you talking about general server benchmarks, HTTP server benchmarks or > PHP-only specific benchmarks (make test?)? General full stack testing ... OS agnostic so we can compare OS's The statement has been made that 32bit PHP is faster than 64bit on windows, and that is what I am currently trying to benchmark. There is 'ab' for testing the Apache installation, and that is giving me a good 10% faster performance for the 64bit VC9 build of Apache 2.2.21 over the 32bit one. php_benchmark seems a little light nowadays since it's difficult to compare performance when it only takes around 34mS to run, I've scaled it to take 10 times longer so I can now actually see a difference, but I'm not seeing a recordable difference between setups. I need to test the database connection as well, and an ADOdb test set will also test PHP performance, but the current benchmark again is only taking 0.1 seconds to complete, so a more substantial one is needed. > > > > […] -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php --- ## #55878 — Lester Caine — 2011-10-20T11:09:15+00:00 URL: https://externals.io/message/55872#55878 · In reply to: #55872 Lester Caine wrote: > So what are people currently using for benchmarking performance? Anybody using the Phoronix Test Suite? Beibg PHP based, and running across all platforms it looks like it should be useful, but it's not 'run out of the box' on my suse system, and I'm not sure about sorting the windows configuration at the moment, so I wonder if it's worth the time to get it working? -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php --- ## #55879 — Mario Brandt — 2011-10-20T20:21:41+00:00 URL: https://externals.io/message/55872#55879 · In reply to: #55876 Hi Lester, you can also run pure command line with the micro bench script http://svn.php.net/viewvc/php/php-src/branches/PHP_5_4/Zend/micro_bench.php?view=co I already made some benchmarking 5.3.8 and 5.4.0beta2 results 5.3.8 http://pastebin.com/raw.php?i=52s1kzMa results 5.4 http://pastebin.com/raw.php?i=1AU10Adf Hardware CPU: Intel T4200 @2GHz RAM: 4GB Best Mario > Bostjan Skufca wrote: > > […] > > General full stack testing ... OS agnostic so we can compare OS's > > The statement has been made that 32bit PHP is faster than 64bit on windows, > and that is what I am currently trying to benchmark. --- ## #55881 — Lester Caine — 2011-10-21T11:00:12+00:00 URL: https://externals.io/message/55872#55881 · In reply to: #55879 Mario Brandt wrote: > I already made some benchmarking 5.3.8 and 5.4.0beta2 OK running it via Apache ;) ( '\n' -> '
' ) Machine is a Dual Core AMD64 laptop (2.1GHz with 1.1GHz memory) On 64bit Windows 7 Home 32bit Apache + 32bit PHP http://pastebin.com/raw.php?i=MS6H4A5g 166.947 Seconds 64bit Apache + 64bit PHP http://pastebin.com/raw.php?i=4RtRyGez 148.839 Seconds So I am happy that the 64bit setup is a good 10% faster, and I've not started using any heavy 64bit stuff like the database But the machine is Dual Boot so on Linux SUSE11.4 http://pastebin.com/raw.php?i=U1WRQzNt 41.688 Seconds So I suppose the question I have to ask is what the f**k am I doing wrong on the windows setup? I've always known that linux was faster, but 4 times faster on the same hardware? My main development machine is giving 27.750 seconds which is even nicer, -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php --- ## #55882 — Reindl Harald — 2011-10-21T11:07:03+00:00 URL: https://externals.io/message/55872#55882 · In reply to: #55881 Am 21.10.2011 13:00, schrieb Lester Caine: > So I suppose the question I have to ask is what the f**k am I doing wrong on the windows setup? > I've always known that linux was faster, but 4 times faster on the same hardware? My main development > machine is giving 27.750 seconds which is even nicer, you do nothing wrong php on windows is horrible slow iterate a directory with 40.000 files on linux 3 seconds after cold boot on windows 3 minutes even after second run and even in a virtual machine on a linux host while in a linux virtual machine on the same host is no performance differenece between vm/bare-metal --- ## #55883 — Stas Malyshev — 2011-10-21T17:10:22+00:00 URL: https://externals.io/message/55872#55883 · In reply to: #55881 Hi! > So I suppose the question I have to ask is what the f**k am I doing wrong on the > windows setup? I've always known that linux was faster, but 4 times faster on > the same hardware? My main development machine is giving 27.750 seconds which is > even nicer, CPU-bound ops should be roughly the same speed on Windows and Linux since the code is basically the same. If they're not I guess the best way would be to profile it. File ops on windows are significantly slower because Windows FS is slower than Linux one. The difference can also be in compiler options - whci optimizations are chosen, etc.. -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #55884 — Lester Caine — 2011-10-21T17:56:13+00:00 URL: https://externals.io/message/55872#55884 · In reply to: #55883 Stas Malyshev wrote: > > […] > > CPU-bound ops should be roughly the same speed on Windows and Linux since the > code is basically the same. If they're not I guess the best way would be to > profile it. File ops on windows are significantly slower because Windows FS is > slower than Linux one. > The difference can also be in compiler options - whci optimizations are chosen, > etc.. The x86 build of PHP is direct off the windows.php web site, and the Linux on is via SUSE package handler. When I get some time I will try a 'command line' run to avoid Apache and see if that is getting in the way. The slower spec Vista box is running a bit faster than on Windows 7, but I need to update to 5.3.8 on that ... again I'll do an x86 install before the x64 and benchmark both if only for my own information. That machine needs updating to W7, but I still have a remote site that 'needs' vista to access it's VPN software :( -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php --- ## #55885 — Stephen Zarkos — 2011-10-21T19:34:44+00:00 URL: https://externals.io/message/55872#55885 · In reply to: #55884 > Stas Malyshev wrote: > > […] > machine > > […] > > The x86 build of PHP is direct off the windows.php web site, and the Linux on > is via SUSE package handler. When I get some time I will try a 'command line' > run to avoid Apache and see if that is getting in the way. > > The slower spec Vista box is running a bit faster than on Windows 7, but I > need to update to 5.3.8 on that ... again I'll do an x86 install before the x64 > and benchmark both if only for my own information. That machine needs > updating to W7, but I still have a remote site that 'needs' vista to access it's > VPN software :( > Apache does tend to be slower on Windows. I think it would be worthwhile to test using the typical IIS/FastCGI configuration or CLI on Windows. For example, on my server I see a total ~44sec with Apache, and ~31sec using either IIS or CLI (W2K8R2, PHP 5.3.8 x86). If you do end up profiling this I would be very interested to see what you find. Steve --- ## #55887 — Richard Riley — 2011-10-21T20:03:48+00:00 URL: https://externals.io/message/55872#55887 · In reply to: #55885 Stephen Zarkos writes: > > […] > > Apache does tend to be slower on Windows. You have some figures to back that up? My experience is that on modern HW its about identical assuming there is spare memory in both instances and the HW is identical. --- ## #55888 — Lester Caine — 2011-10-21T20:29:02+00:00 URL: https://externals.io/message/55872#55888 · In reply to: #55885 Stephen Zarkos wrote: > > […] > Apache does tend to be slower on Windows. I think it would be worthwhile to test using the typical IIS/FastCGI configuration or CLI on Windows. For example, on my server I see a total ~44sec with Apache, and ~31sec using either IIS or CLI (W2K8R2, PHP 5.3.8 x86). If you do end up profiling this I would be very interested to see what you find. My own reasoning for using Apache is that the configuration is identical between the Linux boxes and the Windows ones. I don't want to be having to build different installations especially where we are running multiple site via Apache. That and we still have W2k or XP Home on many sites, so no IIS ;) My single core XP Home machine is giving 77 seconds for the benchmark, which is more in line with what I would expect, and running from the command line, it drops to 75 seconds, so Apache is not getting in the way too much. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php --- ## #55889 — Stephen Zarkos — 2011-10-21T20:39:57+00:00 URL: https://externals.io/message/55872#55889 · In reply to: #55887 > -----Original Message----- > From: Richard Riley [mailto:rileyrg@googlemail.com] > Sent: Friday, October 21, 2011 1:04 PM > To: internals@lists.php.net > Subject: Re: [PHP-DEV] Benchmarking ... > > Stephen Zarkos writes: > > > […] > > You have some figures to back that up? My experience is that on modern HW > its about identical assuming there is spare memory in both instances and the > HW is identical. > Yes, I do :) Although the tests we've done are a bit different from this sort of benchmark. We typically do more load testing, where we have one or more load agent(s) each creating multiple virtual clients accessing one or more PHP pages on another physical server. If you like I can share with you directly - unless you want me to spam the list with spreadsheets ;) Steve --- ## #55892 — Lester Caine — 2011-10-21T21:42:43+00:00 URL: https://externals.io/message/55872#55892 · In reply to: #55889 Stephen Zarkos wrote: > Yes, I do:) Although the tests we've done are a bit different from this sort of benchmark. We typically do more load testing, where we have one or more load agent(s) each creating multiple virtual clients accessing one or more PHP pages on another physical server. I'm trying to factualize this and come up with a benchmark that I can rely on to check performance. The initial claim was that there was no advantage using 64bit builds, but I've re-run everything on the Vista box, and this is giving me 70 seconds on the x86 Apache/PHP but 60 seconds with the x64 build. This is confirming what I thought myself, and I would expect around 40 seconds on a 64 bit linux on the same hardware. I will try a live CD just to confirm that. This would suggest that something else is going on with the Windows7 setup for it to be 4 times slower? At least what all this exercise is doing is confirming nicely that my switch from Windows to Linux on those council sites that are now allowing it IS improving performance. And since this normally happens when we are replacing a machine, the new hardware is faster as well. -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk// Firebird - http://www.firebirdsql.org/index.php --- ## #55893 — William A. Rowe Jr. — 2011-10-21T23:26:25+00:00 URL: https://externals.io/message/55872#55893 · In reply to: #55892 > Stephen Zarkos wrote: > > […] > > I'm trying to factualize this and come up with a benchmark that I can rely on to check > performance. The initial claim was that there was no advantage using 64bit builds, but > I've re-run everything on the Vista box, and this is giving me 70 seconds on the x86 > Apache/PHP but 60 seconds with the x64 build. This is confirming what I thought myself, > and I would expect around 40 seconds on a 64 bit linux on the same hardware. I will try a > live CD just to confirm that. > > This would suggest that something else is going on with the Windows7 setup for it to be 4 > times slower? 4x sounds excessive, but in fact TS code is a significant performance penalty for windows, the only place where thread safety is "supported". I'd challenge you to do a non-TS build on windows and set up an fcgid pool of php workers and compare -that- to linux. I'd expect very nearly identical performance, once you factor out kernel-specific differences in the link-loader/resolver. Those are likely two very different beasts.