Benchmarking ...

14 years ago by Lester Caine — view source — reply

unread

OK ... I've pulled out all the old benchmarking stuff, but being several years
old, they seem to be a little 'light' in what they are doing. I'm getting 34mS
test times on things that were probably a second or more when I first used them.

I've got two Linux machines running public accessible websites, and three
Windows boxes which I can test internally, with various versions of 32 and 64
bit windows so I want a benchmark that will do a reasonable job of testing
PHP5.3+ installations and which other people can compare against.

So what are people currently using for benchmarking performance?

--
Lester Caine - G8HFL

Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk//
Firebird - http://www.firebirdsql.org/index.php

14 years ago by Bostjan Skufca — view source — reply

unread

Are you talking about general server benchmarks, HTTP server benchmarks or
PHP-only specific benchmarks (make test?)?

OK ... I've pulled out all the old benchmarking stuff, but being several
years old, they seem to be a little 'light' in what they are doing. I'm
getting 34mS test times on things that were probably a second or more when I
first used them.

I've got two Linux machines running public accessible websites, and three
Windows boxes which I can test internally, with various versions of 32 and
64 bit windows so I want a benchmark that will do a reasonable job of
testing PHP5.3+ installations and which other people can compare against.

So what are people currently using for benchmarking performance?

--
Lester Caine - G8HFL

Contact - http://lsces.co.uk/wiki/?page=**contact http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk//
Firebird - http://www.firebirdsql.org/**index.php<http://www.firebirdsql.org/index.php

14 years ago by Lester Caine — view source — reply

unread

Bostjan Skufca wrote:

Are you talking about general server benchmarks, HTTP server benchmarks or
PHP-only specific benchmarks (make test?)?

General full stack testing ... OS agnostic so we can compare OS's

The statement has been made that 32bit PHP is faster than 64bit on windows, and
that is what I am currently trying to benchmark.

There is 'ab' for testing the Apache installation, and that is giving me a good
10% faster performance for the 64bit VC9 build of Apache 2.2.21 over the 32bit
one. php_benchmark seems a little light nowadays since it's difficult to compare
performance when it only takes around 34mS to run, I've scaled it to take 10
times longer so I can now actually see a difference, but I'm not seeing a
recordable difference between setups.

I need to test the database connection as well, and an ADOdb test set will also
test PHP performance, but the current benchmark again is only taking 0.1 seconds
to complete, so a more substantial one is needed.

OK ... I've pulled out all the old benchmarking stuff, but being several
years old, they seem to be a little 'light' in what they are doing. I'm
getting 34mS test times on things that were probably a second or more when I
first used them.

I've got two Linux machines running public accessible websites, and three
Windows boxes which I can test internally, with various versions of 32 and
64 bit windows so I want a benchmark that will do a reasonable job of
testing PHP5.3+ installations and which other people can compare against.

So what are people currently using for benchmarking performance?

--
Lester Caine - G8HFL

14 years ago by Mario Brandt — view source — reply

unread

Hi Lester,
you can also run pure command line with the micro bench script
http://svn.php.net/viewvc/php/php-src/branches/PHP_5_4/Zend/micro_bench.php?view=co

I already made some benchmarking 5.3.8 and 5.4.0beta2

results 5.3.8
http://pastebin.com/raw.php?i=52s1kzMa

results 5.4
http://pastebin.com/raw.php?i=1AU10Adf

Hardware
CPU: Intel T4200 @2GHz
RAM: 4GB

Best
Mario

Bostjan Skufca wrote:

Are you talking about general server benchmarks, HTTP server benchmarks or
PHP-only specific benchmarks (make test?)?

General full stack testing ... OS agnostic so we can compare OS's

The statement has been made that 32bit PHP is faster than 64bit on windows,
and that is what I am currently trying to benchmark.

14 years ago by Lester Caine — view source — reply

unread

Mario Brandt wrote:

I already made some benchmarking 5.3.8 and 5.4.0beta2

OK running it via Apache ;) ( '\n' -> '<br />' )

Machine is a Dual Core AMD64 laptop (2.1GHz with 1.1GHz memory)

On 64bit Windows 7 Home
32bit Apache + 32bit PHP
http://pastebin.com/raw.php?i=MS6H4A5g
166.947 Seconds

64bit Apache + 64bit PHP
http://pastebin.com/raw.php?i=4RtRyGez
148.839 Seconds

So I am happy that the 64bit setup is a good 10% faster, and I've not started
using any heavy 64bit stuff like the database

But the machine is Dual Boot so on Linux SUSE11.4
http://pastebin.com/raw.php?i=U1WRQzNt
41.688 Seconds

So I suppose the question I have to ask is what the f**k am I doing wrong on the
windows setup? I've always known that linux was faster, but 4 times faster on
the same hardware? My main development machine is giving 27.750 seconds which is
even nicer,

--
Lester Caine - G8HFL

14 years ago by Reindl Harald — view source — reply

unread

Am 21.10.2011 13:00, schrieb Lester Caine:

So I suppose the question I have to ask is what the f**k am I doing wrong on the windows setup?
I've always known that linux was faster, but 4 times faster on the same hardware? My main development
machine is giving 27.750 seconds which is even nicer,

you do nothing wrong
php on windows is horrible slow

iterate a directory with 40.000 files
on linux 3 seconds after cold boot

on windows 3 minutes even after second run and even in
a virtual machine on a linux host while in a linux
virtual machine on the same host is no performance
differenece between vm/bare-metal

14 years ago by Stas Malyshev — view source — reply

unread

Hi!

So I suppose the question I have to ask is what the f**k am I doing wrong on the
windows setup? I've always known that linux was faster, but 4 times faster on
the same hardware? My main development machine is giving 27.750 seconds which is
even nicer,

CPU-bound ops should be roughly the same speed on Windows and Linux
since the code is basically the same. If they're not I guess the best
way would be to profile it. File ops on windows are significantly slower
because Windows FS is slower than Linux one.
The difference can also be in compiler options - whci optimizations are
chosen, etc..

Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

14 years ago by Lester Caine — view source — reply

unread

Stas Malyshev wrote:

So I suppose the question I have to ask is what the f**k am I doing wrong on the
windows setup? I've always known that linux was faster, but 4 times faster on
the same hardware? My main development machine is giving 27.750 seconds which is
even nicer,

CPU-bound ops should be roughly the same speed on Windows and Linux since the
code is basically the same. If they're not I guess the best way would be to
profile it. File ops on windows are significantly slower because Windows FS is
slower than Linux one.
The difference can also be in compiler options - whci optimizations are chosen,
etc..

The x86 build of PHP is direct off the windows.php web site, and the Linux on is
via SUSE package handler. When I get some time I will try a 'command line' run
to avoid Apache and see if that is getting in the way.

The slower spec Vista box is running a bit faster than on Windows 7, but I need
to update to 5.3.8 on that ... again I'll do an x86 install before the x64 and
benchmark both if only for my own information. That machine needs updating to
W7, but I still have a remote site that 'needs' vista to access it's VPN software :(

--
Lester Caine - G8HFL

14 years ago by Stephen Zarkos — view source — reply

unread

Stas Malyshev wrote:

So I suppose the question I have to ask is what the f**k am I doing
wrong on the windows setup? I've always known that linux was faster,
but 4 times faster on the same hardware? My main development
machine
is giving 27.750 seconds which is even nicer,

CPU-bound ops should be roughly the same speed on Windows and Linux
since the code is basically the same. If they're not I guess the best
way would be to profile it. File ops on windows are significantly
slower because Windows FS is slower than Linux one.
The difference can also be in compiler options - whci optimizations
are chosen, etc..

The x86 build of PHP is direct off the windows.php web site, and the Linux on
is via SUSE package handler. When I get some time I will try a 'command line'
run to avoid Apache and see if that is getting in the way.

The slower spec Vista box is running a bit faster than on Windows 7, but I
need to update to 5.3.8 on that ... again I'll do an x86 install before the x64
and benchmark both if only for my own information. That machine needs
updating to W7, but I still have a remote site that 'needs' vista to access it's
VPN software :(

Apache does tend to be slower on Windows. I think it would be worthwhile to test using the typical IIS/FastCGI configuration or CLI on Windows. For example, on my server I see a total ~44sec with Apache, and ~31sec using either IIS or CLI (W2K8R2, PHP 5.3.8 x86). If you do end up profiling this I would be very interested to see what you find.

Steve

14 years ago by Richard Riley — view source — reply

unread

Stephen Zarkos Stephen.Zarkos@microsoft.com writes:

Stas Malyshev wrote:

So I suppose the question I have to ask is what the f**k am I doing
wrong on the windows setup? I've always known that linux was faster,
but 4 times faster on the same hardware? My main development
machine
is giving 27.750 seconds which is even nicer,

CPU-bound ops should be roughly the same speed on Windows and Linux
since the code is basically the same. If they're not I guess the best
way would be to profile it. File ops on windows are significantly
slower because Windows FS is slower than Linux one.
The difference can also be in compiler options - whci optimizations
are chosen, etc..

The x86 build of PHP is direct off the windows.php web site, and the Linux on
is via SUSE package handler. When I get some time I will try a 'command line'
run to avoid Apache and see if that is getting in the way.

The slower spec Vista box is running a bit faster than on Windows 7, but I
need to update to 5.3.8 on that ... again I'll do an x86 install before the x64
and benchmark both if only for my own information. That machine needs
updating to W7, but I still have a remote site that 'needs' vista to access it's
VPN software :(

Apache does tend to be slower on Windows.

You have some figures to back that up? My experience is that on modern
HW its about identical assuming there is spare memory in both instances
and the HW is identical.

14 years ago by Stephen Zarkos — view source — reply

unread

-----Original Message-----
From: Richard Riley [mailto:rileyrg@googlemail.com]
Sent: Friday, October 21, 2011 1:04 PM
To: internals@lists.php.net
Subject: Re: [PHP-DEV] Benchmarking ...

Stephen Zarkos Stephen.Zarkos@microsoft.com writes:

Apache does tend to be slower on Windows.

You have some figures to back that up? My experience is that on modern HW
its about identical assuming there is spare memory in both instances and the
HW is identical.

Yes, I do :) Although the tests we've done are a bit different from this sort of benchmark. We typically do more load testing, where we have one or more load agent(s) each creating multiple virtual clients accessing one or more PHP pages on another physical server.

If you like I can share with you directly - unless you want me to spam the list with spreadsheets ;)

Steve

14 years ago by Lester Caine — view source — reply

unread

Stephen Zarkos wrote:

Yes, I do:) Although the tests we've done are a bit different from this sort of benchmark. We typically do more load testing, where we have one or more load agent(s) each creating multiple virtual clients accessing one or more PHP pages on another physical server.

I'm trying to factualize this and come up with a benchmark that I can rely on to
check performance. The initial claim was that there was no advantage using 64bit
builds, but I've re-run everything on the Vista box, and this is giving me 70
seconds on the x86 Apache/PHP but 60 seconds with the x64 build. This is
confirming what I thought myself, and I would expect around 40 seconds on a 64
bit linux on the same hardware. I will try a live CD just to confirm that.

This would suggest that something else is going on with the Windows7 setup for
it to be 4 times slower?

At least what all this exercise is doing is confirming nicely that my switch
from Windows to Linux on those council sites that are now allowing it IS
improving performance. And since this normally happens when we are replacing a
machine, the new hardware is faster as well.

--
Lester Caine - G8HFL

14 years ago by William A. Rowe Jr. — view source — reply

unread

Stephen Zarkos wrote:

Yes, I do:) Although the tests we've done are a bit different from this sort of
benchmark. We typically do more load testing, where we have one or more load agent(s)
each creating multiple virtual clients accessing one or more PHP pages on another
physical server.

I'm trying to factualize this and come up with a benchmark that I can rely on to check
performance. The initial claim was that there was no advantage using 64bit builds, but
I've re-run everything on the Vista box, and this is giving me 70 seconds on the x86
Apache/PHP but 60 seconds with the x64 build. This is confirming what I thought myself,
and I would expect around 40 seconds on a 64 bit linux on the same hardware. I will try a
live CD just to confirm that.

This would suggest that something else is going on with the Windows7 setup for it to be 4
times slower?

4x sounds excessive, but in fact TS code is a significant performance
penalty for windows, the only place where thread safety is "supported".

I'd challenge you to do a non-TS build on windows and set up an fcgid
pool of php workers and compare -that- to linux. I'd expect very nearly
identical performance, once you factor out kernel-specific differences
in the link-loader/resolver. Those are likely two very different beasts.

14 years ago by Lester Caine — view source — reply

unread

Stephen Zarkos wrote:

Stas Malyshev wrote:

So I suppose the question I have to ask is what the f**k am I doing
wrong on the windows setup? I've always known that linux was faster,
but 4 times faster on the same hardware? My main development
machine
is giving 27.750 seconds which is even nicer,

CPU-bound ops should be roughly the same speed on Windows and Linux
since the code is basically the same. If they're not I guess the best
way would be to profile it. File ops on windows are significantly
slower because Windows FS is slower than Linux one.
The difference can also be in compiler options - whci optimizations
are chosen, etc..

The x86 build of PHP is direct off the windows.php web site, and the Linux on
is via SUSE package handler. When I get some time I will try a 'command line'
run to avoid Apache and see if that is getting in the way.

The slower spec Vista box is running a bit faster than on Windows 7, but I
need to update to 5.3.8 on that ... again I'll do an x86 install before the x64
and benchmark both if only for my own information. That machine needs
updating to W7, but I still have a remote site that 'needs' vista to access it's
VPN software:(

Apache does tend to be slower on Windows. I think it would be worthwhile to test using the typical IIS/FastCGI configuration or CLI on Windows. For example, on my server I see a total ~44sec with Apache, and ~31sec using either IIS or CLI (W2K8R2, PHP 5.3.8 x86). If you do end up profiling this I would be very interested to see what you find.

My own reasoning for using Apache is that the configuration is identical between
the Linux boxes and the Windows ones. I don't want to be having to build
different installations especially where we are running multiple site via
Apache. That and we still have W2k or XP Home on many sites, so no IIS ;)

My single core XP Home machine is giving 77 seconds for the benchmark, which is
more in line with what I would expect, and running from the command line, it
drops to 75 seconds, so Apache is not getting in the way too much.

--
Lester Caine - G8HFL

14 years ago by Lester Caine — view source — reply

unread

Lester Caine wrote:

So what are people currently using for benchmarking performance?

Anybody using the Phoronix Test Suite? Beibg PHP based, and running across all
platforms it looks like it should be useful, but it's not 'run out of the box'
on my suse system, and I'm not sure about sorting the windows configuration at
the moment, so I wonder if it's worth the time to get it working?

-- Lester Caine - G8HFL

-- Lester Caine - G8HFL

-- Lester Caine - G8HFL

-- Lester Caine - G8HFL

-- Lester Caine - G8HFL

-- Lester Caine - G8HFL

-- Lester Caine - G8HFL

-- Lester Caine - G8HFL

--
Lester Caine - G8HFL

--
Lester Caine - G8HFL

--
Lester Caine - G8HFL

--
Lester Caine - G8HFL

--
Lester Caine - G8HFL

--
Lester Caine - G8HFL

--
Lester Caine - G8HFL

--
Lester Caine - G8HFL