PHP 5.4 Benchmarks

13 years ago by Pierre Joye — view source

unread

hi Andi,

We do automatic perf tests as part of the CI. Some of the results can
be seen here for example here:

http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-20120119-5.3.9-5.4.0RC6.html

for a 5.3.9 vs 5.4.0RC6 comparison. There are other in the same location.

We are going to do add some more perf related tests later.

Cheers,

Hi,

As many of you know we made huge improvements in PHP 5.4 memory usage (enabling higher concurrency) and runtime performance.
We've done our own benchmarks but it'd be great to hear from others re: what you're seeing with your real-world apps.

If you have any experiences to share please do. Benchmarking is not easy and can be misleading (config not being the same, compiler options, different version of byte code caches, etc..) so the more data points we get the better the info re: performance gains.

Thanks,
Andi

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

13 years ago by Andi Gutmans — view source

unread

Cache results are of course important. Memory usage would be great too.

Thanks!!
Andi

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Wednesday, February 01, 2012 10:19 AM
To: Andi Gutmans
Cc: internals@lists.php.net
Subject: Re: [PHP-DEV] PHP 5.4 Benchmarks

hi Andi,

We do automatic perf tests as part of the CI. Some of the results can be seen
here for example here:

http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-20120119-
5.3.9-5.4.0RC6.html

for a 5.3.9 vs 5.4.0RC6 comparison. There are other in the same location.

We are going to do add some more perf related tests later.

Cheers,

Hi,

As many of you know we made huge improvements in PHP 5.4 memory
usage (enabling higher concurrency) and runtime performance.
We've done our own benchmarks but it'd be great to hear from others re:
what you're seeing with your real-world apps.

If you have any experiences to share please do. Benchmarking is not easy
and can be misleading (config not being the same, compiler options,
different version of byte code caches, etc..) so the more data points we get
the better the info re: performance gains.

Thanks,
Andi

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

13 years ago by Pierre Joye — view source

unread

hi!

I could add zend optimizer to the stack but we do not have a license
(or is it free for the optimizer only?).

APC should join the party soon, not sure yet if it is ready yet. We
will run compat tests first.

About the memory usage, yes, this is on my todos. I have a tool for
that (on win) which has less than 1% impact on perf so we can compare
all in one run. But that will have to wait post final.

Cheers,

Cache results are of course important. Memory usage would be great too.

Thanks!!
Andi

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Wednesday, February 01, 2012 10:19 AM
To: Andi Gutmans
Cc: internals@lists.php.net
Subject: Re: [PHP-DEV] PHP 5.4 Benchmarks

hi Andi,

We do automatic perf tests as part of the CI. Some of the results can be seen
here for example here:

http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-20120119-
5.3.9-5.4.0RC6.html

for a 5.3.9 vs 5.4.0RC6 comparison. There are other in the same location.

We are going to do add some more perf related tests later.

Cheers,

Hi,

As many of you know we made huge improvements in PHP 5.4 memory
usage (enabling higher concurrency) and runtime performance.
We've done our own benchmarks but it'd be great to hear from others re:
what you're seeing with your real-world apps.

If you have any experiences to share please do. Benchmarking is not easy
and can be misleading (config not being the same, compiler options,
different version of byte code caches, etc..) so the more data points we get
the better the info re: performance gains.

Thanks,
Andi

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

13 years ago by Andi Gutmans — view source

unread

Optimizer+ is free and packaged in Zend Server Community Edition. Happy to get you and anyone else who wants to run benchmarks a copy out of cycle. We haven't put a preview up yet but have it working.

Re: memory usage, I've seen some great results so definitely will be nice to see what others are seeing.

Thanks,
Andi

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Wednesday, February 01, 2012 10:37 AM
To: Andi Gutmans
Cc: internals@lists.php.net
Subject: Re: [PHP-DEV] PHP 5.4 Benchmarks

hi!

I could add zend optimizer to the stack but we do not have a license (or is it
free for the optimizer only?).

APC should join the party soon, not sure yet if it is ready yet. We will run
compat tests first.

About the memory usage, yes, this is on my todos. I have a tool for that (on
win) which has less than 1% impact on perf so we can compare all in one run.
But that will have to wait post final.

Cheers,

Cache results are of course important. Memory usage would be great too.

Thanks!!
Andi

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Wednesday, February 01, 2012 10:19 AM
To: Andi Gutmans
Cc: internals@lists.php.net
Subject: Re: [PHP-DEV] PHP 5.4 Benchmarks

hi Andi,

We do automatic perf tests as part of the CI. Some of the results can
be seen here for example here:

http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-
20120119

5.3.9-5.4.0RC6.html

for a 5.3.9 vs 5.4.0RC6 comparison. There are other in the same location.

We are going to do add some more perf related tests later.

Cheers,

Hi,

As many of you know we made huge improvements in PHP 5.4 memory
usage (enabling higher concurrency) and runtime performance.
We've done our own benchmarks but it'd be great to hear from others re:
what you're seeing with your real-world apps.

If you have any experiences to share please do. Benchmarking is not
easy
and can be misleading (config not being the same, compiler options,
different version of byte code caches, etc..) so the more data points
we get the better the info re: performance gains.

Thanks,
Andi

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

13 years ago by Sean Coates — view source

unread

APC should join the party soon, not sure yet if it is ready yet. We
will run compat tests first.

I ran into a crashing blocker with APC on 5.4:
https://bugs.php.net/bug.php?id=60845

If anyone has the bandwidth to look into that, I'd love to help out in any way I can.

(Sorry for the thread hijack…)

S

13 years ago by johannes@schlueters.de — view source

unread

We've done our own benchmarks but it'd be great to hear from others

Can we also hear from you? :-)

johannes
who doesn't have proper benchmarks himself, but is interested in
seeing them ;-)

13 years ago by Andi Gutmans — view source

unread

Absolutely. Will gather those and send them over...

Andi

-----Original Message-----
From: Johannes Schlüter [mailto:johannes@schlueters.de]
Sent: Wednesday, February 01, 2012 10:49 AM
To: Andi Gutmans
Cc: internals@lists.php.net
Subject: Re: [PHP-DEV] PHP 5.4 Benchmarks

We've done our own benchmarks but it'd be great to hear from others

Can we also hear from you? :-)

johannes
who doesn't have proper benchmarks himself, but is interested in
seeing them ;-)

13 years ago by Timm Friebe — view source

unread

Hi,

As many of you know we made huge improvements in PHP 5.4 memory usage
(enabling higher concurrency) and runtime performance.

Here are the unit test results for https://github.com/xp-framework/xp-framework and https://github.com/xp-framework/xp-language after applying workarounds for two bugs in current SVN (see https://gist.github.com/1746945). I think one can most definitely say there are some great improvements to be seen.

PHP 5.3.10 (Windows, Non-Thread-Safe)

OK: 4593/4893 run (300 skipped), 4593 succeeded, 0 failed
Memory used: 54871.11 kB (60676.78 kB peak)
Time taken: 16.188 seconds

OK: 1328/1338 run (10 skipped), 1328 succeeded, 0 failed
Memory used: 25259.16 kB (25562.36 kB peak)
Time taken: 4.571 seconds

PHP 5.4.0RC8-DEV (Windows, Non-Thread-Safe)

OK: 4594/4893 run (299 skipped), 4594 succeeded, 0 failed
Memory used: 39910.35 kB (47000.82 kB peak)
Time taken: 15.674 seconds

OK: 1328/1338 run (10 skipped), 1328 succeeded, 0 failed
Memory used: 16318.44 kB (16931.49 kB peak)
Time taken: 3.612 seconds

The memory usage is measured by memory_get_usage() / memory_get_peak_usage()

Timm

13 years ago by Christopher Jones — view source

unread

Hi,

As many of you know we made huge improvements in PHP 5.4 memory usage
(enabling higher concurrency) and runtime performance.

Here are the unit test results for https://github.com/xp-framework/xp-framework and https://github.com/xp-framework/xp-language after applying workarounds for two bugs in current SVN (see https://gist.github.com/1746945). I think one can most definitely say there are some great improvements to be seen.

PHP 5.3.10 (Windows, Non-Thread-Safe)
 OK: 4593/4893 run (300 skipped), 4593 succeeded, 0 failed
 Memory used: 54871.11 kB (60676.78 kB peak)
 Time taken: 16.188 seconds

 OK: 1328/1338 run (10 skipped), 1328 succeeded, 0 failed
 Memory used: 25259.16 kB (25562.36 kB peak)
 Time taken: 4.571 seconds
PHP 5.4.0RC8-DEV (Windows, Non-Thread-Safe)
 OK: 4594/4893 run (299 skipped), 4594 succeeded, 0 failed
 Memory used: 39910.35 kB (47000.82 kB peak)
 Time taken: 15.674 seconds

 OK: 1328/1338 run (10 skipped), 1328 succeeded, 0 failed
 Memory used: 16318.44 kB (16931.49 kB peak)
 Time taken: 3.612 seconds
The memory usage is measured by memory_get_usage() / memory_get_peak_usage()

Timm

Windows PHP 5.4 is built using profile-guided optimizations, but
PHP 5.3 isn't. My reading of Pierre's own benchmarks are that this
seems to be contributing significantly to the 5.4 gains on Windows.
It makes Windows a poor platform to evaluate generic PHP 5.4 improvements.

Chris

--
Email: christopher.jones@oracle.com
Tel: +1 650 506 8630
Blog: http://blogs.oracle.com/opal/

13 years ago by Pierre Joye — view source

unread

On Mon, Feb 6, 2012 at 1:03 AM, Christopher Jones
christopher.jones@oracle.com wrote:

Windows PHP 5.4 is built using profile-guided optimizations, but
PHP 5.3 isn't. My reading of Pierre's own benchmarks are that this
seems to be contributing significantly to the 5.4 gains on Windows.
It makes Windows a poor platform to evaluate generic PHP 5.4 improvements.

Not for memory usage improvements.

Cheers,

Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

13 years ago by Dmitry Stogov — view source

unread

Hi,

I've just rerun some synthetic and real-life benchmarks.
All the test were run on the same box (Linux, Core2 Duo 3GHz, 4GB RAM).
5.3 and 5.4 where configured and build with the same options and ran as
FastCGI server with the same number of processes and set of extensions.

PHP performance [sec]

                      5.3     5.4    speedup

bench.php 3.21 2.56 20%
micro_bench.php 23.15 11.23 50%

PHP + Zend OptimizerPlus performance [req/sec]

                      5.3     5.4    speedup

blog 62.8 71.0 13%
drupal 1074.3 1146.7 7%
zend framework 102.9 124.6 21%
hello 5955.4 7826.5 31%
qdig 267.6 280.4 5%
typo3 357.0 405.6 14%
wordpress 119.8 130.8 9%
xoops 78.4 93.0 19%
scrum 96.3 117.4 22%

PHP/O+ memory usage (memory_get_peak_usage) [KB]

                      5.3     5.4    improvement

blog 1280 768 40%
drupal 512 256 50%
zend framework 7680 5888 23%
hello 512 256 50%
qdig 768 512 33%
typo3 1536 1024 33%
xoops 1536 1280 17%
scrum 1792 1280 19%

Thanks. Dmitry.

13 years ago by Larry Garfield — view source

unread

Yowza. If I were seeing that kind of memory change in just one project
I'd say there was a bug and the code was just dying. :-)

What sort of configuration were you using in each case? A 512 KB Drupal
process seems, um, low, especially compared to the other projects listed
there. (Drupal 7 is normally rather memory hungry.)

--Larry Garfield

Hi,

I've just rerun some synthetic and real-life benchmarks.
All the test were run on the same box (Linux, Core2 Duo 3GHz, 4GB RAM).
5.3 and 5.4 where configured and build with the same options and ran as
FastCGI server with the same number of processes and set of extensions.

PHP performance [sec]

5.3 5.4 speedup
bench.php 3.21 2.56 20%
micro_bench.php 23.15 11.23 50%

PHP + Zend OptimizerPlus performance [req/sec]

5.3 5.4 speedup
blog 62.8 71.0 13%
drupal 1074.3 1146.7 7%
zend framework 102.9 124.6 21%
hello 5955.4 7826.5 31%
qdig 267.6 280.4 5%
typo3 357.0 405.6 14%
wordpress 119.8 130.8 9%
xoops 78.4 93.0 19%
scrum 96.3 117.4 22%

PHP/O+ memory usage (memory_get_peak_usage) [KB]

5.3 5.4 improvement
blog 1280 768 40%
drupal 512 256 50%
zend framework 7680 5888 23%
hello 512 256 50%
qdig 768 512 33%
typo3 1536 1024 33%
xoops 1536 1280 17%
scrum 1792 1280 19%

Thanks. Dmitry.

13 years ago by Jaroslav Hanslik — view source

unread

As many of you know we made huge improvements in PHP 5.4 memory usage (enabling higher concurrency) and runtime performance.
We've done our own benchmarks but it'd be great to hear from others re: what you're seeing with your real-world apps.

Zend Framework API documentation generated by ApiGen, http://apigen.org

PHP 5.3.10 (VC9 x86 Thread Safe) - Total time: 777 seconds, used: 316 MB RAM

PHP 5.4.0RC7 (VC9 x86 Thread Safe) - Total time: 452 seconds, used: 156
MB RAM

Jaroslav Hanslik

13 years ago by Sebastian Bergmann — view source

unread

Am 05.02.2012 21:34, schrieb Jaroslav Hanslik:

PHP 5.3.10 - Total time: 777 seconds, used: 316 MB RAM
PHP 5.4.0RC7 - Total time: 452 seconds, used: 156 MB RAM

phpDox, initial run for Zend Framework 2

PHP 5.4: Time: 54 seconds, Memory: 30.50Mb
PHP 5.3: Time: 01:09, Memory: 42.25Mb

phpDox, cached run for Zend Framework 2

PHP 5.4: Time: 14 seconds, Memory: 4.75Mb
PHP 5.3: Time: 19 seconds, Memory: 7.50Mb

--
Sebastian Bergmann Co-Founder and Principal Consultant
http://sebastian-bergmann.de/ http://thePHP.cc/

13 years ago by Benjamin Eberlei — view source

unread

You cannot use Travis CI to compare for runtime of php5.4 and 5.3 since the
used boxes are propably either different and also under high load due to
virtualization, but you can check for memory differences. And they are huge:

Doctrine 2:
5.3: http://travis-ci.org/#!/doctrine/doctrine2/jobs/593850 85.5 MB
5.4: http://travis-ci.org/#!/doctrine/doctrine2/jobs/593851 50 MB

Symfony 2:
5.3: http://travis-ci.org/#!/symfony/symfony/jobs/624578 143 MB
5.4: http://travis-ci.org/#!/symfony/symfony/jobs/624579 91 MB

On Sun, Feb 5, 2012 at 10:07 PM, Sebastian Bergmann sebastian@php.netwrote:

Am 05.02.2012 21:34, schrieb Jaroslav Hanslik:

PHP 5.3.10 - Total time: 777 seconds, used: 316 MB RAM
PHP 5.4.0RC7 - Total time: 452 seconds, used: 156 MB RAM

phpDox, initial run for Zend Framework 2

PHP 5.4: Time: 54 seconds, Memory: 30.50Mb
PHP 5.3: Time: 01:09, Memory: 42.25Mb

phpDox, cached run for Zend Framework 2

PHP 5.4: Time: 14 seconds, Memory: 4.75Mb
PHP 5.3: Time: 19 seconds, Memory: 7.50Mb

--
Sebastian Bergmann Co-Founder and Principal Consultant
http://sebastian-bergmann.de/ http://thePHP.cc/

PHP 5.4 Benchmarks

http://windows.php.net/downloads/snaps/ostc/pftt/perf/results- 20120119

Cheers,

PHP performance [sec]

PHP + Zend OptimizerPlus performance [req/sec]

PHP/O+ memory usage (memory_get_peak_usage) [KB]

PHP performance [sec]

PHP + Zend OptimizerPlus performance [req/sec]

PHP/O+ memory usage (memory_get_peak_usage) [KB]

http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-
20120119