Benchmark Results for PHP Master 2015-11-19

9 years ago by lp_benchmark_robot@intel.com — view source

unread

Results for project PHP master, build date 2015-11-19 05:26:43+02:00
commit: 84610b1f3904069f6fc0754c807a3f5cd58a4010
revision date: 2015-11-19 00:47:08+03:00
environment: Haswell-EP
cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
mem: 128 GB
os: CentOS 7.1
kernel: Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release php-7.0.0beta3, with hash
1674bd9b151ff389fb8c9fc223bc6aafdd49ff2c from 2015-08-05 04:56:40+00:00

                    benchmark   relative   change since   change since  current rev run
                                std_dev*       last run       baseline         with PGO

:-) Wordpress 4.2.2 cgi -T10000 0.19% -0.63% 2.42% 8.07%
:-) Drupal 7.36 cgi -T10000 0.31% 0.84% 1.67% 3.56%
:-| MediaWiki 1.23.9 cgi -T5000 0.43% 0.10% 0.77% 3.71%
:-) bench.php cgi -T1 0.38% 0.84% 2.30% 5.79%
:-) micro_bench.php cgi -T1 0.05% -0.65% 1.22% 5.23%
:-( mandelbrot.php cgi -T1 0.13% -21.01% -21.64% 23.09%

Note: Benchmark results for Wordpress, Drupal, MediaWiki are measured in
fetches/second while all others are measured in seconds.
More details on measurements methodology at:
https://01.org/lp/documentation/php-environment-setup.

Relative Standard Deviation (Standard Deviation/Average)

Our lab does a nightly source pull and build of the PHP project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

9 years ago by Andrea Faulds — view source

unread

Hi everyone,

lp_benchmark_robot@intel.com wrote:

Results for project PHP master, build date 2015-11-19 05:26:43+02:00
commit: 84610b1f3904069f6fc0754c807a3f5cd58a4010
revision date: 2015-11-19 00:47:08+03:00
environment: Haswell-EP
cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
mem: 128 GB
os: CentOS 7.1
kernel: Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release php-7.0.0beta3, with hash
1674bd9b151ff389fb8c9fc223bc6aafdd49ff2c from 2015-08-05 04:56:40+00:00
                     benchmark   relative   change since   change since  current rev run
                                 std_dev*       last run       baseline         with PGO
:-) Wordpress 4.2.2 cgi -T10000 0.19% -0.63% 2.42% 8.07%
:-) Drupal 7.36 cgi -T10000 0.31% 0.84% 1.67% 3.56%
:-| MediaWiki 1.23.9 cgi -T5000 0.43% 0.10% 0.77% 3.71%
:-) bench.php cgi -T1 0.38% 0.84% 2.30% 5.79%
:-) micro_bench.php cgi -T1 0.05% -0.65% 1.22% 5.23%
:-( mandelbrot.php cgi -T1 0.13% -21.01% -21.64% 23.09%

I'm wondering why Mandelbrot.php has gone down by -21%. Compare the
commits since the previous benchmark:

https://github.com/php/php-src/compare/a6cc322b4da9fa32faeb7d5aa43ddf7202a19ae4...366ba41334870f325f248d8e486e3ebf8bafb984

I wondered if my void return type commit was the culprit for the
Mandelbrot slowdown, but having looked at what Mandelbrot.php does, it
doesn't seem like it would be affected. There's very few function calls
in there.

Dmitry changed some things in opcache, could that be it?

Thanks.

--
Andrea Faulds
http://ajf.me/

9 years ago by Andone — view source

unread

Hi everyone,

-----Original Message-----
From: Andrea Faulds [mailto:ajf@ajf.me]
Sent: Friday, November 20, 2015 12:44 PM
To: internals@lists.php.net
Subject: [PHP-DEV] Re: Benchmark Results for PHP Master 2015-11-19

Hi everyone,

lp_benchmark_robot@intel.com wrote:

Results for project PHP master, build date 2015-11-19 05:26:43+02:00
commit: 84610b1f3904069f6fc0754c807a3f5cd58a4010
revision date: 2015-11-19 00:47:08+03:00
environment: Haswell-EP
cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores,
stepping 2, LLC 45 MB
mem: 128 GB
os: CentOS 7.1
kernel: Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release php-7.0.0beta3, with
hash
1674bd9b151ff389fb8c9fc223bc6aafdd49ff2c from 2015-08-05
04:56:40+00:00
                     benchmark   relative   change since   change
since current rev run
                                 std_dev*       last run
baseline with PGO

:-) Wordpress 4.2.2 cgi -T10000 0.19% -0.63%
2.42% 8.07%
:-) Drupal 7.36 cgi -T10000 0.31% 0.84%
1.67% 3.56%
:-| MediaWiki 1.23.9 cgi -T5000 0.43% 0.10%
0.77% 3.71%
:-) bench.php cgi -T1 0.38% 0.84%
2.30% 5.79%
:-) micro_bench.php cgi -T1 0.05% -0.65%
1.22% 5.23%
:-( mandelbrot.php cgi -T1 0.13% -21.01% -
21.64% 23.09%

I'm wondering why Mandelbrot.php has gone down by -21%. Compare the
commits since the previous benchmark:

https://github.com/php/php-
src/compare/a6cc322b4da9fa32faeb7d5aa43ddf7202a19ae4...366ba41334870f325
f248d8e486e3ebf8bafb984

I wondered if my void return type commit was the culprit for the
Mandelbrot slowdown, but having looked at what Mandelbrot.php does, it
doesn't seem like it would be affected. There's very few function calls
in there.
Please ignore the Mandelbrot result. It seems it was a measurement error, as
we were unable to reproduce it on the same commit with additional runs.

Dmitry changed some things in opcache, could that be it?

Thanks.

--
Andrea Faulds
http://ajf.me/

--

Kind Regards,
Bogdan

9 years ago by Rasmus Lerdorf — view source

unread

Please ignore the Mandelbrot result. It seems it was a measurement error, as
we were unable to reproduce it on the same commit with additional runs.

Perhaps it would be possible to automatically trigger additional runs in case a result shows a drastic change like that? Just to eliminate false outliers. These benchmarks are a great sanity check which can catch small changes that have inadvertent performance hits, but we need to have confidence in the numbers otherwise people will start ignoring them.

-Rasmus