UGLY Benchmark Results for PHP Master 2016-08-05

9 years ago by lp_benchmark_robot@intel.com — view source

unread

Results for project PHP master, build date 2016-08-05 06:26:56+03:00
commit: 65b6f20
previous commit: 69a72df
revision date: 2016-08-04 22:27:42+02:00
environment: Haswell-EP
cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB
mem: 128 GB
os: CentOS 7.1
kernel: Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release php-7.0.0, with hash 60fffd2 from
2015-12-01 04:16:47+00:00

                    benchmark   relative   change since   change since  current rev run
                                std_dev*       last run       baseline         with PGO

:-| Wordpress 4.2.2 cgi -T10000 0.18% -0.20% 5.10% 7.48%
:-| Drupal 7.36 cgi -T10000 0.17% -0.24% -0.86% 5.31%
:-| MediaWiki 1.23.9 cgi -T5000 0.11% 0.22% -0.13% 3.80%
:-) bench.php cgi -T100 0.01% 1.06% 32.28% -7.09%
:-) micro_bench.php cgi -T10 0.01% 4.09% 15.14% 1.66%
:-( mandelbrot.php cgi -T100 0.15% -6.51% 28.51% 5.76%

Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here: http://languagesperformance.intel.com/ugly-benchmark-results-for-php-master-2016-08-05/

Note: Benchmark results for Wordpress, Drupal, MediaWiki are measured in
fetches/second while all others are measured in seconds.
More details on measurements methodology at:
https://01.org/lp/documentation/php-environment-setup.

Subject Label Legend:
Attributes are determined based on the performance evolution of the workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and also
dropped by more than 1% for at least one workload

Our lab does a nightly source pull and build of the PHP project and measures
performance changes against the previous stable version and the previous nightly
measurement. This is provided as a service to the community so that quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration and may
require enabled hardware, software or service activation. Performance varies
depending on system configuration.

9 years ago by Matt Wilmas — view source

unread

Hi all,

----- Original Message -----
From: lp_benchmark_robot@intel.com
Sent: Friday, August 05, 2016

Results for project PHP master, build date 2016-08-05 06:26:56+03:00
commit: 65b6f20
previous commit: 69a72df
revision date: 2016-08-04 22:27:42+02:00
environment: Haswell-EP
cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC
45 MB
mem: 128 GB
os: CentOS 7.1
kernel: Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release php-7.0.0, with hash 60fffd2
from
2015-12-01 04:16:47+00:00
                   benchmark   relative   change since   change since 
current rev run
std_dev* last run baseline
with PGO

:-| Wordpress 4.2.2 cgi -T10000 0.18% -0.20% 5.10%
7.48%
:-| Drupal 7.36 cgi -T10000 0.17% -0.24% -0.86%
5.31%
:-| MediaWiki 1.23.9 cgi -T5000 0.11% 0.22% -0.13%
3.80%
:-) bench.php cgi -T100 0.01% 1.06%
.28% -7.09%
:-) micro_bench.php cgi -T10 0.01% 4.09% 15.14%
1.66%
:-( mandelbrot.php cgi -T100 0.15% -6.51% 28.51%
5.76%

Anyone know what has happened with the Wordpress improvement this week?
This is 3 reports in a row that show ~5%.

Did I miss some substantial commit(s)?

Thanks for any insight!

Matt

Relative Standard Deviation (Standard Deviation/Average)

If this is not displayed properly please visit our results page here:
http://languagesperformance.intel.com/ugly-benchmark-results-for-php-master-2016-08-05/

Note: Benchmark results for Wordpress, Drupal, MediaWiki are measured in
fetches/second while all others are measured in seconds.
More details on measurements methodology at:
https://01.org/lp/documentation/php-environment-setup.

Subject Label Legend:
Attributes are determined based on the performance evolution of the
workloads
compared to the previous measurement iteration.
NEUTRAL: performance did not change by more than 1% for any workload
GOOD: performance improved by more than 1% for at least one workload and
there
is no regression greater than 1%
BAD: performance dropped by more than 1% for at least one workload and
there is
no improvement greater than 1%
UGLY: performance improved by more than 1% for at least one workload and
also
dropped by more than 1% for at least one workload

Our lab does a nightly source pull and build of the PHP project and
measures
performance changes against the previous stable version and the previous
nightly
measurement. This is provided as a service to the community so that
quality
issues with current hardware can be identified quickly.

Intel technologies' features and benefits depend on system configuration
and may
require enabled hardware, software or service activation. Performance
varies
depending on system configuration.

9 years ago by Nikita Popov — view source

unread

Hi all,

----- Original Message -----
From: lp_benchmark_robot@intel.com
Sent: Friday, August 05, 2016

Results for project PHP master, build date 2016-08-05 06:26:56+03:00
commit: 65b6f20
previous commit: 69a72df
revision date: 2016-08-04 22:27:42+02:00
environment: Haswell-EP
cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2,
LLC 45 MB
mem: 128 GB
os: CentOS 7.1
kernel: Linux 3.10.0-229.4.2.el7.x86_64

Baseline results were generated using release php-7.0.0, with hash
60fffd2 from
2015-12-01 04:16:47+00:00
                   benchmark   relative   change since   change since
current rev run
std_dev* last run baseline
with PGO

:-| Wordpress 4.2.2 cgi -T10000 0.18% -0.20%
5.10% 7.48%
:-| Drupal 7.36 cgi -T10000 0.17% -0.24%
-0.86% 5.31%
:-| MediaWiki 1.23.9 cgi -T5000 0.11% 0.22%
-0.13% 3.80%
:-) bench.php cgi -T100 0.01% 1.06% .28%
-7.09%
:-) micro_bench.php cgi -T10 0.01% 4.09%
15.14% 1.66%
:-( mandelbrot.php cgi -T100 0.15% -6.51%
28.51% 5.76%
Anyone know what has happened with the Wordpress improvement this week?
This is 3 reports in a row that show ~5%.

Did I miss some substantial commit(s)?

Thanks for any insight!

Thanks for pointing this out.

This is caused by the change to array_slice() as part of
https://github.com/php/php-src/commit/e730c8fc90299789a7f551cb7142e182952d92e0#diff-497f073aa1ab88afcb8b248fc25d2a12R3014
.

As a consequence of this change, an array_slice() on an array with rc=1
references will now not return these references in the result. (This is the
correct behavior -- previously it instead dropped the references in the
original array, which is not wrong either, but non-standard.)

It looks like Wordpress is passing these arrays to call_user_func_array()
with a function that expects a reference argument:

call_user_func_array($the_['function'], array_slice($args, 0, (int)
$the_['accepted_args']));

And this results in:

nikic@saturn:~/php-src-fast$ sapi/cgi/php-cgi -c php.ini -T1
../wordpress-4.1/index.php | grep Warning
Warning: Parameter 1 to wp_default_styles() expected to be a
reference, value given in
/home/nikic/wordpress-4.1/wp-includes/plugin.php on line
571 
Warning: Parameter 1 to wp_default_scripts() expected to be a
reference, value given in
/home/nikic/wordpress-4.1/wp-includes/plugin.php on line
571

So essentially, we're winning 5% because these two calls do not occur...

Nikita

UGLY Benchmark Results for PHP Master 2016-08-05

current rev run std_dev* last run baseline with PGO

current rev run std_dev* last run baseline with PGO

current rev run
std_dev* last run baseline
with PGO

current rev run
std_dev* last run baseline
with PGO