the experimental jit-branch is impressive

7 years ago by lists@rhsoft.net — view source

unread

https://github.com/zendtech/php-src/tree/jit-dynasm/ext/opcache/jit

that below is a "ab -c 50 -n 100000" on a Intel(R) Core(TM) i7-3770 CPU
@ 3.40GHz on our core cms (pgo-build with heavy compiler optimizations,
only the source tarball deifferent)

currently our production 2x6 core machine with two Intel(R) Xeon(R) CPU
E5-2643 v3 @ 3.40GHz and the same setup is around 4000-5000 per second
which is nearly reached by the 6 years old i7 with the JIT

impressive!

JIT:
Requests per second: 3526.56 [#/sec] (mean)
Time per request: 14.178 [ms] (mean)
Time per request: 0.284 [ms] (mean, across all concurrent requests)

7.1:
Requests per second: 1419.87 [#/sec] (mean)
Time per request: 35.214 [ms] (mean)
Time per request: 0.704 [ms] (mean, across all concurrent requests)

7 years ago by lists@rhsoft.net — view source

unread

and with a demo-page containing all sort of modules and bloat the
difference is even greater - can't wait to see that in production

is there anything known when it is expected to arrive in the official
tree or does Zend even hold it back until the point when they can
suprise with a "we are done, all tests are fine and we can merge it"
announce?

PHP 7.1:
Requests per second: 316.77 [#/sec] (mean)
Time per request: 157.844 [ms] (mean)
Time per request: 3.157 [ms] (mean, across all concurrent requests)
Transfer rate: 12604.11 [Kbytes/sec] received

PHP 7.2 JIT:
Requests per second: 925.96 [#/sec] (mean)
Time per request: 53.998 [ms] (mean)
Time per request: 1.080 [ms] (mean, across all concurrent requests)
Transfer rate: 36842.68 [Kbytes/sec] received

Am 02.05.2017 um 18:00 schrieb lists@rhsoft.net:

https://github.com/zendtech/php-src/tree/jit-dynasm/ext/opcache/jit

that below is a "ab -c 50 -n 100000" on a Intel(R) Core(TM) i7-3770 CPU
@ 3.40GHz on our core cms (pgo-build with heavy compiler optimizations,
only the source tarball deifferent)

currently our production 2x6 core machine with two Intel(R) Xeon(R) CPU
E5-2643 v3 @ 3.40GHz and the same setup is around 4000-5000 per second
which is nearly reached by the 6 years old i7 with the JIT

impressive!

JIT:
Requests per second: 3526.56 [#/sec] (mean)
Time per request: 14.178 [ms] (mean)
Time per request: 0.284 [ms] (mean, across all concurrent requests)

7.1:
Requests per second: 1419.87 [#/sec] (mean)
Time per request: 35.214 [ms] (mean)
Time per request: 0.704 [ms] (mean, across all concurrent requests)

7 years ago by Nikita Popov — view source

unread

and with a demo-page containing all sort of modules and bloat the
difference is even greater - can't wait to see that in production

is there anything known when it is expected to arrive in the official tree
or does Zend even hold it back until the point when they can suprise with a
"we are done, all tests are fine and we can merge it" announce?

PHP 7.1:
Requests per second: 316.77 [#/sec] (mean)
Time per request: 157.844 [ms] (mean)
Time per request: 3.157 [ms] (mean, across all concurrent requests)
Transfer rate: 12604.11 [Kbytes/sec] received

PHP 7.2 JIT:
Requests per second: 925.96 [#/sec] (mean)
Time per request: 53.998 [ms] (mean)
Time per request: 1.080 [ms] (mean, across all concurrent requests)
Transfer rate: 36842.68 [Kbytes/sec] received

These results are very unlikely. I'm 95% sure your benchmark is broken. My
first guess would be that you're benchmarking PHP + JIT against PHP without
opcache. Please share the relevant (opcache-related) portion of the
php.inis you used.

Nikita

Am 02.05.2017 um 18:00 schrieb lists@rhsoft.net:

https://github.com/zendtech/php-src/tree/jit-dynasm/ext/opcache/jit

that below is a "ab -c 50 -n 100000" on a Intel(R) Core(TM) i7-3770 CPU @
3.40GHz on our core cms (pgo-build with heavy compiler optimizations, only
the source tarball deifferent)

currently our production 2x6 core machine with two Intel(R) Xeon(R) CPU
E5-2643 v3 @ 3.40GHz and the same setup is around 4000-5000 per second
which is nearly reached by the 6 years old i7 with the JIT

impressive!

JIT:
Requests per second: 3526.56 [#/sec] (mean)
Time per request: 14.178 [ms] (mean)
Time per request: 0.284 [ms] (mean, across all concurrent requests)

7.1:
Requests per second: 1419.87 [#/sec] (mean)
Time per request: 35.214 [ms] (mean)
Time per request: 0.704 [ms] (mean, across all concurrent requests)

7 years ago by lists@rhsoft.net — view source

unread

Am 02.05.2017 um 20:02 schrieb Nikita Popov:

On Tue, May 2, 2017 at 7:14 PM, lists@rhsoft.net
mailto:lists@rhsoft.net <lists@rhsoft.net mailto:lists@rhsoft.net>
wrote:
and with a demo-page containing all sort of modules and bloat the
difference is even greater - can't wait to see that in production

is there anything known when it is expected to arrive in the
official tree or does Zend even hold it back until the point when
they can suprise with a "we are done, all tests are fine and we can
merge it" announce?

PHP 7.1:
Requests per second: 316.77 [#/sec] (mean)
Time per request: 157.844 [ms] (mean)
Time per request: 3.157 [ms] (mean, across all concurrent requests)
Transfer rate: 12604.11 [Kbytes/sec] received

PHP 7.2 JIT:
Requests per second: 925.96 [#/sec] (mean)
Time per request: 53.998 [ms] (mean)
Time per request: 1.080 [ms] (mean, across all concurrent requests)
Transfer rate: 36842.68 [Kbytes/sec] received
These results are very unlikely. I'm 95% sure your benchmark is broken.
My first guess would be that you're benchmarking PHP + JIT against PHP
without opcache. Please share the relevant (opcache-related) portion of
the php.inis you used

no they are not - since i build RPM packages and even the whole
spec-file is unchanged, only the tarball changed and the build is highly
optimized i can assure you for 100% that i compare PHP 7.1.5RC1 with
https://github.com/zendtech/php-src downloaded today

maybe the PGO-profiling running autotests and fuzzy-calls on the whole
application as well as 2000 cms-requests combined with the compiler
flags improves the JIT itself

without opcache the results for 7.1.5 are dramatically slower

and i repeated the test upgrade/downgrade packages and run "ab" multiple
times on that machine - attached the "php.spec" which is used for the build

in just downloaded the zip from https://github.com/zendtech/php-src,
renamed it to "php-7.2.0", made a tar.xz archive, changed the version on
teh frist line in the spec file and fired the build/profiling - nothing
else changed

phpinfo() of the test machine

PHP Version 7.1.5RC1
Build Date May 2 2017 12:19:59

This program makes use of the Zend Scripting Language Engine: Zend
Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies with Zend
OPcache v7.1.5RC1, Copyright (c) 1999-2017, by Zend Technologies
with Xdebug v2.5.3, Copyright (c) 2002-2017, by Derick Rethans

Directive Local Value Master Value
opcache.blacklist_filename no value no value
opcache.consistency_checks 0 0
opcache.dups_fix Off Off
opcache.enable On On
opcache.enable_cli Off Off
opcache.enable_file_override On On
opcache.error_log /var/log/php_error.log /var/log/php_error.log
opcache.fast_shutdown 1 1
opcache.file_update_protection 2 2
opcache.force_restart_timeout 180 180
opcache.huge_code_pages On On
opcache.inherited_hack On On
opcache.interned_strings_buffer 8 8
opcache.lockfile_path /tmp /tmp
opcache.log_verbosity_level 1 1
opcache.max_accelerated_files 1000 1000
opcache.max_file_size 327680 327680
opcache.max_wasted_percentage 5 5
opcache.memory_consumption 128 128
opcache.opt_debug_level 0 0
opcache.optimization_level 0x7FFFBFFF 0x7FFFBFFF
opcache.preferred_memory_model no value no value
opcache.protect_memory 0 0
opcache.restrict_api /usr/share/php/zendoptimizer.php
/usr/share/php/zendoptimizer.php
opcache.revalidate_freq 5 5
opcache.revalidate_path Off Off
opcache.save_comments 0 0
opcache.use_cwd On On
opcache.validate_permission Off Off
opcache.validate_root Off Off
opcache.validate_timestamps On

7 years ago by lists@rhsoft.net — view source

unread

that would be the numbers of 7.1.5RC1 without opcache

Requests per second: 136.46 [#/sec] (mean)
Time per request: 366.405 [ms] (mean)
Time per request: 7.328 [ms] (mean, across all concurrent requests)
Transfer rate: 5429.78 [Kbytes/sec] received

7.1.5: Requests per second: 136.46
7.1.5 opcache: Requests per second: 316.77
7.2.0 JIT: Requests per second: 925.96

same hardware, same scripts, httpd hard restarted
ab -c 50 -n 20000

believe it or not - i know my php environemt and what i benchmark :-)

Am 02.05.2017 um 20:12 schrieb lists@rhsoft.net:

Am 02.05.2017 um 20:02 schrieb Nikita Popov:
On Tue, May 2, 2017 at 7:14 PM, lists@rhsoft.net
mailto:lists@rhsoft.net <lists@rhsoft.net mailto:lists@rhsoft.net>
wrote:
and with a demo-page containing all sort of modules and bloat the
difference is even greater - can't wait to see that in production

is there anything known when it is expected to arrive in the
official tree or does Zend even hold it back until the point when
they can suprise with a "we are done, all tests are fine and we can
merge it" announce?

PHP 7.1:
Requests per second: 316.77 [#/sec] (mean)
Time per request: 157.844 [ms] (mean)
Time per request: 3.157 [ms] (mean, across all concurrent requests)
Transfer rate: 12604.11 [Kbytes/sec] received

PHP 7.2 JIT:
Requests per second: 925.96 [#/sec] (mean)
Time per request: 53.998 [ms] (mean)
Time per request: 1.080 [ms] (mean, across all concurrent requests)
Transfer rate: 36842.68 [Kbytes/sec] received
These results are very unlikely. I'm 95% sure your benchmark is
broken. My first guess would be that you're benchmarking PHP + JIT
against PHP without opcache. Please share the relevant
(opcache-related) portion of the php.inis you used
no they are not - since i build RPM packages and even the whole
spec-file is unchanged, only the tarball changed and the build is highly
optimized i can assure you for 100% that i compare PHP 7.1.5RC1 with
https://github.com/zendtech/php-src downloaded today

maybe the PGO-profiling running autotests and fuzzy-calls on the whole
application as well as 2000 cms-requests combined with the compiler
flags improves the JIT itself

without opcache the results for 7.1.5 are dramatically slower

and i repeated the test upgrade/downgrade packages and run "ab" multiple
times on that machine - attached the "php.spec" which is used for the build

in just downloaded the zip from https://github.com/zendtech/php-src,
renamed it to "php-7.2.0", made a tar.xz archive, changed the version on
teh frist line in the spec file and fired the build/profiling - nothing
else changed

phpinfo() of the test machine

PHP Version 7.1.5RC1
Build Date May 2 2017 12:19:59

This program makes use of the Zend Scripting Language Engine: Zend
Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies with Zend
OPcache v7.1.5RC1, Copyright (c) 1999-2017, by Zend Technologies
with Xdebug v2.5.3, Copyright (c) 2002-2017, by Derick Rethans

Directive Local Value Master Value
opcache.blacklist_filename no value no value
opcache.consistency_checks 0 0
opcache.dups_fix Off Off
opcache.enable On On
opcache.enable_cli Off Off
opcache.enable_file_override On On
opcache.error_log /var/log/php_error.log /var/log/php_error.log
opcache.fast_shutdown 1 1
opcache.file_update_protection 2 2
opcache.force_restart_timeout 180 180
opcache.huge_code_pages On On
opcache.inherited_hack On On
opcache.interned_strings_buffer 8 8
opcache.lockfile_path /tmp /tmp
opcache.log_verbosity_level 1 1
opcache.max_accelerated_files 1000 1000
opcache.max_file_size 327680 327680
opcache.max_wasted_percentage 5 5
opcache.memory_consumption 128 128
opcache.opt_debug_level 0 0
opcache.optimization_level 0x7FFFBFFF 0x7FFFBFFF
opcache.preferred_memory_model no value no value
opcache.protect_memory 0 0
opcache.restrict_api /usr/share/php/zendoptimizer.php
/usr/share/php/zendoptimizer.php
opcache.revalidate_freq 5 5
opcache.revalidate_path Off Off
opcache.save_comments 0 0
opcache.use_cwd On On
opcache.validate_permission Off Off
opcache.validate_root Off Off
opcache.validate_timestamps On

7 years ago by lists@rhsoft.net — view source

unread

and BTW: for the 7.2.0 build i have not built pecl-extensions so the
application is runnign witout APCu which means loading a single template
in a core-cms-setup takes 1.6% versus 0.06% of the whole page

maybe one missing relevant information:

no 3rd party code and micro-optimized to death over months
100% declare(strict_types=1); after some months of works
99.9% of all functions are using type-hints for all params

could be that the JIT has some benefit from that combined with the
PGP-profiling and heavily CPU optimizations - i will give it a try and
leave out the PGO-profiling and compare if the gain stays similar

Am 02.05.2017 um 20:21 schrieb lists@rhsoft.net:

that would be the numbers of 7.1.5RC1 without opcache

Requests per second: 136.46 [#/sec] (mean)
Time per request: 366.405 [ms] (mean)
Time per request: 7.328 [ms] (mean, across all concurrent requests)
Transfer rate: 5429.78 [Kbytes/sec] received

7.1.5: Requests per second: 136.46
7.1.5 opcache: Requests per second: 316.77
7.2.0 JIT: Requests per second: 925.96

same hardware, same scripts, httpd hard restarted
ab -c 50 -n 20000

believe it or not - i know my php environemt and what i benchmark :-)

Am 02.05.2017 um 20:12 schrieb lists@rhsoft.net:
Am 02.05.2017 um 20:02 schrieb Nikita Popov:
On Tue, May 2, 2017 at 7:14 PM, lists@rhsoft.net
mailto:lists@rhsoft.net <lists@rhsoft.net
mailto:lists@rhsoft.net> wrote:
and with a demo-page containing all sort of modules and bloat the
difference is even greater - can't wait to see that in production

is there anything known when it is expected to arrive in the
official tree or does Zend even hold it back until the point when
they can suprise with a "we are done, all tests are fine and we can
merge it" announce?

PHP 7.1:
Requests per second: 316.77 [#/sec] (mean)
Time per request: 157.844 [ms] (mean)
Time per request: 3.157 [ms] (mean, across all concurrent requests)
Transfer rate: 12604.11 [Kbytes/sec] received

PHP 7.2 JIT:
Requests per second: 925.96 [#/sec] (mean)
Time per request: 53.998 [ms] (mean)
Time per request: 1.080 [ms] (mean, across all concurrent requests)
Transfer rate: 36842.68 [Kbytes/sec] received
These results are very unlikely. I'm 95% sure your benchmark is
broken. My first guess would be that you're benchmarking PHP + JIT
against PHP without opcache. Please share the relevant
(opcache-related) portion of the php.inis you used
no they are not - since i build RPM packages and even the whole
spec-file is unchanged, only the tarball changed and the build is
highly optimized i can assure you for 100% that i compare PHP 7.1.5RC1
with https://github.com/zendtech/php-src downloaded today

maybe the PGO-profiling running autotests and fuzzy-calls on the whole
application as well as 2000 cms-requests combined with the compiler
flags improves the JIT itself

without opcache the results for 7.1.5 are dramatically slower

and i repeated the test upgrade/downgrade packages and run "ab"
multiple times on that machine - attached the "php.spec" which is used
for the build

in just downloaded the zip from https://github.com/zendtech/php-src,
renamed it to "php-7.2.0", made a tar.xz archive, changed the version
on teh frist line in the spec file and fired the build/profiling -
nothing else changed

phpinfo() of the test machine

PHP Version 7.1.5RC1
Build Date May 2 2017 12:19:59

This program makes use of the Zend Scripting Language Engine: Zend
Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies with
Zend OPcache v7.1.5RC1, Copyright (c) 1999-2017, by Zend
Technologies with Xdebug v2.5.3, Copyright (c) 2002-2017, by
Derick Rethans

Directive Local Value Master Value
opcache.blacklist_filename no value no value
opcache.consistency_checks 0 0
opcache.dups_fix Off Off
opcache.enable On On
opcache.enable_cli Off Off
opcache.enable_file_override On On
opcache.error_log /var/log/php_error.log /var/log/php_error.log
opcache.fast_shutdown 1 1
opcache.file_update_protection 2 2
opcache.force_restart_timeout 180 180
opcache.huge_code_pages On On
opcache.inherited_hack On On
opcache.interned_strings_buffer 8 8
opcache.lockfile_path /tmp /tmp
opcache.log_verbosity_level 1 1
opcache.max_accelerated_files 1000 1000
opcache.max_file_size 327680 327680
opcache.max_wasted_percentage 5 5
opcache.memory_consumption 128 128
opcache.opt_debug_level 0 0
opcache.optimization_level 0x7FFFBFFF 0x7FFFBFFF
opcache.preferred_memory_model no value no value
opcache.protect_memory 0 0
opcache.restrict_api /usr/share/php/zendoptimizer.php
/usr/share/php/zendoptimizer.php
opcache.revalidate_freq 5 5
opcache.revalidate_path Off Off
opcache.save_comments 0 0
opcache.use_cwd On On
opcache.validate_permission Off Off
opcache.validate_root Off Off
opcache.validate_timestamps On

7 years ago by Nikita Popov — view source

unread

Am 02.05.2017 um 20:02 schrieb Nikita Popov:
On Tue, May 2, 2017 at 7:14 PM, lists@rhsoft.net mailto:lists@rhsoft.net
<lists@rhsoft.net mailto:lists@rhsoft.net> wrote:
and with a demo-page containing all sort of modules and bloat the
difference is even greater - can't wait to see that in production

is there anything known when it is expected to arrive in the
official tree or does Zend even hold it back until the point when
they can suprise with a "we are done, all tests are fine and we can
merge it" announce?

PHP 7.1:
Requests per second: 316.77 [#/sec] (mean)
Time per request: 157.844 [ms] (mean)
Time per request: 3.157 [ms] (mean, across all concurrent requests)
Transfer rate: 12604.11 [Kbytes/sec] received

PHP 7.2 JIT:
Requests per second: 925.96 [#/sec] (mean)
Time per request: 53.998 [ms] (mean)
Time per request: 1.080 [ms] (mean, across all concurrent requests)
Transfer rate: 36842.68 [Kbytes/sec] received
These results are very unlikely. I'm 95% sure your benchmark is broken.
My first guess would be that you're benchmarking PHP + JIT against PHP
without opcache. Please share the relevant (opcache-related) portion of the
php.inis you used
no they are not - since i build RPM packages and even the whole spec-file
is unchanged, only the tarball changed and the build is highly optimized i
can assure you for 100% that i compare PHP 7.1.5RC1 with
https://github.com/zendtech/php-src downloaded today

maybe the PGO-profiling running autotests and fuzzy-calls on the whole
application as well as 2000 cms-requests combined with the compiler flags
improves the JIT itself

without opcache the results for 7.1.5 are dramatically slower

and i repeated the test upgrade/downgrade packages and run "ab" multiple
times on that machine - attached the "php.spec" which is used for the build

in just downloaded the zip from https://github.com/zendtech/php-src,
renamed it to "php-7.2.0", made a tar.xz archive, changed the version on
teh frist line in the spec file and fired the build/profiling - nothing
else changed

phpinfo() of the test machine

PHP Version 7.1.5RC1
Build Date May 2 2017 12:19:59

This program makes use of the Zend Scripting Language Engine: Zend Engine
v3.1.0, Copyright (c) 1998-2017 Zend Technologies with Zend OPcache
v7.1.5RC1, Copyright (c) 1999-2017, by Zend Technologies with Xdebug
v2.5.3, Copyright (c) 2002-2017, by Derick Rethans

Directive Local Value Master Value
opcache.blacklist_filename no value no value
opcache.consistency_checks 0 0
opcache.dups_fix Off Off
opcache.enable On On
opcache.enable_cli Off Off
opcache.enable_file_override On On
opcache.error_log /var/log/php_error.log /var/log/php_error.log
opcache.fast_shutdown 1 1
opcache.file_update_protection 2 2
opcache.force_restart_timeout 180 180
opcache.huge_code_pages On On
opcache.inherited_hack On On
opcache.interned_strings_buffer 8 8
opcache.lockfile_path /tmp /tmp
opcache.log_verbosity_level 1 1
opcache.max_accelerated_files 1000 1000
opcache.max_file_size 327680 327680
opcache.max_wasted_percentage 5 5
opcache.memory_consumption 128 128
opcache.opt_debug_level 0 0
opcache.optimization_level 0x7FFFBFFF 0x7FFFBFFF
opcache.preferred_memory_model no value no value
opcache.protect_memory 0 0
opcache.restrict_api /usr/share/php/zendoptimizer.php
/usr/share/php/zendoptimizer.php
opcache.revalidate_freq 5 5
opcache.revalidate_path Off Off
opcache.save_comments 0 0
opcache.use_cwd On On
opcache.validate_permission Off Off
opcache.validate_root Off Off
opcache.validate_timestamps On

You have xdebug enabled...

7 years ago by lists@rhsoft.net — view source

unread

Am 02.05.2017 um 20:53 schrieb Nikita Popov:

    These results are very unlikely. I'm 95% sure your benchmark is
    broken. My first guess would be that you're benchmarking PHP +
    JIT against PHP without opcache. Please share the relevant
    (opcache-related) portion of the php.inis you used

no they are not - since i build RPM packages and even the whole
spec-file is unchanged, only the tarball changed and the build is
highly optimized i can assure you for 100% that i compare PHP
7.1.5RC1 with https://github.com/zendtech/php-src
<https://github.com/zendtech/php-src> downloaded today

maybe the PGO-profiling running autotests and fuzzy-calls on the
whole application as well as 2000 cms-requests combined with the
compiler flags improves the JIT itself

without opcache the results for 7.1.5 are *dramatically* slower

and i repeated the test upgrade/downgrade packages and run "ab"
multiple times on that machine - attached the "php.spec" which is
used for the build

in just downloaded the zip from https://github.com/zendtech/php-src
<https://github.com/zendtech/php-src>, renamed it to "php-7.2.0",
made a tar.xz archive, changed the version on teh frist line in the
spec file and fired the build/profiling - nothing else changed

You have xdebug enabled...

loaded, but not enabled

xdebug.default_enable = 0
xdebug.profiler_enable = 0
xdebug.profiler_enable_trigger = 1

7.1.5: Requests per second: 136.46
7.1.5 opcache: Requests per second: 316.77
7.2.0 JIT PGO: Requests per second: 925.96
7.2.0 JIT NON-PGO: Requests per second: 849.99

around 8% are the difference with or without PGO, no idea how much
strict-types and a 100% typehinted codebase makes a difference to the
JIT operations

7 years ago by Nikita Popov — view source

unread

>
> Am 02.05.2017 um 20:53 schrieb Nikita Popov:
>
>> These results are very unlikely. I'm 95% sure your benchmark is
>> broken. My first guess would be that you're benchmarking PHP +
>> JIT against PHP without opcache. Please share the relevant
>> (opcache-related) portion of the php.inis you used
>>
>> no they are not - since i build RPM packages and even the whole
>> spec-file is unchanged, only the tarball changed and the build is
>> highly optimized i can assure you for 100% that i compare PHP
>> 7.1.5RC1 with https://github.com/zendtech/php-src
>> downloaded today
>>
>> maybe the PGO-profiling running autotests and fuzzy-calls on the
>> whole application as well as 2000 cms-requests combined with the
>> compiler flags improves the JIT itself
>>
>> without opcache the results for 7.1.5 are *dramatically* slower
>>
>> and i repeated the test upgrade/downgrade packages and run "ab"
>> multiple times on that machine - attached the "php.spec" which is
>> used for the build
>>
>> in just downloaded the zip from https://github.com/zendtech/php-src
>> , renamed it to "php-7.2.0",
>> made a tar.xz archive, changed the version on teh frist line in the
>> spec file and fired the build/profiling - nothing else changed
>>
> >
>
>> You have xdebug enabled...
>>
>
> loaded, but not enabled
>
> xdebug.default_enable = 0
> xdebug.profiler_enable = 0
> xdebug.profiler_enable_trigger = 1
>
> 7.1.5: Requests per second: 136.46
> 7.1.5 opcache: Requests per second: 316.77
> 7.2.0 JIT PGO: Requests per second: 925.96
> 7.2.0 JIT NON-PGO: Requests per second: 849.99
>
> around 8% are the difference with or without PGO, no idea how much
> strict-types and a 100% typehinted codebase makes a difference to the JIT
> operations
>

This is a common misconception. If you have loaded xdebug you will incur a
major performance hit, regardless of ini settings. xdebug.default_enable is
a badly named option, which controls display of stack traces, not whether
xdebug is enabled. As far as I know, there is no way to disable xdebug once
it has been loaded.

Please unload xdebug and try again.

Nikita

7 years ago by lists@rhsoft.net — view source

unread

Am 02.05.2017 um 21:12 schrieb Nikita Popov:

On Tue, May 2, 2017 at 9:04 PM, lists@rhsoft.net

You have xdebug enabled...
loaded, but not enabled

xdebug.default_enable = 0
xdebug.profiler_enable = 0
xdebug.profiler_enable_trigger = 1

7.1.5:             Requests per second: 136.46
7.1.5 opcache:     Requests per second: 316.77
7.2.0 JIT PGO:     Requests per second: 925.96
7.2.0 JIT NON-PGO: Requests per second: 849.99

around 8% are the difference with or without PGO, no idea how much
strict-types and a 100% typehinted codebase makes a difference to
the JIT operations
This is a common misconception. If you have loaded xdebug you will incur
a major performance hit, regardless of ini settings.
xdebug.default_enable is a badly named option, which controls display of
stack traces, not whether xdebug is enabled. As far as I know, there is
no way to disable xdebug once it has been loaded.

Please unload xdebug and try again

argh - when the trigger is set to generate profiling files runtime is
doubled compared with xdebug just loaded but indeed xdebug has a massive
impact

7.1.5 PGO: Requests per second: 886.63
7.2.0 JIT PGO: Requests per second: 925.96

OK, than it are "only" 5% on a highly optimized codebase

interesting is the impact of PGO-profiling which has likely a
code-coverage nearly to 95% and that without xdebug our cms-system seems
to be that efficient that it nearly runs as fast on a quad-core i7 from
2011 than on a 12-core Xeon from 2015.....

7 years ago by Sara Golemon — view source

unread

OK, than it are "only" 5% on a highly optimized codebase

Awww... that makes me kinda sad to hear. But perhaps with more work
that needle can be moved upwards.

One question which I didn't see answered (in my admittedly quick
scan): What codebase are you testing against? Wordpress stock
install?

-Sara

7 years ago by lists@rhsoft.net — view source

unread

Am 03.05.2017 um 02:37 schrieb Sara Golemon:

OK, than it are "only" 5% on a highly optimized codebase

Awww... that makes me kinda sad to hear. But perhaps with more work
that needle can be moved upwards.

One question which I didn't see answered (in my admittedly quick
scan): What codebase are you testing against? Wordpress stock
install?

100% internal codebase without 3rd party libraries developed over the
last 15 years and in the meantime 100%
strict-types/typehints/return-types only missing some commented
nullable/void return types because i need to wait with 7.1.x for another
busy guy with some of his instances :-)

the profiling system is our internal demo-system with every depolyable
module installed, runs the auto-testsuite inclduing refelection fuzzy-calls

override config and switch to a clean database
create 86 tables running the installer
60 unit tests (heavy sound on a RAID10 HDD system)
1200 reflection-fuzzy calls with random params to methdos
calling every backend url with a configured admin user
spider over 1600 frontend urls (curl, dom)
controlled by a bash/php mix running also the intermediate CLI

that beast is highly optimized and the core-cms with just 4 navigation
points and some lines of text has 12 function calls above 1% in xdebug :-)

well, the demo-install for profiling has magnitudes more running code

so 5% is not that bad - PHP 5.6 to 7.0 brought only 45% on that codebase
while in the meantime the performance of the code got doubled at it's
own (thanks xdebug) - hence someone heard my cry "WTF unbelievebale "
after the first run not realize that the loaded xdebug without JIT makes
most of the factor 2.5 difference while my epectation was 5-10% if any
measureable improvement at all

http://corecms.thelounge.net/show_content.php?sid=2
0.0027 / 0.0013 - cuurently running on 7.0.18

0.0027: Header always set "X-Response-Time" "%D us"
0.0013: microtime(true) first line versus microtime(true) final

3500 requests/second with 7.2-JIT on a 4-core i7 versus 4500/second with
7.8.18 on a 12-core Xeon tells me that PHP itself is only some part of
the runtime of the ab-benchmark given that "ab" on my testmachine eats
nearly one of the 4 cores at it's own and even under 100% load generate
times in the browser stays below 0.01 seconds

250 sequentiell curl requests parsing the "X-Response-Time" average with
identical content

DYNAMIC CMS: 2382 us
STATIC HTM: 242 us
STATIC PHP: 288 us

7 years ago by Sara Golemon — view source

unread

100% internal codebase without 3rd party libraries developed over the last
15 years and in the meantime 100% strict-types/typehints/return-types only
missing some commented nullable/void return types because i need to wait
with 7.1.x for another busy guy with some of his instances :-)

Would you mind running some numbers against "commodity
apps/frameworks" (e.g. Wordpress, ZendFramework, Laravel, etc...) I
expect that will be far more telling (in a potentially positive way).
For example, HHVM performs best on "highly optimized code", which is
why it tends to look a bit tepid against PHP 7 when run on "normal"
codebases (particularly WP, which is in many ways not JIT friendly).
I realize the proof of concept jit branch is probably missing quite a
bit of optimizations compared to where it will land, so it may not
make then same kind of difference, but hey... curiosity killed the
cat.

the profiling system is our internal demo-system with every depolyable
module installed, runs the auto-testsuite inclduing refelection fuzzy-calls

Sounds very comprehensive and I'm glad you're using such a heavy
benchmarking suite. :D

so 5% is not that bad

Apologies if I gave the impression that I felt it was "bad". My
expressed disappointment was only relative to the headline "ZOMGWTFBBQ
100% GAIN" headline (which I think most of us regarded as probably
wrong in some way due to the early-days nature of the jit branch). :)

-Sara

7 years ago by lists@rhsoft.net — view source

unread

Am 03.05.2017 um 05:37 schrieb Sara Golemon:

100% internal codebase without 3rd party libraries developed over the last
15 years and in the meantime 100% strict-types/typehints/return-types only
missing some commented nullable/void return types because i need to wait
with 7.1.x for another busy guy with some of his instances :-)

Would you mind running some numbers against "commodity
apps/frameworks" (e.g. Wordpress, ZendFramework, Laravel, etc...) I
expect that will be far more telling (in a potentially positive way).
For example, HHVM performs best on "highly optimized code", which is
why it tends to look a bit tepid against PHP 7 when run on "normal"

sorry, all my setups have a lot of stuff in the config to kill stuff
like wordpress/joomla/typo3 so nobody comes to the idea installing after
that badly mainatined stuff on my machines (for our own CMS there is a
deplyoment system maintaining 100, 200 or 1000 instances with s single
shell command)

codebases (particularly WP, which is in many ways not JIT friendly).
I realize the proof of concept jit branch is probably missing quite a
bit of optimizations compared to where it will land, so it may not
make then same kind of difference, but hey... curiosity killed the
cat.

WP is not friendly to anything :-)

on the other hand i guess that more the code itself is bloatet the
impact of performance improvements is bigger compared to my code where
httpd and connection handling takes as long as the application while 20%
of my runtime costs are database connections and queries which don't get
faster in any case

the profiling system is our internal demo-system with every depolyable
module installed, runs the auto-testsuite inclduing refelection fuzzy-calls

Sounds very comprehensive and I'm glad you're using such a heavy
benchmarking suite. :D

well, it was more about "how do i feed
https://software.intel.com/en-us/blogs/2015/10/09/pgo-let-it-go-php with
as much as possible runtime informations" and later "how do i get that
beast deployable again after rewrite the whole codebase to native PHP7
with minimized fallout on type-errors"

so 5% is not that bad

Apologies if I gave the impression that I felt it was "bad". My
expressed disappointment was only relative to the headline "ZOMGWTFBBQ
100% GAIN" headline (which I think most of us regarded as probably
wrong in some way due to the early-days nature of the jit branch). :)

yeah, i thought i start a party after the first result which maxed out
at 4200/seconds after a few thousand requests on my workstation looking
in the direction of the 12-core Xeon "and what will you do with that" :-)

what will be interesting with the JIT is how it benefits from newer CPU
architectures, currently we optimize for sandybdrige, with next summer
the cluster nodes will be replaced and the VMware EVC mode raised to
skylake leading to maintain a different PHP PGP-build with -mtune=native

looking at "Use AVX instructions (if enabled and available)"
https://github.com/zendtech/php-src/commit/d9474c784041745a455dd5be06b52c0029e40697
there will be some space of improvement on both sides, the PHP binary
itself as well as what code the JIT can prodcue at the end (skylake Xeon
will intoduce AVX512 as example)

7 years ago by eli@eliw.com — view source

unread

Would you mind running some numbers against "commodity
apps/frameworks" (e.g. Wordpress, ZendFramework, Laravel, etc...) I
expect that will be far more telling (in a potentially positive way).
For example, HHVM performs best on "highly optimized code", which is
why it tends to look a bit tepid against PHP 7 when run on "normal"

sorry, all my setups have a lot of stuff in the config to kill stuff like wordpress/joomla/typo3 so nobody comes to the idea installing after that badly mainatined stuff on my machines (for our own CMS there is a deplyoment system maintaining 100, 200 or 1000 instances with s single shell command)

However the majority of PHP in the wild is in fact running on these commodity frameworks. Especially Wordpress.

So if your numbers don't show the benefit there. Or even show a decrease. Then it's going to be rough moving forward.

Eli

7 years ago by lists@rhsoft.net — view source

unread

Am 03.05.2017 um 13:20 schrieb Eli:

Would you mind running some numbers against "commodity
apps/frameworks" (e.g. Wordpress, ZendFramework, Laravel, etc...) I
expect that will be far more telling (in a potentially positive way).
For example, HHVM performs best on "highly optimized code", which is
why it tends to look a bit tepid against PHP 7 when run on "normal"

sorry, all my setups have a lot of stuff in the config to kill stuff like wordpress/joomla/typo3 so nobody comes to the idea installing after that badly mainatined stuff on my machines (for our own CMS there is a deplyoment system maintaining 100, 200 or 1000 instances with s single shell command)

However the majority of PHP in the wild is in fact running on these commodity frameworks. Especially Wordpress.

So if your numbers don't show the benefit there. Or even show a decrease. Then it's going to be rough moving forward.

well, i think it's also valueable to know that the JIT don't harm or
even improve slightly code which is not mainstream and has already a
small footprint

for wordpress, joomla and other mainstream software people which use it
in their daily workflow are the better ones for setup demosystems
compared to when i loveless setup soemthing probably never used that way
in real life

what i can offer is everytime when the GIT branch get noticeable changes
to fireup a build and watch how it envolves here in case of performance

7 years ago by S.A.N — view source

unread

7.1.5 PGO: Requests per second: 886.63
7.2.0 JIT PGO: Requests per second: 925.96

OK, than it are "only" 5% on a highly optimized codebase

Try PHP script as daemon mode, will speed up 2x