11 years ago by Sebastian Bergmann — view source

unread

Am 05.05.2014 15:07, schrieb Dmitry Stogov:

But we feel, we’ve proven enough out to open it up for review, feedback
and assistance, and wanted to involve the community as soon as we managed
to get on a promising direction. There’s more work to do in finishing
support of all extensions and continue to make some additional engine
improvements.

Tried running PHPUnit 4.1's test suite with phpng:
https://gist.github.com/sebastianbergmann/ad46f8e4e923578a72d5

11 years ago by Dmitry Stogov — view source

unread

It looks like a bug in GC. We will care about it in some point.

Thanks. Dmitry.

On Mon, May 5, 2014 at 5:49 PM, Sebastian Bergmann sebastian@php.netwrote:

Am 05.05.2014 15:07, schrieb Dmitry Stogov:

But we feel, we’ve proven enough out to open it up for review, feedback
and assistance, and wanted to involve the community as soon as we managed
to get on a promising direction. There’s more work to do in finishing
support of all extensions and continue to make some additional engine
improvements.

Tried running PHPUnit 4.1's test suite with phpng:
https://gist.github.com/sebastianbergmann/ad46f8e4e923578a72d5

11 years ago by Andrea Faulds — view source

unread

This seems like something that would be a big feature for PHP 6. How much backwards-compatibility does it break?

--
Andrea Faulds
http://ajf.me/

11 years ago by Dmitry Stogov — view source

unread

I would say it must be 100% compatible at the end, may be with exception
for very rare and unclear cases that worked just because of existing
implementation. (e.g. mixing foreach and next() on the same array).

Thanks. Dmitry.

This seems like something that would be a big feature for PHP 6. How much
backwards-compatibility does it break?

--
Andrea Faulds
http://ajf.me/

11 years ago by Andrea Faulds — view source

unread

I would say it must be 100% compatible at the end, may be with exception for very rare and unclear cases that worked just because of existing implementation. (e.g. mixing foreach and next() on the same array).

Of course it’s compatible with user land, what I meant to ask was how compatible is the ABI? Would this require a lot of changes to extensions and such?

--
Andrea Faulds
http://ajf.me/

11 years ago by Andi Gutmans — view source

unread

Yes it does require changes in the extensions. I must say we’ve been able to hammer through them pretty quickly but it does require change due to the refactoring of the data types. It’s a refactoring exercise though not a rewrite. So relatively speaking not too painful and I think we’ve already worked through many of them so there are a lot of examples.

Andi

I would say it must be 100% compatible at the end, may be with exception for very rare and unclear cases that worked just because of existing implementation. (e.g. mixing foreach and next() on the same array).

Of course it’s compatible with user land, what I meant to ask was how compatible is the ABI? Would this require a lot of changes to extensions and such?

--
Andrea Faulds
http://ajf.me/

11 years ago by Andrea Faulds — view source

unread

Hmm. This is unrelated, but while you’re busy refactoring PHP, could you comment and document the bits you refactor a bit more? More documentation can only be a good thing, and it’s something Zend is sorely lacking.

--
Andrea Faulds
http://ajf.me/

11 years ago by Dmitry Stogov — view source

unread

I'm doing it right now: https://wiki.php.net/phpng-int

Thanks. Dmitry.

Hmm. This is unrelated, but while you’re busy refactoring PHP, could you
comment and document the bits you refactor a bit more? More documentation
can only be a good thing, and it’s something Zend is sorely lacking.

--
Andrea Faulds
http://ajf.me/

11 years ago by Andrea Faulds — view source

unread

I'm doing it right now: https://wiki.php.net/phpng-int

That’s great to see, thanks!

--
Andrea Faulds
http://ajf.me/

11 years ago by johannes@schlueters.de — view source

unread

Hi,

I'm doing it right now: https://wiki.php.net/phpng-int

I haven't looked deeply into it, but a thing I wondered about old zvals
already:

CELL Format (zval)

Have you tried movin the TYPE to the einning of the structure? - I
assume checking/setting the type is the most common operation. By moving
it upfront the address of the type equals the address of the zval, thus
the CPU should be ale to access it without offset, might improve
pipeline throughput.

I haven't measured that, but miht be interesting while already chaning
this.

johannes

11 years ago by Dmitry Stogov — view source

unread

The improvement are made with JIT in mind. In the future, in many places we
might predict the type and don't check it at all.

Thanks. Dmitry.

On Mon, May 5, 2014 at 7:26 PM, Johannes Schlüter johannes@schlueters.dewrote:

Hi,

I'm doing it right now: https://wiki.php.net/phpng-int

I haven't looked deeply into it, but a thing I wondered about old zvals
already:

CELL Format (zval)

+----------------------------------------------------------------------------------+
| VALUE (64-bit)
|

+--------------------------+-------------+-------------+-------------+-------------+
| UNUSED (32-bit) | UNUSED | const_flags | type_flags |
TYPE |

+--------------------------+-------------+-------------+-------------+-------------+
63 32 31 24 23 16 15 8 7
0

Have you tried movin the TYPE to the einning of the structure? - I
assume checking/setting the type is the most common operation. By moving
it upfront the address of the type equals the address of the zval, thus
the CPU should be ale to access it without offset, might improve
pipeline throughput.

I haven't measured that, but miht be interesting while already chaning
this.

johannes

11 years ago by Bob Weinand — view source

unread

Am 5.5.2014 um 16:31 schrieb Dmitry Stogov dmitry@zend.com:

I'm doing it right now: https://wiki.php.net/phpng-int

Thanks. Dmitry.

Hmm. This is unrelated, but while you’re busy refactoring PHP, could you
comment and document the bits you refactor a bit more? More documentation
can only be a good thing, and it’s something Zend is sorely lacking.

--
Andrea Faulds
http://ajf.me/

Nice work!

Just wondering why http://git.php.net/?p=php-src.git;a=commitdiff;h=f614fc68984b2d7fce3f275b8106955b5d910472 was reverted in phpng?

Is there any real reason to still use IS_CONSTANT_ARRAY? The AST variant is much nicer, fixes a bug and removes the IS_CONSTANT_INDEX hack.

Bob

11 years ago by Dmitry Stogov — view source

unread

Hi Bob,

Actually, our patches were orthogonal, resolving the same hack in different
ways.
Sorry, I had a lot of work, and missed your patch when you committed it.
Then I just wasn't able to merge them in reasonable time.

We will need to discuss it in more details and then reimplement on top of
phpng or revert in master.

The thing I don't like in the patch, that you converted every
IS_CONSTANT_ARRAY into IS_CONSTANT_AST.
With this approach all the constant arrays have to be evaluated at run-time.
However, I think about optimization when we won't copy constant arrays
(without symbolic constants) from shared memory at all.
I might be wrong, or misunderstand something in your patch, so we'll
definitely need to talk, but now I would prefer to close the known problems
first.

Let's speak on next week.

Thanks. Dmitry.

Am 5.5.2014 um 16:31 schrieb Dmitry Stogov dmitry@zend.com:

I'm doing it right now: https://wiki.php.net/phpng-int

Thanks. Dmitry.

Hmm. This is unrelated, but while you’re busy refactoring PHP, could you
comment and document the bits you refactor a bit more? More documentation
can only be a good thing, and it’s something Zend is sorely lacking.

--
Andrea Faulds
http://ajf.me/

Nice work!

Just wondering why
http://git.php.net/?p=php-src.git;a=commitdiff;h=f614fc68984b2d7fce3f275b8106955b5d910472was reverted in phpng?

Is there any real reason to still use IS_CONSTANT_ARRAY? The AST variant
is much nicer, fixes a bug and removes the IS_CONSTANT_INDEX hack.

Bob

11 years ago by Andi Gutmans — view source

unread

Yep agree. We started and definitely will need to do mode.

Sent from my iPhone

Hmm. This is unrelated, but while you’re busy refactoring PHP, could you comment and document the bits you refactor a bit more? More documentation can only be a good thing, and it’s something Zend is sorely lacking.

--
Andrea Faulds
http://ajf.me/

11 years ago by Kevin Ingwersen — view source

unread

Am 05.05.2014 um 16:33 schrieb Andi Gutmans andi@zend.com:

Yep agree. We started and definitely will need to do mode.

Sent from my iPhone
Mode? I think auto-correct likes you today =p

Thats awesome, thanks for the replies. Hopefully the people at PHP-CPP will update their API as well to suit the updates, so that these things will continue to work as before.

One thing, as you are on documenting: Documenting class usage from the C side (modifying properties etc) would be good to be documented a little bit more. Last time I looked into it, the object accessing API part was poorly documented.

Going to do testing with phpng now and send results later. I will silently inject it into my currently running site, seeing how the performance will change.

Kind regards, Ingwie.

Hmm. This is unrelated, but while you’re busy refactoring PHP, could you comment and document the bits you refactor a bit more? More documentation can only be a good thing, and it’s something Zend is sorely lacking.

--
Andrea Faulds
http://ajf.me/

11 years ago by Laruence — view source

unread

Hey:

I would say it must be 100% compatible at the end, may be with exception for very rare and unclear cases that worked just because of existing implementation. (e.g. mixing foreach and next() on the same array).

Of course it’s compatible with user land, what I meant to ask was how compatible is the ABI? Would this require a lot of changes to extensions and such?

 Not very much changes.  the most significant changes are zend_hash_* APIs.

 but it's not very difficult to port a old ext into php-ng form

thanks

--
Andrea Faulds
http://ajf.me/

--
Laruence Xinchen Hui
http://www.laruence.com/

11 years ago by Dmitry Stogov — view source

unread

yes. all the extensions will have to be changed.
In some cases it may be difficult, but usually not too difficult.
For example today Xinchen added support for 3 or 5 new extensions.

Thanks. Dmitry.

I would say it must be 100% compatible at the end, may be with exception
for very rare and unclear cases that worked just because of existing
implementation. (e.g. mixing foreach and next() on the same array).

Of course it’s compatible with user land, what I meant to ask was how
compatible is the ABI? Would this require a lot of changes to extensions
and such?

--
Andrea Faulds
http://ajf.me/

11 years ago by Tjerk Meesters — view source

unread

Hi,

I coauthor the pecl extension for oauth; it currently supports >= 5.2 with
only a few compiler directives, but these engine changes are quite
extensive.

My question is how to best approach this to maintain backwards
compatibility?

Thanks!

yes. all the extensions will have to be changed.
In some cases it may be difficult, but usually not too difficult.
For example today Xinchen added support for 3 or 5 new extensions.

Thanks. Dmitry.

I would say it must be 100% compatible at the end, may be with exception
for very rare and unclear cases that worked just because of existing
implementation. (e.g. mixing foreach and next() on the same array).

Of course it’s compatible with user land, what I meant to ask was how
compatible is the ABI? Would this require a lot of changes to extensions
and such?

--
Andrea Faulds
http://ajf.me/

--

Tjerk

11 years ago by Stas Malyshev — view source

unread

Hi!

My question is how to best approach this to maintain backwards
compatibility?

I think this is best targeted for next major version, in which cases the
extensions probably will have to have separate version/branch. With the
depth of the changes done, I really don't see a good way to run both
from the same codebase without making it either completely
incomprehensible or very unstable.

--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

11 years ago by Kevin Ingwersen — view source

unread

Hey!

This sounds like an amazing bit of updatng! :) I am very much looking forward to the new release of PHP. But, there is soemthing I do not understand.

I know that Zend is the current underlying VM, and further, this is not actual work by the PHP team…am I right?
Further, will phpng replace the vm, or even more parts? Will it still need components like TSRM?

I am just asking, as I have been working with PHP within embedded aplications for a while now (currently working on porting autoconf to cDetect). If parts of the source tree structure changed, I would need to refactor the building part as well.

Also, I am currently running a rather small but active community. To test phpng, I was thinking about silently injecting it thru the cgi script, to see how the performance owuld change. Does phpng have any effects on currently existing extensions (PDO, memcached)?

Kind regards, Ingwie!
Am 05.05.2014 um 15:07 schrieb Dmitry Stogov dmitry@zend.com:

For people who know me it's not a secret that PHP performance is my main
responsibility and passion at Zend. Actually, starting from PHP 5.0 we
already made 6 times speedup on synthetic benchmarks and about 2 times
speedup on real-life applications. We endlessly made improvements in PHP
engine and OPCache. However, by PHP 5.5 release we weren’t be able to make
any serious progress, and among other things started to experiment with
memory managers, JIT technologies and other potential ideas.

I spent a significant amount of time experimenting with JIT, and even
created a PoC of transparent LLVM based JIT compiler embedded into OPCache.
The results on bench.php was just amazing – (0.219 seconds against 2.175 – 10
times speedup of PHP 5.5), but on real-life apps we got just few percent
speedup. This made us look much deeper into some of the runtime
characteristics and what was truly the bottleneck to making more
substantial progress. It was clear the VM is already highly optimized, but
works with data structures that require endless memory allocation,
deallocation and reference counting. Typical real-life PHP application
spends about 20% of the CPU time in memory manager, 10% doing hash tables
operations, 30% in internal functions and only 30% in VM. Of course, we
tried to JIT only VM code and in most cases it had to perform the same
memory allocations. So we decided to change focus and work on the big
bottlenecks. The idea was to change our data types to minimize heap
allocations. This was a very difficult decision because we had to start
with a huge refactoring, and we had no idea whether it’s going to have any
impact or not.

Now I'm glad to present you a result of our recent four month work. It's a
refactoring of the PHP engine that significantly improves performance,
memory usage and builds a foundation for a lot more future performance
improvements incl. JIT. I'll avoid technical details (more details will be
published at http://wiki.php.net/phpng http://wiki.php.net/phpng), but
in few words - we changed the basement trying to keep most of the building
unchanged. Right now the new engine already makes 10-30% speedup of
phpnot only on benchmarks but on real-life applications as well!

Some benchmarks we ran so far:

Wordpress 3.6 – 20.0% gain (253 vs 211 req/sec)

Drupal 6.1 – 11.7% gain (1770 vs 1585 req/sec

Qdig – 15.3% gain (555 vs 482 req/sec)

ZF test app – 30.5% gain (217 vs 166 req/sec)

On some apps we show better results than other PHP implementations. It will
be great if others here could test this on their apps and compare to their
existing PHP version to get additional results.

The re-factoring is not finished yet as the focus was to first test whether
this effort would deliver results. Not all extensions are supported, some
tests are failing, and we also have more ideas for additional improvement.

But we feel, we’ve proven enough out to open it up for review, feedback
and assistance, and wanted to involve the community as soon as we managed
to get on a promising direction. There’s more work to do in finishing
support of all extensions and continue to make some additional engine
improvements.

Please try the refactored PHP engine and provide feedback re: performance,
memory usage and any issues that come up. You may find it in phpng branch
at php.net http://php.net. Some instructions may be found at
http://wiki.php.net/phpng
http://wiki.php.net/phpng. As mentioned, there are some missing
extensions so not everything will run.

I would like to say many thanks to Xinchen and Nikita who made significant
part of presented work.

I think that this engine can make the new major version of PHP we’re
talking about a lot more interesting.

Thanks. Dmitry.

11 years ago by Andi Gutmans — view source

unread

Hi Kevin,

The Zend Engine for many years now has been an integral part of the PHP source tree and enjoys contributors from all over the community (not just Zend).
This refactoring exercise doesn’t change any of that and as Dmitry mentioned, is really a refactoring of the Zend Engine vs. a new thing. This started as experimental work and once we figured it was a great, viable direction to head in we decided to share it. It does have implications for PHP extensions because we made changes to the datatype structures, etc. but it’s been relatively straightforward to update the various extensions to conform to the refactored structures.

The benefits do seem to be massive (exceeding our initial expectations). Memory usage is significantly down, overall performance impact on real-world apps is significant and with these changes it’s opened up a lot of additional opportunities for further optimization down the road as we are not bottlenecked by our data structures.

Andi

Hey!

This sounds like an amazing bit of updatng! :) I am very much looking forward to the new release of PHP. But, there is soemthing I do not understand.

I know that Zend is the current underlying VM, and further, this is not actual work by the PHP team…am I right?
Further, will phpng replace the vm, or even more parts? Will it still need components like TSRM?

I am just asking, as I have been working with PHP within embedded aplications for a while now (currently working on porting autoconf to cDetect). If parts of the source tree structure changed, I would need to refactor the building part as well.

Also, I am currently running a rather small but active community. To test phpng, I was thinking about silently injecting it thru the cgi script, to see how the performance owuld change. Does phpng have any effects on currently existing extensions (PDO, memcached)?

Kind regards, Ingwie!
Am 05.05.2014 um 15:07 schrieb Dmitry Stogov dmitry@zend.com:

For people who know me it's not a secret that PHP performance is my main
responsibility and passion at Zend. Actually, starting from PHP 5.0 we
already made 6 times speedup on synthetic benchmarks and about 2 times
speedup on real-life applications. We endlessly made improvements in PHP
engine and OPCache. However, by PHP 5.5 release we weren’t be able to make
any serious progress, and among other things started to experiment with
memory managers, JIT technologies and other potential ideas.

I spent a significant amount of time experimenting with JIT, and even
created a PoC of transparent LLVM based JIT compiler embedded into OPCache.
The results on bench.php was just amazing – (0.219 seconds against 2.175 – 10
times speedup of PHP 5.5), but on real-life apps we got just few percent
speedup. This made us look much deeper into some of the runtime
characteristics and what was truly the bottleneck to making more
substantial progress. It was clear the VM is already highly optimized, but
works with data structures that require endless memory allocation,
deallocation and reference counting. Typical real-life PHP application
spends about 20% of the CPU time in memory manager, 10% doing hash tables
operations, 30% in internal functions and only 30% in VM. Of course, we
tried to JIT only VM code and in most cases it had to perform the same
memory allocations. So we decided to change focus and work on the big
bottlenecks. The idea was to change our data types to minimize heap
allocations. This was a very difficult decision because we had to start
with a huge refactoring, and we had no idea whether it’s going to have any
impact or not.

Now I'm glad to present you a result of our recent four month work. It's a
refactoring of the PHP engine that significantly improves performance,
memory usage and builds a foundation for a lot more future performance
improvements incl. JIT. I'll avoid technical details (more details will be
published at http://wiki.php.net/phpng http://wiki.php.net/phpng), but
in few words - we changed the basement trying to keep most of the building
unchanged. Right now the new engine already makes 10-30% speedup of
phpnot only on benchmarks but on real-life applications as well!

Some benchmarks we ran so far:

Wordpress 3.6 – 20.0% gain (253 vs 211 req/sec)

Drupal 6.1 – 11.7% gain (1770 vs 1585 req/sec

Qdig – 15.3% gain (555 vs 482 req/sec)

ZF test app – 30.5% gain (217 vs 166 req/sec)

On some apps we show better results than other PHP implementations. It will
be great if others here could test this on their apps and compare to their
existing PHP version to get additional results.

The re-factoring is not finished yet as the focus was to first test whether
this effort would deliver results. Not all extensions are supported, some
tests are failing, and we also have more ideas for additional improvement.

But we feel, we’ve proven enough out to open it up for review, feedback
and assistance, and wanted to involve the community as soon as we managed
to get on a promising direction. There’s more work to do in finishing
support of all extensions and continue to make some additional engine
improvements.

Please try the refactored PHP engine and provide feedback re: performance,
memory usage and any issues that come up. You may find it in phpng branch
at php.net http://php.net. Some instructions may be found at
http://wiki.php.net/phpng
http://wiki.php.net/phpng. As mentioned, there are some missing
extensions so not everything will run.

I would like to say many thanks to Xinchen and Nikita who made significant
part of presented work.

I think that this engine can make the new major version of PHP we’re
talking about a lot more interesting.

Thanks. Dmitry.

11 years ago by Dmitry Stogov — view source

unread

On Mon, May 5, 2014 at 6:07 PM, Kevin Ingwersen
ingwie2000@googlemail.comwrote:

Hey!

This sounds like an amazing bit of updatng! :) I am very much looking
forward to the new release of PHP. But, there is soemthing I do not
understand.

I know that Zend is the current underlying VM, and further, this is not
actual work by the PHP team…am I right?
Further, will phpng replace the vm, or even more parts?

I wouldn't talk about replacement. It's the same PHP with the evoluted Zend
Engine (even the less significant change than moving from PHP-4 to PHP-5).

Will it still need components like TSRM?

yes.

I am just asking, as I have been working with PHP within embedded
aplications for a while now (currently working on porting autoconf to
cDetect). If parts of the source tree structure changed, I would need to
refactor the building part as well.

Also, I am currently running a rather small but active community. To test
phpng, I was thinking about silently injecting it thru the cgi script, to
see how the performance owuld change. Does phpng have any effects on
currently existing extensions (PDO, memcached)?

yes. All the extensions need to be "ported" to support phpng.

Thanks. Dmitry.

11 years ago by php@thekid.de — view source

unread

Hi,

Now I'm glad to present you a result of our recent four month work. It's a
refactoring of the PHP engine that significantly improves performance,
memory usage and builds a foundation for a lot more future performance
improvements incl. JIT.

Wow, some great work!

I do see a couple of failures running the XP Framework’s test suite (I’ll have a look into these and try to figure out what’s going on), but comparing these that are working, I can see the following:

PHP 5.4
OK: 1019/1042 run (23 skipped), 1019 succeeded, 0 failed
Memory used: 13877.78 kB (14830.66 kB peak)
Time taken: 0.692 seconds

PHP 5.5
OK: 1019/1042 run (23 skipped), 1019 succeeded, 0 failed
Memory used: 13942.30 kB (14900.96 kB peak)
Time taken: 0.583 seconds

phpng
OK: 1017/1042 run (25 skipped), 1017 succeeded, 0 failed
Memory used: 10790.42 kB (11754.17 kB peak)
Time taken: 0.385 seconds

Timm

11 years ago by Dmitry Stogov — view source

unread

Thanks for results! 34% speed improvement and 22% memory consumption
improvement in comparison to PHP 5.5
I'm always getting better results when bench against competitors myself :)
So the third party benchmarks and comparisons really helps.

phpng is still far away from production quality yet. It actually served the
first wordpress request just a month ago.
I hope we will able to catch most of the bugs soon.

Thanks. Dmitry.

Hi,

Now I'm glad to present you a result of our recent four month work. It's
a
refactoring of the PHP engine that significantly improves performance,
memory usage and builds a foundation for a lot more future performance
improvements incl. JIT.

Wow, some great work!

I do see a couple of failures running the XP Framework’s test suite (I’ll
have a look into these and try to figure out what’s going on), but
comparing these that are working, I can see the following:

PHP 5.4
OK: 1019/1042 run (23 skipped), 1019 succeeded, 0 failed
Memory used: 13877.78 kB (14830.66 kB peak)
Time taken: 0.692 seconds

PHP 5.5
OK: 1019/1042 run (23 skipped), 1019 succeeded, 0 failed
Memory used: 13942.30 kB (14900.96 kB peak)
Time taken: 0.583 seconds

phpng
OK: 1017/1042 run (25 skipped), 1017 succeeded, 0 failed
Memory used: 10790.42 kB (11754.17 kB peak)
Time taken: 0.385 seconds

Timm

11 years ago by Timm Friebe — view source

unread

Hi again,

...

It's been brought to my attention that in my previous comparison, there were
also two skipped tests I'd overlooked, I've commented them so we're actually
comparing apples with apples. Also, I got a couple more tests working, so here
we go:

5.5
OK: 2987/3013 run (26 skipped), 2987 succeeded, 0 failed
Memory used: 37065.45 kB (37350.91 kB peak)
Time taken: 2.133 seconds

phpng
OK: 2987/3013 run (26 skipped), 2987 succeeded, 0 failed
Memory used: 29315.42 kB (29583.73 kB peak)
Time taken: 1.603 seconds

(I’ll have a look into these and try to figure out what’s going on)

Here's my summary of why there are failing tests (all of these work fine with
5.4, 5.6-beta2, 5.7-dev-master). I've tried fixing them but haven't succeeded,
but I thought I'd tell you anyways.

Process control functions
============================
proc_close() seems to always return exit code 1 regardless of the actual exit
code:

$ ./sapi/cli/php -r '$p= proc_open("ls -al buildconf", [0 => ["pipe", "r"], 1 =>
["pipe", "w"], 2 => ["pipe", "w"]], $pipes); var_dump(fread($pipes[1], 1024));
foreach ($pipes as $pipe) { fclose($pipe); } var_dump(proc_close($p));'
string(54) "-rwxr-xr-x 1 friebe friebe 772 Apr 18 16:21 buildconf
"
int(1)

(should be: 0)

$ ./sapi/cli/php -r '$p= proc_open("ls -ü", [0 => ["pipe", "r"], 1 => ["pipe",
"w"], 2 => ["pipe", "w"]], $pipes); var_dump(fread($pipes[1], 1024)); foreach
($pipes as $pipe) { fclose($pipe); } var_dump(proc_close($p));'
string(0) ""
int(1)

(should be: 2)

It works fine in both situations if I don't explicitely fclose() all the pipes.

Fatal error with DateTime
============================
DateTime::date_timezone_set() must be derived from Date::date_timezone_set in
Unknown on line 0
DateTimeInterface::date_format() must be derived from Date::date_format in
Unknown on line 0

Unfortunately I can't exactly produce a small script to reproduce, but the
Date class mentioned above basically does this:

class Date extends Object {
public function __construct($in) {
$this->value= date_create($in, timezone_open(...));
}
}

These errors are both produced by zend_parse_method_parameters(), where this_ptr
points to the class Date and not to DateTime. I've only seen them with the
date extension, nowhere else so far. Maybe that helps you.

Fatal error: Can only throw objects
======================================
References cause this. The ensure() function prevents uninitialized variable
warnings:

$ ./sapi/cli/php -r 'function ensure(&$e) { if ($e) return true; $e= null;
return false; } $e= new Exception(""); if (ensure($e)) { throw $e; }'

Fatal error: Can only throw objects in Command line code on line 1

Expected outcome:

PHP Fatal error: Uncaught exception 'Exception' in Command line code:1

...

Adding a var_dump($e) right before the throw will actually dump the exception,
so I guess this some refcount issue.

-Timm

11 years ago by Pierre Joye — view source

unread

hi!

For people who know me it's not a secret that PHP performance is my main
responsibility and passion at Zend. Actually, starting from PHP 5.0 we
already made 6 times speedup on synthetic benchmarks and about 2 times
speedup on real-life applications. We endlessly made improvements in PHP
engine and OPCache. However, by PHP 5.5 release we weren’t be able to make
any serious progress, and among other things started to experiment with
memory managers, JIT technologies and other potential ideas.

I spent a significant amount of time experimenting with JIT, and even
created a PoC of transparent LLVM based JIT compiler embedded into OPCache.
The results on bench.php was just amazing – (0.219 seconds against 2.175 – 10
times speedup of PHP 5.5), but on real-life apps we got just few percent
speedup. This made us look much deeper into some of the runtime
characteristics and what was truly the bottleneck to making more
substantial progress. It was clear the VM is already highly optimized, but
works with data structures that require endless memory allocation,
deallocation and reference counting. Typical real-life PHP application
spends about 20% of the CPU time in memory manager, 10% doing hash tables
operations, 30% in internal functions and only 30% in VM. Of course, we
tried to JIT only VM code and in most cases it had to perform the same
memory allocations. So we decided to change focus and work on the big
bottlenecks. The idea was to change our data types to minimize heap
allocations. This was a very difficult decision because we had to start
with a huge refactoring, and we had no idea whether it’s going to have any
impact or not.

Now I'm glad to present you a result of our recent four month work. It's a
refactoring of the PHP engine that significantly improves performance,
memory usage and builds a foundation for a lot more future performance
improvements incl. JIT. I'll avoid technical details (more details will be
published at http://wiki.php.net/phpng http://wiki.php.net/phpng), but
in few words - we changed the basement trying to keep most of the building
unchanged. Right now the new engine already makes 10-30% speedup of
phpnot only on benchmarks but on real-life applications as well!

Some benchmarks we ran so far:

Wordpress 3.6 – 20.0% gain (253 vs 211 req/sec)

Drupal 6.1 – 11.7% gain (1770 vs 1585 req/sec

Qdig – 15.3% gain (555 vs 482 req/sec)

ZF test app – 30.5% gain (217 vs 166 req/sec)

On some apps we show better results than other PHP implementations. It will
be great if others here could test this on their apps and compare to their
existing PHP version to get additional results.

The re-factoring is not finished yet as the focus was to first test whether
this effort would deliver results. Not all extensions are supported, some
tests are failing, and we also have more ideas for additional improvement.

But we feel, we’ve proven enough out to open it up for review, feedback
and assistance, and wanted to involve the community as soon as we managed
to get on a promising direction. There’s more work to do in finishing
support of all extensions and continue to make some additional engine
improvements.

It is an awesome step moving forward, thanks to everyone involved in
this work! I will see how fast we can integrate this branch in our
tests.

Please try the refactored PHP engine and provide feedback re: performance,
memory usage and any issues that come up. You may find it in phpng branch
at php.net http://php.net. Some instructions may be found at
http://wiki.php.net/phpng
http://wiki.php.net/phpng. As mentioned, there are some missing
extensions so not everything will run.

Could you move these pages (this one and the internal one) either to
/draft or /rfc please? the latter makes more sense

However I suppose basic windows tests are missing, as well as TS
sapis, right? Is it at least supposed to work in TS mode, by design?
Or should we expect some major issues?

I would like to say many thanks to Xinchen and Nikita who made significant
part of presented work.

I think that this engine can make the new major version of PHP we’re
talking about a lot more interesting.

Do I understand correctly that you target php 6 for these changes, right?

Also one thing is totally missing, and is also caused by something
that becomes a common case, the 64bit support as it should be has been
totally ignored. That makes Anatol work to keep the 64bit branch
almost useless. My take on it is that we should either:

move your patch to this branch and then merge to master
merge the 64bit branche then this patch

That being said, I am not sure what requires less effort but I do not
feel comfortable to scratch the 64bit branch and begins from almost
zero.

There is something that we have to improve, drastically,
communication. Such major changes, developed (and still being worked
on) should pop up in the list and in the radar of other developers
much earlier in the process. For two critical reasons:

Avoid work conflicts, be for similar features/changes or other
areas being deadly affected by such changes
Increase visibility, feedback and cooperations. The earlier other
developers are involved the better it is to get more people involved
in the engine development and maintenance.

I am not sure how to change that or make developers being more open
and stopped hidden developments. Any ideas or thoughts on this topic
are welcome. It is critical and vital for php, a must do.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Levi Morrison — view source

unread

I am not sure how to change that or make developers being more open
and stopped hidden developments. Any ideas or thoughts on this topic
are welcome. It is critical and vital for php, a must do.

This isn't a thought on how to do this, but rather another reason to be
open: there are people out there with the talent and time who want to work
on this stuff if only they knew about it (I know of at least one person).

11 years ago by Dmitry Stogov — view source

unread

I understand.
But now it's open and we still have a lot of problems to solve.
If that person like to join and smart enough to not make slowdown, they are
welcome now.

Thanks. Dmitry.

On Tue, May 6, 2014 at 2:28 AM, Levi Morrison morrison.levi@gmail.comwrote:

I am not sure how to change that or make developers being more open

and stopped hidden developments. Any ideas or thoughts on this topic
are welcome. It is critical and vital for php, a must do.

This isn't a thought on how to do this, but rather another reason to be
open: there are people out there with the talent and time who want to work
on this stuff if only they knew about it (I know of at least one person).

11 years ago by Dmitry Stogov — view source

unread

It is an awesome step moving forward, thanks to everyone involved in

this work! I will see how fast we can integrate this branch in our
tests.

Note, that we even didn't try to run it on Windows.
So it must be some problems.

Please try the refactored PHP engine and provide feedback re:
performance,
memory usage and any issues that come up. You may find it in phpng
branch
at php.net http://php.net. Some instructions may be found at
http://wiki.php.net/phpng
http://wiki.php.net/phpng. As mentioned, there are some missing
extensions so not everything will run.

Could you move these pages (this one and the internal one) either to
/draft or /rfc please? the latter makes more sense

However I suppose basic windows tests are missing, as well as TS
sapis, right? Is it at least supposed to work in TS mode, by design?
Or should we expect some major issues?

ZTS wasn't in our priority list, but Nikita made it more or less work.

I would like to say many thanks to Xinchen and Nikita who made
significant
part of presented work.

I think that this engine can make the new major version of PHP we’re
talking about a lot more interesting.

Do I understand correctly that you target php 6 for these changes, right?

The next major release - 6 or 7, of course, if we port all extensions and
solve all problems.

Also one thing is totally missing, and is also caused by something
that becomes a common case, the 64bit support as it should be has been
totally ignored. That makes Anatol work to keep the 64bit branch
almost useless. My take on it is that we should either:

move your patch to this branch and then merge to master

merge the 64bit branche then this patch

I afraid both ways aren't going to be simple.

Anyway, I think 64-bit integer support on Windows64 makes full sense and
it's possible to add it into "phpng" right now.

I'm not so sure about size_t string length.

That being said, I am not sure what requires less effort but I do not
feel comfortable to scratch the 64bit branch and begins from almost
zero.

There is something that we have to improve, drastically,
communication. Such major changes, developed (and still being worked
on) should pop up in the list and in the radar of other developers
much earlier in the process. For two critical reasons:

Avoid work conflicts, be for similar features/changes or other
areas being deadly affected by such changes

Increase visibility, feedback and cooperations. The earlier other
developers are involved the better it is to get more people involved
in the engine development and maintenance.

I understand your opinion, but when we started this project we didn't know
if we will get any useful results at all.
You know, I can't ask everyone in PHP community to freeze development and
wait while we are thinking :)

I am not sure how to change that or make developers being more open
and stopped hidden developments. Any ideas or thoughts on this topic
are welcome. It is critical and vital for php, a must do.

I don't see problem in hidden development. The formed ideas come as RFC
with patches.
Anyway, I'm open to discussions, if they won't take too mach time.

Thanks. Dmitry.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Stas Malyshev — view source

unread

Hi!

Anyway, I think 64-bit integer support on Windows64 makes full sense and
it's possible to add it into "phpng" right now.

I'm not so sure about size_t string length.

Shouldn't it be easier in phpng with everything using zend_string*
instead of char*+int? As far as I can see, making zend_string use 64-bit
length should be more or less painless, and then we only need to watch
out when we take string length and assign it to some other variable,
which shouldn't be too many places I hope.

--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

11 years ago by Dmitry Stogov — view source

unread

yes. it must be easier to implement (less places have to be changed), but
it may increase memory consumption.

Thanks. Dmitry.

On Tue, May 6, 2014 at 4:16 AM, Stas Malyshev smalyshev@sugarcrm.comwrote:

Hi!

Anyway, I think 64-bit integer support on Windows64 makes full sense and
it's possible to add it into "phpng" right now.

I'm not so sure about size_t string length.

Shouldn't it be easier in phpng with everything using zend_string*
instead of char*+int? As far as I can see, making zend_string use 64-bit
length should be more or less painless, and then we only need to watch
out when we take string length and assign it to some other variable,
which shouldn't be too many places I hope.

--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

11 years ago by Hannes Magnusson — view source

unread

ZTS wasn't in our priority list, but Nikita made it more or less work.

What is the argument to keep it?

It is the leading factor of broken builds
Endless API changes because of it
Noones friend
I am (blissfully?) unaware of any intentional deployment of zts

-Hannes

11 years ago by Dmitry Stogov — view source

unread

hi Hannes,

We didn't come to any agreement to remove ZTS as well.
I don't think we might remove something before opening the code.

Thanks. Dmitry.

On Tue, May 6, 2014 at 4:37 AM, Hannes Magnusson <hannes.magnusson@gmail.com

wrote:

ZTS wasn't in our priority list, but Nikita made it more or less work.

What is the argument to keep it?

It is the leading factor of broken builds

Endless API changes because of it

Noones friend

I am (blissfully?) unaware of any intentional deployment of zts

-Hannes

11 years ago by Pierre Joye — view source

unread

hi Hannes,

We didn't come to any agreement to remove ZTS as well.
I don't think we might remove something before opening the code.

To remove TS support is hardly a question, and would be a very bad
design decision. However to change its implementation (killing TSRM)
could be something I can imagine. But it is a totally different topic
and should be discussed in a separate thread/RFC.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Dmitry Stogov — view source

unread

agree.

Dmitry.

hi Hannes,

We didn't come to any agreement to remove ZTS as well.
I don't think we might remove something before opening the code.

To remove TS support is hardly a question, and would be a very bad
design decision. However to change its implementation (killing TSRM)
could be something I can imagine. But it is a totally different topic
and should be discussed in a separate thread/RFC.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Pierre Joye — view source

unread

ZTS wasn't in our priority list, but Nikita made it more or less work.

Why do I have a deja vu feeling? ;)

Also one thing is totally missing, and is also caused by something
that becomes a common case, the 64bit support as it should be has been
totally ignored. That makes Anatol work to keep the 64bit branch
almost useless. My take on it is that we should either:

move your patch to this branch and then merge to master

merge the 64bit branche then this patch

I afraid both ways aren't going to be simple.

Rght, but again, I really do not feel comfortable to ask Anatal or my
team to redo all the work. This would be very bad.

Anyway, I think 64-bit integer support on Windows64 makes full sense and
it's possible to add it into "phpng" right now.

Again, it is by far not only a windows change, even if Windows is the
platform with the most visible change from a user point of view.

I'm not so sure about size_t string length.

It is hardly imaginable to release php 6 with strings length still
using integer. The memory consumption has been shown to be minimal
during the discussions about the 64bit support. Or did I miss some
important new information?

I understand your opinion, but when we started this project we didn't know
if we will get any useful results at all.
You know, I can't ask everyone in PHP community to freeze development and
wait while we are thinking :)

No, but it is possible to get a branch and work on it, sync it with
the other relevant branch. That is what we have done for the 64bit
branch, providing updated builds and tests results while working on
it. Everyone can test it or contribute. This is the way to do such
things from a technical point of view. It is also the way to do it
from a cooperation point of view, for the reasons I gave earlier.

I am not sure how to change that or make developers being more open
and stopped hidden developments. Any ideas or thoughts on this topic
are welcome. It is critical and vital for php, a must do.

I don't see problem in hidden development. The formed ideas come as RFC with
patches.

Well, I do when it comes to such massive changes, no matter how
positive they are. It is all about being open and transparent. Early
participation eases the understanding of the changes, design or
debugging of the new features or changes. This reduces the bus factor
when it comes to maintain it and we both know that this has been an
issue for too long for everything around the engine area.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Dmitry Stogov — view source

unread

ZTS wasn't in our priority list, but Nikita made it more or less work.

Why do I have a deja vu feeling? ;)

I really, don't see a lot of sense in using ZTS and didn't like to spend
our time.

Also one thing is totally missing, and is also caused by something
that becomes a common case, the 64bit support as it should be has been
totally ignored. That makes Anatol work to keep the 64bit branch
almost useless. My take on it is that we should either:

move your patch to this branch and then merge to master

merge the 64bit branche then this patch

I afraid both ways aren't going to be simple.

Rght, but again, I really do not feel comfortable to ask Anatal or my
team to redo all the work. This would be very bad.

I see your point. Please, look from mine.
Just compare the sizes of diffs between branches.
We won't be able to care about unstable development branches...

We may help, with reimplementation of int64 patch, but actually we come to
ask help ourselves.
I hope we will found some compromise. It can't be a problem if our goals
are not opposite :)

Anyway, I think 64-bit integer support on Windows64 makes full sense and
it's possible to add it into "phpng" right now.

Again, it is by far not only a windows change, even if Windows is the
platform with the most visible change from a user point of view.

OK. Win64 is enough for this patch. I wouldn't care about others, because I
even don't know the names.

I'm not so sure about size_t string length.

It is hardly imaginable to release php 6 with strings length still
using integer. The memory consumption has been shown to be minimal
during the discussions about the 64bit support. Or did I miss some
important new information?

I wouldn't mix size_t string length with 64-bit support. These are two
independent changes.
int64 would make 64-bit PHP version behave more or less consistent with
minimal cost.

size_t string length will increase memory consumption and memory traffic
for the cases that no one web application will use (2GB strings).
I'm trying to save (or even improve) memory consumption, because I know how
it may affect performance because of CPU cache misses.

Lets apply the first part first.

Sorry, I already said my opinion about development.

Thanks. Dmitry.

I understand your opinion, but when we started this project we didn't
know
if we will get any useful results at all.
You know, I can't ask everyone in PHP community to freeze development and
wait while we are thinking :)

No, but it is possible to get a branch and work on it, sync it with
the other relevant branch. That is what we have done for the 64bit
branch, providing updated builds and tests results while working on
it. Everyone can test it or contribute. This is the way to do such
things from a technical point of view. It is also the way to do it
from a cooperation point of view, for the reasons I gave earlier.

I am not sure how to change that or make developers being more open
and stopped hidden developments. Any ideas or thoughts on this topic
are welcome. It is critical and vital for php, a must do.

I don't see problem in hidden development. The formed ideas come as RFC
with
patches.

Well, I do when it comes to such massive changes, no matter how
positive they are. It is all about being open and transparent. Early
participation eases the understanding of the changes, design or
debugging of the new features or changes. This reduces the bus factor
when it comes to maintain it and we both know that this has been an
issue for too long for everything around the engine area.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Pierre Joye — view source

unread

I really, don't see a lot of sense in using ZTS and didn't like to spend our
time.

Well, it is not a taste matter and we all have to take care of issues
on platforms/sapis we do not use daily.

Also one thing is totally missing, and is also caused by something
that becomes a common case, the 64bit support as it should be has been
totally ignored. That makes Anatol work to keep the 64bit branch
almost useless. My take on it is that we should either:

move your patch to this branch and then merge to master

merge the 64bit branche then this patch

I afraid both ways aren't going to be simple.

Rght, but again, I really do not feel comfortable to ask Anatal or my
team to redo all the work. This would be very bad.

I see your point. Please, look from mine.
Just compare the sizes of diffs between branches.
We won't be able to care about unstable development branches...

It is stable as far as we can tell and has been working constantly
since we proposed it. Tests results are public and very positive.
Thing is that every individual here is not alone on working on php,
care has to be taken.

We may help, with reimplementation of int64 patch, but actually we come to
ask help ourselves.
I hope we will found some compromise. It can't be a problem if our goals are
not opposite :)

We have the same goals :) While performance is a lower priority to me
than clean code and maintainability in the long run. But both
performance and ease of maintainability can be achieved together.

Anyway, I think 64-bit integer support on Windows64 makes full sense and
it's possible to add it into "phpng" right now.

Again, it is by far not only a windows change, even if Windows is the
platform with the most visible change from a user point of view.

OK. Win64 is enough for this patch. I wouldn't care about others, because I
even don't know the names.

Err, even Linux has a not so good 64bit implementation. We rely on
single compiler behavior and 20 years old types described by many
leading developers as hacks. PHP is one of the only OSS projects I
know still relying on these types or using int for buffer size. I am
not willing to tell you what you should care about but this is
definitively an area that deserves more attention.

I'm not so sure about size_t string length.

It is hardly imaginable to release php 6 with strings length still
using integer. The memory consumption has been shown to be minimal
during the discussions about the 64bit support. Or did I miss some
important new information?

I wouldn't mix size_t string length with 64-bit support. These are two
independent changes.
int64 would make 64-bit PHP version behave more or less consistent with
minimal cost.

There are part of the same changeset. They are even deeply related.

size_t string length will increase memory consumption and memory traffic for
the cases that no one web application will use (2GB strings).
I'm trying to save (or even improve) memory consumption, because I know how
it may affect performance because of CPU cache misses.

Lets apply the first part first.

I have to disagree here, the patch contains both changes and will be
proposed together. I do not see an appealing reason to split them for
a minimal gain from a memory usage point of view. The importance and
gains brought by these changes are too important. The "new" RFC and
patch are ready, it will be proposed soonish. If accepted, let see
what is the best way to make your patches work with it, we can help
here with the moves anyway :)

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Dmitry Stogov — view source

unread

int64 proposal is still in draft for very long time.
Do you provide support for all other unrelated proposals when you offer
your own?

Lets think how to integrate them. I propose doing it in two steps
(measuring the performance and memory consumption difference).
If the degradation is going to be invisible, I won't object.

Thanks. Dmitry.

I really, don't see a lot of sense in using ZTS and didn't like to spend
our
time.

Well, it is not a taste matter and we all have to take care of issues
on platforms/sapis we do not use daily.

Also one thing is totally missing, and is also caused by something
that becomes a common case, the 64bit support as it should be has
been
totally ignored. That makes Anatol work to keep the 64bit branch
almost useless. My take on it is that we should either:

move your patch to this branch and then merge to master

merge the 64bit branche then this patch

I afraid both ways aren't going to be simple.

Rght, but again, I really do not feel comfortable to ask Anatal or my
team to redo all the work. This would be very bad.

I see your point. Please, look from mine.
Just compare the sizes of diffs between branches.
We won't be able to care about unstable development branches...

It is stable as far as we can tell and has been working constantly
since we proposed it. Tests results are public and very positive.
Thing is that every individual here is not alone on working on php,
care has to be taken.

We may help, with reimplementation of int64 patch, but actually we come
to
ask help ourselves.
I hope we will found some compromise. It can't be a problem if our goals
are
not opposite :)

We have the same goals :) While performance is a lower priority to me
than clean code and maintainability in the long run. But both
performance and ease of maintainability can be achieved together.

Anyway, I think 64-bit integer support on Windows64 makes full sense
and
it's possible to add it into "phpng" right now.

Again, it is by far not only a windows change, even if Windows is the
platform with the most visible change from a user point of view.

OK. Win64 is enough for this patch. I wouldn't care about others,
because I
even don't know the names.

Err, even Linux has a not so good 64bit implementation. We rely on
single compiler behavior and 20 years old types described by many
leading developers as hacks. PHP is one of the only OSS projects I
know still relying on these types or using int for buffer size. I am
not willing to tell you what you should care about but this is
definitively an area that deserves more attention.

I'm not so sure about size_t string length.

It is hardly imaginable to release php 6 with strings length still
using integer. The memory consumption has been shown to be minimal
during the discussions about the 64bit support. Or did I miss some
important new information?

I wouldn't mix size_t string length with 64-bit support. These are two
independent changes.
int64 would make 64-bit PHP version behave more or less consistent with
minimal cost.

There are part of the same changeset. They are even deeply related.

size_t string length will increase memory consumption and memory traffic
for
the cases that no one web application will use (2GB strings).
I'm trying to save (or even improve) memory consumption, because I know
how
it may affect performance because of CPU cache misses.

Lets apply the first part first.

I have to disagree here, the patch contains both changes and will be
proposed together. I do not see an appealing reason to split them for
a minimal gain from a memory usage point of view. The importance and
gains brought by these changes are too important. The "new" RFC and
patch are ready, it will be proposed soonish. If accepted, let see
what is the best way to make your patches work with it, we can help
here with the moves anyway :)

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Pierre Joye — view source

unread

int64 proposal is still in draft for very long time.

Sorry, but it was proposed and rejected because of the changes were
too big for the 5.x serie. It has been said, as arguments for the no,
that it would not be a problem for php 6. We are so far.

Do you provide support for all other unrelated proposals when you offer your
own?

I do not understand the question, what do you mean?

However, about support, we do offer support for almost every proposal,
be for testing, build fixes, etc. Pro actively in most cases, whether
it is in our priorities list or not. My reasoning here is pretty
simple, a stable PHP, for all supported platform or configuration is
critical for the success of PHP. Cooperation too.

Lets think how to integrate them. I propose doing it in two steps (measuring
the performance and memory consumption difference).
If the degradation is going to be invisible, I won't object.

The numbers are available already, using master and 5.5-5.6. We
obviously do not have numbers using your patch but I do not think the
delta will be any different.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Dmitry Stogov — view source

unread

int64 proposal is still in draft for very long time.

Sorry, but it was proposed and rejected because of the changes were
too big for the 5.x serie. It has been said, as arguments for the no,
that it would not be a problem for php 6. We are so far.

I was never glad with size_t string lengths and told about it.

Do you provide support for all other unrelated proposals when you offer
your
own?

I do not understand the question, what do you mean?

I mean that when you develop something big, you have to be concentrated on
the goals you are going to solve and not on compatibility with others
proposals.

However, about support, we do offer support for almost every proposal,
be for testing, build fixes, etc. Pro actively in most cases, whether
it is in our priorities list or not. My reasoning here is pretty
simple, a stable PHP, for all supported platform or configuration is
critical for the success of PHP. Cooperation too.

Lets think how to integrate them. I propose doing it in two steps
(measuring
the performance and memory consumption difference).
If the degradation is going to be invisible, I won't object.

The numbers are available already, using master and 5.5-5.6. We
obviously do not have numbers using your patch but I do not think the
delta will be any different.

phpng completely changes zval structure and reduces overhead of some
subsystems, so the numbers might be different.

Thanks. Dmitry.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Pierre Joye — view source

unread

int64 proposal is still in draft for very long time.

Sorry, but it was proposed and rejected because of the changes were
too big for the 5.x serie. It has been said, as arguments for the no,
that it would not be a problem for php 6. We are so far.

I was never glad with size_t string lengths and told about it.

Do you provide support for all other unrelated proposals when you offer
your
own?

I do not understand the question, what do you mean?

I mean that when you develop something big, you have to be concentrated on
the goals you are going to solve and not on compatibility with others
proposals.

Right, but this is possible to achieve by cooperating with the
authors of other proposals. Early and well organized proposals
minimize the pain for other developers while maximizing the chance of
success.

However, about support, we do offer support for almost every proposal,
be for testing, build fixes, etc. Pro actively in most cases, whether
it is in our priorities list or not. My reasoning here is pretty
simple, a stable PHP, for all supported platform or configuration is
critical for the success of PHP. Cooperation too.

Lets think how to integrate them. I propose doing it in two steps
(measuring
the performance and memory consumption difference).
If the degradation is going to be invisible, I won't object.

The numbers are available already, using master and 5.5-5.6. We
obviously do not have numbers using your patch but I do not think the
delta will be any different.

phpng completely changes zval structure and reduces overhead of some
subsystems, so the numbers might be different.

They will be different, no doubt here, but there should not be that
much differences from a relative point of view.

However I do not think I can go to my team and asked to do that from
scratch based on a work in progress effort (even a very good one :).
The more I look at it the more I think it is easier and faster to
merge the 64bit and then adapt the other one, if accepted (for both
proposals).

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Sebastian Bergmann — view source

unread

Am 06.05.2014 07:43, schrieb Pierre Joye:

Well, it is not a taste matter and we all have to take care of issues
on platforms/sapis we do not use daily.

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

11 years ago by Pierre Joye — view source

unread

Am 06.05.2014 07:43, schrieb Pierre Joye:

Well, it is not a taste matter and we all have to take care of issues
on platforms/sapis we do not use daily.

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

Being in the same mode (Devil's advocate), let make PHP linux+gcc
(some recent versions only), enable only phpdbg, fpm and cli, and be
done with all the other platforms.

Back to normal mode: it is not the way :)

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Zeev Suraski — view source

unread

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

I think you just called me "the devil" ;)

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD).
Things like ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly
brings any tangible benefits over FastCGI (none that can't be
relatively easily replicated in userland anyway, eg htscanner).

I'd support it.

Zeev

11 years ago by Pierre Joye — view source

unread

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

I think you just called me "the devil" ;)

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD).
Things like ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly
brings any tangible benefits over FastCGI (none that can't be
relatively easily replicated in userland anyway, eg htscanner).

I'd support it.

The problem here is not only the SAPIs but as a matter of fact Zend
cares about what they need and that's about it, and pushed things very
hard. We have to keep in mind that PHP success is not only based on
his design ( :-> ) but also because its availability on a large
variety of platforms.

If some likes to have a striped down (to one) set of platforms (in
this case, linux+gcc+fpm+cli), then go for it, the code is open and
there are many places where to host it. Facebook did it with many
other changes. But to restrict the future of PHP to fill some
individual needs (entities or person) is not a good thing, not for php
and certainly not for the communities.

Now, on a more constructive side, there are ways to improve the
situation and ease the overall development and maintenance work.
Droping TSRM or drastically improve it (TLS f.e.) could be one of
them.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Anatol Belski — view source

unread

Hi Zeev,

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

I think you just called me "the devil" ;)

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD). Things like
ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly brings
any tangible benefits over FastCGI (none that can't be relatively easily
replicated in userland anyway, eg htscanner).

I'd support it.

There's an RFC about it https://wiki.php.net/rfc/removal_of_dead_sapis .
It came into being while working on the int64 branch, but now becomes more
relevant with phpng. The simple idea is first to check all those, if it
turns out those SAPIs will be supported by the authors, or at least
there's a minority using them, it might make sense to port them.
Implementing that RFC were a clean and predictable way to go IMHO.

Regards

Anatol

11 years ago by Ferenc Kovacs — view source

unread

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

I think you just called me "the devil" ;)

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD).
Things like ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly
brings any tangible benefits over FastCGI (none that can't be
relatively easily replicated in userland anyway, eg htscanner).

I'd support it.

Zeev

--

You forgot fpm and phpdbg from that list.
A little bit offtopic, but I think it would be nice if we could have some
pecl-like infrastructure to host and install 3rd party sapis without the
need to bundle them with the core.

ps: maybe something like https://bugs.php.net/bug.php?id=66589 could be
also considered then.

--
Ferenc Kovács
@Tyr43l - http://tyrael.hu

11 years ago by Sebastian Bergmann — view source

unread

Am 07.05.2014 11:56, schrieb Ferenc Kovacs:

You forgot fpm and phpdbg from that list.

I still find it weird that phpdbg is a SAPI; and FPM is for me the
same as FastCGI. Sorry about that :)

11 years ago by Dmitry Stogov — view source

unread

I recently converted apache2handler to support phpng and it's appeared
significantly slower than FastCGI and FPM.
So I'm even not sure if it makes sense to support it.
Especially, remembering problems with compatibility between mod_php and
mod_perl for example that use the same libraries.

It's just thoughts, I'm not going to remove something :)

Thanks. Dmitry.

On 7 במאי 2014, at 09:13, Sebastian Bergmann sebastian@php.net
wrote:

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

I think you just called me "the devil" ;)

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD).
Things like ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly
brings any tangible benefits over FastCGI (none that can't be
relatively easily replicated in userland anyway, eg htscanner).

I'd support it.

Zeev

--

You forgot fpm and phpdbg from that list.
A little bit offtopic, but I think it would be nice if we could have some
pecl-like infrastructure to host and install 3rd party sapis without the
need to bundle them with the core.

ps: maybe something like https://bugs.php.net/bug.php?id=66589 could be
also considered then.

--
Ferenc Kovács
@Tyr43l - http://tyrael.hu

11 years ago by Stas Malyshev — view source

unread

Hi!

I recently converted apache2handler to support phpng and it's appeared
significantly slower than FastCGI and FPM.

Do we know why? I would assume since FastCGI has additional data hop
(between webserver and the actual PHP binary) Apache one would be faster
but looks like it's not so. It'd be interesting to know why.

Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

11 years ago by Andi Gutmans — view source

unread

Hi!

I recently converted apache2handler to support phpng and it's appeared
significantly slower than FastCGI and FPM.

Do we know why? I would assume since FastCGI has additional data hop
(between webserver and the actual PHP binary) Apache one would be faster
but looks like it's not so. It'd be interesting to know why.

I think in some cases the benchmarks I’ve seen are not apples-to-apples. I would also like to see this on true apples to apples. [assuming no static files are served].

By the way, at some point there was one reason for FastCGI to be faster - php-cgi was non-PIC while mod_php.so was PIC. But I believe on Linux you can have shared libraries that are non-PIC. [PIC meaning position independent code].

Andi

11 years ago by Jan Ehrhardt — view source

unread

Zeev Suraski in php.internals (Wed, 7 May 2014 09:36:14 +0300):

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD).
Things like ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly
brings any tangible benefits over FastCGI (none that can't be
relatively easily replicated in userland anyway, eg htscanner).

Is Fastcgi capable of supporting 2 versions of PHP? I still need PHP 5.3
for Drupal6, but Drupal7, Wordpress, Modx and the like profit from PHP
5.5. If there was a way to run both versions as Fastcgi, I would happily
go along with you. But I haven't found out how to do that, so one of
the two still is mod_php.

Jan

11 years ago by Andrea Faulds — view source

unread

Zeev Suraski in php.internals (Wed, 7 May 2014 09:36:14 +0300):

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD).
Things like ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly
brings any tangible benefits over FastCGI (none that can't be
relatively easily replicated in userland anyway, eg htscanner).

Is Fastcgi capable of supporting 2 versions of PHP? I still need PHP 5.3
for Drupal6, but Drupal7, Wordpress, Modx and the like profit from PHP
5.5. If there was a way to run both versions as Fastcgi, I would happily
go along with you. But I haven't found out how to do that, so one of
the two still is mod_php.

Correct me if I’m wrong, but don’t you simply need to have two PHP versions installed (5.3 FastCGI and 5.5 FastCGI) and then reference the appropriate ones in your server configuration files? The hard bit would be the dual installation (you’d probably need to build from source), but having the web server handle both should be trivial.

--
Andrea Faulds
http://ajf.me/

11 years ago by Martin Keckeis — view source

unread

Am 07.05.2014 15:18 schrieb "Andrea Faulds" ajf@ajf.me:

Zeev Suraski in php.internals (Wed, 7 May 2014 09:36:14 +0300):

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD).
Things like ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly
brings any tangible benefits over FastCGI (none that can't be
relatively easily replicated in userland anyway, eg htscanner).

Is Fastcgi capable of supporting 2 versions of PHP? I still need PHP 5.3
for Drupal6, but Drupal7, Wordpress, Modx and the like profit from PHP
5.5. If there was a way to run both versions as Fastcgi, I would happily
go along with you. But I haven't found out how to do that, so one of
the two still is mod_php.

Correct me if I’m wrong, but don’t you simply need to have two PHP
versions installed (5.3 FastCGI and 5.5 FastCGI) and then reference the
appropriate ones in your server configuration files? The hard bit would be
the dual installation (you’d probably need to build from source), but
having the web server handle both should be trivial.

--
Andrea Faulds
http://ajf.me/

Yep its easy with vhosts :-)
The only problem is when you need the "php" global command for both
versions...

11 years ago by johannes@schlueters.de — view source

unread

Am 07.05.2014 15:18 schrieb "Andrea Faulds" ajf@ajf.me:

Zeev Suraski in php.internals (Wed, 7 May 2014 09:36:14 +0300):

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD).
Things like ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly
brings any tangible benefits over FastCGI (none that can't be
relatively easily replicated in userland anyway, eg htscanner).

Is Fastcgi capable of supporting 2 versions of PHP? I still need PHP 5.3
for Drupal6, but Drupal7, Wordpress, Modx and the like profit from PHP
5.5. If there was a way to run both versions as Fastcgi, I would happily
go along with you. But I haven't found out how to do that, so one of
the two still is mod_php.

Correct me if I’m wrong, but don’t you simply need to have two PHP
versions installed (5.3 FastCGI and 5.5 FastCGI) and then reference the
appropriate ones in your server configuration files? The hard bit would be
the dual installation (you’d probably need to build from source), but
having the web server handle both should be trivial.

If there is need is relatively easy for distributors to provide
different versions installed to different locations. They can't easily
enable both in the server, though, and the user has to configure vhosts
or such, which is quite obvious.

The only problem is when you need the "php" global command for both
versions...

Either have only one in $PATH and use absolute path for the other or
give explicit names. i.e. by symlinks. Distributions could easily create
such a setup:

/usr/bin/php55 -> /usr/php55/bin/php
/usr/bin/php56 -> /usr/php56/bin/php
/usr/bin/php -> /etc/alternatives/php

johannes

11 years ago by me@rouvenwessling.de — view source

unread

Is Fastcgi capable of supporting 2 versions of PHP? I still need PHP 5.3
for Drupal6, but Drupal7, Wordpress, Modx and the like profit from PHP
5.5. If there was a way to run both versions as Fastcgi, I would happily
go along with you. But I haven't found out how to do that, so one of
the two still is mod_php.

Jan

You can. The trick is running each PHP version on a different port. (i.e. 5.3 on port 9000, 5.4 on 9001 and so on) Than simply use the port for the PHP version you want when configuring the virtual host.

Best regards
Rouven

11 years ago by Dan Cryer — view source

unread

Is Fastcgi capable of supporting 2 versions of PHP? I still need PHP 5.3
for Drupal6, but Drupal7, Wordpress, Modx and the like profit from PHP
5.5. If there was a way to run both versions as Fastcgi, I would happily
go along with you. But I haven't found out how to do that, so one of
the two still is mod_php.

You could install the two different versions of PHP (+ PHP-FPM) to two
different locations on your server, and then run one PHP-FPM pool per
version for your web server to communicate with. That would enable what
you're trying to achieve.

11 years ago by Andi Gutmans — view source

unread

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

I think you just called me "the devil" ;)

Seriously, I can't think of any real reason of why anybody would
need something other than these SAPIs in 2014 (other than FUD).
Things like ISAPI and other more esoteric SAPIs are no brainers, and
while mod_php may be a bit more of a leap of faith, it doesn't truly
brings any tangible benefits over FastCGI (none that can't be
relatively easily replicated in userland anyway, eg htscanner).

I'd support it.

For what it’s worth I don’t think we should stop supporting mod_php at this point in time.
I think there are still plenty of situations where this is the easier, simpler route to go and quite robust.

Andi

11 years ago by Andrea Faulds — view source

unread

For what it’s worth I don’t think we should stop supporting mod_php at this point in time.
I think there are still plenty of situations where this is the easier, simpler route to go and quite robust.

It certainly helps PHP’s popularity. On Debian, sudo apt-get install libapache2-mod-php5 is all you need to do to get PHP functioning on Apache, with no configuration required whatsoever. It may not be the best way to use PHP, but it’s certainly the simplest for new users.

--
Andrea Faulds
http://ajf.me/

11 years ago by Zeev Suraski — view source

unread

For what it’s worth I don’t think we should stop supporting mod_php at this
point in time.
I think there are still plenty of situations where this is the easier,
simpler route to go and quite robust.

It certainly helps PHP’s popularity. On Debian, sudo apt-get install libapache2-mod-php5 is all you need to do to get PHP functioning on
Apache, with no configuration required whatsoever. It may not be the best
way to use PHP, but it’s certainly the simplest for new users.

Again, this is just history/habits.
Nothing prevents creation of apache-fastcgi-php5 package that'll do the
exact same thing using FastCGI.

Zeev

11 years ago by Rasmus Lerdorf — view source

unread

Again, this is just history/habits.
Nothing prevents creation of apache-fastcgi-php5 package that'll do the
exact same thing using FastCGI.

What you are calling history/habits here is also knows as operational
stability. Every change to a stack incurs operational costs which has to
be balanced against the benefits of changing a component. Even though it
seems like a trivial thing to change from Apache/mod_php to
Apache/mod_fcgid or nginx+php-fpm, the operational cost can be quite
high. Consider changes to the Nagios monitoring setup. Dev VM images all
have to be updated. Startup/shutdown/deploy and logging systems might be
affected along with integration tests. And for most environments
changing doesn't actually buy you anything. A typical PHP request spends
so little time in the web server compared to waiting on mysql and
memcache replies that even a major performance boost doesn't actually
give you much.

I fully agree that nginx+php-fpm is a nicer architecture. I like the
process separation that fastcgi brings and the saner and richer
configuration nginx brings along with the slight performance boost, but
even for my own use those benefits haven't outweighed the cost of
switching for me for systems that are running well under mod_php today.

-Rasmus

11 years ago by Zeev Suraski — view source

unread

-----Original Message-----
From: Rasmus Lerdorf [mailto:rasmus@lerdorf.com]
Sent: Thursday, May 08, 2014 7:00 PM
To: Zeev Suraski; Andrea Faulds
Cc: Andi Gutmans; Sebastian Bergmann; internals@lists.php.net
Subject: Re: [PHP-DEV] phpng: Refactored PHP Engine with Big Performance
Improvement

Again, this is just history/habits.
Nothing prevents creation of apache-fastcgi-php5 package that'll do
the exact same thing using FastCGI.

What you are calling history/habits here is also knows as operational
stability.
Every change to a stack incurs operational costs which has to be balanced
against the benefits of changing a component. Even though it seems like a
trivial
thing to change from Apache/mod_php to Apache/mod_fcgid or nginx+php-
fpm, the operational cost can be quite high. Consider changes to the
Nagios
monitoring setup. Dev VM images all have to be updated.
Startup/shutdown/deploy and logging systems might be affected along with
integration tests. And for most environments changing doesn't actually buy
you
anything. A typical PHP request spends so little time in the web server
compared
to waiting on mysql and memcache replies that even a major performance
boost
doesn't actually give you much.

Two things:

You're right it buys operational stability; But then, people who'll go
on a major PHP version upgrade will expect some level of operational
instability (we should hope so at least, because they'll probably get it).
App breakages are an order of magnitude (or two or three) more painful than
relatively simple changes to monitoring tools or administrative scripts.
I never argued that FastCGI/fpm is faster than mod_php; It's probably
not (it's roughly as quick from my experience). But it's a lot more
scalable (in terms of in-machine scalability), since it breaks the 1:1
mapping between Apache processes and PHP processes. Often, for a typical
setup of with hundreds of Apache children, you could use just several dozen
PHP processes when using FastCGI. With mod_php, after a short while you'd
have each Apache process consume a dozen or more megabytes of memory - times
hundreds of processes that's a lot of memory; With FastCGI, you'll only
have several dozen processes consuming that much memory, while the Apache
processes can stay lean. Not to mention you get to use more efficient
mpm's, ones mod_php doesn't support - and save even more memory. With
requests growing shorter, simpler while also rapidly growing in frequency -
this sort of density is very important IMHO.

I fully agree that nginx+php-fpm is a nicer architecture. I like the
process
separation that fastcgi brings and the saner and richer configuration
nginx brings
along with the slight performance boost, but even for my own use those
benefits haven't outweighed the cost of switching for me for systems that
are
running well under mod_php today.

The question is whether that cost doesn't become marginal as you assess the
cost of moving to the next major version of PHP, as I'm obviously not
arguing we do this in 5.x. I don't think people generally expect to just
'yum update' and be done with it when they deal with a major version of PHP.
Now, I'll be the first person to admit (and wholeheartedly argue) that
compatibility breakage adds up as opposed to being boolean and must be
minimized - but I think the benefits here far outweigh the costs. We've
done much more painful transitions.

I view mod_php as a relic of the past which served its purpose. Same goes
to all other server-specific SAPIs. It's time to move on.

My 2c.

Zeev

11 years ago by Rasmus Lerdorf — view source

unread

You're right it buys operational stability; But then, people who'll go
on a major PHP version upgrade will expect some level of operational
instability (we should hope so at least, because they'll probably get it).
App breakages are an order of magnitude (or two or three) more painful than
relatively simple changes to monitoring tools or administrative scripts.

We tend to provide very good forward compatibility with E_DEPRECATED
warnings and a detailed UPGRADING document. This means you can prepare a
codebase for the next major release well ahead of time and run it in
production on the previous version while it is easy for the developers
to run a newer PHP version on a dev vm to make sure everything gets
caught. At the point of the upgrade it becomes quite painless. That is,
the work is nicely spread out and there is no jarring company-wide
disruption since all app breakages have been pre-fixed.

When swapping out an infrastructure element like the web server or at
least an architectural aspect of the web server, it is something that
touches a lot of moving parts and hits not just the dev team but also
the test and ops teams quite hard. It is hard to prepare for because you
don't really know if you missed something until you swap out the
production implementation that is running at scale. So the angst
involved is much higher.

I never argued that FastCGI/fpm is faster than mod_php; It's probably
not (it's roughly as quick from my experience). But it's a lot more
scalable (in terms of in-machine scalability), since it breaks the 1:1
mapping between Apache processes and PHP processes. Often, for a typical
setup of with hundreds of Apache children, you could use just several dozen
PHP processes when using FastCGI. With mod_php, after a short while you'd
have each Apache process consume a dozen or more megabytes of memory - times
hundreds of processes that's a lot of memory; With FastCGI, you'll only
have several dozen processes consuming that much memory, while the Apache
processes can stay lean. Not to mention you get to use more efficient
mpm's, ones mod_php doesn't support - and save even more memory. With
requests growing shorter, simpler while also rapidly growing in frequency -
this sort of density is very important IMHO.

Yup, I don't disagree that it is a more efficient model, especially for
smaller shops that don't have the luxury of using CDNs and having
multiple server pools for different purposes. However, for larger sites,
every request that makes it through to the Apache+mod_php process tends
to be an actual heavy PHP request. Static files aren't handled by that
pool of servers. In that scenario the server density differences between
FastCGI and mod_php becomes much smaller. If every request is heavy you
are going to need the same number of each. You gain a bit by not having
the Apache piece in each PHP process, but it isn't all that significant
and when weighed against the operational cost of switching it doesn't
seem all that attractive. I would love to be able to convince sites like
that to switch, but I haven't been able to come up with enough technical
benefits to outweigh the operational costs yet.

-Rasmus

11 years ago by Zeev Suraski — view source

unread

You're right it buys operational stability; But then, people
who'll go on a major PHP version upgrade will expect some level of
operational instability (we should hope so at least, because they'll
probably get
it).
App breakages are an order of magnitude (or two or three) more painful
than relatively simple changes to monitoring tools or administrative
scripts.

We tend to provide very good forward compatibility with E_DEPRECATED
warnings and a detailed UPGRADING document. This means you can prepare a
codebase for the next major release well ahead of time and run it in
production
on the previous version while it is easy for the developers to run a newer
PHP
version on a dev vm to make sure everything gets caught. At the point of
the
upgrade it becomes quite painless. That is, the work is nicely spread out
and
there is no jarring company-wide disruption since all app breakages have
been
pre-fixed.

While that's true in theory, in practice I think it's rarely the case. My
experience has been that when companies decided to move to a new version -
that's the time they did the code migration, and not a moment sooner.
Beforehand, they'd just turn off E_DEPRECATED to quiet it down. That may
not be the nice or right thing to do, but it appears to be the most popular
course of action.

That said - we can do something along the same lines here, to be nice to
those who play nice. We can issue an official 'mod_php is being deprecated'
notice when the server starts up - even as soon as 5.6 - and then actually
deprecate it in the next major version. Depending on the kind of feedback
we see, we can choose whether to actually pull it out or just keep the
deprecation warning.

Last, I think we're ignoring the fact that the work involved in actually
doing the move to FastCGI is very simple and centralized, as opposed to a
full app audit - sometimes a source code audit - that is needed when we
change constructs or function behavior. It's difficult to compare the
complexity of changing codebases of tens, hundreds and sometimes millions of
lines of code - to that of relatively minor setup issues.

When swapping out an infrastructure element like the web server or at
least an
architectural aspect of the web server, it is something that touches a lot
of
moving parts and hits not just the dev team but also the test and ops
teams quite
hard. It is hard to prepare for because you don't really know if you
missed
something until you swap out the production implementation that is running
at
scale. So the angst involved is much higher.

I think that if we publish it sufficiently ahead of time; Plus point the
die hards to where they can get the old mod_php; The amount of angst would
actually be much lower than some of the bigger changes we've made over the
course of PHP's history.
I find it difficult to believe that a change that can be implemented within
a couple of hours on a bad day - and with perhaps a couple of extra days to
account for scripts and monitoring configurations - would create that much
backlash. I think people will appreciate we're pushing them towards the
right choice.

I never argued that FastCGI/fpm is faster than mod_php; It's
probably not (it's roughly as quick from my experience). But it's a
lot more scalable (in terms of in-machine scalability), since it
breaks the 1:1 mapping between Apache processes and PHP processes.
Often, for a typical setup of with hundreds of Apache children, you
could use just several dozen PHP processes when using FastCGI. With
mod_php, after a short while you'd have each Apache process consume a
dozen or more megabytes of memory - times hundreds of processes that's
a lot of memory; With FastCGI, you'll only have several dozen
processes consuming that much memory, while the Apache processes can
stay lean. Not to mention you get to use more efficient mpm's, ones
mod_php doesn't support - and save even more memory. With requests
growing shorter, simpler while also rapidly growing in frequency - this
sort of
density is very important IMHO.

Yup, I don't disagree that it is a more efficient model, especially for
smaller
shops that don't have the luxury of using CDNs and having multiple server
pools
for different purposes. However, for larger sites, every request that
makes it
through to the Apache+mod_php process tends to be an actual heavy PHP
request. Static files aren't handled by that pool of servers. In that
scenario the
server density differences between FastCGI and mod_php becomes much
smaller. If every request is heavy you are going to need the same number
of
each. You gain a bit by not having the Apache piece in each PHP process,
but it
isn't all that significant and when weighed against the operational cost
of
switching it doesn't seem all that attractive.

I still come across a lot of sites of large companies (not the top Internet
companies, but still quite large) where Apache does end up serving at least
some of the static files. We needn't assume that every medium-large website
has experts working to squeeze every last bit of performance out of it,
since unfortunately it's not true. Also, maybe it's me but my experience is
that Apache really mishandles a situation where it runs out of processes -
so more often than not, configurations allow for a lot more processes than
may actually be needed. FastCGI/fpm seem to do a much better job at
handling a large number of requests with just a handful (or dozenful) of
processes.

I would love to be able to convince
sites like that to switch, but I haven't been able to come up with enough
technical benefits to outweigh the operational costs yet.

Well, I think deprecating it would be pretty convincing :)

Zeev

11 years ago by Pierre Joye — view source

unread

Well, I think deprecating it would be pretty convincing :)

I am -1 on that, in general. But we have RFCs so feel free to write
one to kill all SAPIs Zend does not care about in php-next :)

However I would rather focus on developing next instead, and discussed
that in this thread or other :)

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Andi Gutmans — view source

unread

Well, I think deprecating it would be pretty convincing :)

I am -1 on that, in general. But we have RFCs so feel free to write
one to kill all SAPIs Zend does not care about in php-next :)

With all due respect Pierre trying to win arguments by always pulling out a “Zend” card is irrelevant and wrong.
We don’t pull out the Microsoft card on you every time you have an opinion.
This is purely Zeev’s personal opinion. I respect it but I disagree with it myself (personal opinion too).

However I would rather focus on developing next instead, and discussed
that in this thread or other :)

And it’d be nice if you guys renamed the “Subject:” because this discussion has nothing to do with the refactored PHP engine.

Would be great if folks could give it a spin, do some benchmarks and send in results! :)

Andi

11 years ago by Pierre Joye — view source

unread

Well, I think deprecating it would be pretty convincing :)

I am -1 on that, in general. But we have RFCs so feel free to write
one to kill all SAPIs Zend does not care about in php-next :)

With all due respect Pierre trying to win arguments by always pulling out
a “Zend” card is irrelevant and wrong.
We don’t pull out the Microsoft card on you every time you have an
opinion.
This is purely Zeev’s personal opinion. I respect it but I disagree with
it myself (personal opinion too).

Sorry but no. It is based on public statement on this list or other
channels. Confirmed by the numerous issues in patches in configuration not
in the company priority. I am not blaming Zend but it is slightly bad to
push this direction based on a limited view of the market.

However I would rather focus on developing next instead, and discussed
that in this thread or other :)

And it’d be nice if you guys renamed the “Subject:” because this
discussion has nothing to do with the refactored PHP engine.

Ack :) moving on :)

Would be great if folks could give it a spin, do some benchmarks and send
in results! :)

We are on it, once we fixed it to run in all supported configs :)

Andi

11 years ago by Patrick Schaaf — view source

unread

Am 08.05.2014 23:25 schrieb "Zeev Suraski" zeev@zend.com:

You're right it buys operational stability; But then, people who'll go
on a major PHP version upgrade will expect some level of operational
instability (we should hope so at least, because they'll probably get it).
App breakages are an order of magnitude (or two or three) more painful
than
relatively simple changes to monitoring tools or administrative scripts.

In practise, I would never ever do both in the same timeframe. Otherwise I
wouldn't know whether apparent problems stem from monitoring changes,
deployment changes, or the PHP upgrade itself.

Right now I can pretty easily test new PHP versions, wrt. app
compatibility, by throwing them onto one or the other development server,
and/or onto one of a dozen deployment servers, because I can easily revert
that when it does not work.

With mod_php missing, I won't do that. And that will probably shift
adoption of a PHP++ into the future by 6 months, or something like that.

Not that any of that matters in the grand scheme of things. I'm just one
small site in the whole Internet....

best regards
Patrick

11 years ago by Patrick Schaaf — view source

unread

Am 08.05.2014 23:25 schrieb "Zeev Suraski" zeev@zend.com:

I never argued that FastCGI/fpm is faster than mod_php; It's probably
not (it's roughly as quick from my experience). But it's a lot more
scalable (in terms of in-machine scalability), since it breaks the 1:1
mapping between Apache processes and PHP processes. Often, for a typical
setup of with hundreds of Apache children, you could use just several
dozen
PHP processes when using FastCGI.

Is that really the usual case? For me it's about 90% PHP requests, and the
remaining CSS/JS/Image requests wont be sped up a bit by being served
through a smaller apache process, because they are already fast anyway.

Going FPM would thus mean about double the numberr of processes, and double
the number of context switches.

best regards
Patrick

11 years ago by Zeev Suraski — view source

unread

Is that really the usual case? For me it's about 90% PHP requests, and the
remaining CSS/JS/Image requests wont be sped up a bit by being served
through a smaller apache process, because they are already fast anyway.

I think it's a very common case, yes. Let me ask you a question - what is
your MaxClients set to? If it's over a hundred (or even over 50), chances
are you're not using your server's memory as efficiently as you could with
FastCGI/fpm.

Going FPM would thus mean about double the numberr of processes, and double
the number of context switches.

Not really, as Apache could be down to a single process and consume a lot
less memory. Regarding context switches - I agree with Dmitry's principle
that it's real life tests that are the only thing that matters, and real
world tests prove that FastCGI is as fast and typically marginally faster
than mod_php (before we take into account the memory savings benefits)

Zeev

11 years ago by Dmitry Stogov — view source

unread

the performance difference may be significant.
For example qdig.

mod_php (PIC) 440 req/sec
FastCGI (non-PIC) 537 req/sec

The bechmark was done with 32-bit build of phpng.

I didn't get time to investigate it in all details, but it'll really make
sense.

Thanks. Dmitry.

Again, this is just history/habits.
Nothing prevents creation of apache-fastcgi-php5 package that'll do the
exact same thing using FastCGI.

What you are calling history/habits here is also knows as operational
stability. Every change to a stack incurs operational costs which has to
be balanced against the benefits of changing a component. Even though it
seems like a trivial thing to change from Apache/mod_php to
Apache/mod_fcgid or nginx+php-fpm, the operational cost can be quite
high. Consider changes to the Nagios monitoring setup. Dev VM images all
have to be updated. Startup/shutdown/deploy and logging systems might be
affected along with integration tests. And for most environments
changing doesn't actually buy you anything. A typical PHP request spends
so little time in the web server compared to waiting on mysql and
memcache replies that even a major performance boost doesn't actually
give you much.

I fully agree that nginx+php-fpm is a nicer architecture. I like the
process separation that fastcgi brings and the saner and richer
configuration nginx brings along with the slight performance boost, but
even for my own use those benefits haven't outweighed the cost of
switching for me for systems that are running well under mod_php today.

-Rasmus

11 years ago by Rasmus Lerdorf — view source

unread

the performance difference may be significant.
For example qdig.

mod_php (PIC) 440 req/sec
FastCGI (non-PIC) 537 req/sec

The bechmark was done with 32-bit build of phpng.

I didn't get time to investigate it in all details, but it'll really
make sense.

But on 32-bit mod_php (libphp5.so) doesn't need to be PIC, does it? We
went through this many years ago and worked out that a non-PIC
libphp5.so works fine.

Things are a bit different on 64-bit because of 32-bit immediate operands.

However, I don't really believe that PIC vs. non-PIC accounts for
anywhere near a 20% performance difference in a real-world scenario. Can
you build phpng non-pic and run the same test again?

-Rasmus

11 years ago by Dmitry Stogov — view source

unread

FastCGI (non-PIC) 537 req/sec
mod_php (non-PIC) 555 req/sec
mod_pgp (PIC) 463 req/sec

with non-PIC code mod_php is a bit faster than FastCGI (as expected).

PIC makes a big difference on x86.

Thanks. Dmitry.

the performance difference may be significant.
For example qdig.

mod_php (PIC) 440 req/sec
FastCGI (non-PIC) 537 req/sec

The bechmark was done with 32-bit build of phpng.

I didn't get time to investigate it in all details, but it'll really
make sense.

But on 32-bit mod_php (libphp5.so) doesn't need to be PIC, does it? We
went through this many years ago and worked out that a non-PIC
libphp5.so works fine.

Things are a bit different on 64-bit because of 32-bit immediate operands.

However, I don't really believe that PIC vs. non-PIC accounts for
anywhere near a 20% performance difference in a real-world scenario. Can
you build phpng non-pic and run the same test again?

-Rasmus

11 years ago by Rasmus Lerdorf — view source

unread

FastCGI (non-PIC) 537 req/sec
mod_php (non-PIC) 555 req/sec
mod_pgp (PIC) 463 req/sec

with non-PIC code mod_php is a bit faster than FastCGI (as expected).

PIC makes a big difference on x86.

I wonder if the difference is as large on x86_64. You tested only on
32-bit x86, right?

Also, even with FastCGI, you are going to be linking in a whole bunch of
PIC shared libraries. So in a realworld app that makes lots of calls
into these libraries you are still going to have jump table function
call overhead.

Another thought. Since PIC overhead is on function calls and our default
VM is CALL, perhaps using SWITCH or GOTO instead might help here? And if
it does it might make sense to change the default now that the world is
64-bit and we can't compile non-PIC anymore.

-Rasmus

11 years ago by Dmitry Stogov — view source

unread

FastCGI (non-PIC) 537 req/sec
mod_php (non-PIC) 555 req/sec
mod_pgp (PIC) 463 req/sec

with non-PIC code mod_php is a bit faster than FastCGI (as expected).

PIC makes a big difference on x86.

I wonder if the difference is as large on x86_64. You tested only on
32-bit x86, right?

yes, 64-bit is almost always a bit slower because of bigger memory transfer
and more often cache misses.

FastCGI (x86_64) 527 req/sec

(all the benchmarks were done on the same machine with phpng built from the
same sources and with the similar configure options).

Also, even with FastCGI, you are going to be linking in a whole bunch of
PIC shared libraries. So in a realworld app that makes lots of calls
into these libraries you are still going to have jump table function
call overhead.

external functions are not so affected by PIC, because they usually do some
significant work and not just addition or comparison of two values, that
our VM handlers do.

Another thought. Since PIC overhead is on function calls and our default
VM is CALL, perhaps using SWITCH or GOTO instead might help here? And if
it does it might make sense to change the default now that the world is
64-bit and we can't compile non-PIC anymore.

Even in this 64-bit world, I would suggest to try 32-bit PHP. In most cases
it shows better performance.
X32 ABI would be ideal, but it's not widely supported yet.

Anyway, PIC doesn't make significant slowdown on x86_64, because of
additional CPU registers and special PIC support.

We tried GOTO executor and, of course, it's faster.
Actually, after all the optimizations, instruction dispatch overhead became
more and more significant,
so we will have to think about it.

Thanks. Dmitry.

-Rasmus

11 years ago by Rasmus Lerdorf — view source

unread

Even in this 64-bit world, I would suggest to try 32-bit PHP. In most
cases it shows better performance.
X32 ABI would be ideal, but it's not widely supported yet.

Anyway, PIC doesn't make significant slowdown on x86_64, because of
additional CPU registers and special PIC support.

We tried GOTO executor and, of course, it's faster.
Actually, after all the optimizations, instruction dispatch overhead
became more and more significant,
so we will have to think about it.

But I don't think people are going to be building PHP 32-bit if the rest
of their system is all 64-bit. And PIC vs. non-PIC isn't really an issue
anymore as I suspected. We should be optimizing for 64-bit. By the time
phpng hits the world there will be even fewer 32-bit deploys out there.

-Rasmus

11 years ago by Andrea Faulds — view source

unread

But I don't think people are going to be building PHP 32-bit if the rest
of their system is all 64-bit. And PIC vs. non-PIC isn't really an issue
anymore as I suspected. We should be optimizing for 64-bit. By the time
phpng hits the world there will be even fewer 32-bit deploys out there.

This is something I’m curious about. While I’d be surprised if the vast majority of servers in use lacked 64-bit CPUs, I can’t seem to find any statistics on this. Does anyone have any?

Andrea Faulds
http://ajf.me/

11 years ago by Dmitry Stogov — view source

unread

In case you care about few additional req/sec it must not be a problem to
do it on most major Linux distributions.

$ CC="gcc -m32" ./configure --without-pic ....

anyway, 64-bit phpng gets much more improvement because of reduced memory
consumption.

Thanks. Dmitry,

Even in this 64-bit world, I would suggest to try 32-bit PHP. In most
cases it shows better performance.
X32 ABI would be ideal, but it's not widely supported yet.

Anyway, PIC doesn't make significant slowdown on x86_64, because of
additional CPU registers and special PIC support.

We tried GOTO executor and, of course, it's faster.
Actually, after all the optimizations, instruction dispatch overhead
became more and more significant,
so we will have to think about it.

But I don't think people are going to be building PHP 32-bit if the rest
of their system is all 64-bit. And PIC vs. non-PIC isn't really an issue
anymore as I suspected. We should be optimizing for 64-bit. By the time
phpng hits the world there will be even fewer 32-bit deploys out there.

-Rasmus

11 years ago by Pierre Joye — view source

unread

the performance difference may be significant.
For example qdig.

mod_php (PIC) 440 req/sec
FastCGI (non-PIC) 537 req/sec

The bechmark was done with 32-bit build of phpng.

I didn't get time to investigate it in all details, but it'll really make
sense.

Not sure on linux, but a good configured apache+mod_php is fast enough
for many users. I often seen apache well configured performing better
than nginx+fpm.

On windows, things are slightly different, because of TSRM. Without it
(LTS), it is almost as fast (basic config).

However, as much as I like to get php faster, it should not be the
main argument for everything. And definitively not in this case, I do
not see any gain for us to kill mod_php support.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Zeev Suraski — view source

unread

Playing devil's advocate: does anyone really need SAPIs other than

FastCGI, CLI, and embed? And if so: do we care enough to burden

ourselves with their maintenance?

I think you just called me "the devil" ;)

Seriously, I can't think of any real reason of why anybody would

need something other than these SAPIs in 2014 (other than FUD).

Things like ISAPI and other more esoteric SAPIs are no brainers, and

while mod_php may be a bit more of a leap of faith, it doesn't truly

brings any tangible benefits over FastCGI (none that can't be

relatively easily replicated in userland anyway, eg htscanner).

I'd support it.

For what it’s worth I don’t think we should stop supporting mod_php at this
point in time.
I think there are still plenty of situations where this is the easier,
simpler route to go and quite robust.

Such as?

FastCGI (or fpm) is just as robust, it's a lot more scalable and has mostly
just advantages. The main thing mod_php has going for it is history.
There are no real technological advances and the rise of PHP on nginx can
attest to that.

Zeev

11 years ago by Andi Gutmans — view source

unread

For what it’s worth I don’t think we should stop supporting mod_php at this point in time.
I think there are still plenty of situations where this is the easier, simpler route to go and quite robust.

Such as?

FastCGI (or fpm) is just as robust, it's a lot more scalable and has mostly just advantages. The main thing mod_php has going for it is history. There are no real technological advances and the rise of PHP on nginx can attest to that.

There are tools and know-how out there on configuring various aspects of PHP through http.conf/.htaccess. Sure you can achieve most of that via FastCGI but I do not believe it’s as simple and straightforward and I still believe the majority of the PHP community is using mod_php. Re: nginx, the reality is, that while popular, it’s still not being used in the majority of environments. It’s being used more on the big, scale-out properties than on the massive of smaller PHP deployments.

I guess we’ll need to agree to disagree on this one :)

Andi

11 years ago by Zeev Suraski — view source

unread

There are tools and know-how

That's what I meant by "history".
I don't think that should deter us from discontinuing an inferior option if
there's an easy migration path.

out there on configuring various aspects of PHP through http.conf/.htaccess.

Httpd.conf should work unchanged, and if we make htscanner a standard part
of PHP - so will htaccess.

I think this actually shows how little understanding there is in the market
in terms of how easy migrating to FastCGI is.

Sure you can achieve most of that via FastCGI but I do not believe it’s as
simple and straightforward

It actually is, very. And we can make it even more so if we decided that
was the way to go.

I guess we’ll need to agree to disagree on this one :)

No problem :)
But let's disagree for the right reasons. Saying it's hard to switch is
simply not true. Heck, even our very own Zend Server runs on mod_php, two
flavors of FastCGI and one fpm - with no complexities stemming from these
different setups.

Zeev

11 years ago by Ferenc Kovacs — view source

unread

There are tools and know-how

That's what I meant by "history".
I don't think that should deter us from discontinuing an inferior option if
there's an easy migration path.

out there on configuring various aspects of PHP through
http.conf/.htaccess.

Httpd.conf should work unchanged, and if we make htscanner a standard part
of PHP - so will htaccess.

stuff like php_value in the apache config will be ignored AFAIK, but yeah,
if you put those in your fastcgi/fpm conf or .htaccess with htscanner then
it is pretty straightforward migration.

I think this actually shows how little understanding there is in the market
in terms of how easy migrating to FastCGI is.

setting up fastcgi is a bit more complicated for the less tech savy people.
it adds another step of indirection, stuff like using mod_fcgid or
mod_fastcgi, using unix sockets or tcp, setting up the correct timeout
settings through the pipe, max request length, etc.

--
Ferenc Kovács
@Tyr43l - http://tyrael.hu

11 years ago by Andrea Faulds — view source

unread

I think this actually shows how little understanding there is in the market
in terms of how easy migrating to FastCGI is.

setting up fastcgi is a bit more complicated for the less tech savy people.
it adds another step of indirection, stuff like using mod_fcgid or
mod_fastcgi, using unix sockets or tcp, setting up the correct timeout
settings through the pipe, max request length, etc.

It’s also easier for new users to configure it wrong. A lot of servers have been misconfigured with FastCGI and have PATH_INFO-related exploits. You can’t make the same mistake with mod_php.

Andrea Faulds
http://ajf.me/

11 years ago by Zeev Suraski — view source

unread

I think this actually shows how little understanding there is in the market
in terms of how easy migrating to FastCGI is.

setting up fastcgi is a bit more complicated for the less tech savy people.
it adds another step of indirection, stuff like using mod_fcgid or
mod_fastcgi, using unix sockets or tcp, setting up the correct timeout
settings through the pipe, max request length, etc.

It’s also easier for new users to configure it wrong. A lot of servers have
been misconfigured with FastCGI and have PATH_INFO-related exploits. You
can’t make the same mistake with mod_php.

But all of this is very manageable. Distros and stacks (which is what the
vast majority of users use, especially new users) will have the right setup
out of the box. To the average new user it'll work just as it did before,
non extra setup needed.

Heck, we can even get "make install" to work properly with Apache FastCGI
if we wanted to, and create a base setup that's fast and secure.

Zeev

11 years ago by Hannes Magnusson — view source

unread

I think this actually shows how little understanding there is in the market
in terms of how easy migrating to FastCGI is.

setting up fastcgi is a bit more complicated for the less tech savy people.
it adds another step of indirection, stuff like using mod_fcgid or
mod_fastcgi, using unix sockets or tcp, setting up the correct timeout
settings through the pipe, max request length, etc.

It’s also easier for new users to configure it wrong. A lot of servers have
been misconfigured with FastCGI and have PATH_INFO-related exploits. You
can’t make the same mistake with mod_php.

But all of this is very manageable. Distros and stacks (which is what the
vast majority of users use, especially new users) will have the right setup
out of the box. To the average new user it'll work just as it did before,
non extra setup needed.

Heck, we can even get "make install" to work properly with Apache FastCGI
if we wanted to, and create a base setup that's fast and secure.

Could you spend a day or two on that please?
And then document the crap out of fpm?

That can even be part of 5.6 and will be one of the major things we
can announce.

The current installation instructions of PHP have barely changed at
all in the past 10 years, other then tiny maintenance tweaks here and
there. That sucks balls. And means fcgi and fpm are poorly documented.

That means people will just follow what they are used to and know,
since every single installation instruction ever talks about mod_php
anyway.

If you can help us spend some time on making fpm more inviting for
people, make the installation easy flowing - and with some tweaks to
"make install" we are golden.

-Hannes

11 years ago by Zeev Suraski — view source

unread

Could you spend a day or two on that please?

I think it makes sense if & when we go with FastCGI as the default
implementation for Apache. It's premature at this point.

That can even be part of 5.6 and will be one of the major things we can
announce.

I think this is too big a change to introduce in 5.x, and we're far from
reaching consensus about it being suitable for even the next major version
of PHP.

Zeev

11 years ago by Hannes Magnusson — view source

unread

Could you spend a day or two on that please?

I think it makes sense if & when we go with FastCGI as the default
implementation for Apache. It's premature at this point.

That can even be part of 5.6 and will be one of the major things we can
announce.

I think this is too big a change to introduce in 5.x, and we're far from
reaching consensus about it being suitable for even the next major version
of PHP.

Oh sorry, I didn't mean to imply removal of mod_php or anything. Thats
a bit drastic at this point.
I was talking about fixing the mess of
http://www.php.net/manual/en/install.unix.commandline.php saying
"don't use this sapi" and the confusing
http://www.php.net/manual/en/install.fpm.php

You may also notice the wtf apache2 install guide:
http://www.php.net/manual/en/install.unix.apache2.php

But if "make install" could do most of the formalities
http://www.php.net/manual/en/install.unix.nginx.php tells you to do,
then we are golden and we can make that as one of the headline
features of 5.6;
"Installing PHP is once again a simple task!"

-Hannes

11 years ago by Ferenc Kovacs — view source

unread

I think this actually shows how little understanding there is in the market
in terms of how easy migrating to FastCGI is.

setting up fastcgi is a bit more complicated for the less tech savy people.
it adds another step of indirection, stuff like using mod_fcgid or
mod_fastcgi, using unix sockets or tcp, setting up the correct timeout
settings through the pipe, max request length, etc.

It’s also easier for new users to configure it wrong. A lot of servers
have been misconfigured with FastCGI and have PATH_INFO-related exploits.
You can’t make the same mistake with mod_php.

Andrea Faulds
http://ajf.me/

apache+mod_php also had it's fair bit of insecure default configurations
until distros figured out how to properly set it up:
http://ilia.ws/archives/226-Beware-of-the-default-Apache-2-config-for-PHP.html

--
Ferenc Kovács
@Tyr43l - http://tyrael.hu

11 years ago by Pierre Joye — view source

unread

There are tools and know-how

That's what I meant by "history".
I don't think that should deter us from discontinuing an inferior option if
there's an easy migration path.

out there on configuring various aspects of PHP through
http.conf/.htaccess.

Httpd.conf should work unchanged, and if we make htscanner a standard part
of PHP - so will htaccess.

stuff like php_value in the apache config will be ignored AFAIK, but yeah,
if you put those in your fastcgi/fpm conf or .htaccess with htscanner then
it is pretty straightforward migration.

htscanner features, from a php point of view is fully supported via
.user.ini. The main missing points are the htaccess syntax but not
really a blocker.

--
Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Zeev Suraski — view source

unread

Btw, I'm of course advocating doing this as a part of PHP 7, and not
in 5.6 or a minor version. There's going to be a migration process
from 5.6 to 7, and I think this part would be one of the easiest to
handle.

Zeev

For what it’s worth I don’t think we should stop supporting mod_php at this point in time.
I think there are still plenty of situations where this is the easier, simpler route to go and quite robust.

Such as?

FastCGI (or fpm) is just as robust, it's a lot more scalable and has mostly just advantages. The main thing mod_php has going for it is history. There are no real technological advances and the rise of PHP on nginx can attest to that.

There are tools and know-how out there on configuring various aspects of PHP through http.conf/.htaccess. Sure you can achieve most of that via FastCGI but I do not believe it’s as simple and straightforward and I still believe the majority of the PHP community is using mod_php. Re: nginx, the reality is, that while popular, it’s still not being used in the majority of environments. It’s being used more on the big, scale-out properties than on the massive of smaller PHP deployments.

I guess we’ll need to agree to disagree on this one :)

Andi

11 years ago by Pierre Joye — view source

unread

Btw, I'm of course advocating doing this as a part of PHP 7, and not
in 5.6 or a minor version. There's going to be a migration process
from 5.6 to 7, and I think this part would be one of the easiest to
handle.

There is no php7, or 6 for what matters at this time. Let keep
discussions focused.

The main disadvantage about focusing only on some SAPIs (fcgi+fpm and
cli, dbg) is that we are mostly heading to a nts only versions of PHP,
and that is per design a very bad choice. Also maintaining mod_php
does not require much resources, or at least none of yours as it is
not on your priority list since quite some time already.

Cheers,

11 years ago by Park Framework — view source

unread

2014-05-07 9:12 GMT+03:00 Sebastian Bergmann sebastian@php.net:

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

pthreads requires a build of PHP with ZTS (Zend Thread Safety) enabled (
--enable-maintainer-zts or --enable-zts on Windows )
http://docs.php.net/manual/en/pthreads.requirements.php

11 years ago by Pierre Joye — view source

unread

2014-05-07 9:12 GMT+03:00 Sebastian Bergmann sebastian@php.net:

Playing devil's advocate: does anyone really need SAPIs other than
FastCGI, CLI, and embed? And if so: do we care enough to burden
ourselves with their maintenance?

pthreads requires a build of PHP with ZTS (Zend Thread Safety) enabled (
--enable-maintainer-zts or --enable-zts on Windows )
http://docs.php.net/manual/en/pthreads.requirements.php

And talking about embed without TS is wrong. Totally.

11 years ago by Sebastian Bergmann — view source

unread

Am 07.05.2014 16:54, schrieb Pierre Joye:

And talking about embed without TS is wrong. Totally.

I was not talking about removing ZTS.

11 years ago by Anatol Belski — view source

unread

Anyway, I think 64-bit integer support on Windows64 makes full sense
and it's possible to add it into "phpng" right now.

Again, it is by far not only a windows change, even if Windows is the
platform with the most visible change from a user point of view.

OK. Win64 is enough for this patch. I wouldn't care about others, because
I
even don't know the names.

I'm not so sure about size_t string length.

It is hardly imaginable to release php 6 with strings length still
using integer. The memory consumption has been shown to be minimal during
the discussions about the 64bit support. Or did I miss some important
new information?

I wouldn't mix size_t string length with 64-bit support. These are two
independent changes. int64 would make 64-bit PHP version behave more or
less consistent with minimal cost.

size_t string length will increase memory consumption and memory traffic
for the cases that no one web application will use (2GB strings). I'm
trying to save (or even improve) memory consumption, because I know how it
may affect performance because of CPU cache misses.

just two cents about the size_t part.

The possible memory increase on 64 bit is unproportional to the increase
of the available memory on the 64 bit system itself. While it's less then
4bg on 32 bit systems, it's usually at least 8gb or more on a 64 bit
system (otherwise there were no reasons to use a 64 bit machine IMHO). So
from this POV it's almost uncritical. Furthermore, the big part of that
will be going on the image size, not on the heap (on linux zval didn't
increase).

Regarding the possible performance issue you mentioned - yes, it can be as
well, but not worse as it is. However as discussed before, the usage of
the mem types like size_t and ptrdiff_t is mentioned by CPU manufacturers
as a good thing to do as apart from cache it'll lead to more optimal
memory and registers usage. The same is about the other cases like usage
of right data types in the loops, security with the string lengths coming
as 64 bit int from the user space, pointer casts to int instead of
ptrdiff_t, etc. This topic was and is on the plate and was just defered to
the time the int64 patch were accepted. One can see what exactly the
bottle necks are using the tools from the CPU manufacturers like AMD and
Intel. That is definitely an outstanding task which were far in advance
now if the int64 patch were in 5.6 now.

More on memory usage is that right now many memory structures can be
optimized for better alignment which would decrease their allocated sizes.
Of course that has to be revisited again in the new patches. But that
could free an amount of memory.

IMHO it's much more important to do a clean start now with both patches
and drive them to the top. That's why we should have coordinated our works
on that which would save our efforts on merging it and allow a better
result from the start on.

Best regards

Anatol

11 years ago by Dmitry Stogov — view source

unread

Hi Anatol,

i agree that coordination from beginning would make some thing easier, but
I hardly believe we would able to do the PoC in short time with endless
discussions.

Lets take what we have now.
I understand what you are not glad to do the same work once again.
I make take some part of this work, if we come to agreement.

According to performance, I wouldn't believe to anything except tests.
and it's why I propose to start with the part that can't make any harm.

Thanks. Dmitry.

On Tue, May 6, 2014 at 11:20 AM, Anatol Belski anatol.php@belski.netwrote:

Anyway, I think 64-bit integer support on Windows64 makes full sense
and it's possible to add it into "phpng" right now.

Again, it is by far not only a windows change, even if Windows is the
platform with the most visible change from a user point of view.

OK. Win64 is enough for this patch. I wouldn't care about others, because
I
even don't know the names.

I'm not so sure about size_t string length.

It is hardly imaginable to release php 6 with strings length still
using integer. The memory consumption has been shown to be minimal
during
the discussions about the 64bit support. Or did I miss some important
new information?

I wouldn't mix size_t string length with 64-bit support. These are two
independent changes. int64 would make 64-bit PHP version behave more or
less consistent with minimal cost.

size_t string length will increase memory consumption and memory traffic
for the cases that no one web application will use (2GB strings). I'm
trying to save (or even improve) memory consumption, because I know how
it
may affect performance because of CPU cache misses.

just two cents about the size_t part.

The possible memory increase on 64 bit is unproportional to the increase
of the available memory on the 64 bit system itself. While it's less then
4bg on 32 bit systems, it's usually at least 8gb or more on a 64 bit
system (otherwise there were no reasons to use a 64 bit machine IMHO). So
from this POV it's almost uncritical. Furthermore, the big part of that
will be going on the image size, not on the heap (on linux zval didn't
increase).

Regarding the possible performance issue you mentioned - yes, it can be as
well, but not worse as it is. However as discussed before, the usage of
the mem types like size_t and ptrdiff_t is mentioned by CPU manufacturers
as a good thing to do as apart from cache it'll lead to more optimal
memory and registers usage. The same is about the other cases like usage
of right data types in the loops, security with the string lengths coming
as 64 bit int from the user space, pointer casts to int instead of
ptrdiff_t, etc. This topic was and is on the plate and was just defered to
the time the int64 patch were accepted. One can see what exactly the
bottle necks are using the tools from the CPU manufacturers like AMD and
Intel. That is definitely an outstanding task which were far in advance
now if the int64 patch were in 5.6 now.

More on memory usage is that right now many memory structures can be
optimized for better alignment which would decrease their allocated sizes.
Of course that has to be revisited again in the new patches. But that
could free an amount of memory.

IMHO it's much more important to do a clean start now with both patches
and drive them to the top. That's why we should have coordinated our works
on that which would save our efforts on merging it and allow a better
result from the start on.

Best regards

Anatol

11 years ago by Pierre Joye — view source

unread

Hi Anatol,

i agree that coordination from beginning would make some thing easier, but I
hardly believe we would able to do the PoC in short time with endless
discussions.

Lets take what we have now.
I understand what you are not glad to do the same work once again.
I make take some part of this work, if we come to agreement.

According to performance, I wouldn't believe to anything except tests.
and it's why I propose to start with the part that can't make any harm.

Indeed. However I would not agree to suspend/discard the 64bit RFC or
anything along this line at this point, "only" because there is a
(very welcome and good) patch being worked on.

I think we can help to make it work with the 64bit patch as well as
porting extensions, including pecl ones.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Anatol Belski — view source

unread

Hi Dmitry,

Hi Anatol,

i agree that coordination from beginning would make some thing easier,
but I hardly believe we would able to do the PoC in short time with
endless discussions.

Lets take what we have now.
I understand what you are not glad to do the same work once again.
I make take some part of this work, if we come to agreement.

Yep, we have what we have now, so lets take it as the basis. I do not shy
to take that effort as well, not a question at all.

According to performance, I wouldn't believe to anything except tests.
and it's why I propose to start with the part that can't make any harm.

Yeah, there is almost no diff if you look here
http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-20140411-masterr6c2f7bc-str_size_and_int64raa0c920.html
(that's also linked from the RFC). Many scripting languages do support
that like python and ruby and seem to have no harm. AFAIR Python even
implements int64 on 32 bit platforms. So IMHO it really makes sense too
keep both as then we have a balance between the functionality and
performance, both will have a gain and both can get further improvement in
the future.

Best regards

Anatol

11 years ago by Dmitry Stogov — view source

unread

On Tue, May 6, 2014 at 12:21 PM, Anatol Belski anatol.php@belski.netwrote:

Hi Dmitry,

Hi Anatol,

i agree that coordination from beginning would make some thing easier,
but I hardly believe we would able to do the PoC in short time with
endless discussions.

Lets take what we have now.
I understand what you are not glad to do the same work once again.
I make take some part of this work, if we come to agreement.

Yep, we have what we have now, so lets take it as the basis. I do not shy
to take that effort as well, not a question at all.

great!

According to performance, I wouldn't believe to anything except tests.
and it's why I propose to start with the part that can't make any harm.

Yeah, there is almost no diff if you look here

http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-20140411-masterr6c2f7bc-str_size_and_int64raa0c920.html
(that's also linked from the RFC). Many scripting languages do support
that like python and ruby and seem to have no harm. AFAIR Python even
implements int64 on 32 bit platforms. So IMHO it really makes sense too
keep both as then we have a balance between the functionality and
performance, both will have a gain and both can get further improvement in
the future.

It's the reason why we are faster :)

support for both int32 and int64 is going to be a big overhead even on
64-bit platforms. (I implemented and tested it about 8 years ago).
Each math opcdoe (ZEND_ADD, ZEND_MUL etc) are going to be complicated
finding the right operand types combination and overflow handling.

Thanks. Dmitry.

Best regards

Anatol

11 years ago by Anatol Belski — view source

unread

On Tue, May 6, 2014 at 12:21 PM, Anatol Belski
anatol.php@belski.netwrote:

Hi Dmitry,

Hi Anatol,

i agree that coordination from beginning would make some thing
easier, but I hardly believe we would able to do the PoC in short time
with endless discussions.

Lets take what we have now.
I understand what you are not glad to do the same work once again.
I make take some part of this work, if we come to agreement.

Yep, we have what we have now, so lets take it as the basis. I do not
shy to take that effort as well, not a question at all.

great!

According to performance, I wouldn't believe to anything except
tests. and it's why I propose to start with the part that can't make
any harm.

Yeah, there is almost no diff if you look here

http://windows.php.net/downloads/snaps/ostc/pftt/perf/results-20140411-
masterr6c2f7bc-str_size_and_int64raa0c920.html (that's also linked from
the RFC). Many scripting languages do support that like python and ruby
and seem to have no harm. AFAIR Python even implements int64 on 32 bit
platforms. So IMHO it really makes sense too keep both as then we have a
balance between the functionality and performance, both will have a gain
and both can get further improvement in the future.

It's the reason why we are faster :)

support for both int32 and int64 is going to be a big overhead even on
64-bit platforms. (I implemented and tested it about 8 years ago).
Each math opcdoe (ZEND_ADD, ZEND_MUL etc) are going to be complicated
finding the right operand types combination and overflow handling.

I wasn't suggesting to do that, especially not both 32 and 64 bit ints in
the same build :). Just to ilustrate how the equilibrium could be kept. As
you've seen, the discussion on the int64 RFC was started again. Lets see
how we can proceed further after it. And i'll be testing the phpng branch
the next days, too.

Cheers

Anatol

11 years ago by Pierre Joye — view source

unread

I wasn't suggesting to do that, especially not both 32 and 64 bit ints in
the same build :).

FYI, There is no plan to support 32 and 64 bit integer on 64 bit
architecture but only 64bit integer. 32 bit architecture will only have
32bit as of now as well.

11 years ago by Dmitry Stogov — view source

unread

it's fine. sorry, for misunderstanding.

Dmitry.

I wasn't suggesting to do that, especially not both 32 and 64 bit ints in
the same build :).

FYI, There is no plan to support 32 and 64 bit integer on 64 bit
architecture but only 64bit integer. 32 bit architecture will only have
32bit as of now as well.

11 years ago by Stas Malyshev — view source

unread

Hi!

It is hardly imaginable to release php 6 with strings length still
using integer. The memory consumption has been shown to be minimal

Why btw it would be unimaginable? I.e., how many PHP apps really need to
load strings of 2g+ size? I'm not advocating for not doing this, just
curious why it is that big of a deal.

--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

11 years ago by Pierre Joye — view source

unread

Hi!

It is hardly imaginable to release php 6 with strings length still
using integer. The memory consumption has been shown to be minimal

Why btw it would be unimaginable? I.e., how many PHP apps really need to
load strings of 2g+ size? I'm not advocating for not doing this, just
curious why it is that big of a deal.

Because it is really a bad way to support 64bit architecture. Many
compilers, analyzer or related tools can do a much better job when
modern and clean typing are used, let alone the safe typing (no, using
int for buffer length is not safe nor clean :).

In the last thread about this topic we posted many external links to
documentation or tutorials explaining in details why size_t and co
should be used. If you are interested to go deeper in this topic,

--
Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Pierre Joye — view source

unread

hi,

Some comments about the future ideas:

. replace the memory manager

If you are working on it now, please do it in public so other can
contribute and ensure that it will work on all supported platforms and
SAPIs. Debugging a huge patch is much harder and time consuming than
constantly improving something.

refactoring zpp

this idea fits very well with what we need for the 64 bit branch as
well. Some coop are necessary here as well.

jsond

Unless there are some very bad BC breaks, I would really like to see
it merged in master very soon. This is the best way to get it stable
and tested.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Dmitry Stogov — view source

unread

hi,

Some comments about the future ideas:

. replace the memory manager

If you are working on it now, please do it in public so other can
contribute and ensure that it will work on all supported platforms and
SAPIs. Debugging a huge patch is much harder and time consuming than
constantly improving something.

I sent xx_malloc prototype on December (nobody was really interested).
I didn't make any related work (just tested that it works with refactored
php and gives some improvement).
I don't plan to work on it in nearest weeks, because MM changes must be
completely transparent for extensions and I like to think about the parts
that might affect others first.

refactoring zpp

this idea fits very well with what we need for the 64 bit branch as
well. Some coop are necessary here as well.

I think it's unrelated to 64 bit. Just an API designed without performance
in mind. :(

Thanks. Dmitry.

jsond

Unless there are some very bad BC breaks, I would really like to see
it merged in master very soon. This is the best way to get it stable
and tested.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Pierre Joye — view source

unread

I sent xx_malloc prototype on December (nobody was really interested).
I didn't make any related work (just tested that it works with refactored
php and gives some improvement).
I don't plan to work on it in nearest weeks, because MM changes must be
completely transparent for extensions and I like to think about the parts
that might affect others first.

refactoring zpp

this idea fits very well with what we need for the 64 bit branch as
well. Some coop are necessary here as well.

I think it's unrelated to 64 bit. Just an API designed without performance
in mind. :(

There was quite some interest if I remember correctly. But you defined
it as highly experimental and on going work. I think it would have
made sense to have a branch with it or use master with a flag to ease
testing and contributions (and still makes sense).

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Dmitry Stogov — view source

unread

I'm going to integrate it into phpng later.
It's probably a too big change for "master".

Thanks. Dmitry.

I sent xx_malloc prototype on December (nobody was really interested).
I didn't make any related work (just tested that it works with refactored
php and gives some improvement).
I don't plan to work on it in nearest weeks, because MM changes must be
completely transparent for extensions and I like to think about the parts
that might affect others first.

refactoring zpp

this idea fits very well with what we need for the 64 bit branch as
well. Some coop are necessary here as well.

I think it's unrelated to 64 bit. Just an API designed without
performance
in mind. :(

There was quite some interest if I remember correctly. But you defined
it as highly experimental and on going work. I think it would have
made sense to have a branch with it or use master with a flag to ease
testing and contributions (and still makes sense).

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

11 years ago by Laruence — view source

unread

hi,

Some comments about the future ideas:

. replace the memory manager

If you are working on it now, please do it in public so other can
contribute and ensure that it will work on all supported platforms and
SAPIs. Debugging a huge patch is much harder and time consuming than
constantly improving something.

refactoring zpp

this idea fits very well with what we need for the 64 bit branch as
well. Some coop are necessary here as well.

jsond

Unless there are some very bad BC breaks, I would really like to see
it merged in master very soon. This is the best way to get it stable
and tested.
Hey:
I was working on it yesterday, but, it turns out, the parser of it
assume array and object properties are all Hashtables..

unfortunately, in ng, array is zend_array now... so, it will need
a little more works to do...

anyway: I will keep working on it, but not now... :)

thanks

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

--

--
Laruence Xinchen Hui
http://www.laruence.com/

11 years ago by Jakub Zelenka — view source

unread

hi,

Some comments about the future ideas:

. replace the memory manager

If you are working on it now, please do it in public so other can
contribute and ensure that it will work on all supported platforms and
SAPIs. Debugging a huge patch is much harder and time consuming than
constantly improving something.

refactoring zpp

this idea fits very well with what we need for the 64 bit branch as
well. Some coop are necessary here as well.

jsond

Unless there are some very bad BC breaks, I would really like to see
it merged in master very soon. This is the best way to get it stable
and tested.
Hey:
I was working on it yesterday, but, it turns out, the parser of it
assume array and object properties are all Hashtables..

unfortunately, in ng, array is zend_array now... so, it will need
a little more works to do...

anyway: I will keep working on it, but not now... :)

thanks

Yeah it looks that it will require some changes in the parser. I'll
definitely have a look... ;)

It could make sense to replace json with jsond in the phpng branch? There
are no bad BC breaks... :) It would also help to stabilize it as this
branch probably will be quite heavily tested... I could also work more on
this branch and help to fix some other issues...

Regards

Jakub

11 years ago by Adam Harvey — view source

unread

It could make sense to replace json with jsond in the phpng branch? There
are no bad BC breaks... :) It would also help to stabilize it as this
branch probably will be quite heavily tested... I could also work more on
this branch and help to fix some other issues...

I like the idea of having jsond in the longer term, but do we want
phpng to be the de facto "every PHP 5++ feature" branch?

Spitballing:

I presume the end game for phpng is going to be to vote on it, (if
passed) merge it to master, add a delta to the major version number
(whether that delta is 1 or 2 is probably another vote), and then open
PHP 5++ for RFCs, which would then presumably allow jsond to be voted
and merged soon after.

I don't know what timeframe Dmitry and co are looking at for having
this merged, but it seems like it might dovetail nicely with 5.6: we
can get 5.6 out the door, then turn our attention to 5++ for a while —
start refining that list of ideas on the Wiki, getting stuff in,
figuring out what should be removed/deprecated, and so on.

Adam, who promises he'll write some actual code at some point rather
than just prognosticating on Internals.

phpng: Refactored PHP Engine with Big Performance Improvement

--

...

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Do we know why? I would assume since FastCGI has additional data hop (between webserver and the actual PHP binary) Apache one would be faster but looks like it's not so. It'd be interesting to know why.

Cheers,

This is something I’m curious about. While I’d be surprised if the vast majority of servers in use lacked 64-bit CPUs, I can’t seem to find any statistics on this. Does anyone have any?

Cheers,

It’s also easier for new users to configure it wrong. A lot of servers have been misconfigured with FastCGI and have PATH_INFO-related exploits. You can’t make the same mistake with mod_php.

It’s also easier for new users to configure it wrong. A lot of servers have been misconfigured with FastCGI and have PATH_INFO-related exploits. You can’t make the same mistake with mod_php.

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Do we know why? I would assume since FastCGI has additional data hop
(between webserver and the actual PHP binary) Apache one would be faster
but looks like it's not so. It'd be interesting to know why.

It’s also easier for new users to configure it wrong. A lot of servers
have been misconfigured with FastCGI and have PATH_INFO-related exploits.
You can’t make the same mistake with mod_php.