ZTS - why are you using it?

12 years ago by Zeev Suraski — view source

unread

I didn’t want to hijack the Optimizer+ thread so I’m creating a new one,
based on the apparent level of interest in ZTS. This isn’t an RFC to
remove ZTS by any stretch, but I am a bit confused about why people are
still using ZTS.

A bit of background. I started the ZTS project (based on initial work by
Shane Caraveo more than a decade ago), and invested countless hours in
pushing it all throughout PHP. So I really hold absolutely no grudge
against it, quite the contrary… However, many years ago, I came to the
conclusion that it was a bit of a lost cause trying to weed out all the
thread safety issues from everywhere – and that a simpler, more elegant
solution exists in the form of FastCGI.

Which brings me to the subject of this mail – why are you using ZTS PHP
instead of single threaded PHP? The reasons not to use it are few but
fairly major – it’s significantly slower than the non-ZTS PHP, and it’s
significantly less robust in the sense that a single bug somewhere can
bring down an entire server (or at least a bunch of many different
threads). What are your reasons to choose it over FastCGI?

Zeev

12 years ago by netroby — view source

unread

I didn’t want to hijack the Optimizer+ thread so I’m creating a new one,
based on the apparent level of interest in ZTS. This isn’t an RFC to
remove ZTS by any stretch, but I am a bit confused about why people are
still using ZTS.

A bit of background. I started the ZTS project (based on initial work by
Shane Caraveo more than a decade ago), and invested countless hours in
pushing it all throughout PHP. So I really hold absolutely no grudge
against it, quite the contrary… However, many years ago, I came to the
conclusion that it was a bit of a lost cause trying to weed out all the
thread safety issues from everywhere – and that a simpler, more elegant
solution exists in the form of FastCGI.

Which brings me to the subject of this mail – why are you using ZTS PHP
instead of single threaded PHP? The reasons not to use it are few but
fairly major – it’s significantly slower than the non-ZTS PHP, and it’s
significantly less robust in the sense that a single bug somewhere can
bring down an entire server (or at least a bunch of many different
threads). What are your reasons to choose it over FastCGI?

Zeev

Old behavior not easily to be changed.
let it as it be, all of us like to keep on old behavior as it was.

should put banner on php.net, say: do not using ZTS, that's old way.

--
Appreciate your time.

Netroby

12 years ago by Alexey Zakhlestin — view source

unread

Which brings me to the subject of this mail – why are you using ZTS PHP
instead of single threaded PHP? The reasons not to use it are few but
fairly major – it’s significantly slower than the non-ZTS PHP, and it’s
significantly less robust in the sense that a single bug somewhere can
bring down an entire server (or at least a bunch of many different
threads). What are your reasons to choose it over FastCGI?

I use it for long-running processes, when I need something like
http://pecl.php.net/package/pthreads
But, those are scenarios, when I don't need bytecode cache, because
everything is preloaded in "real" memory

--
Alexey Zakhlestin,
http://github.com/indeyets

12 years ago by Laruence — view source

unread

I didn’t want to hijack the Optimizer+ thread so I’m creating a new one,
based on the apparent level of interest in ZTS. This isn’t an RFC to
remove ZTS by any stretch, but I am a bit confused about why people are
still using ZTS.

A bit of background. I started the ZTS project (based on initial work by
Shane Caraveo more than a decade ago), and invested countless hours in
pushing it all throughout PHP. So I really hold absolutely no grudge
against it, quite the contrary… However, many years ago, I came to the
conclusion that it was a bit of a lost cause trying to weed out all the
thread safety issues from everywhere – and that a simpler, more elegant
solution exists in the form of FastCGI.

Which brings me to the subject of this mail – why are you using ZTS PHP
instead of single threaded PHP? The reasons not to use it are few but
fairly major – it’s significantly slower than the non-ZTS PHP, and it’s
significantly less robust in the sense that a single bug somewhere can
bring down an entire server (or at least a bunch of many different
threads). What are your reasons to choose it over FastCGI?

Hey:

It's not we choose ZTS, it is there are many users run with them (IIS,
Apache+workers, and pthreads extension require it)

and all PHP extensions supports it, so if O+ doesn't, it feel a little
inconsistent.

thanks

Zeev

--
Laruence Xinchen Hui
http://www.laruence.com/

12 years ago by Laruence — view source

unread

I didn’t want to hijack the Optimizer+ thread so I’m creating a new one,
based on the apparent level of interest in ZTS. This isn’t an RFC to
remove ZTS by any stretch, but I am a bit confused about why people are
still using ZTS.

A bit of background. I started the ZTS project (based on initial work by
Shane Caraveo more than a decade ago), and invested countless hours in
pushing it all throughout PHP. So I really hold absolutely no grudge
against it, quite the contrary… However, many years ago, I came to the
conclusion that it was a bit of a lost cause trying to weed out all the
thread safety issues from everywhere – and that a simpler, more elegant
solution exists in the form of FastCGI.

Which brings me to the subject of this mail – why are you using ZTS PHP
instead of single threaded PHP? The reasons not to use it are few but
fairly major – it’s significantly slower than the non-ZTS PHP, and it’s
significantly less robust in the sense that a single bug somewhere can
bring down an entire server (or at least a bunch of many different
threads). What are your reasons to choose it over FastCGI?

Hey:

It's not we choose ZTS, it is there are many users run with them (IIS,
Apache+workers, and pthreads extension require it)

and all PHP extensions supports it, so if O+ doesn't, it feel a little
inconsistent.

And, if O+ aims only to be a PECL extension, I think it's okey.

but if it is going to PHP src, I think the consistent should be
considered, agree?

thanks

thanks

Zeev

--
Laruence Xinchen Hui
http://www.laruence.com/

--
Laruence Xinchen Hui
http://www.laruence.com/

12 years ago by Zeev Suraski — view source

unread

Hey:

It's not we choose ZTS, it is there are many users run with them (IIS,
Apache+workers, and pthreads extension require it)

For pthreads I can understand it, but why would users be using it on
IIS/Apache instead of using FastCGI? FastCGI is both faster and more
robust. Is it a matter of lack of knowledge, or is there a real reason to
use it?

and all PHP extensions supports it, so if O+ doesn't, it feel a little
inconsistent.

My question was outside the context of O+ - it should support it for as long
as ZTS is available...

Zeev

12 years ago by Laruence — view source

unread

Hey:

It's not we choose ZTS, it is there are many users run with them (IIS,
Apache+workers, and pthreads extension require it)

For pthreads I can understand it, but why would users be using it on
IIS/Apache instead of using FastCGI? FastCGI is both faster and more
robust. Is it a matter of lack of knowledge, or is there a real reason to
use it?

and all PHP extensions supports it, so if O+ doesn't, it feel a little
inconsistent.

My question was outside the context of O+ - it should support it for as long
as ZTS is available...

Oh, in that case, I prefer fcgi(eg: nginx fpm) too. :)

thanks

Zeev

--
Laruence Xinchen Hui
http://www.laruence.com/

12 years ago by Pierre Joye — view source

unread

hi Zeev,

Which brings me to the subject of this mail – why are you using ZTS PHP
instead of single threaded PHP? The reasons not to use it are few but
fairly major – it’s significantly slower than the non-ZTS PHP, and it’s
significantly less robust in the sense that a single bug somewhere can
bring down an entire server (or at least a bunch of many different
threads). What are your reasons to choose it over FastCGI?

There are situations where FPM/FCGI are not appropriate, or the server
used does not support NTS (Apache windows for example, when fcgi is
not an option).

However, in the long run, I would like to kill TSRM altogether, moving
the file/IO related parts where they should be implemented, either the
engine or main. Does it mean we will kill thread safety by doing so?
No. My idea is to replace it with LTS (thread local storage) which
drastically reduce the thread safety issues due to its exact nature,
store local thread data per thread and not per process. Issues include
well know crashes due to non thread safe code (almost none left in
core, but some libs have issues) and brings a huge performance
improvement.

The other main reason from my side to keep ZTS is Windows. Windows
cannot perform well using process based SAPI. It won't match linux as
long as it runs within a webserver using a process based
implementation (but CLI does not match linux performance and is even
faster in a couple of use cases). I've made some tests to write new
ISAPIs for IIS8 and with the httpkernel (1). The performance are very
promising, given that it is still only a proof of concepts. Having an
ISAPI also allows a tighter integration to IIS8 (pipelines, auths,
scripting, etc.), which is not possible to do now using fastcgi (no
direct access but using RMI or the IIS shell, but horrible, in all
possible ways).

Conclusion:

Yes, TSRM is horrible and does not match modern thread safe
implementation (APC does it better for its usage f.e. using rwlock).
However it would be a very bad idea to give up on thread safety only
because the relatively new trend is all about single threaded/process
based implementation. These systems have limits and I'm sure we will
see some big change in this area in the couple of years. TLS will let
us to do the transition and we can 'easily' support extension
developers to port their extensions.

(1) similar to our builtin webserver except that it can be used in
production servers. It is what IIS used under the hood. Having a SAPI
relying directly on HttpKernel will allow one to create many app pools
using PHP only. It will work smoothly even if IIS runs on the same
host, sharing the same IPs or ports, even the cache will work.

Cheers.

Pierre

@pierrejoye

12 years ago by Pierre Joye — view source

unread

The other main reason from my side to keep ZTS is Windows. Windows
cannot perform well using process based SAPI. It won't match linux as
long as it runs within a webserver using a process based
implementation (but CLI does not match linux performance and is even
faster in a couple of use cases).

I mean:

but CLI does match linux performance and is even faster in a couple of
use cases :)

--
Pierre

@pierrejoye

12 years ago by Zeev Suraski — view source

unread

The other main reason from my side to keep ZTS is Windows. Windows
cannot
perform well using process based SAPI.

Windows actually works quite well with FastCGI. So well Microsoft even
created their own version for IIS. It's outperforming the ISAPI module by
a wide margin.

Other than Apache/Windows not having FastCGI support(*), I really can't
imagine any situation where using ZTS inside of a Web Server context makes
any sense at all. I wouldn't call it a new trend, it's both old (I've
been pushing for it since at least 2006, probably earlier) and with very
solid technical reasons (faster, more reliable).

Yes, TSRM is horrible and does not match modern thread safe
implementation
(APC does it better for its usage f.e. using rwlock).

Note that I wasn't talking about the implementation of ZTS, but why you
would want to use it in the first place. I actually think that using
thread local storage is much better than using locks - but if you can make
the whole problem disappear because there's no need for thread safety,
that's even better. Why heavily invest in something unless there's a very
good reason to use it?

Zeev

(*) Apache actually does have a good FastCGI implementation available in
Zend Server for Windows (including the free CE version). Using it is
faster and more reliable than using mod_php on Windows.

12 years ago by Pierre Joye — view source

unread

The other main reason from my side to keep ZTS is Windows. Windows
cannot
perform well using process based SAPI.

Windows actually works quite well with FastCGI. So well Microsoft even
created their own version for IIS. It's outperforming the ISAPI module by
a wide margin.

Laziness and design mistake. Everything on windows (AD,IIS, asp.net, etc)
uses thread.

And no, nuts is not faster. I am not talking about PHP zts, but in general.

Other than Apache/Windows not having FastCGI support(*), I really can't
imagine any situation where using ZTS inside of a Web Server context makes
any sense at all. I wouldn't call it a new trend, it's both old (I've
been pushing for it since at least 2006, probably earlier) and with very
solid technical reasons (faster, more reliable).

Miss the rest of my mail or? Current implementation is outdated and slow.

Yes, TSRM is horrible and does not match modern thread safe
implementation
(APC does it better for its usage f.e. using rwlock).

Note that I wasn't talking about the implementation of ZTS, but why you
would want to use it in the first place. I actually think that using
thread local storage is much better than using locks - but if you can make
the whole problem disappear because there's no need for thread safety,
that's even better. Why heavily invest in something unless there's a very
good reason to use it?

See what I wrote in my previous reply.

Cheers,

12 years ago by johannes@schlueters.de — view source

unread

Laziness and design mistake. Everything on windows (AD,IIS, asp.net,
etc)
uses thread.

Well, most other things don't create "shared-nothing" environments like
PHP does. ASP, not only due to the Application object, for instance
isn't shared-nothing like PHP but sharing state. So what we're doing,
basically, is emulate a separation of threads, similar to what an
operating system should do with processes.

Of course an opcode cache isn't shred-nothing either, and maybe sharing
opcodes within a process is faster than doing this in shared memory.

Miss the rest of my mail or? Current implementation is outdated and
slow.

That is true. Many modern compilers and environments provide better
support for thread local storages ....

What I see is that TSRM brings a lot of complexity and introduces risk
of bugs in there as many developers don't think about. Most of us are
focused on the nice and simple world of single-threaded computing. And I
assume that we can still find quite some threading bugs when going on a
hunt for them ... so besides of pure executable performance there is
also the cost of developer power for (proper) maintenance ...

johannes

12 years ago by Pierre Joye — view source

unread

hi,

On Tue, Jan 29, 2013 at 12:45 PM, Johannes Schlüter
johannes@schlueters.de wrote:

Laziness and design mistake. Everything on windows (AD,IIS, asp.net,
etc)
uses thread.

Well, most other things don't create "shared-nothing" environments like
PHP does. ASP, not only due to the Application object, for instance
isn't shared-nothing like PHP but sharing state. So what we're doing,
basically, is emulate a separation of threads, similar to what an
operating system should do with processes.

Of course an opcode cache isn't shred-nothing either, and maybe sharing
opcodes within a process is faster than doing this in shared memory.

It is inter process sharing and is very expensive, nothing to compare
with shared memory within a single process, accross many threads.

Miss the rest of my mail or? Current implementation is outdated and
slow.

That is true. Many modern compilers and environments provide better
support for thread local storages ....

Exactly, or more exactly CRTs (libc, crt and the likes)>

What I see is that TSRM brings a lot of complexity and introduces risk
of bugs in there as many developers don't think about. Most of us are
focused on the nice and simple world of single-threaded computing. And I
assume that we can still find quite some threading bugs when going on a
hunt for them ... so besides of pure executable performance there is
also the cost of developer power for (proper) maintenance ...

The amount of bugs is rather low. We do test in TS and TS on a daily
basis. The last extension causing more critical issues was mysql, we
have discussed these bugs for a long time.

However let me state my point clear: Giving up on thread safety would
be a mistake as big as safemode and other insane choices we made. That
does mean we have to keep going with the current state but we must
keep it. Other languages support it because most of them are thread
safe by design and allows extensions to rely on that.

Cheers,

Pierre

@pierrejoye

12 years ago by johannes@schlueters.de — view source

unread

That is true. Many modern compilers and environments provide better
support for thread local storages ....

Exactly, or more exactly CRTs (libc, crt and the likes)>

That's what I called "environment" - some of these things depend on
kernels, some on libraries, ... anyway - irrelevant terminology :-p

What I see is that TSRM brings a lot of complexity and introduces risk
of bugs in there as many developers don't think about. Most of us are
focused on the nice and simple world of single-threaded computing. And I
assume that we can still find quite some threading bugs when going on a
hunt for them ... so besides of pure executable performance there is
also the cost of developer power for (proper) maintenance ...

The amount of bugs is rather low. We do test in TS and TS on a daily
basis. The last extension causing more critical issues was mysql, we
have discussed these bugs for a long time.

There were mysqli threading bugs, the last one of those actually had
been engine bugs which affected other extensions, too. See i.e.
http://news.php.net/php.internals/59353

Such bugs identified a year after the release of 5.4 gives me quite a
good indication of the little actual testing being done overall.

johannes

12 years ago by Pierre Joye — view source

unread

On Tue, Jan 29, 2013 at 1:01 PM, Johannes Schlüter
johannes@schlueters.de wrote:

There were mysqli threading bugs, the last one of those actually had
been engine bugs which affected other extensions, too. See i.e.
http://news.php.net/php.internals/59353

Such bugs identified a year after the release of 5.4 gives me quite a
good indication of the little actual testing being done overall.

Can you stay fair, check your backlog and the verbose mail exchanges
we had with the mysql connectivity team about these issues? Before and
after the bugs were reported? thanks.

--
Pierre

@pierrejoye

12 years ago by johannes@schlueters.de — view source

unread

On Tue, Jan 29, 2013 at 1:01 PM, Johannes Schlüter
johannes@schlueters.de wrote:

There were mysqli threading bugs, the last one of those actually had
been engine bugs which affected other extensions, too. See i.e.
http://news.php.net/php.internals/59353

Such bugs identified a year after the release of 5.4 gives me quite a
good indication of the little actual testing being done overall.

Can you stay fair, check your backlog and the verbose mail exchanges
we had with the mysql connectivity team about these issues? Before and
after the bugs were reported? thanks.

I said "There were mysqli threading bugs" and yes, it took some time to
fix those. But to be fair it also has to be mentioned that not all bugs
pushed to "yet another mysqli threading bug" had actually been mysqli
issues.

Out of curiosity I had a quick test: Running wordpress with a bunch of
parallel threads. Immediately found some minor threading issues of this
form:

    ==19088== Possible data race during read of size 1 at 0x4c82820 by thread #6
    ==19088==    at 0x47747D: pcre_get_compiled_regex_cache (php_pcre.c:387)
    ==19088==    by 0x4777CB: php_pcre_replace (php_pcre.c:986)
    ==19088==    by 0x47794E: php_replace_in_subject (php_pcre.c:1311)
    ==19088==    by 0x478022: preg_replace_impl (php_pcre.c:1409)
    ==19088==    by 0x478377: zif_preg_replace (php_pcre.c:1429)
    ==19088==    by 0x76EC36: zend_do_fcall_common_helper_SPEC (zend_vm_execute.h:546)
    ==19088==    by 0x75EA8A: execute_ex (zend_vm_execute.h:356)
    ==19088==    by 0x6E38D2: zend_execute_scripts (zend.c:1316)
    ==19088==    by 0x67C180: php_execute_script (main.c:2479)
    ==19088==    by 0x79E4B1: pconn_do_request (pconnect-sapi.c:208)
    ==19088==    by 0x79E299: php_thread (main.c:63)
    ==19088==    by 0x4A0C01D: mythread_wrapper (hg_intercepts.c:221)
    ==19088==  Address 0x4c82820 is 0 bytes inside a block of size 11 alloc'd
    ==19088==    at 0x4A0626E: malloc (vg_replace_malloc.c:236)
    ==19088==    by 0x339F280871: strdup (in /lib64/libc-2.12.so)
    ==19088==    by 0x339F228FE3: setlocale (in /lib64/libc-2.12.so)
    ==19088==    by 0x449074: seek_to_tz_position (parse_tz.c:283)
    ==19088==    by 0x4490B6: timelib_timezone_id_is_valid (parse_tz.c:312)
    ==19088==    by 0x422231: zif_date_default_timezone_set (php_date.c:4350)
    ==19088==    by 0x76EC36: zend_do_fcall_common_helper_SPEC (zend_vm_execute.h:546)
    ==19088==    by 0x75EA8A: execute_ex (zend_vm_execute.h:356)
    ==19088==    by 0x6E38D2: zend_execute_scripts (zend.c:1316)
    ==19088==    by 0x67C180: php_execute_script (main.c:2479)
    ==19088==    by 0x79E4B1: pconn_do_request (pconnect-sapi.c:208)
    ==19088==    by 0x79E299: php_thread (main.c:63)
    ==19088== 
    ==19088== Possible data race during read of size 1 at 0x4c82820 by thread #3
    ==19088==    at 0x4A07823: strcmp (hg_intercepts.c:2342)
    ==19088==    by 0x4772D7: pcre_get_compiled_regex_cache (php_pcre.c:264)
    ==19088==    by 0x4777CB: php_pcre_replace (php_pcre.c:986)
    ==19088==    by 0x47794E: php_replace_in_subject (php_pcre.c:1311)
    ==19088==    by 0x478022: preg_replace_impl (php_pcre.c:1409)
    ==19088==    by 0x478377: zif_preg_replace (php_pcre.c:1429)
    ==19088==    by 0x76EC36: zend_do_fcall_common_helper_SPEC (zend_vm_execute.h:546)
    ==19088==    by 0x75EA8A: execute_ex (zend_vm_execute.h:356)
    ==19088==    by 0x6E38D2: zend_execute_scripts (zend.c:1316)
    ==19088==    by 0x67C180: php_execute_script (main.c:2479)
    ==19088==    by 0x79E4B1: pconn_do_request (pconnect-sapi.c:208)
    ==19088==    by 0x79E299: php_thread (main.c:63)
    ==19088==  Address 0x4c82820 is 0 bytes inside a block of size 11 alloc'd
    ==19088==    at 0x4A0626E: malloc (vg_replace_malloc.c:236)
    ==19088==    by 0x339F280871: strdup (in /lib64/libc-2.12.so)
    ==19088==    by 0x339F228FE3: setlocale (in /lib64/libc-2.12.so)
    ==19088==    by 0x449074: seek_to_tz_position (parse_tz.c:283)
    ==19088==    by 0x4490B6: timelib_timezone_id_is_valid (parse_tz.c:312)
    ==19088==    by 0x422231: zif_date_default_timezone_set (php_date.c:4350)
    ==19088==    by 0x76EC36: zend_do_fcall_common_helper_SPEC (zend_vm_execute.h:546)
    ==19088==    by 0x75EA8A: execute_ex (zend_vm_execute.h:356)
    ==19088==    by 0x6E38D2: zend_execute_scripts (zend.c:1316)
    ==19088==    by 0x67C180: php_execute_script (main.c:2479)
    ==19088==    by 0x79E4B1: pconn_do_request (pconnect-sapi.c:208)
    ==19088==    by 0x79E299: php_thread (main.c:63)

So at least on my Linux box there is an issue around the usage of
setlocale(). Gues this won't show on Windows as Windows locale handling
is different, but again a data point showing that most developers don't
spend attention on these things. (and well, handlin locales in a
threaded environment is tough ... as different scripts might use
different locales, similar to the CWD things ....)

johannes

--
Pierre

@pierrejoye

12 years ago by Pierre Joye — view source

unread

On Tue, Jan 29, 2013 at 1:01 PM, Johannes Schlüter

So at least on my Linux box there is an issue around the usage of
setlocale(). Gues this won't show on Windows as Windows locale handling
is different,

but again a data point showing that most developers don't
spend attention on these things. (and well, handlin locales in a
threaded environment is tough ... as different scripts might use
different locales, similar to the CWD things ....)

It is very well known that locale are not TS. Nothing to do with PHP itself.

12 years ago by Zeev Suraski — view source

unread

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Tuesday, January 29, 2013 1:49 PM
To: Johannes Schlüter
Cc: Zeev Suraski; PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

On Tue, Jan 29, 2013 at 12:45 PM, Johannes Schlüter
johannes@schlueters.de wrote:

Of course an opcode cache isn't shred-nothing either, and maybe
sharing opcodes within a process is faster than doing this in shared
memory.

It is inter process sharing and is very expensive, nothing to compare
with shared
memory within a single process, accross many threads.

What are you basing that assertion on? Shared memory should have
identical performance to regular process memory. It's mapped in very
similar way.

There are effectively no performance advantages to using multiple threads
over multiple processes, none whatsoever. It might help you a bit with
memory consumption - but in the case of PHP, FastCGI with it's n:m mapping
is much superior to it.

Using multiple threads only makes sense if you take functional advantage
of that, like shared objects and such - which we don't.

Miss the rest of my mail or? Current implementation is outdated and
slow.

That is true. Many modern compilers and environments provide better
support for thread local storages ....

Exactly, or more exactly CRTs (libc, crt and the likes)>

Can we stop arguing about the implementation - which I didn't even bring
up in my original mail - and focus on the original question - why would
you want thread safe PHP in the first place?
So far the only real use case that was brought up was for using pthreads
in a long-running app. I've yet to hear about a single real reason of why
someone would want to use it in web server context that is not based on
misconceptions or myths.

However let me state my point clear: Giving up on thread safety would
be a
mistake as big as safemode and other insane choices we made.

As big as giving up on safe mode or creating it in the first place? If
it's the former, I say let's do it right now :)

Zeev

12 years ago by Pierre Joye — view source

unread

It is inter process sharing and is very expensive, nothing to compare
with shared
memory within a single process, accross many threads.

What are you basing that assertion on? Shared memory should have
identical performance to regular process memory. It's mapped in very
similar way.

There are effectively no performance advantages to using multiple threads
over multiple processes, none whatsoever. It might help you a bit with
memory consumption - but in the case of PHP, FastCGI with it's n:m mapping
is much superior to it.

Using multiple threads only makes sense if you take functional advantage
of that, like shared objects and such - which we don't.

Testing maybe?

Miss the rest of my mail or? Current implementation is outdated and
slow.

That is true. Many modern compilers and environments provide better
support for thread local storages ....

Exactly, or more exactly CRTs (libc, crt and the likes)>

Can we stop arguing about the implementation

No, we can't. The main reason of the bugs and performance issue is due
to the current implementation and how extensions developers have to
deal with it. It is not separable from discussing what could be the
worst decision we ever made, droping thread safety.

which I didn't even bring
up in my original mail - and focus on the original question - why would
you want thread safe PHP in the first place?

I mentioned one which is alone a good enough reason. LTS brings zts in
line with NTS in term of performance while reducing massively the
shared/locks access related bugs for many areas.

So far the only real use case that was brought up was for using pthreads
in a long-running app. I've yet to hear about a single real reason of why
someone would want to use it in web server context that is not based on
misconceptions or myths.

Thanks, it is really nice to say something and being told that what
has been done is misconceptions or myths, especially if you gave up on
thread safety years ago within your company. That's all good, as long
as it only affects your company products and not PHP as a whole.

However let me state my point clear: Giving up on thread safety would
be a
mistake as big as safemode and other insane choices we made.

As big as giving up on safe mode or creating it in the first place? If
it's the former, I say let's do it right now :)

both :) But with a small difference, back then TSRM was not bad. But
it was never updated to what is available now on any modern OS.

Cheers,

Pierre

@pierrejoye

12 years ago by Zeev Suraski — view source

unread

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Tuesday, January 29, 2013 2:19 PM
To: Zeev Suraski
Cc: Johannes Schlüter; PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

It is inter process sharing and is very expensive, nothing to compare
with shared
memory within a single process, accross many threads.

What are you basing that assertion on? Shared memory should have
identical performance to regular process memory. It's mapped in very
similar way.

Testing maybe?

Retest then, you've got it wrong. Shared memory (or memory mapped files
as they'd be on Windows) are simply memory, it's available to the process
in exactly the same way and while the OS does slightly different
book-keeping for it in how it populates the pages - it's negligible.

Miss the rest of my mail or? Current implementation is outdated
and slow.

That is true. Many modern compilers and environments provide
better support for thread local storages ....

Exactly, or more exactly CRTs (libc, crt and the likes)>

Can we stop arguing about the implementation

No, we can't. The main reason of the bugs and performance issue is due
to the
current implementation and how extensions developers have to deal with
it. It is
not separable from discussing what could be the worst decision we ever
made,
droping thread safety.

Well, I'm not interested in discussing the implementation when the need
isn't clear to me.
For example, you could say that thread-safe PHP would be 2x faster. Or 2x
more memory efficient. Or will enable us to do XYZ which we otherwise
can't.
For now, an utopian ZTS mode would, at best, not give us anything.
Even if we put aside the performance/reliability issues - why would
someone want that mode over other modes?

So far the only real use case that was brought up was for using
pthreads in a long-running app. I've yet to hear about a single real
reason of why someone would want to use it in web server context that
is not based on misconceptions or myths.

Thanks, it is really nice to say something and being told that what has
been done
is misconceptions or myths, especially if you gave up on thread safety
years ago
within your company. That's all good, as long as it only affects your
company
products and not PHP as a whole.

Pierre, I'm sorry, but when you state something like '[shared memory] is
inter process sharing and is very expensive', there's no other way to
describe it other than a myth. Better theoretical performance is a
misconception. The best PHP performance/density available today is on
nginx, which couldn't be farther away from
multithreaded-PHP-in-process-arch.

Even after what you just said, you've yet to bring up a single reason for
why one would want to use ZTS inside a web server context. You seem to
love ZTS with a vengeance but you can't quite explain why. My 'company'
didn't decide anything regarding ZTS. It was me, probably the person with
the most experience with ZTS back in the day, and it was probably much
more difficult for me than anybody else to realize that the ZTS approach
was the wrong path to go to as it was my baby project.

BTW, for the love of [insert here], I'm not talking about killing thread
safety. It's useful for long running processes, and who knows, maybe in
the future we'll have threads inside PHP. But I am talking about
telling people that using ZTS in production is not recommended and that
they should be using FastCGI instead.

Zeev

12 years ago by damian@tylczynski.pl — view source

unread

As a developer I really enjoyed that discussion, it was very
informative for me. I'm always stuck with question which version of
PHP to use.

2013/1/29 Zeev Suraski zeev@zend.com:

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Tuesday, January 29, 2013 2:19 PM
To: Zeev Suraski
Cc: Johannes Schlüter; PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

It is inter process sharing and is very expensive, nothing to compare
with shared
memory within a single process, accross many threads.

What are you basing that assertion on? Shared memory should have
identical performance to regular process memory. It's mapped in very
similar way.

Testing maybe?

Retest then, you've got it wrong. Shared memory (or memory mapped files
as they'd be on Windows) are simply memory, it's available to the process
in exactly the same way and while the OS does slightly different
book-keeping for it in how it populates the pages - it's negligible.

Miss the rest of my mail or? Current implementation is outdated
and slow.

That is true. Many modern compilers and environments provide
better support for thread local storages ....

Exactly, or more exactly CRTs (libc, crt and the likes)>

Can we stop arguing about the implementation

No, we can't. The main reason of the bugs and performance issue is due
to the
current implementation and how extensions developers have to deal with
it. It is
not separable from discussing what could be the worst decision we ever
made,
droping thread safety.

Well, I'm not interested in discussing the implementation when the need
isn't clear to me.
For example, you could say that thread-safe PHP would be 2x faster. Or 2x
more memory efficient. Or will enable us to do XYZ which we otherwise
can't.
For now, an utopian ZTS mode would, at best, not give us anything.
Even if we put aside the performance/reliability issues - why would
someone want that mode over other modes?

So far the only real use case that was brought up was for using
pthreads in a long-running app. I've yet to hear about a single real
reason of why someone would want to use it in web server context that
is not based on misconceptions or myths.

Thanks, it is really nice to say something and being told that what has
been done
is misconceptions or myths, especially if you gave up on thread safety
years ago
within your company. That's all good, as long as it only affects your
company
products and not PHP as a whole.

Pierre, I'm sorry, but when you state something like '[shared memory] is
inter process sharing and is very expensive', there's no other way to
describe it other than a myth. Better theoretical performance is a
misconception. The best PHP performance/density available today is on
nginx, which couldn't be farther away from
multithreaded-PHP-in-process-arch.

Even after what you just said, you've yet to bring up a single reason for
why one would want to use ZTS inside a web server context. You seem to
love ZTS with a vengeance but you can't quite explain why. My 'company'
didn't decide anything regarding ZTS. It was me, probably the person with
the most experience with ZTS back in the day, and it was probably much
more difficult for me than anybody else to realize that the ZTS approach
was the wrong path to go to as it was my baby project.

BTW, for the love of [insert here], I'm not talking about killing thread
safety. It's useful for long running processes, and who knows, maybe in
the future we'll have threads inside PHP. But I am talking about
telling people that using ZTS in production is not recommended and that
they should be using FastCGI instead.

Zeev

--

--
Pozdrawiam,
Damian Tylczyński

12 years ago by Pierre Joye — view source

unread

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Tuesday, January 29, 2013 2:19 PM
To: Zeev Suraski
Cc: Johannes Schlüter; PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

It is inter process sharing and is very expensive, nothing to compare
with shared
memory within a single process, accross many threads.

What are you basing that assertion on? Shared memory should have
identical performance to regular process memory. It's mapped in very
similar way.

Testing maybe?

Retest then, you've got it wrong. Shared memory (or memory mapped files
as they'd be on Windows) are simply memory, it's available to the process
in exactly the same way and while the OS does slightly different
book-keeping for it in how it populates the pages - it's negligible.

It is not negligible, also the locking is different. Also, if I
remember correctly there is no rwlock support either, which is a huge
cons.

No, we can't. The main reason of the bugs and performance issue is due
to the
current implementation and how extensions developers have to deal with
it. It is
not separable from discussing what could be the worst decision we ever
made,
droping thread safety.

Well, I'm not interested in discussing the implementation when the need
isn't clear to me.

For example, you could say that thread-safe PHP would be 2x faster. Or 2x
more memory efficient. Or will enable us to do XYZ which we otherwise
can't.

There are quite a few things I can do on Windows only in thread safe
environments (f.e. ESENT, builtin DB, used by AD for example, damned
fast).

For now, an utopian ZTS mode would, at best, not give us anything.
Even if we put aside the performance/reliability issues - why would
someone want that mode over other modes?

I must be really unclear or something along this line. Alone on IIS,
there are dozen of features you can't do nor use using fastcgi. Apache
could also be used in a better way (windows here too, linux has a
module for both builds of apache).

btw, have you tried to compare IIS/FastCGI (I won't even try with
apache+fastcgi) on Windows vs nginx-fpm on linux? I would be very
surprised if your numbers show that they are equivalent.

Pierre, I'm sorry, but when you state something like '[shared memory] is
inter process sharing and is very expensive', there's no other way to
describe it other than a myth. Better theoretical performance is a
misconception. The best PHP performance/density available today is on
nginx, which couldn't be farther away from
multithreaded-PHP-in-process-arch.

Even after what you just said, you've yet to bring up a single reason for
why one would want to use ZTS inside a web server context. You seem to
love ZTS with a vengeance but you can't quite explain why. My 'company'
didn't decide anything regarding ZTS. It was me, probably the person with
the most experience with ZTS back in the day, and it was probably much
more difficult for me than anybody else to realize that the ZTS approach
was the wrong path to go to as it was my baby project.

It could have been the wrong path for some situation back then, but it
is definitively not wrong to improve thread safety.

I would rather have one single build instead, and enable the ts
related stuff when necessary. It will add some more functions in the
engine but it would be much more clear and easier than what we have
now.

BTW, for the love of [insert here], I'm not talking about killing thread
safety.

That's how it looks and that's where this discussion is going, tbh.

It's useful for long running processes, and who knows, maybe in
the future we'll have threads inside PHP. But I am talking about
telling people that using ZTS in production is not recommended and that
they should be using FastCGI instead.

As far as I remember, we already do that for a couple of web servers.
And in the long run, I will rather tell not to use FastCGI for
dedicated hosting and the likes. That being said, I also met many ISPs
which are not happy with the all-fastcgi, memory usageand performance
were the issues (have to read my backlog for other details :).

Cheers,

Pierre

@pierrejoye

12 years ago by Rasmus Lerdorf — view source

unread

As far as I remember, we already do that for a couple of web servers.
And in the long run, I will rather tell not to use FastCGI for
dedicated hosting and the likes. That being said, I also met many ISPs
which are not happy with the all-fastcgi, memory usageand performance
were the issues (have to read my backlog for other details :).

Those ISPs are probably stuck in old fastcgi-land and haven't figured
out FPM's ondemand pooling. If you idle out the ondemand children
somewhat quickly you can support a lot of vhosts without using much
memory since each one doesn't need to have a running process. Only hosts
actually being accessed will have running processes.

-Rasmus

12 years ago by Pierre Joye — view source

unread

Those ISPs are probably stuck in old fastcgi-land and haven't figured
out FPM's ondemand pooling. If you idle out the ondemand children
somewhat quickly you can support a lot of vhosts without using much
memory since each one doesn't need to have a running process. Only hosts
actually being accessed will have running processes.

Not talking about Linux here, but IIS have something similar, but
really not even close to how FPM performs or how it works (having
basically the pros of TS and NTS).

Cheers,

Pierre

@pierrejoye

12 years ago by Zeev Suraski — view source

unread

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Tuesday, January 29, 2013 3:37 PM
To: Rasmus Lerdorf
Cc: PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

On Tue, Jan 29, 2013 at 2:33 PM, Rasmus Lerdorf rasmus@lerdorf.com
wrote:

Those ISPs are probably stuck in old fastcgi-land and haven't figured
out FPM's ondemand pooling. If you idle out the ondemand children
somewhat quickly you can support a lot of vhosts without using much
memory since each one doesn't need to have a running process. Only
hosts actually being accessed will have running processes.

Not talking about Linux here, but IIS have something similar, but really
not even
close to how FPM performs or how it works (having basically the pros of
TS and
NTS).

On Windows with impersonation you're actually in a better situation than
you are in Linux. You could hold a small pool of processes and handle as
many different users as you'd like.

Zeev

12 years ago by Pierre Joye — view source

unread

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Tuesday, January 29, 2013 3:37 PM
To: Rasmus Lerdorf
Cc: PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

On Tue, Jan 29, 2013 at 2:33 PM, Rasmus Lerdorf rasmus@lerdorf.com
wrote:

Those ISPs are probably stuck in old fastcgi-land and haven't figured
out FPM's ondemand pooling. If you idle out the ondemand children
somewhat quickly you can support a lot of vhosts without using much
memory since each one doesn't need to have a running process. Only
hosts actually being accessed will have running processes.

Not talking about Linux here, but IIS have something similar, but really
not even
close to how FPM performs or how it works (having basically the pros of
TS and
NTS).

On Windows with impersonation you're actually in a better situation than
you are in Linux. You could hold a small pool of processes and handle as
many different users as you'd like.

Works fine with ZTS too btw, IIS takes care about managing the
respective application pools.

But yes, impersonation is a huge on Windows/IIS, get ride of
openbasedir and all that in one go (and again, not NTS specific).

Cheers,

Pierre

@pierrejoye

12 years ago by keisial@gmail.com — view source

unread

On Windows with impersonation you're actually in a better situation than
you are in Linux. You could hold a small pool of processes and handle as
many different users as you'd like.
Works fine with ZTS too btw, IIS takes care about managing the
respective application pools.

But yes, impersonation is a huge on Windows/IIS, get ride of
openbasedir and all that in one go (and again, not NTS specific).

Wouldn't that be equivalent security-wise to running the server on linux
as root for
calling setuid() to each user?

12 years ago by Pierre Joye — view source

unread

On Windows with impersonation you're actually in a better situation than
you are in Linux. You could hold a small pool of processes and handle as
many different users as you'd like.
Works fine with ZTS too btw, IIS takes care about managing the
respective application pools.

But yes, impersonation is a huge on Windows/IIS, get ride of
openbasedir and all that in one go (and again, not NTS specific).

Wouldn't that be equivalent security-wise to running the server on linux
as root for
calling setuid() to each user?

No, not the same, while the results look the same.

Cheers,

Pierre

@pierrejoye

12 years ago by Zeev Suraski — view source

unread

On Windows with impersonation you're actually in a better situation than
you are in Linux. You could hold a small pool of processes and handle as
many different users as you'd like.
Works fine with ZTS too btw, IIS takes care about managing the
respective application pools.

But yes, impersonation is a huge on Windows/IIS, get ride of
openbasedir and all that in one go (and again, not NTS specific).

Wouldn't that be equivalent security-wise to running the server on linux
as root for
calling setuid() to each user?

No, because the process is reusable and can be switched to other creds.

Zeev

12 years ago by Zeev Suraski — view source

unread

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Tuesday, January 29, 2013 3:19 PM
To: Zeev Suraski
Cc: Johannes Schlüter; PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Tuesday, January 29, 2013 2:19 PM
To: Zeev Suraski
Cc: Johannes Schlüter; PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

It is inter process sharing and is very expensive, nothing to
compare
with shared
memory within a single process, accross many threads.

What are you basing that assertion on? Shared memory should have
identical performance to regular process memory. It's mapped in
very similar way.

Testing maybe?

Retest then, you've got it wrong. Shared memory (or memory mapped
files as they'd be on Windows) are simply memory, it's available to
the process in exactly the same way and while the OS does slightly
different book-keeping for it in how it populates the pages - it's
negligible.

It is not negligible, also the locking is different. Also, if I remember
correctly
there is no rwlock support either, which is a huge cons.

It is too negligible. Once the pages get allocated and mapped properly -
it's exactly the same thing as accessing memory.
Locking in between processes is indeed slower than locking inside
processes, but at the risk of sounding like a broken record, in something
like an opcode cache (few writes, mostly reads), it's negligible.

For example, you could say that thread-safe PHP would be 2x faster.
Or 2x more memory efficient. Or will enable us to do XYZ which we
otherwise can't.

There are quite a few things I can do on Windows only in thread safe
environments (f.e. ESENT, builtin DB, used by AD for example, damned
fast).

OK, now we're getting somewhere. So ESENT is one legit use case that
would benefit from a multithreaded PHP.

For now, an utopian ZTS mode would, at best, not give us anything.
Even if we put aside the performance/reliability issues - why would
someone want that mode over other modes?

I must be really unclear or something along this line. Alone on IIS,
there are
dozen of features you can't do nor use using fastcgi. Apache could also
be used
in a better way (windows here too, linux has a module for both builds of
apache).

Which ones?

btw, have you tried to compare IIS/FastCGI (I won't even try with
apache+fastcgi) on Windows vs nginx-fpm on linux? I would be very
surprised if your numbers show that they are equivalent.

Not recently. But I'd be surprised if it wasn't equivalent at least as
far PHP is concerned. Nginx is marginally faster in pure PHP performance
than Apache; Where it really shines is that it can do it with fewer
processes -> much less memory consumption. That attribute stems from
breaking the 1:1 mapping and turning it into an n:m mapping.

It could have been the wrong path for some situation back then, but it
is
definitively not wrong to improve thread safety.

I didn't suggest that. My original question was 'Why do you use
thread-safe ZTS'. It was explicitly not an RFC to stop developing or
remove ZTS mode.

I do think that we'll do our users a much better service if we tell them
that unless they have a very good reason to use a web server plugin on a
multithreaded server, they should use FastCGI instead.

BTW, for the love of [insert here], I'm not talking about killing
thread safety.

That's how it looks and that's where this discussion is going, tbh.

Well, it wasn't my intention, at all. I was genuinely surprised there was
a high level of interest about ZTS when we discussed Optimizer+, which
made me wonder why that is the case.

It's useful for long running processes, and who knows, maybe in the
future we'll have threads inside PHP. But I am talking about
telling people that using ZTS in production is not recommended and
that they should be using FastCGI instead.

As far as I remember, we already do that for a couple of web servers.
And in the long run, I will rather tell not to use FastCGI for dedicated
hosting and
the likes. That being said, I also met many ISPs which are not happy
with the all-
fastcgi, memory usageand performance were the issues (have to read my
backlog for other details :).

I think that properly configured FastCGI will be superior to a
server-plugin in mostly every way. The one downside is that you lose the
server-specific features, but that's typically not a huge price to pay.

Zeev

12 years ago by johannes@schlueters.de — view source

unread

As far as I remember, we already do that for a couple of web servers.
And in the long run, I will rather tell not to use FastCGI for
dedicated hosting and the likes. That being said, I also met many ISPs
which are not happy with the all-fastcgi, memory usageand performance
were the issues (have to read my backlog for other details :).

I myself wouldn't use current-PHP for shared hosting in ZTS mode. Stack
overflows (functrion nesting, pcre, ...) will kill the whole process so
one bad guy can hurt other parallel requests for other vhosts quite
easily. With a multi-process-setup such bad user can hurt performance
but not completely kill other requests.

That aside: I agree that a system written for threaded environments can
benefit from threads vs. processes (not only memory access, but also
context switches etc.) at least on some operating systems. But PHP
(currently) is not that kind of a system. (I went through this quite a
bit with Solaris devs in the past, which led to some improvements which
were available via http://hg.genunix.org/php-experiment.hg/ which seems
to be down ...)

My primary perspective is that I claim that barely anybody is doing more
than compile-testing in ZTS mode. TSRM macros are also often one of the
biggest causes of confusion when teaching extension writing/patching of
PHP, even though for 95% of the cases it can be seen as a pure blackbox.

I for one would vote for a simplification of the codebase over a small
performance win.

johannes

12 years ago by Stas Malyshev — view source

unread

Hi!

Of course an opcode cache isn't shred-nothing either, and maybe sharing
opcodes within a process is faster than doing this in shared memory.

I don't think so. IIRC main time is spent of two things: building
runtime structures from storage formats (because we mess with our
structures in runtime, we can not run engine on shm copies of
everything) and ensuring writing new scripts does not mess with existing
ones. These will have to be done regardless of how shared storage is
organized.

--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

12 years ago by Martin Keckeis — view source

unread

(*) Apache actually does have a good FastCGI implementation available in
Zend Server for Windows (including the free CE version). Using it is
faster and more reliable than using mod_php on Windows.

Absolute right. Zend Server works great on Windows with FastCGI.
Using it since 3 years and got never problems.

12 years ago by Stas Malyshev — view source

unread

Hi!

There are situations where FPM/FCGI are not appropriate, or the server
used does not support NTS (Apache windows for example, when fcgi is
not an option).

Why Apache can't use FCGI? There's no proper driver os something in
Apache architecture prevents it from using FCGI?

No. My idea is to replace it with LTS (thread local storage) which
drastically reduce the thread safety issues due to its exact nature,

I'm not sure it will. Our threading issues mainly related to libraries
using global context and either not protecting it at all (e.g. ICU uses

or at least did couple of years ago - some global variables as caches
which are locale-dependent, and once these cache vars are set, it's
almost impossible to change and it leads to context leaking between
threads) or protecting it improperly so that we get race conditions or
crashes. Moving our data to LTS would make our part faster but won't
make these libraries work better.

The other main reason from my side to keep ZTS is Windows. Windows
cannot perform well using process based SAPI. It won't match linux as

I don't see why. With proper prefork model, cost of process creation
does not matter too much, and I've seen decent performance results with
FCGI. Actually, IIRC, performance hit from ZTS is more than performance
hit from multiprocess model. There are other things that make windows
performance lag (Unicode APIs and filesystem being one of the factors,
for example) but I do not think process model is a major factor here.

promising, given that it is still only a proof of concepts. Having an
ISAPI also allows a tighter integration to IIS8 (pipelines, auths,
scripting, etc.), which is not possible to do now using fastcgi (no
direct access but using RMI or the IIS shell, but horrible, in all
possible ways).

This is true, however I am not convinced it should be in PHP as opposed
to separate user modules... If you'd want to run PHP code in IIS
pipeline, I imagine performance-wise it would be expensive - setting up
PHP request is not a cheap thing, compared to what auth stage usually does.

--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

12 years ago by Pierre Joye — view source

unread

Hi!

There are situations where FPM/FCGI are not appropriate, or the server
used does not support NTS (Apache windows for example, when fcgi is
not an option).

Why Apache can't use FCGI? There's no proper driver os something in
Apache architecture prevents it from using FCGI?

No, more by choice (out of the developer control).

No. My idea is to replace it with LTS (thread local storage) which
drastically reduce the thread safety issues due to its exact nature,

I'm not sure it will. Our threading issues mainly related to libraries
using global context and either not protecting it at all (e.g. ICU uses

or at least did couple of years ago - some global variables as caches
which are locale-dependent, and once these cache vars are set, it's
almost impossible to change and it leads to context leaking between
threads) or protecting it improperly so that we get race conditions or
crashes. Moving our data to LTS would make our part faster but won't
make these libraries work better.

I did not check latest ICU code base but we never had any issues in
intl in ZTS. However you are right, since 5.3.0 most TS issues were in
libs, on windows I fixed a couple by patching the libraries when
possible (the oss ones).

The other main reason from my side to keep ZTS is Windows. Windows
cannot perform well using process based SAPI. It won't match linux as

I don't see why. With proper prefork model, cost of process creation
does not matter too much, and I've seen decent performance results with
FCGI.

I never said fcgi performs badly. What I say is that Windows design is
not process based, none of the hiperformence tools available on this
platform is based on threads (AD, IIS, asp.net, etc.).

Actually, IIRC, performance hit from ZTS is more than performance
hit from multiprocess model. There are other things that make windows
performance lag (Unicode APIs and filesystem being one of the factors,
for example) but I do not think process model is a major factor here.

Filesystem is getting better, it is already better in 5.3.0+ and will
be a bit better in 5.5.0 (if I got the time to do it :/). We do not
use the unicode API yet (for IO or FS), so no real impact yet but a
few functions that still have to use it in some libs or in core.

promising, given that it is still only a proof of concepts. Having an
ISAPI also allows a tighter integration to IIS8 (pipelines, auths,
scripting, etc.), which is not possible to do now using fastcgi (no
direct access but using RMI or the IIS shell, but horrible, in all
possible ways).

This is true, however I am not convinced it should be in PHP as opposed
to separate user modules... If you'd want to run PHP code in IIS
pipeline, I imagine performance-wise it would be expensive

To bring zts in line with NTS in term of performance is possible, look
at Arnaud's 1st draft for a TLS support in php:

https://wiki.php.net/rfc/tls

setting up
PHP request is not a cheap thing, compared to what auth stage usually does.

The IIS pipeline architecture allows much more than only auth, much more.

Let me repeat it a last time, I do think that giving up on ZTS
altogether is a very bad idea, to say it in a nice way. That's the
kind of choice that will shoot at us back very soon. I agree that it
adds some work, but 99% of us do not care but there are solutions to
ease the developer work and reduce the wtf factors. TLS is one of
them.

Cheers,

Pierre

@pierrejoye

12 years ago by Stas Malyshev — view source

unread

Hi!

I did not check latest ICU code base but we never had any issues in
intl in ZTS. However you are right, since 5.3.0 most TS issues were in

One of them has to do with number formatting, so if you have a number of
apps that use different locale settings on the same server, which have
different number separators, you may encounter it. Not a frequent use
case, though. See: http://bugs.icu-project.org/trac/ticket/6711
Not unique to TS, but there might be ones that are, if they use globals
in this manner. Just showing that modern libraries are still having this
kind of issues, and it's very hard to get maintainers to fix them.

I never said fcgi performs badly. What I say is that Windows design is
not process based, none of the hiperformence tools available on this
platform is based on threads (AD, IIS, asp.net, etc.).

ITYM based or processes, or s/none/all/. But so what? For IIS case
threads are fine, but for our case - shared-nothing - they don't help us
much. No idea what asp.net does but they probably designed it to be
threaded from scratch, so it probably has very different architecture
than PHP.

Filesystem is getting better, it is already better in 5.3.0+ and will
be a bit better in 5.5.0 (if I got the time to do it :/). We do not
use the unicode API yet (for IO or FS), so no real impact yet but a
few functions that still have to use it in some libs or in core.

AFAIR all APIs that go into filesystem, C library, etc. ultimately end
up with Unicode - i.e.

To bring zts in line with NTS in term of performance is possible, look
at Arnaud's 1st draft for a TLS support in php:

https://wiki.php.net/rfc/tls

TLS is nice, I have nothing against TLS, and if somebody wants to get
behind the effort to get ZTS to use TLS, great. But I was talking about
cost of setting up envt for PHP script, which is not going to change.
You need to initialize all the globals, create all the structures, etc.
etc. - it's all not free.

The IIS pipeline architecture allows much more than only auth, much more.

I know, it's just an example. I'm just saying PHP request init is
expensive, and I'm not even sure we can do proper init on early stages
of IIS request... So PHP may not be the best thing to use inside IIS
pipeline in general case.

ease the developer work and reduce the wtf factors. TLS is one of
them.

Fine, so who's going to do this? This RFC is marked with 2008, now it's
2013 and it didn't happen. Why?

Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

12 years ago by Pierre Joye — view source

unread

hi,

Hi!

I did not check latest ICU code base but we never had any issues in
intl in ZTS. However you are right, since 5.3.0 most TS issues were in

One of them has to do with number formatting, so if you have a number of
apps that use different locale settings on the same server, which have
different number separators, you may encounter it. Not a frequent use
case, though. See: http://bugs.icu-project.org/trac/ticket/6711

" I can confirm the ICU version 3.8 worked well"

Every maintained software (with active users ;) has bugs, which get fixed.

Not unique to TS, but there might be ones that are, if they use globals
in this manner. Just showing that modern libraries are still having this
kind of issues, and it's very hard to get maintainers to fix them.

Yes, but they are very open to get these issues fixed.

I never said fcgi performs badly. What I say is that Windows design is
not process based, none of the hiperformence tools available on this
platform is based on threads (AD, IIS, asp.net, etc.).

ITYM based or processes, or s/none/all/. But so what? For IIS case
threads are fine, but for our case - shared-nothing - they don't help us
much. No idea what asp.net does but they probably designed it to be
threaded from scratch, so it probably has very different architecture
than PHP.

We do only do share nothing for the engine (related to script
execution). There are other parts where we do share (realpath for one,
pcre, etc.). Also I think that's something we should investigate
instead of taking (yet again) the easiest or trendy way.

Filesystem is getting better, it is already better in 5.3.0+ and will
be a bit better in 5.5.0 (if I got the time to do it :/). We do not
use the unicode API yet (for IO or FS), so no real impact yet but a
few functions that still have to use it in some libs or in core.

AFAIR all APIs that go into filesystem, C library, etc. ultimately end
up with Unicode - i.e.

It is not unicode per se and there is not necessary a conversion
happening, at least with NTFS filesystems. But that's another topic :)

To bring zts in line with NTS in term of performance is possible, look
at Arnaud's 1st draft for a TLS support in php:

https://wiki.php.net/rfc/tls

TLS is nice, I have nothing against TLS, and if somebody wants to get
behind the effort to get ZTS to use TLS, great. But I was talking about
cost of setting up envt for PHP script, which is not going to change.
You need to initialize all the globals, create all the structures, etc.
etc. - it's all not free.

But you ignore that TLS brings ZTS in line with NTS from a performance
point of view. And it is only one of the possible gain.

The IIS pipeline architecture allows much more than only auth, much more.

I know, it's just an example. I'm just saying PHP request init is
expensive, and I'm not even sure we can do proper init on early stages
of IIS request... So PHP may not be the best thing to use inside IIS
pipeline in general case.

I'd to disagree. And giving up only because it may require some more
work sounds really bad to me.

ease the developer work and reduce the wtf factors. TLS is one of
them.

Fine, so who's going to do this? This RFC is marked with 2008, now it's
2013 and it didn't happen. Why?

Nice to totally ignore the results you were asking for... And as of
why, these changes could not be done in 5.x. We discussed the idea
with the engine guys back then, and even last year again.

Anyway, I have given enough arguments and this discussion is being in
a loop now.

Cheers,

Pierre

@pierrejoye

12 years ago by johannes@schlueters.de — view source

unread

I didn’t want to hijack the Optimizer+ thread so I’m creating a new one,
based on the apparent level of interest in ZTS. This isn’t an RFC to
remove ZTS by any stretch, but I am a bit confused about why people are
still using ZTS.

I doubt many people use it in production, especially under "high" load.
About a year ago I've spent quite some time fixing threading issues in
different parts of PHP (object instantiation in the engine, mysqli, ...)
there were a few bug reports but considering how simple it was to
exploit some of the bugs (<?php new stdclass; ?>, or any other internal
class, was all that was needed for one of the bugs) there were way too
few reports.

johannes

12 years ago by johannes@schlueters.de — view source

unread

exploit some of the bugs (<?php new stdclass; ?>, or any other internal
class, was all that was needed for one of the bugs) there were way too
few reports.

Ah, that specific bug was 5.4-only, back then 5.4 was quite new. But
there were other bugs, too.

johannes

12 years ago by Derick Rethans — view source

unread

Which brings me to the subject of this mail – why are you using ZTS
PHP instead of single threaded PHP? The reasons not to use it are few
but fairly major – it’s significantly slower than the non-ZTS PHP, and
it’s significantly less robust in the sense that a single bug
somewhere can bring down an entire server (or at least a bunch of many
different threads). What are your reasons to choose it over FastCGI?

I have no reasons to ever use it. I've thought about dropping the
support of ZTS in Xdebug for a while now. I might just do that when 5.5
comes along...

cheers,
Derick

12 years ago by Martin Nicholls — view source

unread

I’m creating a new one,
based on the apparent level of interest in ZTS. This isn’t an RFC to
remove ZTS by any stretch, but I am a bit confused about why people are
still using ZTS.

Personally because runkit sandbox requires it, amongst other extensions.
Haven't had a chance to play with it yet but the new pthreads extension
requires it which I'm probably going to be using extensively in future too.

Why does it need to be enabled for most people? It probably doesn't, but
there are many third part extensions that can't function without it with
no obvious workarounds that would cease to exist if it hypothetically
disappeared. This for me, and I suppose many others would be a fairly
major problem.

12 years ago by Pierre Joye — view source

unread

Laziness and design mistake. Everything on windows (AD,IIS, asp.net, etc)
uses thread.

And no, nuts is not faster. I am not talking about PHP zts, but in
general.

Of course everything that is Windows native uses threads, but I don't
believe any language that is single-threaded on other platforms - has a
thread safe version for Windows.

Python, for example, is thread safe by default. Extensions developers
can rely on its APIs to write TS code. Our core is designed in a way
that it is painful to have to deal with two code bases. That's what
causes troubles, not really something else.

I disagree that laziness and design mistakes are what makes ZTS slower.
It's slower by design, since instead of relying on operating system
isolation - you have to emulate it yourself in userland code.

It was and still is a lazy and design mistake to have focused on
FastCGI to support PHP on IIS more easily, while everything else in
this stack uses what the whole OS stack does.

Contrary to popular belief FastCGI actually works very very well on Windows,
and has substantial advantages over using a server module - even on Linux
(it's no coincidence that there are hoards of users migrating from Apache to
nginx, where a server module isn't even an option - and they're seeing
better performance, not worse).

I never said that FastCGI does not work well. But that doing so will
never allow us to reach the performance we could get using a more
suited implementation.

About nginx vs Apache, I'm also quite sure that we can get an Apache
SAPI module performing as good as nging with FPM. It is only almost
impossible with the current implementation of PHP, we both know that.
We have seen examples in the past already where well optimized php and
apache were as fast as lighttpd (back then) with a correct setup.

Again, giving up on thread safety is a major mistake.

Cheers,

Pierre

@pierrejoye

12 years ago by Stas Malyshev — view source

unread

Hi!

Python, for example, is thread safe by default. Extensions developers

Doesn't Python have global engine lock?

It was and still is a lazy and design mistake to have focused on
FastCGI to support PHP on IIS more easily, while everything else in
this stack uses what the whole OS stack does.

That is your opinion, which I am afraid many people do not share. I, for
example, think that since PHP follows shared-nothing model, it is much
more efficient to allow the OS to do the separation than try to simulate
what the OS is doing in userspace. The latter would always be less
efficient. While thread-locals may allow some workarounds with regard to
internal engine data structures, libraries still remain the problem
which is hard to deal with unless you let the OS do it.

I never said that FastCGI does not work well. But that doing so will
never allow us to reach the performance we could get using a more
suited implementation.

I am still confused as for what exactly is supposed to make threaded
implementation much faster than FCGI one. Generally, FCGI has one factor
that slows it down a bit - it has one additional data passing stage -
but beyond that, we'd have to do the same and probably more if we'd have
to lock resources that aren't thread-local out of the box (example:
current directory, locales, etc.).

Again, giving up on thread safety is a major mistake.

I'd like to hear a good data-backed explanation why. So far in practice
NTS setups proven to be more robust without sacrificing performance.

Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

12 years ago by Pierre Joye — view source

unread

Hi!

Python, for example, is thread safe by default. Extensions developers

Doesn't Python have global engine lock?

Right, but they do not give up thread safety. See "Thread State and
the Global Interpreter Lock" in:

http://docs.python.org/2/c-api/init.html

The TS model in php should be redesigned in the next major version,
instead of simply giving it up.

Cheers,

Pierre

@pierrejoye

12 years ago by Florian Anderiasch — view source

unread

On Wed, Jan 30, 2013 at 2:15 AM, Stas Malyshev
smalyshev@sugarcrm.com wrote:

Hi!

Python, for example, is thread safe by default. Extensions developers

Doesn't Python have global engine lock?

Right, but they do not give up thread safety. See "Thread State and
the Global Interpreter Lock" in:

http://docs.python.org/2/c-api/init.html

The TS model in php should be redesigned in the next major version,
instead of simply giving it up.

You make it sound as if the GIL was a good thing...

And yes, it might have been introduced for that exact reason, but I
haven't read anything pro-GIL in the last few years, according to most
Python developers (mostly those who want to scale) it's basically their
number one pain point.

Doesn't sound like a fair deal at all. Plus, mod_python is dead,
everyone's using FastCGI/UWSGI anyway, so I'm not sure what you're
hinting at... (Or am I missing something?)

NB: Deliberately staying away from any pro/contra PHP-ZTS arguments,
this was just a quick blurb about Python.

Greetings,
Florian

12 years ago by Stas Malyshev — view source

unread

Hi!

Right, but they do not give up thread safety. See "Thread State and
the Global Interpreter Lock" in:

http://docs.python.org/2/c-api/init.html

That's whole different concept of thread safety. It's basically saying
"you can do anything you want outside of Python engine but only one
thread can do Python at the same time". If we created GIL in PHP, most
of our threading problems probably would go away, but it's not likely it
would be of any use for IIS or any webserver.

The TS model in php should be redesigned in the next major version,
instead of simply giving it up.

Again, I'd not mind seeing this redesign, but do we have somebody who's
actually going to do that?

Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

12 years ago by Lester Caine — view source

unread

Stas Malyshev wrote:

The TS model in php should be redesigned in the next major version,

instead of simply giving it up.
Again, I'd not mind seeing this redesign, but do we have somebody who's
actually going to do that?

Ignoring the problem of 'someone to do it', in this age of multi-core systems,
the idea of sending requests to a database server while building the page is an
area where threading could provide a speed gain in returning a page? Something
else to go on the wish list for PHP6?

--
Lester Caine - G8HFL

Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

12 years ago by Pierre Joye — view source

unread

Stas Malyshev wrote:

The TS model in php should be redesigned in the next major version,

instead of simply giving it up.

Again, I'd not mind seeing this redesign, but do we have somebody who's
actually going to do that?

Ignoring the problem of 'someone to do it', in this age of multi-core
systems, the idea of sending requests to a database server while building
the page is an area where threading could provide a speed gain in returning
a page?

Not really, async queries are way better and easier to deal with. Even
if some OS implementation uses thread under the hood, it is totally
hidden from an API point of view.

--
Pierre

@pierrejoye

12 years ago by Zeev Suraski — view source

unread

-----Original Message-----
From: Pierre Joye [mailto:pierre.php@gmail.com]
Sent: Wednesday, January 30, 2013 8:10 AM
To: Stas Malyshev
Cc: Zeev Suraski; PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

On Wed, Jan 30, 2013 at 2:15 AM, Stas Malyshev smalyshev@sugarcrm.com
wrote:

Hi!

Python, for example, is thread safe by default. Extensions developers

Doesn't Python have global engine lock?

Right, but they do not give up thread safety. See "Thread State and the
Global
Interpreter Lock" in:

http://docs.python.org/2/c-api/init.html

Pierre,

As Stas pointed out a global lock has very little to do with the topic of
discussion here. What PHP is trying to do is much more ambitious, so
ambitious that at least I concluded it's not an achievable goal. This has
nothing to do with laziness, much like deprecating safe_mode wasn't about
laziness - it was about facing reality. The Python approach is
exceptionally lazy (in software development that's often a Good Thing),
and can be implemented very quickly if we wanted to. It wouldn't help you
one bit with your use cases, though.

The TS model in php should be redesigned in the next major version,
instead of
simply giving it up.

I don't mind that we redesign TS for the next version, with two key
issues:

I'm not sure that relying on TLS (beyond what we already do on TSRM
today) is feasible. That was the naďve approach that I first tried back
in 1999 and ended up ditching since it wasn't suitable. The two main ones
are: [a] limited TLS space (we have lots of globals) [b] Inability to
access globals from another image (e.g.., you won't be able to access
sapi_globals which belong to php5.dll from mysqli.dll). Point [b] is
mentioned in the RFC that you referred to. Point [a] might have been
solved over the course of the last decade. TSRM wasn't designed for fun,
it was there to solve real problems that TLS did not solve.
I think it's not very good use of our time. Based on what I heard in
this thread, I would give up right here and now on the goal of having PHP
run in-process in multithreaded web server. The pros pale in comparison
to the cons. I would not remove ZTS - this was never the point of this
thread - but I would recalibrate people's expectations about it. It
sounds as if it could be useful for advanced 'I know what I'm doing' use
cases like long-running threaded processes, and may be useful for the
future if we ever want to support threads within a single request - but
that's about it.

Based on the feedback on this thread, I think we should instruct users
that using thread-safe versions of PHP is slower, and that actually
putting thread-safe PHP inside a multithreaded server is not a very
reliable solution. We should recommend FastCGI/fpm instead. We should do
that tomorrow morning (figuratively speaking).

Zeev

12 years ago by Bas van Beek — view source

unread

Hi Guys,

As a heavy user of ZTS in multi threaded C/C++ applications, here are my $0.02.

Removing ZTS would be a bad idea for all those custom multi-threaded applications out there that allow some form of internal/embedded PHP scripting. These applications are not web-servers but do make use of threads for reasons of their own. Building a FastCGI set-up would be out of the question if these apps are deployed customer / client side and need concurrent PHP script runs.

Based on the feedback on this thread, I think we should instruct users
that using thread-safe versions of PHP is slower, and that actually
putting thread-safe PHP inside a multithreaded server is not a very
reliable solution. We should recommend FastCGI/fpm instead. We should do
that tomorrow morning (figuratively speaking).

Zeev

--

This makes a lot of sense, I see where FastCGI/fpm is the best solution for the majority of users of PHP which deploy large scale web applications. The tone in this conversation looked like ZTS had no place in the PHP future. I think it does, but maybe not for the regular web server / web application use case. For custom applications with embedded PHP, I see it making more and more sense as a lot of native applications need to interface with web services and often SDK's for these services are either not available or tedious to use for native application development. Being able to use the PHP SDK's and quickly build conduits between custom apps and services using PHP is very beneficial. Developers of these kind of apps are well aware of the potential pitfalls in using threads and potential non TS libraries as long as documentation is clear about it.

My biggest worry is when we signal that ZTS is a relic or not important enough to support in extensions we end up with developers building extensions not trying to be ZTS compatible even if they could be made thread safe easily. For the majority of extensions it's a simple question of adding the TSRM macros. If the underlying library doesn't support it or it would impact performance too much, I would at least want to know that the extension is not thread safe so I can leave it aside and come up with another solution.

Bas van Beek

12 years ago by Zeev Suraski — view source

unread

-----Original Message-----
From: Bas van Beek [mailto:bas@tobin.nl]
Sent: Wednesday, January 30, 2013 2:29 PM
To: Zeev Suraski
Cc: Pierre Joye; Stas Malyshev; PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

Hi Guys,

As a heavy user of ZTS in multi threaded C/C++ applications, here are my
$0.02.

Removing ZTS would be a bad idea for all those custom multi-threaded
applications out there that allow some form of internal/embedded PHP
scripting. These applications are not web-servers but do make use of
threads for
reasons of their own. Building a FastCGI set-up would be out of the
question if
these apps are deployed customer / client side and need concurrent PHP
script
runs.

Based on the feedback on this thread, I think we should instruct users
that using thread-safe versions of PHP is slower, and that actually
putting thread-safe PHP inside a multithreaded server is not a very
reliable solution. We should recommend FastCGI/fpm instead. We
should do that tomorrow morning (figuratively speaking).

In case I wasn't sufficiently clear, I'm talking about putting PHP inside
a multithreaded web server, not being a good idea. The use case you
specify is exactly the use case for keeping ZTS support in the code...

Zeev

12 years ago by Bas van Beek — view source

unread

Op 30 jan. 2013, om 13:42 heeft Zeev Suraski het volgende geschreven:

-----Original Message-----
From: Bas van Beek [mailto:bas@tobin.nl]
Sent: Wednesday, January 30, 2013 2:29 PM
To: Zeev Suraski
Cc: Pierre Joye; Stas Malyshev; PHP internals
Subject: Re: [PHP-DEV] ZTS - why are you using it?

Hi Guys,

As a heavy user of ZTS in multi threaded C/C++ applications, here are my
$0.02.

Removing ZTS would be a bad idea for all those custom multi-threaded
applications out there that allow some form of internal/embedded PHP
scripting. These applications are not web-servers but do make use of
threads for
reasons of their own. Building a FastCGI set-up would be out of the
question if
these apps are deployed customer / client side and need concurrent PHP
script
runs.

Based on the feedback on this thread, I think we should instruct users
that using thread-safe versions of PHP is slower, and that actually
putting thread-safe PHP inside a multithreaded server is not a very
reliable solution. We should recommend FastCGI/fpm instead. We
should do that tomorrow morning (figuratively speaking).

In case I wasn't sufficiently clear, I'm talking about putting PHP inside
a multithreaded web server, not being a good idea. The use case you
specify is exactly the use case for keeping ZTS support in the code...

Zeev

--

Yes I did get that and my response to your post was not meant as an argument against those points but more as an addition. I just wanted to re-enforce that ZTS has it's purpose and we should be careful about which signals we give to the PHP user-land and extension developer community as most of the previous posts to this thread seemed to discard usefulness of ZTS all together as if the only important use case is running PHP code on a web server. PHP is so much more than that and I do not want to see PHP developers exclude or forget about these other use cases because prominent people in the PHP community make blunt statements as the above topic implies.

Bas van Beek

12 years ago by Pierre Joye — view source

unread

In case I wasn't sufficiently clear, I'm talking about putting PHP inside
a multithreaded web server, not being a good idea.

It makes no sense where FPM is supported, or little sense.

The use case you
specify is exactly the use case for keeping ZTS support in the code...

This use case fits in both new SAPIs I was talking about earlier, as
the way they work will be pretty much the same.

Cheers,

Pierre

@pierrejoye

ZTS - why are you using it?

-- Appreciate your time.

Cheers.

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Cheers,

Fine, so who's going to do this? This RFC is marked with 2008, now it's 2013 and it didn't happen. Why?

Cheers,

Cheers,

I'd like to hear a good data-backed explanation why. So far in practice NTS setups proven to be more robust without sacrificing performance.

Cheers,

Again, I'd not mind seeing this redesign, but do we have somebody who's actually going to do that?

-- Lester Caine - G8HFL

Cheers,

--
Appreciate your time.

Fine, so who's going to do this? This RFC is marked with 2008, now it's
2013 and it didn't happen. Why?

I'd like to hear a good data-backed explanation why. So far in practice
NTS setups proven to be more robust without sacrificing performance.

Again, I'd not mind seeing this redesign, but do we have somebody who's
actually going to do that?

--
Lester Caine - G8HFL