[VOTE] Fibers

5. Not compatible with Swoole

In my opinion, Swoole is an important part of the PHP ecosystem. But now, Fiber cannot work with Swoole. And based on the above reasons, Swoole will not consider compatible fiber.

We would expect some ZendAPI rather than Fiber extensions to provide support for coroutine switching.

Hi Twosee,

Are you saying that by adding this Fiber code to core it will prevent Swoole from functioning? If so, that is concerning. Or are you simply saying that Swoole doesn't like this implementation and will not use it with their own code?

One thing to consider when comparing the Fiber implementation is that the vast majority of PHP applications are still run behind a web server in short-lived requests. Unlike Swoole, Swow, and Parallel, it isn't limited to ZTS or CLI. It might not be the ideal solution, but IMO it is a step in the right direction for PHP to allow for better async support.

And I think you are right that "Fiber is just an enhancement of Generator'', but so what? The Generator addition in 5.5 opened up PHP to a hacky way of doing async, but it worked everywhere. Fiber allows that to be much more convenient in switching contexts deep inside a call stack. I don't think it needs to be the end-all-be-all coroutine solution to rival Goroutines, and I'm pretty sure it's not trying to be.

Thanks,
Peter

Hello,

This Fiber implementation does not conflict with Swoole. Both could coexist. However, Swoole's implementation does provide some other features that this does not, so I believe Twosee was simply saying that Swoole cannot directly use this fiber implementation. My focus was to provide a minimalistic implementation in core so user space code can make other API decisions as they saw fit.

Regarding some of the other concerns raised:

• Switch notification: The observer notification is triggered before switching to a fiber and after suspending from a fiber. The timing here was an open question I've discussed with others and we planned to examine it later. Moving the second notification before suspending the second fiber was already being considered. As this was an internal API I felt it was not needed directly in the RFC.
• Fiber is final, as is Generator. I believe it is better to compose classes rather than extend them. Of course it is always possible to remove final.
• Using Fiber::resume() and Fiber::suspend() models fibers as a stack and simplifies logic of resuming/suspending fibers. Ruby fibers share this exact API, though their API uses the term yield instead of suspend.
• State functions: Separate methods (i.e., isRunning, isTerminated) returning a boolean are preferable IMO to comparing integer constants to a value returned by a status method. This is really a matter of opinion than one being strictly better than the other.
• Pure C coroutine: The extension code provides user space fibers. Adding functions to create fiber C-only fibers can certainly be done if the need arises.
• Being marked experimental: I was hoping this would be a good compromise between allowing people to experiment with fibers and providing the ability to make minor BC breaks if the need arose. I realize this is generally not the PHP way, however this is, IMO, not entirely without precedent. JIT and FFI were not widely used before being added to PHP.

Thanks for the feedback!

Aaron Piotrowski

Hi Dan,

We have no commercial purpose on the swoole open source project. This is a purely technical project. 
If possible, we can remove the name of swoole, contribute the source code of swoole-src to php-src, and transfer the copyright.
Here is just a technical discussion. My opinion is that if PHP will support fiber/green-threads, this is a major change, should redesign language syntax, standard library, ZendVM.
This proposal must be discussed extensively and in-depth, should not make a hasty decision.

There are 7 parts to consider with my experience:

EventLoop API

Fiber/Coroutine/GreenThread

IO-Scheduler (Socket/FileSystem/ChildProcess/Signal/Timer/Stdout/Stdin)

CPU-Scheduler

Compatible with existing extesions  (php_streams/ext-sockets/ext-redis/ext-mysqli/ext-pdo-mysql/ext-curl ...) and builtin-functions (sleep/gethostbyname/proc_open/file_get_contents)

Service container, How to support php-fpm and provide a coroutine version of http server

Coroutine communication, How to pass messages between two coroutines

It’s nice to see php has such topic. This may be the key technology in the next generation of PHP.

Thanks

Tianfeng.Han

------------------ Original ------------------
From:  "Dan Ackroyd"<Danack@basereality.com>;
Date:  Fri, Mar 12, 2021 05:50 AM
To:  &quot;twosee"<twose@qq.com>; &quot;韩天峰"<rango@swoole.com>;
Cc:  "Aaron Piotrowski"<aaron@trowski.com>; "php internals"<internals@lists.php.net>;
Subject:  Re: [PHP-DEV] [VOTE] Fibers

Hi Twosee, Tianfeng.Han

I was drafting a longer reply to you both, but I realised I might be
missing some information.

Please could you disclose the commercial interests of the Swoole
maintainers, and the ties to the for profit companies that provide
services implementing Swoole?

Having people vote on RFCs who have ties to companies that provide
services via commercial entities isn't unprecedented. For example
zend.com has had employees who are involved in voting on RFCs, but to
a large extent that was done openly, with all of them using zend.com
email addresses.

twosee wrote:
> Of course, the Swoole community still expects to merge it into php-src at an appropriate time.

What's your idea of an appropriate time and how do you plan to start
this conversation?

If something hasn't been merged back after 8 years, it sounds
reasonably unlikely that it's ever going to be merged back.

cheers
Dan
Ack

2021年3月12日 上午5:50，Dan Ackroyd Danack@basereality.com 写道：

Hi Twosee, Tianfeng.Han

I was drafting a longer reply to you both, but I realised I might be
missing some information.

Please could you disclose the commercial interests of the Swoole
maintainers, and the ties to the for profit companies that provide
services implementing Swoole?

Having people vote on RFCs who have ties to companies that provide
services via commercial entities isn't unprecedented. For example
zend.com has had employees who are involved in voting on RFCs, but to
a large extent that was done openly, with all of them using zend.com
email addresses.

twosee wrote:

Of course, the Swoole community still expects to merge it into php-src at an appropriate time.

What's your idea of an appropriate time and how do you plan to start
this conversation?

If something hasn't been merged back after 8 years, it sounds
reasonably unlikely that it's ever going to be merged back.

cheers
Dan
Ack

--

To unsubscribe, visit: https://www.php.net/unsub.php

Hi all,

The following are some of my personal views, any comment is welcome:

What's your idea of an appropriate time and how do you plan to start
this conversation?

Maybe, I misunderstood the rules of the PHP RFC voting. In my opinion, the PHP RFC drafters need to have considerable experience in PHP kernel development (I mean that I don’t think I have met such requirements yet). And the RFC content needs to be fully verified by unit tests and actual projects, and the API or code implementation should be very stable. Once determined, no major changes will be made in principle.

But now, there is another way shown in front of me. We can draft an RFC first, then complete it based on community feedback, mark it as experimental, and continue to iterate in the PHP core. What I have to admit here is that drafting Swow's RFC will be more difficult than Fiber, because the content of the former is many times than that of the latter. However, to my surprise, it seems that there are not as many people who care about the details of the internal implementation as I thought. Most people care more about the content in the user space, and things under the user space are very easy to modify.

In addition, due to language and network barriers, people active in the Swoole community rarely participate in discussions in the PHP core community. In fact, I started subscribing to the PHP internal mailing list in July 2020 and occasionally show my opinion there. I often worry that incorrect translations will cause my intentions to be misinterpreted, and other members of the Swoole community never participated in discussions. We hardly use non-local social media. I just registered an account on Reddit, a great community exchange website yesterday, and I haven't had time to read all the discussions on that.

For all of the above reasons, it is extremely difficult for me to draft the RFC, and I have always hoped that I can make more contributions to other aspects of the PHP kernel, and then try to propose a huge coroutine draft as a goal of PHP9. And I have not taken the first step so far, I am very sorry about that.

Anyway, I will share content about Swow soon and try to draft a proposal.

If something hasn't been merged back after 8 years, it sounds
reasonably unlikely that it's ever going to be merged back.

Swoole does have a long history, what I have to point out is that the early Swoole did not really participate in the international open-source community. And Swoole switched from the pure asynchronous mode to the coroutine mode in 2018. At that time, we found that the PHP kernel has a good wrapping for various IO operations. We only need to implement the coroutine version of php_stream_wrapper to turn all relevant synchronous clients into coroutine clients or make synchronization blocking file operations into coroutine way. In this case, we can use a very small amount of work to achieve the reuse of most of the original libraries of the PHP native ecosystem. But Swoole gradually becomes a C++ project with some technical plan, so that it makes Swoole hard to be merged into php-src.

I started programming in 2017 and contributed to the Swoole project in 2018. In 2019, I learned and mastered almost all the technical content of Swoole. In 2020, I obtained a php.net account. At the same time, I also created a new systematic pure C coroutine project called Swow (for sure, it can be called any other name, but the name Swow is only respect for Swoole because Swow leans in the implementation principle of Swoole and its technical experience) , and I plan to propose a systematic draft of coroutine this year. I have paid my best to accelerate this process. For the PHP community, everyone really needs a real asynchronous/coroutine solution. Therefore, I totally understand the criticism of the slow progress. But, sorry from the bottom of my heart. I can't do it faster.

Are you saying that by adding this Fiber code to core it will prevent
Swoole from functioning? If so, that is concerning. Or are you
simply saying that Swoole doesn't like this implementation and will not use
it with their own code?

First of all, please let me pay tribute to your work. I fully understand that the implementation of Fiber could be indeed compatible with Swoole/Swow because theoretically nothing is incompatible. However, undoubtedly, for some reasons I mentioned or not mentioned, this requires a considerable amount of work. Of course, I am not completely denying the feature of Fiber, I just cannot agree with the merger of ext-fiber now. We always regard merging the coroutine system into php-src as a systematic plan. Even if the implementation of ext-fiber is merged, I am still willing to overcome difficulties to make more changes to promote the merger of Swow into php-src, it's my pleasure.

One thing to consider when comparing the Fiber implementation is that the
vast majority of PHP applications are still run behind a web server in
short-lived requests. Unlike Swoole, Swow, and Parallel, it isn't limited
to ZTS or CLI. It might not be the ideal solution, but IMO it is a step in
the right direction for PHP to allow for better async support.

There is a misunderstanding here. Swow has made improvements in this area. It supports NTS, ZTS, supports running under all SAPIs (including FPM), and also supports running under all operating systems.

Now, Swow's working principle is a bit like Opcache. Opcache will optimize and replace the CPU instructions, which executed by the code, and Swow will replace the blocking system calls with the corresponding coroutine operation. They hardly change the behavior of the code. Opcache makes The program runs faster, and Swow can make the program's IO concurrency better.

Therefore, all PHP programs will benefit from it. We can directly use the synchronous-blocking guzzle. Any network IO will only block one coroutine instead of the entire process. You just need to open more coroutines to get better concurrency performance. For amphp and reactphp, after that, stream_select() and curl_multi_select() will become the coroutine version, and even they will no longer block the entire program.

Generally, the solution provided by Swow is completely a superset of Fiber.

By the way, Tianfeng’s translation may have caused some misunderstandings. I think what he wants to express is that Fiber is not a feature that can directly benefit all PHP projects. Of course, I know that the RFC has mentioned that Fiber is mainly provided for the author of the framework and library uses. Sorry for this that caused misunderstanding.

Regards,
Twosee

Thank you, and everyone involved, for your effort.

Greetings everyone!

The vote has started on the fiber RFC: https://wiki.php.net/rfc/fibers https://wiki.php.net/rfc/fibers

Voting will run through March 22nd.

I voted /no/, because of the dependency on Boost.
If my assumptions are wrong, I may reconsider my vote.

Only asm files are used[1], and these can be bundled, so there is no
dependency on boost or C++ in general.

[1] https://github.com/amphp/ext-fiber/tree/master/boost

--
Christoph M. Becker

Hi Mike, Christoph, and Derick,

To add a bit more information:

These asm files are part of the low-level boost.context lib, found here: https://github.com/boostorg/context

This library has infrequent releases. Some of the files for old architectures have not changed in several years. Keeping these files up-to-date will not be a burden (and I plan to assume this responsibility).

The Boost license is extremely permissive and approved by the OSI, so there is no problem bundling with PHP.

Hopefully that provides some clarification and you’ll reconsider your vote Mike.

Cheers!
Aaron Piotrowski

--

To unsubscribe, visit: https://www.php.net/unsub.php

Apologies, this is a long one!

This RFC strikes me as being very dangerous. Implicitly allowing code
which is synchronous by design to be executed asynchronously seems
sure to lead to very subtle, unpredictable, confusing and dangerous
bugs.

The issue is that executing code asynchronously introduces race
conditions and, in code which is synchronous by design, those race
conditions will lead to bugs which will never occur during QA or
development and will be incredibly hard to identify once they do occur
in production.

These are the kind of bugs where user sessions get mixed up, the wrong
package is installed by a package manager, or an ecommerce website
captures the wrong amount of money from a customer's credit card. And
developers will have no realistic way of tracking down why these
things are happening in real world, multi developer, multi vendor PHP
applications.

For example, the PHP ecosystem is full of code like this:

private function capturePayment()
{
    $paymentRequest = preparePaymentRequest($this->currentOrder);
    $this->paymentGateway->capturePayment($paymentRequest);
    $this->currentOrder->setTransactionId($paymentRequest->getTransactionId());
}

In the above code snippet, the developer has relied on the fact that
the mutable value $this->currentOrder will be the same before and
after the call to $this->capturePayment(), because capturePayment() is
synchronous. In all likelihood, the authoring PHP developer is not
even familiar with asynchronicity, and the assumption that
capturePayment() is synchronous is therefore not even a conscious one.
Note that, if capturePayment is executed asynchronously, eventually we
will see payments attributed to the wrong orders.

Note that, if fibers land, there is no know whether the code above is
correct by looking at it in isolation. You can only know that by
knowing the environment in which the code is executed, as well as
understanding all of the internals of capturePayment(), and its
dependencies, many of which will probably managed by other developers
and will involve a bunch of injected services, many of which will do
IO and therefore are candidates for using fibers, directly or
indirectly, such as logging, HTTP, and database.

The worst characteristics to me are the following:

• These race conditions will cause problems so rarely that developers
  will not even realise that their programs contain dangerous bugs. Code
  which is not safe to use with fibers will appear to be functional
  because the happy path will work fine, and so developers will assume
  it's okay.
• Changes very, very far away from your code can cause your code to
  become unexpectedly asynchronous. Imagine you are using the DynamoDB
  monolog logger in your application, and that library indirectly uses
  Guzzle, which gets fiber support. Any place you are logging anything
  now becomes asynchronous, as well as any code which invokes that code.
  If you are using some kind of event dispatch pattern then pretty much
  everything in your program becomes asynchronous.
• As a library author, there is no way of knowing how your code is
  going to be used, and what injected dependencies might use fibers
  internally.

The counter arguments:

• "The above code snippet is suboptimal, it shouldn't assume that
  mutable state remains unchanged following some IO, and the order
  should be passed in as an arg." -- Whether or not this is true is
  irrelevant -- developers have relied on the invariants of the PHP
  language when writing their code, such as synchronous code being
  synchronous, and this is how lots of code is written in the wild. If
  this feature lands, that won't change -- very few userland developers
  will understand this feature, including library authors.
• "Fibers should only be enabled in async first programs." and "We
  could flag that certain libraries are fiber compatible or
  incompatible." If code appears to work in a fiber environment, people
  will just use it in a fiber environment, without understanding all of
  the security issues and horrible bugs they're introducing. This
  problem should rather be solved at the language level by making async
  explicit.

Consider the following:

private async function capturePayment()
{
    $paymentRequest = preparePaymentRequest($this->currentOrder);
    await $this->paymentGateway->capturePayment($paymentRequest);
    $this->currentOrder->setTransactionId($paymentRequest->getTransactionId());
}

Looking at the above code, I can see the race condition. It becomes
explicit. So I change the code and communicate explicitly via the type
system that this function is asynchronous and that my code expects
PaymentGateway::capturePayment() to be asynchronous.

private async function capturePayment()
{
    $order = $this->currentOrder;
    $paymentRequest = preparePaymentRequest($order);
    await $this->paymentGateway->capturePayment($paymentRequest);
    $order->setTransactionId($paymentRequest->getTransactionId());
}

Regarding PHP's type system not supporting generics, and the
associated advantages of this RFC over promises; I agree that this is
a problem with promises. But the lack of generics in PHP already means
we have no language-level type safety on data structures, which is
certainly worse than not having type safety on promises. Psalm et al
have provided reasonable solutions to this issue.

Sorry for the criticism of the idea -- I've used React and Amp in
production on a product which has been in prod for many years -- and
I'd be happy to see first class asynchronous support within the
language, but I think this RFC would make the language worse, not
better. Technically, I'm sure this is great, and I do appreciate that
a lot of work has probably been put into this.

Thanks

Hey Levi,

Greetings everyone!

The vote has started on the fiber RFC: https://wiki.php.net/rfc/fibers https://wiki.php.net/rfc/fibers

Voting will run through March 22nd.

Cheers,
Aaron Piotrowski

This is selfish, but I would like to kindly request lengthening the
voting window to allow me more time to play with it. I feel like I
can't vote "yes" on something like this without more experience with
it (which is why I currently have voted "no"). I hope others would
play with it more as well if we had more time. Any objections?

How much time do you think you need?

Another week seems reasonable; enough time to evaluate it more
thoroughly but not delay things seriously.

This is fine with me. Let's extend voting for about another week, ending on 3/28 at about 11 PM EDT.

Thanks!
Aaron Piotrowski

I hope others would
play with it more as well if we had more time. Any objections?

Yes, I object.

You've been around PHP internals long enough to see the drama has
occurred on other RFCs where people have been
cajoled/pressured/threatened to either suspend votes or withdraw RFCs.

Although this RFC is reasonably free from drama, I think it was wrong
for you to ask for an extension regardless of the current voting. Not
only is it improper on its face, but it sets a bad precedent for more
drama filled RFCs.

cheers
Dan
Ack

btw I would prefer that all RFC votes were longer (e.g. 4 weeks for
all decisions that don't have an external time constraint), but
changing the end time during the vote is bogus.

I agree with Dan. Also, there was plenty of discussion time for this
RFC. It’s not like it was rushed.

Hey all,

given the wording in the voting rules, the fact that the RFC hasn't been
edited to include the new end date, and due to the objections, I think the
best way forward is closing the voting on the RFC soon. Given no timezone
is mentioned in the RFC, we can probably go with either UTC or UTC-12.
We'll close the vote after UTC-12 passed.

Please create a new thread for discussions about the voting rules in
general that aren't specific for this RFC to keep this thread focused.

Thanks,
Niklas