Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:113638
MIME-Version: 1.0
References: <D376DEFE-26A4-4F11-8548-D9454087E9FD@trowski.com>
 <70951423-5e77-c150-6dce-dd3c62f8dc8b@php.net> <0b994a60-7970-605b-1657-d6ee732690e5@gmx.de>
 <5C73A1DE-E563-4F69-B8C7-6506F81D7345@trowski.com> <CAKiSzdDVR+joZQQtAS03rAZE_HBZir1CkaBG-BJm6Da1xLnu6A@mail.gmail.com>
 <CANUQDCg-TQCC6LBhDBRvXse2CpVODmM5cK5==a-yMOS09VYeGg@mail.gmail.com>
 <CAKiSzdA+6YK69ez_9kZGACTNMi_e44xX_UPLfDCoU1i0qrF-ew@mail.gmail.com>
 <cc0e0683-2dda-4340-a3b0-6e601882a65f@www.fastmail.com> <CAKiSzdCzXkx_e2ZSbFRFNCd_9F-N_ZZ3-qDMxVZt-jOMMuyVuA@mail.gmail.com>
 <CAPrKfG5ZoiC02RdgNSP294OFq=kDBDPPOpmN7UTDKhzQah0GhA@mail.gmail.com>
 <CAKiSzdAXXX6d=tSyskbM5R3jc8-BQSYpT0g8TQPLpGDtuL+Sgw@mail.gmail.com>
 <CAPrKfG79Y4=SVFDFipXBpaEh+o_-kgQ38-=evR4GvgKZaOSmRw@mail.gmail.com>
 <CAKiSzdCPPM-s7yA75dJqS7N4j-VQHCfuXc5K6Zrh+sm_bdSUHQ@mail.gmail.com>
 <CAPrKfG6hbu2K1ans4+a2ajm1TCBCM+TZSr4ESBXj9BHH3gaiCQ@mail.gmail.com>
 <CAKiSzdBHotq1qMYmOoZek7=URpVJT528FtZVxCQhuu3J5fEkdg@mail.gmail.com> <c8a2d855-5c88-b2a6-c267-d72460eaa448@gmail.com>
In-Reply-To: <c8a2d855-5c88-b2a6-c267-d72460eaa448@gmail.com>
Date: Sat, 20 Mar 2021 07:56:02 +0000
Message-ID: <CAKiSzdCjuL9AgBtdYO7zeDMDfwvsky6PrcGH=_q=iHRnjF+e6w@mail.gmail.com>
To: Rowan Tommins <rowan.collins@gmail.com>
Cc: PHP internals <internals@lists.php.net>
Content-Type: text/plain; charset="UTF-8"
Subject: Re: [PHP-DEV] [VOTE] Fibers
From: joshdifabio@gmail.com (Josh Di Fabio)

HI Rowan,

Thanks for sharing your thoughts.

On Fri, Mar 19, 2021 at 7:55 PM Rowan Tommins <rowan.collins@gmail.com> wrote:
>
> On 18/03/2021 09:20, Josh Di Fabio wrote:
> > "If you want to enable fibers in your application, you must be
> > confident about the implementation details of all of the code in your
> > application, including that of your dependencies, which are written
> > and maintained by other developers."
> >
> > I don't have anything to add to my previous point in that I disagree
> > that this is practical.
>
>
> While I agree that this is extremely difficult, and slows adoption of
> asynchronous technologies, I think the challenge is not identifying
> asynchronous code, it's identifying shared state.

I agree. If we take it back to first principles, so to speak, then the
problems all involve shared (mutable) state. Thanks for highlighting
that.

Perhaps focusing on shared mutable state might be helpful in
identifying more popular and even more effective solutions to the kind
of problems I'm concerned about. I.e. perhaps solutions which
target/reduce/identify mutability via language features or static
analysis might be more elegant, more effective, and almost certainly
more popular (if added via new language features) than solutions which
rather make it harder to utilise fibers.

If developers could identify that their program contains only a few
hundred state mutations (and they could see where they are), then I
think reviewing and understanding those mutations asks something a bit
more realistic of developers than reviewing and understanding the
internals of their entire codebase.

Having said that, it does seem like it would be very challenging to
really encourage strict immutability at the language level in PHP
given we lack most of the requisite building blocks, such as;
ergonomic higher order functions, immutable data structures, generic
data structures, local constants, etc. I suppose static analysis is a
more realistic place to attack these problems, although that approach
has various drawbacks.

> In your example, you show code that was written to use shared state
> unwittingly calling code that was written to be asynchronous:
>
>      private function capturePayment()
>      {
>          $paymentRequest = preparePaymentRequest($this->currentOrder);
> $this->paymentGateway->capturePayment($paymentRequest);
> $this->currentOrder->setTransactionId($paymentRequest->getTransactionId());
>      }
>
> However, the same problem exists the other way around - code written to
> be asynchronous unwittingly calling code written to use shared state:
>
>      private async function capturePayment()
>      {
>          $paymentRequest = $this->someDependency->preparePaymentRequest();
>          await $this->paymentGateway->capturePayment($paymentRequest);
> $this->someDependency->setTransactionId($paymentRequest->getTransactionId());
>      }
>
> This all looks fine - but what if someDependency is actually calling
> into a library which stores the current order in a static variable? Now
> you have exactly the same race condition for the opposite reason. And
> the solution is the same: carefully vet all your dependencies.
>
>
> I can think of a few ways of solving this:
>
> 1) Require all the code to be synchronous. This is easy in PHP, even if
> Fibers are supported: just don't run in an asynchronous framework.
>
> 2) Require most of the code to be synchronous. This is the common
> approach of labelling functions as "async" or converting return values
> to Generators or Promises. The big disadvantage is that it requires
> rewriting a lot of code that doesn't care one way or the other if it's
> called synchronously.
>
> 3) Require all the code to be free of shared state. This is ultimately
> the only way you'll get the full advantage of asynchronous code.
>
> 4) Require most of the code to be free of shared state. Having some
> primitives in the language to mark out code that definitely *can't* be
> asynchronous would probably be useful. Perhaps you could mark a function
> as "no-interrupt"; this could then be used as a wrapper when calling
> into a library you know or suspect of using state in unsafe ways.

Niklas has suggested something similar as far as I understand. Some
other people have suggested that vendors could somehow declare at the
package level that they are ~"fiber safe" (perhaps in composer.json).

>
> > Perhaps we could rather make fibers*opt in*  at the*callsite*
> > (similar to goroutine calls) in order to prevent functions
> > unexpectedly being executed asynchronously due to faraway changes.
> > This would be safe and predictable while also avoiding the "What color
> > is your function" problem.
>
>
> Although this would avoid keeping both synchronous and asynchronous
> versions of the same function, it would require adding that keyword to
> every single function call, just in case somewhere inside it wants to
> take advantage of your asynchronous environment.
>
>  From my limited understanding, goroutines are a completely different
> concept. Saying "go someFunction()" in Go immediately starts a new
> thread-like thing, and doesn't return anything. The runtime manages the
> scheduling of the thread, but if you want to get any data out of the
> goroutine, you have to pass it explicitly, e.g. via a channel.
>
>
> Regards,
>
> --
> Rowan Tommins
> [IMSoP]
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: https://www.php.net/unsub.php
>

Thanks for taking the time to share your thoughts. I found them very
insightful and helpful.

As a side note, GitHub published a blog post a couple of days ago
which provides an interesting real world example of this kind of issue
if people are interested.

https://github.blog/2021-03-18-how-we-found-and-fixed-a-rare-race-condition-in-our-session-handling/

Cheers