Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:126817
Feedback-ID: i8414410d:Fastmail
Precedence: bulk
MIME-Version: 1.0
Date: Tue, 18 Mar 2025 00:59:07 -0500
To: "php internals" <internals@lists.php.net>
Message-ID: <fdeeec46-55c0-409e-a230-f04c3af23464@app.fastmail.com>
In-Reply-To: 
 <CAMW7n8DQqUhnb+U+WcA7bJ-APRDTtzuJC4m8s-q_2LH7bEuH+A@mail.gmail.com>
References: 
 <CAMW7n8DQqUhnb+U+WcA7bJ-APRDTtzuJC4m8s-q_2LH7bEuH+A@mail.gmail.com>
Subject: Re: [PHP-DEV] PHP True Async RFC - Stage 2
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From: larry@garfieldtech.com ("Larry Garfield")

On Sun, Mar 16, 2025, at 4:24 AM, Edmond Dantes wrote:
> Good day, everyone. I hope you're doing well.
>
> https://wiki.php.net/rfc/true_async
>
> Here is a new version of the RFC dedicated to asynchrony.
>
> Key differences from the previous version:
>
> * The RFC is not based on Fiber; it introduces a separate class=20
> representation for the asynchronous context.

I'm unclear here.  It doesn't expose Fibers at all, or it's not even tou=
ching the C code for fibers internally?  Like, would this render the exi=
sting Fiber code entirely vestigial, not just its API?

> * All low-level elements, including the Scheduler and Reactor, have=20
> been removed from the RFC.
> * The RFC does not include Future, Channel, or any other primitives,=20
> except those directly related to the implementation of structured=20
> concurrency.
>
> The new RFC proposes more significant changes than the previous one;=20
> however, all of them are feasible for implementation.
>
> I have also added PHP code examples to illustrate how it could look=20
> within the API of this RFC.
>
> I would like to make a few comments right away. In the end, the Kotlin=20
> model lost, and the RFC includes an analysis of why this happened. The=20
> model that won is based on the Actor approach, although, in reality,=20
> there are no Actors, nor is there an assumption of implementing=20
> encapsulated processes.
>
> On an emotional level, the chosen model prevailed because it forces=20
> developers to constantly think about how long coroutines will run and=20
> what they should be synchronized with. This somewhat reminded me of=20
> Rust=E2=80=99s approach to lifetime management.

Considering that lifetime management is one of the hardest things in Rus=
t to learn, that's not a ringing endorsement.

> Another advantage I liked is that there is no need for complex syntax=20
> like in Kotlin, nor do we have to create separate entities like=20
> Supervisors and so on. Everything is achieved through a simple API tha=
t=20
> is quite intuitive.

I'll be honest... intuitive is not the term I'd use.  In fact, I didn't =
make it all the way through the RFC before I got extremely confused abou=
t how it all worked.

First off, it desperately needs an "executive summary" section up at the=
 top.  There's a *lot* going on, and having a big-picture overview would=
 help a ton.  (For examples, see property hooks[1] and pattern matching[=
2].) =20

Second, please include realistic examples.  Nearly all of the examples a=
re contrived, which doesn't help me see how I would actually use async r=
outines or what the common patterns would be, and I therefore cannot eva=
luate how well the proposal treats those common cases.  The first non-fo=
obar example includes a comment "of course you should never do it like t=
his", which makes the example rather useless.  And the second is built a=
round a code model that I would never, ever accept into a code base, so =
it's again unhelpful.  Most of the RFC also uses examples that... have n=
o return values.  So from reading the first half of it, I honestly could=
n't tell you how return values work, or if they're wrapped in a Future o=
r something.

Third, regarding syntax, I largely agree with Tim that keywords are bett=
er than functions.  This is very low-level functionality, so we can and =
should build dedicated syntax to make it as robust and self-evident (and=
 IDE friendly) as possible.

That said, even allowing for the async or await or spawn keywords, I got=
 super confused when the Scope object was introduced.  So would the func=
tions/keywords be shortcuts for some of the common functionality of a Sc=
ope object?  If not, what's the actual difference?  I got lost at that p=
oint.

The first few sections of the RFC seem to read as "this RFC doesn't actu=
ally work at all, until some future RFC handles this other part."  Which=
... no, that's not how this works. :-)

As someone that has not built an async framework before (which is 99.9% =
of PHP developers, including those on this list), I do not see the point=
 of half the functionality here.  Especially the BoundedScope.  I see no=
 reason for it to be separate from just any other Scope.  What is the di=
fference between scope and context?  I have no clue. =20

My biggest issue, though, is that I honestly can't tell what the mental =
model is supposed to be.  The RFC goes into detail about three different=
 async models.  Are those standard terms you're borrowing from elsewhere=
, or your own creation?  If the former, please include citations.  I can=
not really tell which one the "playpen" model would fit into.  I... thin=
k bottom up, but I'm not sure.  Moreover, I then cannot tell which of th=
ose models is in use in the RFC.  There's a passing reference to it bein=
g bottom up, I think, but it certainly looks like the No Limit model.  T=
here's a section called structured concurrency, but what it describes do=
esn't look a thing like the playpen-definition of structured concurrency=
, which as noted is my preference.  It's not clear why the various posit=
ives and negatives are there; it's just presented as though self-evident=
.  Why does bottom up lead to high memory usage, for instance?  That's n=
ot clear to me.  So really... I have no idea how to think about any of i=
t.

Sorry, I'm just totally lost at this point.

As an aside: I used "spawn" as a throw-away keyword to avoid using "awai=
t" in a previous example.  It's probably not the right word to use in mo=
st of these cases.

I know some have expressed the sentiment that tightly structured concurr=
ency is just us not trusting developers and babysitting them.  To which =
I say... YES!  The overwhelming majority of PHP developers have no exper=
ience writing async code.  Their odds of getting it wrong and doing some=
thing inadvertently stupid by accident through not understanding some nu=
ance are high.  And I include myself in that.  MY chances of inadvertent=
ly doing something stupid by accident are high.  I *want* a design that =
doesn't let me shoot myself in the foot, or at least makes it difficult =
to do.  If that means I cannot do everything I want to... GOOD!  Humans =
are not to be trusted with manually coordinating parallelism.  We're jus=
t not very good at it, as a species.

Broadly speaking, I can think of three usage patterns for async in PHP (=
speaking, again, as someone who doesn't have a lot of async experience, =
so I may be missing some):

1. Fan-out.  This is the "fetch all these URLs at once" type use case, w=
hich in most cases could be wrapped up into a para_map() function.  (Whi=
ch is exactly what Rust does.)
2. Request handlers, for persistent-process servers.  Would also apply f=
or a queue worker.
3. Throw it over the wall.  This would be the logging example, or sendin=
g an email on some trigger, etc.  Importantly, these are cases where the=
re is no result needed from the sub-routine.

I feel like those three seem to capture most reasonable use cases, give =
or take some details.  (And, of course, many apps will include all three=
 in various places.)  So any proposal should include copious examples of=
 how those three cases would look, and why they're sufficiently ergonomi=
c.

A playpen model can handle both 1 and 2.  In fan out, you want the "Wait=
 all" logic, but then you also need to think about a Future object or si=
milar.  In a request handler, you're spawning an arbitrary number of cor=
outines that will terminate, and you probably don't care if they have a =
return value.

It's the "throw over the wall" cases where a playpen takes more work.  A=
s I showed previously, it can be done.  It just takes a bit more setup. =
 But if that is too much for folks, I offer a compromise position.  Agai=
n, just spitballing the syntax specifics:

// Creates an async scope, in which you can create coroutines.
async {

  // Creates a new coroutine that MAY last beyond the scope of this bloc=
k.
  // However, it MUST be a void-return function, indicating that it's go=
ing to
  // do work that is not relevant to the rest of this block.
  spawn func_call(1, 2, 3);

  // Creates a new coroutine that will block at the end of this async bl=
ock.
  // The return value is a future for whatever other_function() will ret=
urn.
  // $future may be used as though it were the type returned, but trying=20
  // to read it will block until the function completes.  It may also ha=
ve other
  // methods on it, not sure.
  $future =3D start other_function(4, 5, 6);

  // Queues a coroutine to get called after all "start"ed coroutines hav=
e completed
  // and this block is about to end.  Its return value is discarded. Per=
haps it should be
  // restricted to void-return, not sure.  In this case it doesn't hurt =
anything.
  defer cleanup(7, 8, 9);

  // Do nothing except allow other coroutines to switch in here if they =
want.
  suspend;

  // Enqueues this coroutine to run in 100 ms, or slightly thereafter wh=
enever the scheduler gets to it.
  timeout 100ms something(4, 5, 6);

} // There is an implicit wait-all here for anything start-ed, but not f=
or spawn-ed.

I honestly cannot see a use case at this point for starting coroutines i=
n arbitrary scopes.  Only "current scope" and "global scope, let it esca=
pe."  That maps to "start" and "spawn" above.  If internally "spawn" get=
s translated to "start in the implicit async block that is the entire ap=
plication", so that those coroutines will still block the whole script f=
rom terminating, that is not a detail most devs will care about.  (Which=
 also means in the global async scope, spawn and start are basically syn=
onymous.)

I can see the need for cancellation, which means probably we do need a s=
cope object to represent the current async block.  However, that's just =
a cancel() method, which would propagate to any child.  Scheduling it ca=
n be handled by the timeout command.  At this point, I do not see the us=
e case for anything more advanced than the above (except for channels, w=
hich as I argued before could make spawn unnecessary).  There may be a g=
ood reason for it, but I don't know what it is and the RFC does not make=
 a compelling argument for why anything more is needed.

I could see an argument that async $scope { ... } lets you call all of t=
he above keywords as methods on $scope, and the keywords are essentially=
 a shorthand for "this method on the current scope".  But you could also=
 pass the scope object around to places if you want to do dangerous thin=
gs.  I'm not sure if I like that, honestly, but it seems like an option.

Elsewhere in the thread, Tim noted that we should unify the function cal=
l vs closure question.  I used straight function calls above for simplic=
ity, but standardizing on a closure also makes sense.  Related, I've bee=
n talking with Arnaud about trying to put Partial Function Application f=
orward again[3], assuming pipes[4] pass.  If we follow the previous mode=
l, then it would implicitly provide a way to turn any function call into=
 a delayed function call:

function foo(int $a, int $b) { ... }

foo(4, 5); // Calls foo() right now
foo(4, 5, ...); // Creates a 0-argument closure that will call foo(4, 5)=
 when invoked.

Basically the latter is equivalent to:
fn() =3D> foo(4, 5);

A 0-argument closure (because all arguments are already captured) goes b=
y the delightful name "thunk" (as in the past tense of think, if you don=
't know English very well.)  That likely wouldn't be ideal, but it would=
 make standardizing start/spawn on "thou shalt provide a closure" fairly=
 straightforward, as any function could trivially be wrapped into one.

That's not necessarily the best way, but I mention it to show that there=
 are options available if we allow related features to support each othe=
r synergistically, which I always encourage.

Like Tim, I applaud you're commitment to this topic and willingness to w=
ork with feedback.  But the RFC text is still a long way from a model th=
at I can wrap my head around, much less support.

[1] https://wiki.php.net/rfc/property-hooks
[2] https://wiki.php.net/rfc/pattern-matching
[3] https://wiki.php.net/rfc/partial_function_application
[4] https://wiki.php.net/rfc/pipe-operator-v3

--Larry Garfield