Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:126583
Feedback-ID: i8414410d:Fastmail
Precedence: bulk
MIME-Version: 1.0
Date: Wed, 05 Mar 2025 12:30:07 -0600
To: "php internals" <internals@lists.php.net>
Message-ID: <78a03dd0-fd4a-4f4a-ad8a-37e5704f06fc@app.fastmail.com>
In-Reply-To: 
 <CAMW7n8CM7oBfXCDsKtV4hTFs40UmLCU3183WjYE2exLNqKDWLQ@mail.gmail.com>
References: 
 <CAMW7n8AJckEDzhGv9BdjNhq8zAdCqb4HsVr56vGi+izw50X6Dg@mail.gmail.com>
 <9964db8c-0ffe-43d5-8246-47fc76b07180@app.fastmail.com>
 <CAMW7n8CM7oBfXCDsKtV4hTFs40UmLCU3183WjYE2exLNqKDWLQ@mail.gmail.com>
Subject: Re: [PHP-DEV] PHP True Async RFC
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From: larry@garfieldtech.com ("Larry Garfield")

On Wed, Mar 5, 2025, at 3:37 AM, Edmond Dantes wrote:
> Good day, Larry.
>
>> First off, as others have said, thank you for a thorough and detailed=
 proposal.
> Thanks!
>
>> * A series of free-standing functions.
>> * That only work if the scheduler is active.
>> * The scheduler being active is a run-once global flag.
>> * So code that uses those functions is only useful based on a global =
state not present in that function.
>> * And a host of other seemingly low-level objects that have a myriad =
of methods on them that do, um, stuff.
>> * Oh, and a lot of static methods, too, instead of free-standing func=
tions.
>
> Suppose these shortcomings don=E2=80=99t exist, and we have implemente=
d the=20
> boldest scenario imaginable. We introduce Structured Concurrency,=20
> remove low-level elements, and possibly even get rid of `Future`. Of=20
> course, there are no functions like `startScheduler` or anything like=20
> that.
>
>  1. In this case, how should PHP handle `Fiber` and all the behavior=20
> associated with it? Should `Fiber` be declared deprecated and removed=20
> from the language? What should the flow be?

I'm not sure yet.  I was quite hesitant about Fibers when they went in b=
ecause they were so low-level, but the authors were confident that it wa=
s enough for a user-space toolchain to be iterated on quickly that every=
one could use.  That clearly didn't pan out as intended (Revolt exists, =
but usage of it is still rare), so here we are with a half-finished API.

Thinking aloud, perhaps we could cause `new Fiber` to create an automati=
c async block?  Or we do deprecate it and discourage its use.  Something=
 to think through, certainly.

>  2. What should be done with I/O functions? Should they remain=20
> blocking, with a separate API provided as an extension?

The fact that IO functions become transparently async when appropriate i=
s the best part of the current RFC.  Please keep that. :-)

>  3. Would it be possible to convince the maintainers of XDEBUG and=20
> other extensions to rewrite their code to support the new model? ( *If=20
> you're reading this question now, please share your opinion.* )

I cannot speak for Derick.

>  4. If transparent concurrency is introduced for I/O in point 2, what=20
> should be done with `Revolt` + `AMPHP`? This would break their code.=20
> Should an additional function or option be introduced to switch PHP=20
> into "legacy mode"?

Also an excellent question, to which I do not yet have an answer.  (See =
previous point about Fibers being half-complete.)  I would want to invol=
ve Aaron, Christian, and Ces-Jan before trying to make any suggestions h=
ere.


> Structured concurrency is a great thing. However, I=E2=80=99d like to =
avoid=20
> changing the language syntax and make something closer to Go=E2=80=99s=20
> semantics. I=E2=80=99ll think about it and add this idea to my TODO.

Well, as noted in the article, structured concurrency done right means *=
not* having unstructured concurrency.  Having Go-style async and then bu=
ilding a structured nursery system on top of it means you cannot have an=
y of the guarantees of the structured approach, because the other one is=
 still poking out the side and leaking.  We're already stuck with mutabl=
e-by-default, global variables, and other things that prevent us from ma=
king helpful assumptions.  Please, let's try to avoid that for async.  W=
e don't need more gotos.

>> async $context {
>> // $context is an object of AsyncContext, and can be passed around as=
 such.
>> // It is the *only* way to span anything async, or interact with the =
async controls.
>> // If a function doesn't take an AsyncContext param, it cannot contro=
l async.  This is good.
>
> This is a very elegant solution. Theoretically.
>
> However, in practice, if you require explicitly passing the context to=20
> all functions, it leads to the following consequences:
>
>  1. The semantics of all functions increase by one additional paramete=
r=20
> (*Signature bloat*).

No, just those functions/objects that necessarily involve running async =
control commands.  Most wouldn't.  They would just silently context swit=
ch when they hit an IO operation (which as noted above is transparency s=
upported, which is what makes this work) and otherwise behave the same.

But if something does actively need to do async stuff, it should have a =
context to work within.  It's the same discussion as:

A: "Pass/inject a DB connection to a class that needs it, don't just cal=
l a global db() function."
B: "But then I have to pass it to all these places explicitly!"
A: "That's a sign your SQL is too scattered around the code base. Fix th=
at first and your problem goes away."

Explicit flow control is how you avoid bugs.  It's also self-documenting=
, as it's patently obvious what code expects to run in an async context =
and which doesn't care.

>  2. If an asynchronous call needs to be added to a function, and other=20
> functions depend on it, then the semantics of all dependent functions=20
> must be changed as well.=20

This is no different than DI of any other service.  I have restructured =
code to handle temporary contexts before.  (My AttributeUtils and Serde =
libraries.)  The result was... much better code than I had before.  I'm =
glad I made those refactors.

> In this example, there is another aspect: the fact that async executio=
n=20
> is explicitly limited to a specific scope. This is essentially the sam=
e=20
> as `startScheduler`, and it is one of the options I was considering.
>
> Of course, `startScheduler` can be replaced with a construction like=20
> `async(function() { ... })`.
> This means that async execution is only active within the closure, and=20
> coroutines can only be created inside that closure.
>
> This is one of the semantic solutions that allows removing=20
> `startScheduler`, but at the implementation level, it is exactly the=20
> same.
>
> What do you think about this?

That looks mostly like the async block syntax I proposed, spelled differ=
ently.  The main difference is that the body of the wrapped function wou=
ld need to explicitly `use` any variables from scope that it wanted, rat=
her than getting them implicitly.  Whether that's good or bad is probabl=
y subjective.

But it would allow for a syntax like this for the context, which is quit=
e similar to how database transactions are often done:

$val =3D async(function(AsyncContext $ctx) use ($stuff, $fn) {
  $result =3D [];
  foreach ($stuff as $item) {
    $result[] =3D $ctx->run($fn);
  }

  // We block/wait here until all subtasks are complete, then the async(=
) call returns this value.
  return $result;
});

And of course in both cases you could use a pre-defined callable instead=
 of inlining one.  At this point I think it's mostly a stylistic differe=
nce, function vs block.

>> I'm not convinced that sticking arbitrary key/value pairs into the Co=
ntext object is wise;
>
> Why not?=20
>
>> that's global state by another name
>
>   Static variables inside a function are also global state. Are you=20
> against static variables?

Vocally, in fact. :-)

>> But if we must, the above would handle all the inheritance and overri=
de stuff quite naturally. Possibly with:
>
>  How will a context with open string keys help preserve service data=20
> that the service doesn't want to expose to anyone? The `Key()` solutio=
n=20
> is essentially the same as `Symbol` in JS, which is used for the same=20
> purpose. Of course, we could add a `coroutine static $var` construct t=
o=20
> the language syntax. But it's all the same just syntactic sugar that=20
> would require more code to support.=20

I cannot speak to JS Symbols as I haven't used them.  I am just vhementl=
y opposed to globals, no matter how many layers they're wrapped in. :-) =
 Most uses could be replaced by proper DI or partial application.

>> [$in, $out] =3D Channel::create($buffer_size);
>
> This semantics require the programmer to remember that two variables=20
> actually point to the same object. If a function has multiple channels=
,=20
> this makes the code quite verbose. Additionally, such channels are=20
> inconvenient to store in lists because their structure becomes more=20
> complex.
>
> I would suggest a slightly different solution:
>
> <code php>
> $in =3D new Channel()->getProducer();
> async myFunction($in->getConsumer());
> <code>
>
> This semantics do not restrict the programmer in usage patterns while=20
> still allowing interaction with the channel through a well-defined=20
> contract.

I'd go slightly differently if you wanted to go that route:

$ch =3D new Channel($buffer_size);
$in =3D $ch->producer();
$out =3D $ch->consumer();

// You do most interaction with $in and $out.

I could probably work with that as well.

(Or even just $ch->inPipe and $ch->outPipe, now that we have nice proper=
ty support.)

But the overall point, I think, is avoiding implicit modal logic.  If my=
 code doesn't need to care if it's in an async world, it doesn't care.  =
If it does, then I need an explicit async world to work within, rather t=
han relying on one implicitly existing, I hope.  And I shouldn't have to=
 think about "who owns this end of this channel".  I just have an in and=
 out hose I stick stuff into and pull out from, kthxbye.

> Thanks for the great examples, and a special thanks for the article.
> I also like the definition of context.
>
> Ed

--Larry Garfield