Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:126545 X-Original-To: internals@lists.php.net Delivered-To: internals@lists.php.net Received: from php-smtp4.php.net (php-smtp4.php.net [45.112.84.5]) by qa.php.net (Postfix) with ESMTPS id BC6151A00BC for ; Sun, 2 Mar 2025 14:08:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=php.net; s=mail; t=1740924382; bh=rPPPspj6E0kI66ZfwNaLgHln7PA2nG6G4sn8j8/26M0=; h=Date:From:To:In-Reply-To:References:Subject:From; b=KdiwVdJsFUE/WpobFEzwGTsGdhd1bNLphtt7MS9S7FcXK+jsfO9Bxn4aqZM8Yrgnv ovLy5VRV1Sjol5ApCOTWcYK/VhBhSY4HBAh4VCd6ASexaVT1On9xis+dmnySNTdjZY mrg4nSpTdsC/MMga/dQ1d1eq3JOvPTWEGWNPFhKC9nOvftADWd+8pOqrjaD4Jawt06 yYInJ/Kyq03nBh+McfZ/XhlVHx9JZ2tyqp9tTxCHBgBiZRs+kiHxeqwzHnRITvNu7d wncCMAsvZaQRJJZHaJPj1Lk0GoyCOTpYk9g7posaZQGs3fmAy6SYPP5UcLzY90O/73 /LuVeQ3LDKnwA== Received: from php-smtp4.php.net (localhost [127.0.0.1]) by php-smtp4.php.net (Postfix) with ESMTP id 1515318006D for ; Sun, 2 Mar 2025 14:06:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on php-smtp4.php.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,DMARC_MISSING,HTML_MESSAGE, RCVD_IN_DNSWL_LOW,SPF_HELO_PASS,SPF_PASS autolearn=no autolearn_force=no version=4.0.0 X-Spam-Virus: No X-Envelope-From: Received: from fhigh-b1-smtp.messagingengine.com (fhigh-b1-smtp.messagingengine.com [202.12.124.152]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by php-smtp4.php.net (Postfix) with ESMTPS for ; Sun, 2 Mar 2025 14:06:20 +0000 (UTC) Received: from phl-compute-12.internal (phl-compute-12.phl.internal [10.202.2.52]) by mailfhigh.stl.internal (Postfix) with ESMTP id EAF2E254012D for ; Sun, 2 Mar 2025 09:08:56 -0500 (EST) Received: from phl-imap-09 ([10.202.2.99]) by phl-compute-12.internal (MEProxy); Sun, 02 Mar 2025 09:08:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bottled.codes; h=cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1740924536; x=1741010936; bh=Zb5J4/+7IH IlXhPq3SC02kpSEgLlXVQcq6LdGPP76BM=; b=KVgcZCh06ZqbTxvy1Iyn1t5zZY felHNEHgajqc0Hh5MF4jPMtmeWXhgtsZ6oFA64FpihnoV/PPkjFKrl7L6TdeGz/D M7h8HRCm3ClEQqtU2571zmXjVL/r8HI8y62Q+bqLzpr/Li+8Ne6XhTDxhAbd8m7k o6vy8RL/oPVZtjF6IFPiuYNzskBiDtIl6RiOzR75WwlU2vBMUH1KE9fA5az4iQEd 6TDrPoJ4DrM6gIR7bbwfoZqO3Db8Bx3H4hhdfwURcmnxb6AaAROKo/Igei7y9URZ G2dVXVW0RrRgV1ovNG8pQoEaHuOBmm4HJOKdgHPKmxUANw4LreF0mVfa4k9A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1740924536; x=1741010936; bh=Zb5J4/+7IHIlXhPq3SC02kpSEgLlXVQcq6L dGPP76BM=; b=GGCbwOap6trFmQuHNS16l6uffG4alqp76RjiesXb7qetEjYWwzS 6ayVhUGvzxECAIzwETkGYOrrI1mpRvUHLzyiJkxhNJrunufW2CkzHC6OxL3348jR xdCJOWTZqHS9RzGSLkqRAt0cP+hj8sEmp3xI64hX+oq+/nH+0Koaly5Rz3a3dGJ5 Td1vB+hL1GvrlOT+yZVsrcZgsXIyyrqpbwKAfIioBDWL/0ZjzNet4CBl1HTW+7rr c6la1KwOlBEcN3fkLhew8omGWDVa6bLJueYiF++B7HTTE3MewQklBL3qDwl1Mg+o 3BxQ0g5TzA0EUCzYK9cPOmTTU7a1Gg84zCw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgdelieegvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecuogfuuhhsphgvtghtff homhgrihhnucdlgeelmdenucfjughrpefoggffhffvkfgjfhfutgesrgdtreerredtjeen ucfhrhhomhepfdftohgsucfnrghnuggvrhhsfdcuoehrohgssegsohhtthhlvggurdgtoh guvghsqeenucggtffrrghtthgvrhhnpeelkeehtdfgfefhleeilefggeeihfekvdelfeej tdfflefhheehfffgudetuddutdenucffohhmrghinhepphhhphdrnhgvthenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehrohgssegsohhtthhl vggurdgtohguvghspdhnsggprhgtphhtthhopedupdhmohguvgepshhmthhpohhuthdprh gtphhtthhopehinhhtvghrnhgrlhhssehlihhsthhsrdhphhhprdhnvght X-ME-Proxy: Feedback-ID: ifab94697:Fastmail Received: by mailuser.phl.internal (Postfix, from userid 501) id 64EB6780068; Sun, 2 Mar 2025 09:08:56 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface Precedence: bulk list-help: list-post: List-Id: internals.lists.php.net x-ms-reactions: disallow MIME-Version: 1.0 Date: Sun, 02 Mar 2025 15:08:35 +0100 To: internals@lists.php.net Message-ID: In-Reply-To: References: Subject: Re: [PHP-DEV] PHP True Async RFC Content-Type: multipart/alternative; boundary=ab8c71c44ab64b6a96dedf5bf5850612 From: rob@bottled.codes ("Rob Landers") --ab8c71c44ab64b6a96dedf5bf5850612 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Sat, Mar 1, 2025, at 10:11, Edmond Dantes wrote: > Good day, everyone. I hope you're doing well. >=20 > I=E2=80=99d like to introduce a draft version of the RFC for the True = Async component. >=20 > https://wiki.php.net/rfc/true_async >=20 > I believe this version is not perfect and requires analysis. And I str= ongly believe that things like this shouldn't be developed in isolation.= So, if you think any important (or even minor) aspects have been overlo= oked, please bring them to attention. >=20 > The draft status also highlights the fact that it includes doubts abou= t the implementation and criticism. The main global issue I see is the l= ack of "future experience" regarding how this API will be used=E2=80=94a= nother reason to bring it up for public discussion. >=20 > Wishing you all a great day, and thank you for your feedback! >=20 Hey Edmond: I find this feature quite exciting! I've got some feedback so far, thoug= h most of it is for clarification or potential optimizations: > A PHP developer *SHOULD NOT* make any assumptions about the order in w= hich Fibers will be executed, as this order may change or be too complex= to predict. There should be a defined ordering (or at least, some guarantees). Being= able to understand what things run in what order can help with understa= nding a complex system. Even if it is just a vague notion (user tasks ar= e processed before events, or vice versa), it would still give developer= s more confidence in the code they write. You actually mention a bit of = the order later (microtasks happen before fibers/events), so this senten= ce maybe doesn't make complete sense. Personally, I feel as though an async task should run as though it were = a function call until it hits a suspension. This is mostly an optimizati= on though (C# does this), but it could potentially reduce overhead of qu= eueing a function that may never suspend (which you mention as a potenti= al problem much later on): Async\run(*function*() { =20 $fiber =3D Async\async(*function*() { sleep (1); // this gets enqueued now return "Fiber completed!"; }); =20 *// Execution is paused until the fiber completes* $result =3D Async\await($fiber); // immediately enter $fiber without = queuing =20 echo $result . "*\n*"; =20 echo "Done!*\n*"; }); > Until it is activated, PHP code behaves as before: calls to blocking f= unctions will block the execution thread and will not switch the *Fiber*= context. Thus, code written without the *Scheduler* component will func= tion exactly the same way, without side effects. This ensures backward c= ompatibility. I'm not sure I understand this. Won't php code behave exactly the same a= s it did before once enabling the scheduler? Will libraries written befo= re this feature existed suddenly behave differently? Do we need to worry= about the color of functions because it changes the behavior? > `True Async` prohibits initializing the `Scheduler` twice. How will a library take advantage of this feature if it cannot be certai= n the scheduler is running or not? Do I need to write a library for asyn= c and another version for non-async? Or do all the async functions with = this feature work without the scheduler running, or do they throw a catc= hable error? > This is crucial because the process may handle an OS signal that impos= es a time limit on execution (for example, as Windows does). Will this change the way os signals are handled then? Will it break comp= atibility if a library uses pcntl traps and I'm using true async traps t= oo? Note there are several different ways (timeout) signals are handled = in PHP -- so if (per-chance) the scheduler could always be running, mayb= e we can unify the way signals are handled in php. > Code that uses *Resume* cannot rely on when exactly the *Fiber* will r= esume execution. What if it never resumes at all? Will it call a finally block if it is t= ry/catched or will execution just be abandoned? Is there some way to ens= ure cleanup of resources? It should probably mention this case and how a= bandoning execution works. > If an exception is thrown inside a fiber and not handled, it will stop= the Scheduler and be thrown at the point where `Async\launchScheduler()= ` is called. The RFC doesn't mention the stack trace. Will it throw away any informat= ion about the inner exception? > The *Graceful Shutdown* mode can also be triggered using the function: What will calling `exit` or `die` do? > A concurrent runtime allows handling requests using Fibers, where each= Fiber can process its own request. In this case, storing request-associ= ated data in global variables is no longer an option. Why is this the case? Furthermore, if it inherits from the fiber that st= arted its current fiber, won't using Resume/Notifier potentially cause p= roblems when used manually? There are examples over the RFC using global= variables in closures; so do these examples not actually work? Will sha= ring instances of objects in scope of the functions break things? For ex= ample: Async\run($obj->method1(...)); Async\run($obj->method2(...)); This is technically sharing global variables (well, global to that scope= -- global is just a scope after all) -- so what happens here? Would it = make sense to delegate this fiber-local storage to user-land libraries i= nstead? > Objects of the `Future` class are high-level patterns for handling def= erred results.=20 By this point we have covered FiberHandle, Resume, and Contexts. Now we = have Futures? Can we simplify this to just Futures? Why do we need all t= hese different ways to handle execution? > A channel is a primitive for message exchange between `Fibers`. Why is there an `isEmpty` and `isNotEmpty` function? Wouldn't `!$channel= ->isEmpty()` suffice? It's also not clear what the value of most of these function is. For exa= mple: if ($chan->isFull()) { doSomething(); // suspends at some point inside? We may not know when = we write the code. // chan is no longer full, or maybe it is -- who knows, but the origin= al assumption entering this branch is no longer true. ... } Whether a channel is full or not is not really important, and if you rel= y on that information, this is usually an architectural smell (at least = in other languages). Same thing with empty or writable, or many others o= f these functions. You basically just write to a channel and eventually = (or not, which is a bug and causes a deadlock) something will read it. T= he entire point is to use channels to decouple async code, but most of t= he functions here allow for code to become strongly coupled. As for the single producer method, I am not sure why you would use this.= I can see some upside for the built-in constraints (potentially in a de= v-mode environment) but in a production system, single-producer bottlene= cks are a real thing that can cause serious performance issues. This is = usually something you explicitly want to avoid. > In addition to the `send/receive` methods, which suspend the execution= of a `Fiber`, the channel also provides non-blocking methods: `trySend`= , `tryReceive`, and auxiliary explicit blocking methods: `waitUntilWrita= ble` and `waitUntilReadable`.=20 It isn't clear what happens when `trySend` fails. Is this an error or do= es nothing?=20 Thinking through it, there may be cases where `trySend` is valid, but mo= re often than not, it is probably an antipattern. I cannot think of a va= lid reason for `tryReceive` and it's usage is most likely guaranteed to = cause a deadlock in real code. For true multi-threaded applications, it = makes more sense, but not for single-threaded concurrency like this. In other words, the following code is likely to be more robust, and not = depend on execution order (which we are told at the beginning not to do): Async\run(*function*() { $channel =3D *new* Async\Channel(); =20 $reader =3D Async\async(*function*() *use*($channel) { while ($data =3D $channel->read() && $data !=3D=3D NULL) { echo "receive: *$data**\n*"; } }); =20 for ($i =3D 0; $i < 4; $i++) { echo "send: event data *$i**\n*"; $data =3D $channel->send("event data *$i*"); } =20 $reader->cancel(); // clean up our reader // or $channel->close(); // will receive NULL I believe? }); A `trySend` is still useful when you want to send a message but don't wa= nt to block if it is full. However, this is going to largely depend on h= ow long is has been since the developer last suspended the current fiber= , and nothing else -- thus it is probably an antipattern since it totall= y depends on the literal structure of the code, not the structure of the= program -- if that makes sense. > This means that `trapSignal` is not intended for =E2=80=9Cregular code= =E2=80=9D and should not be used =E2=80=9Canywhere=E2=80=9D. Can you expand on what this means in the RFC? Why expose it if it should= n't be used? ----- I didn't go into the low level api details yet -- this email is already = pretty long. But I would suggest maybe thinking about how to unify Notif= iers/Resume/FiberHandle/Future into a single thing. These things are pre= tty similar to one another (from a developer's standpoint) -- a way to c= ontinue execution, and they all offer a slightly different api. I also noticed that you seem to be relying heavily on the current implem= entation to define behavior. Ideally, the RFC should define behavior and= the implementation implement that behavior as described in the RFC. In = other words, the RFC is used as a reference point as to whether somethin= g is a bug or an enhancement in the future. There has been more than onc= e where the list looks back at an old RFC to try and determine the inten= t for discovering if something is working as intended or a bug. RFCs are= also used to write documentation, so the more detailed the RFC, the bet= ter the documentation will be for new users of PHP. =E2=80=94 Rob --ab8c71c44ab64b6a96dedf5bf5850612 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

=
On Sat, Mar 1, 2025, at 10:11, Edmond Dantes wrote:

Good day, everyone. I hope you're doing well.

I=E2=80=99d = like to introduce a draft version of the RFC for the True Async componen= t.

https://wik= i.php.net/rfc/true_async

I believe this version is not per= fect and requires analysis. And I strongly believe that things like this= shouldn't be developed in isolation. So, if you think any important (or= even minor) aspects have been overlooked, please bring them to attentio= n.

The draft status also highlights the fact that it includes = doubts about the implementation and criticism. The main global issue I s= ee is the lack of "future experience" regarding how this API will be use= d=E2=80=94another reason to bring it up for public discussion.

Wishing you all a great day, and thank you for your feedback!


Hey Edmond:
I find this feature quite exciting! I've got some feedback s= o far, though most of it is for clarification or potential optimizations= :

A PHP developer S= HOULD NOT&nbs= p;make any assumptions about the order in which Fibers will be executed,= as this order may change or be too complex to predict.
=

There should be a defined ordering (or = at least, some guarantees). Being able to understand what things run in = what order can help with understanding a complex system. Even if it is j= ust a vague notion (user tasks are processed before events, or vice vers= a), it would still give developers more confidence in the code they writ= e. You actually mention a bit of the order later (microtasks happen befo= re fibers/events), so this sentence maybe doesn't make complete sense.

Personally, I feel as though an async task s= hould run as though it were a function call until it hits a suspension. = This is mostly an optimization though (C# does this), but it could poten= tially reduce overhead of queueing a function that may never suspend (wh= ich you mention as a potential problem much later on):
Async\run(function() {=0A =0A   $fiber =3D Async\async(function() {=0A       sleep(1); // this gets enqueued now=0A       return "Fiber completed!";=0A   });=0A =0A   // Execution is paused=
 until the fiber completes=0A   $result =3D Async\await($fiber); // immediately enter $fiber without queuing<=
/span>=0A =0A   echo "\n"=
;=0A&nbs=
p;=0A   echo "Done!\n";=0A});

Until i= t is activated, PHP code behaves as before: calls to blocking functions = will block the execution thread and will not switch the F= iber con= text. Thus, code written without the Scheduler component will fun= ction exactly the same way, without side effects. This ensures backward = compatibility.

I'm not = sure I understand this. Won't php code behave exactly the same as it did= before once enabling the scheduler? Will libraries written before this = feature existed suddenly behave differently? Do we need to worry about t= he color of functions because it changes the behavior?
True Async prohibits initializing the Scheduler twice.

How will a library take advantage of this feature = if it cannot be certain the scheduler is running or not? Do I need to wr= ite a library for async and another version for non-async? Or do all the= async functions with this feature work without the scheduler running, o= r do they throw a catchable error?

This is crucial because the process may handle an OS signal t= hat imposes a time limit on execution (for example, as Windows does).

Will this change the way = os signals are handled then? Will it break compatibility if a library us= es pcntl traps and I'm using true async traps too? Note there are severa= l different ways (timeout) signals are handled in PHP -- so if (per-chan= ce) the scheduler could always be running, maybe we can unify the way si= gnals are handled in php.

Code tha= t uses Resume cannot rely on when exactly the Fibe= r will r= esume execution.

What i= f it never resumes at all? Will it call a finally block if it is try/cat= ched or will execution just be abandoned? Is there some way to ensure cl= eanup of resources? It should probably mention this case and how abandon= ing execution works.

<= div>If an excepti= on is thrown inside a fiber and not handled, it will stop the Scheduler = and be thrown at the point where Async\launchScheduler() is called.

The RFC doesn't mention the stack trace. Will= it throw away any information about the inner exception?
=
The Graceful Shutdown mode can also be tr= iggered using the function:

=
What will calling `exit` or `die` do?

A concurrent runtime allows handling requests using Fibers, whe= re each Fiber can process its own request. In this case, storing request= -associated data in global variables is no longer an option.
<= /div>

Why is this the case? Furthermore,= if it inherits from the fiber that started its current fiber, won't usi= ng Resume/Notifier potentially cause problems when used manually? There = are examples over the RFC using global variables in closures; so do thes= e examples not actually work? Will sharing instances of objects in scope= of the functions break things? For example:

Async\run($obj->method1(...));
Async\run($obj->met= hod2(...));

This is technically sharing glo= bal variables (well, global to that scope -- global is just a scope afte= r all) -- so what happens here? Would it make sense to delegate this fib= er-local storage to user-land libraries instead?

Objects of the Future class are high-level patterns for handling deferred r= esults. 

By this p= oint we have covered FiberHandle, Resume, and Contexts. Now we have Futu= res? Can we simplify this to just Futures? Why do we need all these diff= erent ways to handle execution?

A = channel is a primitive for message exchange between Fibers.
<= br>
Why is there an `isEmpty` and `isNotEmpty` function? Would= n't `!$channel->isEmpty()` suffice?

It's= also not clear what the value of most of these function is. For example= :

i=
f ($chan->isFull()) {=0A  doSomething(); // suspends at some point in=
side? We may not know when we write the code.=0A  // chan is no longer f=
ull, or maybe it is -- who knows, but the original assumption entering t=
his branch is no longer true.=0A  ...=0A}

W= hether a channel is full or not is not really important, and if you rely= on that information, this is usually an architectural smell (at least i= n other languages). Same thing with empty or writable, or many others of= these functions. You basically just write to a channel and eventually (= or not, which is a bug and causes a deadlock) something will read it. Th= e entire point is to use channels to decouple async code, but most of th= e functions here allow for code to become strongly coupled.

As for the single producer method, I am not sure why yo= u would use this. I can see some upside for the built-in constraints (po= tentially in a dev-mode environment) but in a production system, single-= producer bottlenecks are a real thing that can cause serious performance= issues. This is usually something you explicitly want to avoid.

In addition to the send/receive methods, which suspend the = execution of a Fiber, t= he channel also provides non-blocking methods: trySend, tryReceive, and auxiliary explicit blocking methods: waitUntilWritable and <= /span>waitUntilReadable.&nbs= p;

It isn't clear what happens= when `trySend` fails. Is this an error or does nothing?
=
Thinking through it, there may be cases where `trySend` i= s valid, but more often than not, it is probably an antipattern. I canno= t think of a valid reason for `tryReceive` and it's usage is most likely= guaranteed to cause a deadlock in real code. For true multi-threaded ap= plications, it makes more sense, but not for single-threaded concurrency= like this.

In other words, the following c= ode is likely to be more robust, and not depend on execution order (whic= h we are told at the beginning not to do):

Async\run(function() {=0A    $channel =3D new Async\Channel();=0A =0A    $reader=
 =3D Async\async(fun=
ction() u=
se(<=
/span>$channel<=
/span>) =
{=0A    =
    while=
 ($data =3D $channel->read()=
 && $data !=3D=3D NULL) {=0A            echo "receive: $data\n";=0A        }=0A    });=0A =0A    for ($i =3D 0; $i 4<=
/span>; =
$i++) {=0A        echo "send: event data $i\n";=0A        =
$data =3D $channel->send("event data $i");=0A    }=0A   =20=0A    $rea=
der->cancel(); // clean up our reader=0A    // or=0A    $channel->=
close(); // will receive NULL I believe?=0A});

A `tr= ySend` is still useful when you want to send a message but don't want to= block if it is full. However, this is going to largely depend on how lo= ng is has been since the developer last suspended the current fiber, and= nothing else -- thus it is probably an antipattern since it totally dep= ends on the literal structure of the code, not the structure of the prog= ram -- if that makes sense.

This m= eans that tra= pSignal&nb= sp;is not intended for =E2=80=9Cregular code=E2=80=9D and should not be = used =E2=80=9Canywhere=E2=80=9D.

<= /div>
Can you expand on what this means in the RFC? Why expose it if= it shouldn't be used?

-----
=
I didn't go into the low level api details yet -- this em= ail is already pretty long. But I would suggest maybe thinking about how= to unify Notifiers/Resume/FiberHandle/Future into a single thing. These= things are pretty similar to one another (from a developer's standpoint= ) -- a way to continue execution, and they all offer a slightly differen= t api.

I also noticed that you seem to be r= elying heavily on the current implementation to define behavior. Ideally= , the RFC should define behavior and the implementation implement that b= ehavior as described in the RFC. In other words, the RFC is used as a re= ference point as to whether something is a bug or an enhancement in the = future. There has been more than once where the list looks back at an ol= d RFC to try and determine the intent for discovering if something is wo= rking as intended or a bug. RFCs are also used to write documentation, s= o the more detailed the RFC, the better the documentation will be for ne= w users of PHP.

=E2=80=94 R= ob
--ab8c71c44ab64b6a96dedf5bf5850612--