Good day, everyone. I hope you're doing well.
I’m happy to present the fourth version of the RFC. It wasn’t just me
who worked on it — members of the PHP community contributed as well.
Many thanks to everyone for your input!
https://wiki.php.net/rfc/true_async
What has changed in this version?
The RFC has been significantly simplified:
- Components (such as TaskGroup) that can be discussed in separate
RFCs have been removed from the current one. - Coroutines can now be created anywhere — even inside shutdown_function.
- Added Memory Management and Garbage Collection section
Although work on the previous API RFC was interrupted and we weren’t
able to include it in PHP 8.5, it still provided valuable feedback on
the Async API code.
During this time, I managed to refactor and optimize the TrueAsync
code, which showed promising performance results in I/O scenarios.
A test integration between NGINX UNIT and the TrueAsync API
was implemented to evaluate the possibility of using PHP as an
asynchronous backend for a web server:
https://github.com/EdmondDantes/nginx-unit/tree/true-async/src/true-async-php
During this time, the project has come very close to beta status.
Once again, I want to thank everyone who supported me during difficult
times, offered advice, and helped develop this project.
Given the maturity of both the code and the RFC, this time I hope to
proceed with a vote.
Wishing you all a great day, and thank you for your feedback!
Good day, everyone. I hope you're doing well.
I’m happy to present the fourth version of the RFC. It wasn’t just me
who worked on it — members of the PHP community contributed as well.
Many thanks to everyone for your input!https://wiki.php.net/rfc/true_async
What has changed in this version?
The RFC has been significantly simplified:
- Components (such as TaskGroup) that can be discussed in separate
RFCs have been removed from the current one.- Coroutines can now be created anywhere — even inside shutdown_function.
- Added Memory Management and Garbage Collection section
Although work on the previous API RFC was interrupted and we weren’t
able to include it in PHP 8.5, it still provided valuable feedback on
the Async API code.During this time, I managed to refactor and optimize the TrueAsync
code, which showed promising performance results in I/O scenarios.A test integration between NGINX UNIT and the TrueAsync API
was implemented to evaluate the possibility of using PHP as an
asynchronous backend for a web server:https://github.com/EdmondDantes/nginx-unit/tree/true-async/src/true-async-php
During this time, the project has come very close to beta status.
Once again, I want to thank everyone who supported me during difficult
times, offered advice, and helped develop this project.Given the maturity of both the code and the RFC, this time I hope to
proceed with a vote.Wishing you all a great day, and thank you for your feedback!
Hi, I am so looking forward to this capability!
Just a quick question - other methods that tried to provide async/parallel
type functionality previously were only available via the CLI.
I can see a big opportunity for people running websites with Apache +
PHP-FPM where on each page request you do stuff like:
Call API 1 (e.g. external auth component)
Call API 2 (e.g. product catalogue)
Call API 3 (e.g. setup payment processor)
Am hoping that you could put these three calls within a Scope and therefore
have all three calls run at the same time, and only have to wait as long as
the slowest API, rather than the combination of all 3 response times.
I didn't see anything in the RFC about this, so just wanted to check.
Thanks,
Adam
Hello.
Just a quick question - other methods that tried to provide async/parallel type functionality previously were only available via the CLI.
TrueAsync itself is integrated into PHP in such a way that it is
always active. The scenario you described is technically possible (Of
course, this can also be useful for sending telemetry in a way that
doesn’t interfere with request processing.), but it’s not particularly
relevant in the context of modern development.
Why?
Because client requests are usually processed sequentially, step by
step. Parallel tasks are rare. Therefore, from the server’s
perspective, the main benefit of concurrency is the ability to handle
multiple requests within a single process. The same thing that Swoole,
AMPHP, and other modern backend solutions do.
And this is one of the reasons why FPM is morally outdated and
therefore not used in stateful backends. That’s why you encounter CLI
so often.
Hello.
Just a quick question - other methods that tried to provide async/parallel type functionality previously were only available via the CLI.
TrueAsync itself is integrated into PHP in such a way that it is
always active. The scenario you described is technically possible (Of
course, this can also be useful for sending telemetry in a way that
doesn’t interfere with request processing.), but it’s not particularly
relevant in the context of modern development.Why?
Because client requests are usually processed sequentially, step by
step. Parallel tasks are rare.
This is simply not true. The example you're replying to is quite common.
It's even more common for the database. WordPress, Drupal, and many other such systems frequently run different DB queries to build different components of a page. (Blocks, widgets, components, the names differ.) Being able to do those in parallel is a natural optimization that we were thinking about in Drupal nearly 15 years ago, but it wasn't viable at the time.
Therefore, from the server’s
perspective, the main benefit of concurrency is the ability to handle
multiple requests within a single process.
That is A benefit. It is not the only benefit. Being able to compress the time of each request in a shared-nothing model is absolutely valuable.
Remember, in the wild, PHP-FPM and mod_php are by orders of magnitude the most common ways PHP is executed. React, Swoole, etc. are rounding errors in most of the market. And the alternate runtime with the most momentum is FrankenPHP, which reuses processes but is still "one request in a process at a time."
The same thing that Swoole,
AMPHP, and other modern backend solutions do.And this is one of the reasons why FPM is morally outdated and
I am going to assume this is a translation issue, because "morally outdated" is the wrong term here. "Morally outdated" is how you'd describe "racial segregation is good, actually." Not "this technology is slower than we need it to be." You probably mean "severely outdated" or something along those lines.
Which, as I explained above, is simply not true. PHP is going to be running in a mostly shared-nothing environment for the foreseeable future. Those use cases still would benefit from async support.
--Larry Garfield
Hi.
This is simply not true. The example you're replying to is quite common.
It’s probably my poor English. So I’ll try to rephrase the idea:
The majority of database queries are executed sequentially, step by
step. Not all queries. Not always. But most of them.
This is true even in languages that already have async.
That is A benefit. It is not the only benefit. Being able to compress the time of each request in a shared-nothing model is absolutely valuable.
(It’s important not to overestimate this model, otherwise lately you
sometimes hear complaints that the ultra-trendy immutable philosophy
leads to terrible performance :))
A stateful worker does not automatically mean active sharing of state
between requests. It gives the developer the choice of what can and
cannot be shared. You have a choice. If you want all services to
follow the immutable model — you can do that. But now you don’t have
to pay for compilation or initialization. You have complete creative
freedom.
Remember, in the wild, PHP-FPM and mod_php are by orders of magnitude the most common ways PHP is executed. React, Swoole, etc. are rounding errors in most of the market. And the alternate runtime with the most momentum is FrankenPHP, which reuses processes but is still "one request in a process at a time."
Almost no one wants to spend time building code with a technology that
isn’t supported. So when people want to do things like that, they
simply choose another language.
I’m not saying that async isn’t supported in CGI mode... but..
it’s just that a gain of a few milliseconds is unlikely to be noticeable.
I am going to assume this is a translation issue, because "morally outdated" is the wrong term here.
Thank you! That’s true. But a more accurate translation would be: it’s
a technology that has become outdated not because of time, but because
the circumstances and requirements have changed. Back in the years
when CGI was evolving, things were different. There were no servers
with a dozen cores.
Hi!
Hi.
This is simply not true. The example you're replying to is quite common.
It’s probably my poor English. So I’ll try to rephrase the idea:
The majority of database queries are executed sequentially, step by
step. Not all queries. Not always. But most of them.
This is true even in languages that already have async.
I find this a bit confusing to contextualize. I agree that most code
written was probably written following the principle of 1-query-at-a-time,
even in languages that already support async. But at the same time, if
you're tasked with optimizing the time it takes for a certain HTTP Endpoint
to execute then caching data OR rethinking query execution flow are among
the top contenders for change. What I'm trying to say is that I would look
at this from a different lensis. You're right that just because async is
already available doesn't mean that queries will take advantage of it by
default. But for the critical parts of a system that requires optimization
of the execution duration, having async capabilities can easily drive the
decision of how the code will be restructured to fulfill the need for
performance improvements.
That is A benefit. It is not the only benefit. Being able to
compress the time of each request in a shared-nothing model is absolutely
valuable.
(It’s important not to overestimate this model, otherwise lately you
sometimes hear complaints that the ultra-trendy immutable philosophy
leads to terrible performance :))A stateful worker does not automatically mean active sharing of state
between requests. It gives the developer the choice of what can and
cannot be shared. You have a choice. If you want all services to
follow the immutable model — you can do that. But now you don’t have
to pay for compilation or initialization. You have complete creative
freedom.
Talking about stateful workers and shared-state here is a bit ambiguous, at
least for me, tbh. When you say the developer has a choice, my
interpretation is that you mean to say that the PHP Developer can choose
what to share and what not to share by defining static variables, much like
most other languages implement the Singleton Pattern. In PHP, especially
with the share-nothing model, even static variables are cleared out. While
there's no denying that there is value in creating a shareable space for
performance gains, and this is seen in popularization of Swoole, Laravel
Octane, FrankenPHP Worker Mode, etc; there's still a point to be made about
the fact that 30 years worth of PHP code exists in the wild assuming that
static variables gets cleared out between requests and as such are a
non-trivial task to port them to newer execution models. This is where I
think Larry's point comes strong with the fact that most these new "modern
/ non-legacy" execution models are just a rounding error in the amount of
PHP code being executed everyday and where support of async execution for
FPM would be a game changer for code that is too hard to lift-and-shift
into worker mode, but not so hard to make adjustments in the next PHP
upgrade to e.g. parallelize database queries.
Remember, in the wild, PHP-FPM and mod_php are by orders of magnitude
the most common ways PHP is executed. React, Swoole, etc. are rounding
errors in most of the market. And the alternate runtime with the most
momentum is FrankenPHP, which reuses processes but is still "one request in
a process at a time."Almost no one wants to spend time building code with a technology that
isn’t supported. So when people want to do things like that, they
simply choose another language.
I’m not saying that async isn’t supported in CGI mode... but..
it’s just that a gain of a few milliseconds is unlikely to be noticeable.
If you have a report that executes 3 queries and each query averages
between 4 to 5 seconds, this report takes up to 15 seconds to run in PHP.
The capability of executing async code in FPM would mean a 3x performance
gain on a report like this. That is far from just a few milliseconds gain.
And to be honest, the biggest gain for me would be the ability to keep the
applications contextual logic within a single execution unit. One very
common route that is taken with today's options is to break those 3 queries
into separate HTTP endpoints and let the frontend stitch them together
which provides a very similar performance gain by taking advantage of
JS/Browser parallel requests since PHP is unable to do so.
--
Marco Deleu
Deleu deleugyn@gmail.com hat am 06.10.2025 19:29 CEST geschrieben:
Hi!
On Mon, Oct 6, 2025 at 1:43 PM Edmond Dantes <edmond.ht@gmail.com mailto:edmond.ht@gmail.com> wrote: > > Hi.
This is simply not true. The example you're replying to is quite common.
It’s probably my poor English. So I’ll try to rephrase the idea:
The majority of database queries are executed sequentially, step by
step. Not all queries. Not always. But most of them.
This is true even in languages that already have async. >
I find this a bit confusing to contextualize. I agree that most code written was probably written following the principle of 1-query-at-a-time, even in languages that already support async. But at the same time, if you're tasked with optimizing the time it takes for a certain HTTP Endpoint to execute then caching data OR rethinking query execution flow are among the top contenders for change. What I'm trying to say is that I would look at this from a different lensis. You're right that just because async is already available doesn't mean that queries will take advantage of it by default. But for the critical parts of a system that requires optimization of the execution duration, having async capabilities can easily drive the decision of how the code will be restructured to fulfill the need for performance improvements.That is A benefit. It is not the only benefit. Being able to compress the time of each request in a shared-nothing model is absolutely valuable.
(It’s important not to overestimate this model, otherwise lately you
sometimes hear complaints that the ultra-trendy immutable philosophy
leads to terrible performance :))A stateful worker does not automatically mean active sharing of state
between requests. It gives the developer the choice of what can and
cannot be shared. You have a choice. If you want all services to
follow the immutable model — you can do that. But now you don’t have
to pay for compilation or initialization. You have complete creative
freedom. >
Talking about stateful workers and shared-state here is a bit ambiguous, at least for me, tbh. When you say the developer has a choice, my interpretation is that you mean to say that the PHP Developer can choose what to share and what not to share by defining static variables, much like most other languages implement the Singleton Pattern. In PHP, especially with the share-nothing model, even static variables are cleared out. While there's no denying that there is value in creating a shareable space for performance gains, and this is seen in popularization of Swoole, Laravel Octane, FrankenPHP Worker Mode, etc; there's still a point to be made about the fact that 30 years worth of PHP code exists in the wild assuming that static variables gets cleared out between requests and as such are a non-trivial task to port them to newer execution models. This is where I think Larry's point comes strong with the fact that most these new "modern / non-legacy" execution models are just a rounding error in the amount of PHP code being executed everyday and where support of async execution for FPM would be a game changer for code that is too hard to lift-and-shift into worker mode, but not so hard to make adjustments in the next PHP upgrade to e.g. parallelize database queries.Remember, in the wild, PHP-FPM and mod_php are by orders of magnitude the most common ways PHP is executed. React, Swoole, etc. are rounding errors in most of the market. And the alternate runtime with the most momentum is FrankenPHP, which reuses processes but is still "one request in a process at a time."
Almost no one wants to spend time building code with a technology that
isn’t supported. So when people want to do things like that, they
simply choose another language.
I’m not saying that async isn’t supported in CGI mode... but..
it’s just that a gain of a few milliseconds is unlikely to be noticeable. >
If you have a report that executes 3 queries and each query averages between 4 to 5 seconds, this report takes up to 15 seconds to run in PHP. The capability of executing async code in FPM would mean a 3x performance gain on a report like this. That is far from just a few milliseconds gain. And to be honest, the biggest gain for me would be the ability to keep the applications contextual logic within a single execution unit. One very common route that is taken with today's options is to break those 3 queries into separate HTTP endpoints and let the frontend stitch them together which provides a very similar performance gain by taking advantage of JS/Browser parallel requests since PHP is unable to do so.
--Marco Deleu
I'd like to mention that running queries in parallel can give better performance, but it depends on the resources available on the database server. In case disk IO is the bottleneck and a single database server is used, parallel execution can be even slower in worst case.
For MySQL/MariaDB, parallel execution normally helps (see https://dev.mysql.com/doc/refman/8.0/en/faqs-general.html#faq-mysql-support-multi-core).
For modern analytical databases using by default multiple CPU cores per query, column stores, simd and many other optimizations, parallelization is mostly not necessary since the connection time for a new connection is often slower than executing a query.
Regards
Thomas
Hi.
But for the critical parts of a system that requires optimization of the execution duration
If you want to improve performance, you need to optimize SQL queries,
not try to execute them in parallel. This can bring down the entire
database (like it did today :) )
There are only a few patterns where multiple asynchronous queries can
actually be useful. Hedged Requests for example. Question: how often
have you seen this pattern in PHP FPM applications? Probably never :)
I know it.
Right now, there are only two significant PHP frameworks that are
ready for stateful execution. And only one of them supports
asynchronous stateful execution. This situation is caused by several
reasons, and one of them is whether or not the language itself
provides support for it.
Why is stateful execution the primary environment for async? Because
async applications are servers. And FPM is not a client-server
application. It's a plugin for a server. For a very long time, PHP was
essentially just a plugin for a web server. And a client-server
application differs from a plugin in that it starts up and processes
data streams while staying in memory. Such a process has far more use
cases for async than a process that is born and dies immediately. This
is the distinction I’m referring to.
As for the issue with frameworks: a project with several tens of
thousands of lines of code was adapted for Swoole in 2–3 weeks. It
didn’t work perfectly, sometimes it would hang, but to say that it was
really difficult… no, it wasn’t. Yes, there is a problem, yes, there
are global states in places. But if the code was written with at least
some respect for SOLID principles, this can be solved using the
Context pattern. And in reality, there isn’t that much work involved,
provided the abstractions were written reasonably well.
If you have a report that executes 3 queries and each query averages between 4 to 5 seconds,
If an SQL query takes 3...5 seconds to execute, just find another developer :)
Developers of network applications (I’m not talking about PHP) have
accumulated a lot of optimization experience over many years of trial
and error — everything has long been known. Swoole, for example, has a
huge amount of experience, having essentially made the classic R/W
worker architecture a standard in its ecosystem.
Of course, you might say that there are simple websites for which FPM
is sufficient. But over the last two years, even for simple sites,
there’s TypeScript — and although its ecosystem may be weaker, the
language may be more complex for some people, and its performance
slightly worse — it comes with async, WebSockets, and a single
language for both frontend and backend out of the box (a killer
feature). And this trend is only going to grow stronger.
Commercial development of mid-sized projects is the only niche that
cannot be lost. These guys need Event-Driven architecture, telemetry,
services. And they ask the question: why choose a language that
doesn’t support modern technologies. Async is needed specifically for
those technologies, not for FPM.
Of course, you might say that there are simple websites for which FPM
is sufficient. But over the last two years, even for simple sites,
there’s TypeScript — and although its ecosystem may be weaker, the
language may be more complex for some people, and its performance
slightly worse — it comes with async, WebSockets, and a single
language for both frontend and backend out of the box (a killer
feature). And this trend is only going to grow stronger.Commercial development of mid-sized projects is the only niche that
cannot be lost. These guys need Event-Driven architecture, telemetry,
services. And they ask the question: why choose a language that
doesn’t support modern technologies. Async is needed specifically for
those technologies, not for FPM.
We must have a different definition of mid-sized, because FPM has been used for numerous mission critical large sites, like government and university sites, and has been fine. And such sites still benefit from faster telemetry, logging, etc.
Regardless, we can quibble about the percentages and what people "should" do; those are all subjective debates.
The core point is this: Any async approach in core needs to treat the FPM use case as a first-class citizen, which works the same way, just as reliably, as it would in a persistent CLI command. That is not negotiable.
If for no other reason than avoiding splitting the ecosystem into async/CLI and sync/FPM libraries, which would be an absolute disaster.
--Larry Garfield
The core point is this: Any async approach in core needs to treat the FPM use case as a first-class citizen, which works the same way, just as reliably, as it would in a persistent CLI command. That is not negotiable.
If for no other reason than avoiding splitting the ecosystem into async/CLI and sync/FPM libraries, which would be an absolute disaster.
I 100% agree. In fact, perhaps the single biggest benefit of having a
core async model would be to reverse the current fragmentation of
run-times and libraries.
If you want to improve performance, you need to optimize SQL queries,
not try to execute them in parallel. This can bring down the entire
database (like it did today 🙂 )
You talk as though "the database" is a single resource, which can't be
scaled out. That's not the case if you have a scalable cluster of
SQL/relational databases, or a dynamically sharded NoSQL/document-based
data store, or are combining data from unconnected sources.
a project with several tens of
thousands of lines of code was adapted for Swoole in 2–3 weeks. It
didn’t work perfectly, sometimes it would hang ...
2-3 weeks of development to get to something that's not even production
ready is a significant investment. If your application's performance is
bottlenecked on external I/O (e.g. data stores, API access), the
immediate gain is probably not worth it.
For those applications, the only justification for that investment is
that it unlocks a further round of development to use asynchronous I/O
on those bottlenecks. What would excite me is if we can get extensions
and libraries to a point where we can skip the first part, and just add
async I/O to a shared-nothing application.
--
Rowan Tommins
[IMSoP]
In my opinion, PHP must add asynchronous and concurrent support as soon as possible, and asynchronous IO must be regarded as a first-class citizen.
Over the past few decades, the one-process-one-request model of PHP-FPM has been remarkably successful; it is simple and reliable. However, modern web applications do much more than merely reading from databases or caches, or handling internal HTTP requests—frequent cross-domain requests have become the norm.
The response times for these external HTTP calls are often unpredictable. Under the PHP-FPM model, delays or timeouts from certain external APIs can easily trigger a cascading failure, bringing down the entire system.
Since the emergence of ChatGPT in 2024, many software systems have been trying to integrate AI models from OpenAI, Anthropic, Google Gemini, and others.
These APIs often take tens of seconds to respond, and PHP-FPM with multi-process is almost unavailable in such scenarios.
Only asynchronous I/O offers a real solution to these challenges.
Wordpress, as the PHP application with the largest number of users, may need to add LLM capabilities in the future.
If PHP cannot provide support, Wordpress developers may also consider abandoning PHP and using other programming languages that support asynchronous IO for refactoring.
PHP must set aside its past achievements and fully embrace the Async IO tech stack.
Tianfeng Han
10/10/2025
------------------ Original ------------------
From: "Rowan Tommins [IMSoP]"<imsop.php@rwec.co.uk>;
Date: Tue, Oct 7, 2025 05:26 AM
To: "php internals"<internals@lists.php.net>;
Subject: Re: [PHP-DEV] PHP True Async RFC Stage 4
> The core point is this: Any async approach in core needs to treat the FPM use case as a first-class citizen, which works the same way, just as reliably, as it would in a persistent CLI command. That is not negotiable.
>
> If for no other reason than avoiding splitting the ecosystem into async/CLI and sync/FPM libraries, which would be an absolute disaster.
I 100% agree. In fact, perhaps the single biggest benefit of having a
core async model would be to reverse the current fragmentation of
run-times and libraries.
> If you want to improve performance, you need to optimize SQL queries,
> not try to execute them in parallel. This can bring down the entire
> database (like it did today 🙂 )
You talk as though "the database" is a single resource, which can't be
scaled out. That's not the case if you have a scalable cluster of
SQL/relational databases, or a dynamically sharded NoSQL/document-based
data store, or are combining data from unconnected sources.
> a project with several tens of
> thousands of lines of code was adapted for Swoole in 2–3 weeks. It
> didn’t work perfectly, sometimes it would hang ...
2-3 weeks of development to get to something that's not even production
ready is a significant investment. If your application's performance is
bottlenecked on external I/O (e.g. data stores, API access), the
immediate gain is probably not worth it.
For those applications, the only justification for that investment is
that it unlocks a further round of development to use asynchronous I/O
on those bottlenecks. What would excite me is if we can get extensions
and libraries to a point where we can skip the first part, and just add
async I/O to a shared-nothing application.
--
Rowan Tommins
[IMSoP]
Hello.
In my opinion, PHP must add asynchronous and concurrent support as soon as possible, and asynchronous IO must be regarded as a first-class citizen.
Thank you for your words.
And especially thank you for your major contribution to the
development of asynchrony in PHP. My words may sound clichéd, but now
is a good moment to say them.
If it weren’t for the Swoole project, many PHP developers wouldn’t
have had the opportunity to try asynchronous PHP out of the box along
with a full set of tools. It was fantastic.
The experience of working with Swoole became a key source of knowledge
when creating this RFC.
I would also like to express my gratitude to the maintainer of Swow, twose.
The energy and persistence with which you have tried to make the
language better out of love for PHP deserves the utmost respect —
especially because it is backed by professionalism and technical
competence. That’s an awesome combination.
Thanks again!
Hi,
I 100% agree. In fact, perhaps the single biggest benefit of having a
core async model would be to reverse the current fragmentation of
run-times and libraries.
This is PHP FPM + TrueAsync in Docker.
You can try it. It runs with a single command, the build takes a bit
of time, but that’s because it compiles from C.
https://github.com/true-async/fpm
It’s not that I had any doubts that async would work with FPM, but it
was still necessary to verify it.
I would, of course, also like to try things like PHP Native or
something similar.
But in principle, there shouldn’t be any difference in where or how
PHP is run, because the SAPI itself doesn’t change from the
perspective of the external consumer.
Thanks, Ed.
Of course, you might say that there are simple websites for which FPM
is sufficient. But over the last two years, even for simple sites,
there’s TypeScript — and although its ecosystem may be weaker, the
language may be more complex for some people, and its performance
slightly worse — it comes with async, WebSockets, and a single
language for both frontend and backend out of the box (a killer
feature). And this trend is only going to grow stronger.Commercial development of mid-sized projects is the only niche that
cannot be lost. These guys need Event-Driven architecture, telemetry,
services. And they ask the question: why choose a language that
doesn’t support modern technologies. Async is needed specifically for
those technologies, not for FPM.We must have a different definition of mid-sized, because FPM has been
used for numerous mission critical large sites, like government and
university sites, and has been fine. And such sites still benefit
from faster telemetry, logging, etc.Regardless, we can quibble about the percentages and what people
"should" do; those are all subjective debates.The core point is this: Any async approach in core needs to treat the
FPM use case as a first-class citizen, which works the same way, just
as reliably, as it would in a persistent CLI command. That is not
negotiable.If for no other reason than avoiding splitting the ecosystem into
async/CLI and sync/FPM libraries, which would be an absolute disaster.
I also agree with this.
I tried reading the RFC today, but I ran out of time. It is 59 page
printed (I didn't).
I think we need to be very careful that we do not introduce a feature
that allows our users to run into all sorts of problems. The symantics
of such a complex feature are going to be really important. Especially
about reasoning in which direction the code runs and flows, and how
errors are treated.
I recently read
https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
which seems like an entirely sensible way of proceeding. Although the
title talks about Go and its problems, the "Structured Concurrency"
approach is more of a way of doing concurrency right, without the
possibility of our users getting into trouble.
I don't think the RFC as-is is close to this at all — but I have mostly
skimmed it so far.
I would also believe that discussion how this should work would work
better with a group of people - preferably in real-time - and not as an
idea and implementation of a single person. I know others have been
reviewing and commenting on it, but I don't think that's quite the same.
Concurrency in all its forms is a complex subject, and we can't really
get this wrong as we'll have to live with the concepts for a long time.
cheers,
Derick
--
https://derickrethans.nl | https://xdebug.org | https://dram.io
Author of Xdebug. Like it? Consider supporting me: https://xdebug.org/support
mastodon: @derickr@phpc.social @xdebug@phpc.social
Hello.
I tried reading the RFC today, but I ran out of time. It is 59 page printed (I didn't).
...
I don't think the RFC as-is is close to this at all — but I have mostly skimmed it so far.
Thank you for the feedback.
This time there will be a vote. If this RFC is not accepted, I promise
that I will not create a fifth version. So if anyone has something to
say, please feel free to speak openly. Please.
Hello.
I tried reading the RFC today, but I ran out of time. It is 59 page printed (I didn't).
...
I don't think the RFC as-is is close to this at all — but I have mostly skimmed it so far.Thank you for the feedback.
This time there will be a vote. If this RFC is not accepted, I promise
that I will not create a fifth version. So if anyone has something to
say, please feel free to speak openly. Please.
Like Derick, I am still highly skeptical about this design. It's vastly improved from the first version back in the spring, but there are still numerous footguns in the design that will lead me to voting No on its current iteration. Mainly, we should not be allowing anything but structured, guaranteed async blocks (as described in the article Derick linked). It is still perfectly possible to build completely-async systems that way, but it prevents writing code that would only work in such an all-encompassing system.
I very much want to see it evolve further in that direction before a vote is called and we're locked into a system with so many foot guns built in.
--Larry Garfield
It's vastly improved from the first version back in the spring, but there are still numerous footguns
Which specific footguns? $scope var?
Mainly, we should not be allowing anything but structured
If you always follow the “nursery” rules, you always have to define a
coroutine just to create a nursery, even when the coroutine itself
isn’t actually needed.
That’s why we end up with hacks like “Task.detached.” (Swift). And
Kotlin keeps trying to invent workarounds.
TrueAsync RFC takes a different approach and gives the programmer
maximum flexibility while still complying with every principle of
structured concurrency.
At the same time, the programmer gains two styles of structured
concurrency organization, one of which fully matches Trio.
I very much want to see it evolve further in that direction before a vote is called and we're locked into a system with so many foot guns built in.
Such an approach would require more changes to the code, and I don’t
see how it would protect the programmer from mistakes any better than
this RFC does.
Of course, the with-style syntax would allow for maximum safety when
working with tasks, but that’s not an issue with this RFC.
The Trio model is not perfect; Kotlin and other languages do not
adopt it (maybe by accident — or maybe not).
It’s not suitable for all types of tasks, which is why the criticism is valid.
Although Kotlin is criticized for storing Scope inside objects like:
“Long-living CoroutineScope stored in objects almost always lead to
resource leaks or forgotten jobs.”
However, there is no other way to solve the problem when coroutines
need to be launched within a Scope gradually rather than all at once.
But... Ok...
async def background_manager():
async with trio.open_nursery() as nursery:
while True:
event = await get_next_event()
nursery.start_soon(handle_event, event)
^^^
The example of how the pursuit of an “ideal” ends up producing ugly code.
My position is this: TrueAsync should support the best patterns
for specific use cases while still remaining convenient for the
majority of tasks.
The fact that certain tools require careful handling applies to all
programming languages. That doesn’t mean those tools shouldn’t exist.
It's vastly improved from the first version back in the spring, but there are still numerous footguns
Which specific footguns? $scope var?
Mainly, we should not be allowing anything but structured
If you always follow the “nursery” rules, you always have to define a
coroutine just to create a nursery, even when the coroutine itself
isn’t actually needed.
That’s why we end up with hacks like “Task.detached.” (Swift). And
Kotlin keeps trying to invent workarounds.TrueAsync RFC takes a different approach and gives the programmer
maximum flexibility while still complying with every principle of
structured concurrency.
At the same time, the programmer gains two styles of structured
concurrency organization, one of which fully matches Trio.I very much want to see it evolve further in that direction before a vote is called and we're locked into a system with so many foot guns built in.
Such an approach would require more changes to the code, and I don’t
see how it would protect the programmer from mistakes any better than
this RFC does.
Of course, the with-style syntax would allow for maximum safety when
working with tasks, but that’s not an issue with this RFC.The Trio model is not perfect; Kotlin and other languages do not
adopt it (maybe by accident — or maybe not).
It’s not suitable for all types of tasks, which is why the criticism is valid.Although Kotlin is criticized for storing Scope inside objects like:
“Long-living CoroutineScope stored in objects almost always lead to
resource leaks or forgotten jobs.”
However, there is no other way to solve the problem when coroutines
need to be launched within a Scope gradually rather than all at once.But... Ok...
async def background_manager(): async with trio.open_nursery() as nursery: while True: event = await get_next_event() nursery.start_soon(handle_event, event)^^^
The example of how the pursuit of an “ideal” ends up producing ugly code.My position is this: TrueAsync should support the best patterns
for specific use cases while still remaining convenient for the
majority of tasks.
The fact that certain tools require careful handling applies to all
programming languages. That doesn’t mean those tools shouldn’t exist.
--
https://derickrethans.nl | https://xdebug.org | https://dram.io
Author of Xdebug. Like it? Consider supporting me: https://xdebug.org/support
mastodon: @derickr@phpc.social @xdebug@phpc.social
Hello.
It seems there was supposed to be some text here… but perhaps an error
occurred. I only see a quote.
Best regards, Ed
Hello.
It seems there was supposed to be some text here… but perhaps an error
occurred. I only see a quote.Best regards, Ed
Yeah, sorry. I sent it by mistake when starting to write it. The real email will follow.
cheers
Derick
Hi,
I have now read more thoroughly through the whole RFC, and I am less
happier with it than I was.
It's vastly improved from the first version back in the spring, but
there are still numerous footgunsWhich specific footguns? $scope var?
There are many as I see them, to name a few:
- A new ini setting — we would be much better off picking a value, and
stick with with it. - Showing warnings when all sort of issues happen.
- Indeed, the passing around of "$scope".
- "A good practice is to ensure that a Scope object has only ONE owner."
— that really ought to be enforced - "Passing $scope as a parameter to other functions or assigning it to multiple
objects is a potentially dangerous operation that can lead to complex bugs."
— so this isn't something a design should even allow. - "Awaiting a Scope is a potentially dangerous operation that should be
performed consciously, not accidentally. " — a design should prevent PHP
users from making potentially dangerous situations. - Disposing etc, I don't think I even understand what you're trying to do with
this, considering warnings, zombies, etc. - "Warning: You should not attempt to suppress CancellationError exception, as
it may cause application malfunctions." — again, a good design should not
require our users to have to think of this. - Hierarchies of scopes make for a lot of complexity. I doubt whether this is
useful in most cases.
There is a lot of complexity and I still also think this RFC is trying to do
way too much. PHP is primarily used as a web language, and the goal of adding
concurrency into it should make the live of web developers easier. This seems
to be designed fo running PHP as a long time running script/event loop. We have
web servers and FrankenPHP for that.
In addition:
The RFC also doesn't describe how scheduling works. You call it "out of
scope", but as this is needed for this whole concept to work, it very
much ought to be in scope.
In the second example of "Coroutine::onFinally" there is an onFinally() call
without object.
Under "Tools", there is: "getSuspendFileAndLine():array Returns an array of
two elements: the file name and the line number where the coroutine was last
suspended. If the coroutine has not been suspended, it may return empty
string,0." Returning bogus values (empty string, and 0) isn't a good idea.
Instead, the method should probably have a "?array" return type.
"The format of this array depends on the implementation of the Scheduler and
the Reactor." — Spell out what these are, and how they're supposed to work. We
cannot make decisions on non-complete information.
"Therefore, using concurrency is reasonable only for long-life scenarios
implemented via CLI." — I thought the whole point of this was so that
developers can use concurrent things during their web requests — for example
running several queries at the same time, or doing multiple http requests.
Isn't that what you would expect people to use concurrency for?
I don't understand why the "NGINX Unit integration example" is part of this RFC.
I very much want to see it evolve further in that direction before a vote
is called and we're locked into a system with so many foot guns built in.Such an approach would require more changes to the code, and I don’t
see how it would protect the programmer from mistakes any better than
this RFC does.
In my opinion, it was not wise to spend so much time on the code (beyond
prototyping). The role of the RFC process is to hammer out a good feature for
PHP, with the code being (mostly) an implementation detail.
My position is this: TrueAsync should support the best patterns
for specific use cases while still remaining convenient for the
majority of tasks.
The "Goals" describe what you would like this feature to be, but not why. The
RFC does not describe why this feature is important for PHP for specific use
cases at all.
cheers,
Derick
--
https://derickrethans.nl | https://xdebug.org | https://dram.io
Author of Xdebug. Like it? Consider supporting me: https://xdebug.org/support
mastodon: @derickr@phpc.social @xdebug@phpc.social
Hello
Hierarchies of scopes make for a lot of complexity. I doubt whether this is useful in most cases.
https://dl.acm.org/doi/10.1145/3547276.3548519
Here’s a short article about the basics of asynchrony and structured
concurrency.
Please look at icppworkshops22-13-fig4.jpg
The logic of a Scope largely mirrors that of variable scopes. There’s
a direct logical parallel here: a scope defines a variable’s lifetime.
In asynchronous programming, it’s often necessary to have multiple
scopes that depend on each other in a parent → child relationship.
That’s what structured concurrency is.
There is a lot of complexity and I still also think this RFC is trying to do way too much.
Why is that even a problem? There are many complex things in
programming, and languages support complex features for example,
classes, inheritance, and traits. PHP even plans to introduce
generics, which are very sophisticated abstractions.
If we follow the logic of “nothing complex,” then PHP should remove
all advanced abstractions and become a language only for "simple
websites".
But I think many developers would disagree because they want to make money.
And if a language doesn’t support complex abstractions, it limits their career.
Of course, I could have made this RFC similar to Go: with only
coroutines and channels.
But then there would be no way to limit the lifetime of coroutines
from code that doesn’t know about them.
I could also have made it more like Kotlin: but then it would have
been even more complex.
Instead, this RFC was designed specifically to fit PHP.
The current approach Scope + Cancellation policy makes
asynchronous code safer and simpler.
It allows monitoring all coroutines from a single place in the code
(One point of responsibility).
Just look at the code examples with CancellationToken to see the
benefits these tools provide.
Without Scope, the programmer would have to manually track different
coroutines and pass cancellation tokens into each of them to limit
their lifetime.
I’ve had that experience and I know the true cost of such complexity.
and the goal of adding concurrency into it should make the live of web developers easier
Asynchronous programming doesn’t make a developer’s life easier, quite
the opposite.
Asynchrony is one of the most complex areas that programmers have
successfully avoided for many years.
If a developer approaches asynchrony thinking it will make their life
easier, it means they don’t understand it at all.
Concurrency is used to squeeze the maximum performance out of the CPU
by reducing the number of kernel API calls and keeping the processor
busy with useful work.
That’s the main reason. Asynchrony requires additional effort from
developers because modern backend solutions operate under much heavier
loads and use far more complex interaction algorithms than they did
ten years ago. The real truth is that it didn’t matter much before,
but now it does.
This seems to be designed fo running PHP as a long time running script/event loop. We have web servers and FrankenPHP for that.
FrankenPHP doesn’t make PHP asynchronous.
The RFC also doesn't describe how scheduling works. You call it "out of
scope", but as this is needed for this whole concept to work, it very
much ought to be in scope.
but as this is needed for this whole concept to work
Could you explain with an example what exactly wouldn’t work?
I’ll try to help you work through this question.
When you write a program, how do you account for the operating
system’s scheduler algorithm?
In the current implementation, the Scheduler is a queue based on a
circular buffer.
Swoole, for example, implements a scheduler with preemptive multitasking.
In principle, this could be done at any time and it would not affect
this RFC in any way.
"The format of this array depends on the implementation of the Scheduler and the Reactor."
I didn’t define the format of debugging information in the RFC because
it may change over time.
Besides that, the Scheduler and Reactor components can be replaced
with others the ABI allows this.
Different implementations might also want to include different
metadata, which is perfectly normal.
Isn't that what you would expect people to use concurrency for?
These questions are answered in other messages in this thread.
I don't understand why the "NGINX Unit integration example" is part of this RFC.
Sorry but I don’t understand the point of this question. What’s wrong
with the example?
In my opinion, it was not wise to spend so much time on the code (beyond
prototyping). The role of the RFC process is to hammer out a good feature for
PHP, with the code being (mostly) an implementation detail.
Years of design experience show that any complex system must be
developed together with the code, because purely abstract ideas are
always flawed.
This RFC was created using an iterative approach, where design phases
alternate with implementation phases.
For example, the rules defining where coroutines can be launched were
determined by the implementation, not by the RFC.
It was also important to assess whether the project was justified
whether it should be implemented in the PHP core at all.
Today, I know the answers to those questions.
The "Goals" describe what you would like this feature to be, but not why. The
RFC does not describe why this feature is important for PHP for specific use
cases at all.
The RFC contains many explanations of what each abstraction is used for.
Of course, I might have made a mistake or forgotten to describe something.
So if you have any questions, feel free to ask.
Best regards, Ed
Good day, everyone. I hope you're doing well.
I’m happy to present the fourth version of the RFC. It wasn’t just me
who worked on it — members of the PHP community contributed as well.
Many thanks to everyone for your input!https://wiki.php.net/rfc/true_async
What has changed in this version?
The RFC has been significantly simplified:
- Components (such as TaskGroup) that can be discussed in separate
RFCs have been removed from the current one.- Coroutines can now be created anywhere — even inside shutdown_function.
- Added Memory Management and Garbage Collection section
Although work on the previous API RFC was interrupted and we weren’t
able to include it in PHP 8.5, it still provided valuable feedback on
the Async API code.During this time, I managed to refactor and optimize the TrueAsync
code, which showed promising performance results in I/O scenarios.A test integration between NGINX UNIT and the TrueAsync API
was implemented to evaluate the possibility of using PHP as an
asynchronous backend for a web server:
https://github.com/EdmondDantes/nginx-unit/tree/true-async/src/true-async-phpDuring this time, the project has come very close to beta status.
Once again, I want to thank everyone who supported me during difficult
times, offered advice, and helped develop this project.Given the maturity of both the code and the RFC, this time I hope to
proceed with a vote.Wishing you all a great day, and thank you for your feedback!
Hello,
I’m not even half way done with a list of comments and questions, but I have one that continues to bother me while reading, so I figure I will just ask it.
Why doesn’t scope implement Awaitable?
— Rob
Hello.
Why doesn’t scope implement Awaitable?
Let’s say there’s a programmer named John who is writing a library,
and he has a function that calls an external handler.
Programmer Robert wrote the externalHandler.
function processData(string $url, callable $externalHandler): void
{
...
$externalHandler($result);
}
John knows which contracts are inside processData, but he knows
nothing about $externalHandler.
From the perspective of processData, $externalHandler acts as a
black box.
If John uses await() on that black box, it may lead to an infinite wait.
There are two solutions to this problem:
- Delegate full responsibility for the program’s behavior to
Robert, meaning to$externalHandler - Establish a limiting contract
Therefore, if a Scope needs to be awaited, it can only be done
together with a cancellation token.
In real-world scenarios, awaiting a Scope during normal execution
makes no sense, because you have a cancellation policy.
This means that at any necessary moment you can dispose() the Scope
and thus interrupt the execution of tasks inside the black box.
For the TaskGroup pattern, which exists in the third version of the
RFC, awaiting is a relatively safe operation, because in this case we
assume that the code is written by someone who has direct access to
the agreements and bears full responsibility for any errors.
So, Scope is intended for components where responsibility is shared
between different programmers, while TaskGroup should be used when
working with a clearly defined set of coroutines.
Hello.
Why doesn’t scope implement Awaitable?
Let’s say there’s a programmer named John who is writing a library,
and he has a function that calls an external handler.
Programmer Robert wrote theexternalHandler.function processData(string $url, callable $externalHandler): void { ... $externalHandler($result); }John knows which contracts are inside processData, but he knows
nothing about$externalHandler.
From the perspective ofprocessData,$externalHandleracts as a
black box.
If John usesawait()on that black box, it may lead to an infinite wait.There are two solutions to this problem:
- Delegate full responsibility for the program’s behavior to
Robert, meaning to$externalHandler- Establish a limiting contract
Therefore, if a
Scopeneeds to be awaited, it can only be done
together with a cancellation token.In real-world scenarios, awaiting a
Scopeduring normal execution
makes no sense, because you have acancellation policy.
This means that at any necessary moment you can dispose() theScope
and thus interrupt the execution of tasks inside the black box.For the
TaskGrouppattern, which exists in the third version of the
RFC, awaiting is a relatively safe operation, because in this case we
assume that the code is written by someone who has direct access to
the agreements and bears full responsibility for any errors.So,
Scopeis intended for components where responsibility is shared
between different programmers, whileTaskGroupshould be used when
working with a clearly defined set of coroutines.
I don’t get it. What does different programmers working on a program have to do with whether or not scopes implements Awaitable? Scope has an await method, it should be Awaitable. await() takes a cancellation and thus anything Awaitable can be cancelled at any time. I don’t see why scope is special in that regard.
If John uses
await()on that black box, it may lead to an infinite wait.
This is true of any software or code. Knowing whether or not something will ever complete is called The Halting Problem. It is unsolvable, in the general sense. You can await() a read of an infinite file, or a remote file that will take 5y to read because it is being read at 1 bps. Your clock can fry on your motherboard, preventing timeouts from ever firing. Your disk can die mid-read, preventing it from ever sending you any data. There is so much that can go wrong. To say that something that has an await method isn’t Awaitable because it may never return is true for ALL Awaitable tasks as well. It isn’t special.
— Rob
I don’t get it. What does different programmers working
My main point was about contracts.
Developers were used to demonstrate breaches of agreements.
A properly defined contract with a black box helps identify errors and
limit their impact.
I don’t know how to explain it more simply. These are fundamental
elements of design in IT.
I don’t get it. What does different programmers working
My main point was about contracts.
Developers were used to demonstrate breaches of agreements.
A properly defined contract with a black box helps identify errors and
limit their impact.
I don’t know how to explain it more simply. These are fundamental
elements of design in IT.
You've provided examples and said that it violates design and fundamental elements, but not which design and fundamentals, ie, the evidence for the decision. I would expect more than "because I said so" for such a huge language feature, but rather arguments grounded in computer science. People are going to ask this question, it will probably be in the docs, so, there needs to be a good answer.
— Rob
I would expect more than "because I said so"
In my response I provided arguments, but they were ignored. If you
tell me exactly what is unclear to you, I can give a concrete,
well-reasoned answer.
I would expect more than "because I said so"
In my response I provided arguments, but they were ignored. If you
tell me exactly what is unclear to you, I can give a concrete,
well-reasoned answer.
This is all I have to go off of, and my explicit rebuttals as to why they are not reasons:
Therefore, if a
Scopeneeds to be awaited, it can only be done
together with a cancellation token.
This is effectively not true. I can pass a cancellation token that never cancels. Sometimes, this is exactly the behaviour that is desired.
In real-world scenarios, awaiting a
Scopeduring normal execution
makes no sense, because you have acancellation policy.
This means that at any necessary moment you can dispose() theScope
and thus interrupt the execution of tasks inside the black box.
That sounds like a feature, not a reason.
For the
TaskGrouppattern, which exists in the third version of the
RFC, awaiting is a relatively safe operation, because in this case we
assume that the code is written by someone who has direct access to
the agreements and bears full responsibility for any errors.
We aren't talking about TaskGroups, but regardless, I fail to understand how that makes scopes 'dangerous' to await and why it should not be Awaitable.
So,
Scopeis intended for components where responsibility is shared
between different programmers, whileTaskGroupshould be used when
working with a clearly defined set of coroutines.
My question is "why does this mean something with an await method isn't Awaitable?" -- it seems that this is orthogonal to the interface and the reason why it should/should not be Awaitable.
— Rob
The quotes you provided were not my arguments.
- There is an
await()function. This function has a cancellation
token that allows limiting its lifetime. - There is a
Scope. It is undesirable to wait on the Scope without
an explicit limit, because while waiting, a reference to it is held,
and if the tasks within the Scope contain an error, the application
ends up in an undefined state. - However, if the developer explicitly defines a contract that limits
the lifetime using a cancellation token, such an operation becomes
safe. - But in await(), the cancellation token is an optional parameter.
Is that clear?
That sounds like a feature, not a reason.
There is almost nothing in this RFC or perhaps nothing at all that
exists by accident. Every class, function, every policy was created to
solve real problems.
This document is the result of long-term analysis and studying cases,
not just a “because I felt like it” approach.
You've provided examples and said that it violates design and fundamental
elements, but not which design and fundamentals, ie, the evidence for the
decision. I would expect more than "because I said so" for such a huge
language feature, but rather arguments grounded in computer science.
He very explicitly described the issue, in objective terms: a breach of
agreement in the context of the invocation of a foreign interface.
In simpler terms, you can't and should not be able to mess with the
internals of code you didn't write: this is similar to the principle of
encapsulation in OOP, where you cannot modify private properties of an
object.
Cancellation should be part of the contract of an async function: it is
safe, and most async languages already implement it explicitly by passing a
context or a cancellation token, the current approach of the RFC does it
implicitly, which is also fine.
Awaiting for the completion of spawned coroutines should not be part of
the contract of an async function: it is an incredibly easy footgun, as
it's incredibly easy to spawn a coroutine meant to i.e. run until the
object is destroyed, and then encounter a deadlock when the linked scope is
awaited for before the object is destroyed (even messier if cycles and thus
the GC are involved).
Languages like Kotlin that do implement await on scopes have already
realized that it is a mistake, as can be seen by the many warnings against
using await on a scope, as I already linked in previous emails.
On the other hand, making the same mistake described above, by cancelling a
scope, will produce a very easy to debug exception instead of a deadlock,
easily fixable by (warning the author of the library class) to use a new
scope to spawn coroutines within the object.
Awaiting a scope leads to deadlocks in case where a separate scope is
needed but not used, cancelling them leads to a simple exception.
Awaiting on multiple tasks can already be done, explicitly, with TaskGroup.
Regards,
Daniil Gentili.
You've provided examples and said that it violates design and fundamental elements, but not which design and fundamentals, ie, the evidence for the decision. I would expect more than "because I said so" for such a huge language feature, but rather arguments grounded in computer science.
He very explicitly described the issue, in objective terms: a breach of agreement in the context of the invocation of a foreign interface.
In simpler terms, you can't and should not be able to mess with the internals of code you didn't write: this is similar to the principle of encapsulation in OOP, where you cannot modify private properties of an object.
Cancellation should be part of the contract of an async function: it is safe, and most async languages already implement it explicitly by passing a context or a cancellation token, the current approach of the RFC does it implicitly, which is also fine.
Awaiting for the completion of spawned coroutines should not be part of the contract of an async function: it is an incredibly easy footgun, as it's incredibly easy to spawn a coroutine meant to i.e. run until the object is destroyed, and then encounter a deadlock when the linked scope is awaited for before the object is destroyed (even messier if cycles and thus the GC are involved).
Languages like Kotlin that do implement await on scopes have already realized that it is a mistake, as can be seen by the many warnings against using await on a scope, as I already linked in previous emails.
On the other hand, making the same mistake described above, by cancelling a scope, will produce a very easy to debug exception instead of a deadlock, easily fixable by (warning the author of the library class) to use a new scope to spawn coroutines within the object.
Awaiting a scope leads to deadlocks in case where a separate scope is needed but not used, cancelling them leads to a simple exception.
Awaiting on multiple tasks can already be done, explicitly, with TaskGroup.Regards,
Daniil Gentili.
Hey Daniil and Edmond,
I think I understand the intention. Would it be better to instead of having ->awaitCompletion (which feels like an implicit implementation of Awaitable without an explicit implmentation -- which is the part that was bothering me), maybe having something like ->joinAll() or ->joinOnCancellation()? That way someone like me won't come along and wrap them in an Awaitable because it looks Awaitable.
It also might be a good idea to make specifying a timeout in ms mandatory, instead of a taking an Awaitable/Cancellation. This would also prevent people from simply passing a "never returning" Awaitable thinking they're being clever.
It also might be good to provide a realistic looking example showing a "bad case" of how this is dangerous instead of simply saying that it is, showing how a scope is not a 'future', but a container, and preemptively mention TaskGroups, linking to the future scope (which it should also probably be listed there as well).
— Rob
Would it be better to instead of having ->awaitCompletion (which feels like an implicit implementation of Awaitable without an explicit implmentation -- which is the part that was bothering me), maybe having something like ->joinAll() or ->joinOnCancellation()?
That way someone like me won't come along and wrap them in an Awaitable because it looks Awaitable.
For example, you have an interface DataBaseAdmin with a removeDB()
method and a class that implements it.
You need to be able to delete the database, but with an additional
condition. So you create a decorator class with a method
ContextualDBAdmin::removeDBWhen($extraRules).
As a result, the decorator class is logically associated with the
DataBaseAdmin interface.
The question is: so what?
It also might be a good idea to make specifying a timeout in ms mandatory, instead of a taking an Awaitable/Cancellation.
The idea is correct, but there will definitely be someone who says
they need more flexibility.
And it's true you can create a DeferredCancellation and forget to
finish it. :)
There are a lot of such subtle points, and they can be discussed endlessly.
But I wouldn’t spend time on them.
It also might be good to provide a realistic looking example showing a "bad case" of how this is dangerous instead of simply saying that it is, showing how a scope is not a 'future',
but a container, and preemptively mention TaskGroups, linking to the future scope (which it should also probably be listed there as well).
- It’s very difficult to write a realistic example that’s still small.
- The
TaskGrouporCoroutineGroupclass is left for future
discussion. In the final documentation, it will be exactly as you
suggested.
Would it be better to instead of having ->awaitCompletion (which feels like an implicit implementation of Awaitable without an explicit implmentation -- which is the part that was bothering me), maybe having something like ->joinAll() or ->joinOnCancellation()?
That way someone like me won't come along and wrap them in an Awaitable because it looks Awaitable.For example, you have an interface
DataBaseAdminwith aremoveDB()
method and a class that implements it.
You need to be able to delete the database, but with an additional
condition. So you create a decorator class with a method
ContextualDBAdmin::removeDBWhen($extraRules).As a result, the decorator class is logically associated with the
DataBaseAdmininterface.The question is: so what?
I think we might be talking past each other a little.
You said earlier that one of the goals here is to prevent misuse (e.g. unbounded or foreign awaits). I completely agree and that’s exactly why I’m suggesting a rename. From the outside, a method called awaitCompletion() looks and behaves like an Awaitable, even though the type explicitly isn’t one. That contradiction encourages people to wrap it to “make it awaitable,” which re-introduces the very problem you’re trying to avoid.
Renaming it to something like joinAll() or joinAfterCancellation() keeps the semantics intact while communicating the intent: this isn’t a future; it’s a container join. That small change would make the interface self-documenting and harder to misuse.
It also might be a good idea to make specifying a timeout in ms mandatory, instead of a taking an Awaitable/Cancellation.
The idea is correct, but there will definitely be someone who says
they need more flexibility.
And it's true you can create aDeferredCancellationand forget to
finish it. :)There are a lot of such subtle points, and they can be discussed endlessly.
But I wouldn’t spend time on them.
That's what we are here to do, no? Discussion is useful if it makes things better than the sum of their parts.
It also might be good to provide a realistic looking example showing a "bad case" of how this is dangerous instead of simply saying that it is, showing how a scope is not a 'future',
but a container, and preemptively mention TaskGroups, linking to the future scope (which it should also probably be listed there as well).
- It’s very difficult to write a realistic example that’s still small.
I'll give it a go:
$scope = new Scope();
// Library code spawns in my scope (transitively)
$scope->spawn(fn() => thirdPartyOperation()); // may spawn more
// Looks innocent, but this can wait on foreign work:
$scope->awaitCompletion(Async\timeout(60000)); // rename -> joinAll(...)?
- The
TaskGrouporCoroutineGroupclass is left for future
discussion. In the final documentation, it will be exactly as you
suggested.
My point is that it wasn't listed in "future scope" of the RFC, though they're mentioned throughout the document, in passing.
— Rob
That contradiction encourages people to wrap it to “make it awaitable,” which re-introduces the very problem you’re trying to avoid.
Hm...
Renaming it to something like joinAll() or joinAfterCancellation() keeps the semantics intact while communicating the intent: this isn’t a future; it’s a container join.
That small change would make the interface self-documenting and harder to misuse.
There’s also an issue with the method name.
The awaitCompletion method waits not for the Scope to close, but for
all tasks to complete. That’s not the same thing.
Maybe joinAll really is better, since it resembles the way threads
are expected to work?
But does joinAll fully reflect the meaning? After all, it’s not
about waiting for all tasks in general, but only for those that
exist while the function is active. That’s a very important
distinction. Because in addition to this function, there could also be
something like awaitFinally or similar.
That's what we are here to do, no? Discussion is useful if it makes things better than the sum of their parts.
Typically, when developers start medium or large projects, they begin
with the key components intentionally leaving out details that can be
easily changed later.
There’s definitely something to discuss here, but is it really worth
doing it right now? :)
// Looks innocent, but this can wait on foreign work:
$scope->awaitCompletion(Async\timeout(60000)); // rename -> joinAll(...)?
If the developer wanted to start a task and limit its execution time,
the code looks fine.
And I would add a try/catch to handle errors.
My point is that it wasn't listed in "future scope" of the RFC, though they're mentioned throughout the document, in passing.
Yes, that was done intentionally on the advice of the PHP Team.
Hello everyone,
Tomorrow marks two weeks since the RFC was published, which means
that, formally, it is now eligible to be submitted for voting.
Allow me to briefly summarize the current state of the project:
- The implementation has been delivered both as core changes to
PHP and as a separate extension. -
More than 50 PHP functions have been adapted to work in
non-blocking mode, all of which are fully covered by tests. - There is a working integration example with a web server using
the model “react-process + embedded PHP process”. - The project is currently in an Alpha+ stage, and is available
for testing and experimentation. - The RFC v3 has been implemented almost entirely, and RFC v4
is fully covered. - The second version of the PHP Stream integration demonstrates
strong I/O performance in web server scenarios. - The code introduces asynchrony into all stages of PHP execution,
including ensuring correct GC operation. It successfully passes the
standard PHP test suite. - Supports both ZTS and non-ZTS builds.
- Separate testing was performed with XDebug (with a small patch applied).
At this stage, all technical objectives of the project have been achieved 100%.
To emphasize: the project is not yet production-ready, but it has
fully achieved its demonstration goals.
Around this time, the project turned one year old.
RFC
- A high-level API for PHP land has been developed, based on the
experience of other programming languages. - Known development pitfalls have been taken into account, and
fault-tolerant solutions have been incorporated.
At this point, all RFC objectives have been achieved.
My special thanks go to Roman Pronskiy, Jakub Zelenka, Arnaud Le
Blanc, Valentin Udaltsov as well as to everyone who supported the
project in various ways.
Dear PHP community, we are now at a decision point: what should be done next?
I am obliged to draw the attention of the PHP community to the fact
that the Swow project has existed for several years. Unlike TrueAsync,
it is more mature, and its author Twose has previously expressed
an intention to integrate it into the core.
This RFC can be used as a foundation regardless of the implementation.
So I believe that Twose’s opinion is important and should be taken
into consideration.
I recommend voting for the TrueAsync RFC with the status of “experimental”.
Although this status is not formally defined in PHP rules, it would
allow framework authors, library developers, and other maintainers to
treat this RFC and its implementation as something expected to be
adopted in the future.
Without this step, further development makes little sense.
There is no point in creating an equivalent of Swoole or Swow. These
projects already exist, are excellently built, and fulfill their
purpose.
For the moderators
It is possible that voting on this RFC may not make sense. If that is
the case, please state so within the next 2-3 days.
This would be a rational and respectful approach toward the
participants of the vote.
Thank you all, and best of luck.
Hi,
Hello everyone,
Tomorrow marks two weeks since the RFC was published, which means
that, formally, it is now eligible to be submitted for voting.
I just want to say that the amount of effort put into this is really
impressive. So, thank you!
At the same time, the reviewing effort is also big, and I can hope you will
keep this in mind. There is no need to go with the minimum period of two
weeks.
I would very much want to see this proposal succeed. I think you should
allow for a lengthier review period so that people will have enough time to
contribute their thoughts and ideas, to make sure all unclarities are
cleared up, and so, to have a better chance of being accepted.
Thank you again,
Alex
Hello.
I think you should allow for a lengthier review period so that people will have enough time to contribute their thoughts and ideas,
to make sure all unclarities are cleared up, and so, to have a better chance of being accepted.
I’m happy to allow as much time as needed. How about we extend the
review period by another two weeks?
Thank you, Ed
Hello.
I think you should allow for a lengthier review period so that people will have enough time to contribute their thoughts and ideas,
to make sure all unclarities are cleared up, and so, to have a better chance of being accepted.I’m happy to allow as much time as needed. How about we extend the
review period by another two weeks?Thank you, Ed
That would be great,
As mentioned in my last email, I was only half way through (now closer to 60%) of making notes and reviewing the proposal. This isn't a simple proposal but rather a complex one with interleaving behaviours. It takes awhile to digest and understand, plus run through various scenarios and understand how it will work.
— Rob
Good day, everyone. I hope you're doing well.
I’m happy to present the fourth version of the RFC. It wasn’t just me
who worked on it — members of the PHP community contributed as well.
Many thanks to everyone for your input!https://wiki.php.net/rfc/true_async
What has changed in this version?
The RFC has been significantly simplified:
- Components (such as TaskGroup) that can be discussed in separate
RFCs have been removed from the current one.- Coroutines can now be created anywhere — even inside shutdown_function.
- Added Memory Management and Garbage Collection section
Although work on the previous API RFC was interrupted and we weren’t
able to include it in PHP 8.5, it still provided valuable feedback on
the Async API code.During this time, I managed to refactor and optimize the TrueAsync
code, which showed promising performance results in I/O scenarios.A test integration between NGINX UNIT and the TrueAsync API
was implemented to evaluate the possibility of using PHP as an
asynchronous backend for a web server:
https://github.com/EdmondDantes/nginx-unit/tree/true-async/src/true-async-phpDuring this time, the project has come very close to beta status.
Once again, I want to thank everyone who supported me during difficult
times, offered advice, and helped develop this project.Given the maturity of both the code and the RFC, this time I hope to
proceed with a vote.Wishing you all a great day, and thank you for your feedback!
Hey Edmond,
I'm not quite finished notating the whole thing, but let's start with this:
AWAITABLE
There's something here that bothers me. The RFC says:
he
Awaitableinterface is a contract that allows objects to be used in theawaitexpression.
TheAwaitableinterface does not impose limitations on the number of state changes.
In the general case, objects implementing theAwaitableinterface can act as triggers — that is, they can change their state an unlimited number of times. This means that multiple calls toawait <Awaitable>may produce different results.
But then coroutines say:
Coroutines behave like Futures:
once a coroutine completes (successfully, with an exception, or through cancellation),
it preserves its final state.
Multiple calls toawait()on the same coroutine will always return the same result or
throw the same exception.
This seems a bit contradictory and confuses things. When I await(), do I need to do it in a loop, or just once? It might be a good idea to make a couple subtypes: Signal and Future. Coroutines become Future that only await once, while Signal is something that can be awaited many times.
It probably won't change much from the C point of view, but it would change how we implement them in general libraries:
if ($awaitable instanceof Trigger) {
// throw or maybe loop over the trigger value?
}
CANCELLATIONS
The RFC says:
In the context of coroutines, it is not recommended to use
catch \Throwableorcatch CancellationError.
But the example setChildScopeExceptionHandler does exactly this! Further, much framework/app code uses the $previous to wrap exceptions as they bubble up, so it might be nice to have an Async\isCancellation(Throwable): bool function that can efficiently walk the exception chain and tell us if any cancellation was involved.
Minor nit: in the Async\protect section, it would be nice to say that cancellations being AFTER the protect() are guaranteed, and also specify reentry/nesting of protect(). Like what happens here:
Async\protect(foo(...)); // foo also calls protect()
And for reentrancy, foo() -> bar() -> foo() -> bar() and foo() calls protect().
Which one gets the cancellation? I also think that calling it a "critical section" is a misnomer, as that traditionally indicates a "lock" (only one thread can execute that section at a time) and not "this can't be cancelled".
Also, if I'm reading this correctly, a coroutine can mark itself as canceled, yet run to completion; however anyone await()'ing it, will get a CancellationException instead of the completed value?
DESTRUCTORS
Allowing destructors to spawn feels extremely dangerous to me (but powerful). These typically -- but not always -- run between the return statement and the next line (typically best to visualize that as the "}" since it runs in the original scope IIRC). That could make it 'feel like' methods/functions are hanging or never returning if a library abuses this by suspending or awaiting something.
ZOMBIES
async.zombie_coroutine_timeout says 2 seconds in the text, but 5 seconds in the php.ini section.
The RFC says:
Once the application is considered finished, zombie coroutines are given a time limit within which they must complete execution. If this limit is exceeded, all zombie coroutines are canceled.
What is defined as "application considered finished?" FrankenPHP workers, for instance, don’t "finish" — is there a way to reap zombies manually?
Then there is dispose() and disposeSafely(), it would be good to specify ordering and finally/onFinally execution here. ie, in nested scopes, does it go from inner -> outer scopes, in the order they are created? When does finally/onFinally execute in that context?
FIBERS
Fibers are proliferant in existing code. It would be a good idea to provide a few helpers to allow code to migrate. Maybe something like Async\isEnabled() to know whether I should use fibers or not.
Nit: the error message has a grammatical error: "Cannot create a fiber while an True Async is active" should be "Cannot create a fiber while True Async is active"?
SHUTDOWN
Is there also a timeout on Phase 1 shutdown? Otherwise, if it is only an exception, then this could hang forever.
EXCEPTION IDENTITY
The RFC says:
Multiple calls to
await()on the same coroutine will always return the same result or
throw the same exception.
Is this "same exception" mean this literally, or is it a clone? If it is the same, what prevents another code path from mutating the original exception before it gets to me?
TYPOS
- fix
file_get_contenttofile_get_contentsin examples. - I think get_last_error() should be
error_get_last()? - you call "suspend" a keyword in several places but it is actually a function.
- examples use
sleep(), but don't clarify whethersleep()will be blocking or non-blocking. - Sometimes you use AwaitCancelledException and other times CancellationError. Which is it?
— Rob
Good day, everyone. I hope you're doing well.
I’m happy to present the fourth version of the RFC. It wasn’t just me
who worked on it — members of the PHP community contributed as well.
Many thanks to everyone for your input!https://wiki.php.net/rfc/true_async
What has changed in this version?
The RFC has been significantly simplified:
- Components (such as TaskGroup) that can be discussed in separate
RFCs have been removed from the current one.- Coroutines can now be created anywhere — even inside shutdown_function.
- Added Memory Management and Garbage Collection section
Although work on the previous API RFC was interrupted and we weren’t
able to include it in PHP 8.5, it still provided valuable feedback on
the Async API code.During this time, I managed to refactor and optimize the TrueAsync
code, which showed promising performance results in I/O scenarios.A test integration between NGINX UNIT and the TrueAsync API
was implemented to evaluate the possibility of using PHP as an
asynchronous backend for a web server:
https://github.com/EdmondDantes/nginx-unit/tree/true-async/src/true-async-phpDuring this time, the project has come very close to beta status.
Once again, I want to thank everyone who supported me during difficult
times, offered advice, and helped develop this project.Given the maturity of both the code and the RFC, this time I hope to
proceed with a vote.Wishing you all a great day, and thank you for your feedback!
Hey Edmond,
I'm not quite finished notating the whole thing, but let's start with this:
AWAITABLE
There's something here that bothers me. The RFC says:
he
Awaitableinterface is a contract that allows objects to be used in theawaitexpression.
TheAwaitableinterface does not impose limitations on the number of state changes.
In the general case, objects implementing theAwaitableinterface can act as triggers — that is, they can change their state an unlimited number of times. This means that multiple calls toawait <Awaitable>may produce different results.But then coroutines say:
Coroutines behave like Futures:
once a coroutine completes (successfully, with an exception, or through cancellation),
it preserves its final state.
Multiple calls toawait()on the same coroutine will always return the same result or
throw the same exception.This seems a bit contradictory and confuses things. When I await(), do I need to do it in a loop, or just once? It might be a good idea to make a couple subtypes: Signal and Future. Coroutines become Future that only await once, while Signal is something that can be awaited many times.
It probably won't change much from the C point of view, but it would change how we implement them in general libraries:
if ($awaitable instanceof Trigger) {
// throw or maybe loop over the trigger value?
}CANCELLATIONS
The RFC says:
In the context of coroutines, it is not recommended to use
catch \Throwableorcatch CancellationError.But the example setChildScopeExceptionHandler does exactly this! Further, much framework/app code uses the $previous to wrap exceptions as they bubble up, so it might be nice to have an Async\isCancellation(Throwable): bool function that can efficiently walk the exception chain and tell us if any cancellation was involved.
Minor nit: in the Async\protect section, it would be nice to say that cancellations being AFTER the protect() are guaranteed, and also specify reentry/nesting of protect(). Like what happens here:
Async\protect(foo(...)); // foo also calls protect()
And for reentrancy, foo() -> bar() -> foo() -> bar() and foo() calls protect().
Which one gets the cancellation? I also think that calling it a "critical section" is a misnomer, as that traditionally indicates a "lock" (only one thread can execute that section at a time) and not "this can't be cancelled".
Also, if I'm reading this correctly, a coroutine can mark itself as canceled, yet run to completion; however anyone await()'ing it, will get a CancellationException instead of the completed value?
DESTRUCTORS
Allowing destructors to spawn feels extremely dangerous to me (but powerful). These typically -- but not always -- run between the return statement and the next line (typically best to visualize that as the "}" since it runs in the original scope IIRC). That could make it 'feel like' methods/functions are hanging or never returning if a library abuses this by suspending or awaiting something.
ZOMBIES
async.zombie_coroutine_timeout says 2 seconds in the text, but 5 seconds in the php.ini section.
The RFC says:
Once the application is considered finished, zombie coroutines are given a time limit within which they must complete execution. If this limit is exceeded, all zombie coroutines are canceled.
What is defined as "application considered finished?" FrankenPHP workers, for instance, don’t "finish" — is there a way to reap zombies manually?
Then there is dispose() and disposeSafely(), it would be good to specify ordering and finally/onFinally execution here. ie, in nested scopes, does it go from inner -> outer scopes, in the order they are created? When does finally/onFinally execute in that context?
FIBERS
Fibers are proliferant in existing code. It would be a good idea to provide a few helpers to allow code to migrate. Maybe something like Async\isEnabled() to know whether I should use fibers or not.
Nit: the error message has a grammatical error: "Cannot create a fiber while an True Async is active" should be "Cannot create a fiber while True Async is active"?
SHUTDOWN
Is there also a timeout on Phase 1 shutdown? Otherwise, if it is only an exception, then this could hang forever.
EXCEPTION IDENTITY
The RFC says:
Multiple calls to
await()on the same coroutine will always return the same result or
throw the same exception.Is this "same exception" mean this literally, or is it a clone? If it is the same, what prevents another code path from mutating the original exception before it gets to me?
TYPOS
- fix
file_get_contenttofile_get_contentsin examples.- I think get_last_error() should be
error_get_last()?- you call "suspend" a keyword in several places but it is actually a function.
- examples use
sleep(), but don't clarify whethersleep()will be blocking or non-blocking.- Sometimes you use AwaitCancelledException and other times CancellationError. Which is it?
— Rob
For bike shedding purposes:
It's also worth pointing out that "cancelled" is the British spelling, while "canceled" is the American spelling. PHP is already inconsistent in the docs/error messages, but I don't see any types/functions with either spelling. Generally, PHP tends to follow American spelling for the standard lib (color vs. colour, behavior vs behaviour, analyzes vs analyses, initialize vs initialise, serialize vs serialise, etc). So, IMHO, we should probably be using the american spelling here ...
— Rob
When I await(), do I need to do it in a loop, or just once?
It depends on what is being awaited.
On one hand, it would probably be convenient to have many different
operations for different cases, but then we make the language
semantics more complex.
Coroutines become Future that only await once, while Signal is something that can be awaited many times.
At the moment, only such objects exist. It’s hard to say whether there
will be others.
Although one can imagine an Interval object, there are some doubts
about whether such an object should be used in a while await loop,
because from a performance standpoint, it’s not very efficient.
But the example setChildScopeExceptionHandler does exactly this!
The Scope-level handler does not interfere with coroutine completion.
And it is not called because the cancellation exception is "absorbed"
by the coroutine.
Further, much framework/app code uses the $previous to wrap exceptions as they bubble up,
If a programmer wants to wrap an exception in their own one let them.
No one forbids catching exceptions; they just shouldn’t be suppressed.
Async\isCancellation(Throwable): bool
Why make a separate function if you can just walk through the chain?
Minor nit: in the Async\protect section, it would be nice to say that cancellations being AFTER the protect() are guaranteed, and also specify reentry/nesting of protect(). Like what happens here:
That’s a good case! Re-entering protect should be forbidden that must
not be allowed.
Also, if I'm reading this correctly, a coroutine can mark itself as canceled, yet run to completion; however anyone await()'ing it, will get a CancellationException instead of the completed value?
If a coroutine is canceled, its return value will be ignored.
However, of course, it can still call return, and that will work
without any issues.
I considered issuing a warning for such behavior but later removed it,
since I don’t see it as particularly dangerous.
This point requires attention, because there’s a certain “flexibility”
here that can be confusing. However, the risk in this case is low.
Allowing destructors to spawn feels extremely dangerous to me (but powerful). These typically -- but not always -- run between the return statement and the next line (typically best to visualize that > as the "}" since it runs in the original scope IIRC). That could make it 'feel like' methods/functions are hanging or never returning if a library abuses this by suspending or awaiting something.
Launching coroutines in destructors is indeed a relatively dangerous
operation, but for different reasons mainly related to who owns such
coroutines. However, I didn’t quite understand what danger you were
referring to?
Asynchronous operations, as well as coroutine launching, are indeed
used in practice. The code executes properly, so I don’t quite see
what risks there could be, apart from potential resource leaks caused
by faulty coroutines.
async.zombie_coroutine_timeout says 2 seconds in the text, but 5 seconds in the php.ini section.
Thanks.
What is defined as "application considered finished?" FrankenPHP workers, for instance, don’t "finish" — is there a way to reap zombies manually?
The Scheduler keeps track of the number of coroutines being executed.
When the number of active coroutines reaches zero, the Scheduler stops
execution. Zombie coroutines are not counted among those that keep the
execution running. If PHP is running in worker mode, then the worker
code must correctly keep the execution active. But even workers
sometimes need to shutdown.
it would be good to specify ordering and finally/onFinally execution here
Doesn’t the RFC define the order of onFinally handler execution?
onFinally handlers are executed after the coroutine or the Scope has completed.
onFinally is not directly related to dispose() in any way.
When dispose() is called, coroutine cancellation begins. This process
may take some time. Only after the last coroutine has stopped will
onFinally be invoked. In other words, you should not attempt to link
the calls of these methods in any way.
Maybe something like Async\isEnabled() to know whether I should use fibers or not.
Good idea!,
considering that such a function actually exists at the C code level.
Is this "same exception" mean this literally, or is it a clone? If it is the same, what prevents another code path from mutating the original exception before it gets to me?
It’s the exact same object that is, a reference to the same instance.
So if someone modifies it, those changes will, of course, take effect.
Is there also a timeout on Phase 1 shutdown? Otherwise, if it is only an exception, then this could hang forever.
That’s true! A hang is indeed possible. I’m still not sure whether
it’s worth adding an auxiliary mechanism to handle such cases, because
that would effectively make PHP “smarter” than the programmer. I
believe that a language should not try to be smarter than the
programmer if the application runs in a certain way, then it’s
probably meant to be that way.
- Sometimes you use AwaitCancelledException and other times CancellationError. Which is it?
The old exception name apparently hasn’t been updated to the new one everywhere.
Nit: the error message has a grammatical error: "Cannot create a fiber while an True Async is active" should be "Cannot create a fiber while True Async is active"?
Thanks!
When I await(), do I need to do it in a loop, or just once?
It depends on what is being awaited.
On one hand, it would probably be convenient to have many different
operations for different cases, but then we make the language
semantics more complex.
Coroutines become Future that only await once, while Signal is something that can be awaited many times.
At the moment, only such objects exist. It’s hard to say whether there
will be others.
Although one can imagine an Interval object, there are some doubts
about whether such an object should be used in a while await loop,
because from a performance standpoint, it’s not very efficient.
It might be good to clarify this when we talk about the Awaitable Interface then? Maybe something like:
"In PHP 8.6 the only awaitables are single-completion. Future versions may add multi-event awaitables." just to clear it up for early adopters?
But the example setChildScopeExceptionHandler does exactly this!
The Scope-level handler does not interfere with coroutine completion.
And it is not called because the cancellation exception is "absorbed"
by the coroutine.
:thumbsup:
Further, much framework/app code uses the $previous to wrap exceptions as they bubble up,
If a programmer wants to wrap an exception in their own one let them.
No one forbids catching exceptions; they just shouldn’t be suppressed.
Async\isCancellation(Throwable): bool
Why make a separate function if you can just walk through the chain?
If everyone writes their own isCancellation() we risk divergence (not to mention, it will be faster in C and basically need to be checked on every catch that might await); having one blessed function guarantees consistent detection and can allow for static-analysis support.
Minor nit: in the Async\protect section, it would be nice to say that cancellations being AFTER the protect() are guaranteed, and also specify reentry/nesting of protect(). Like what happens here:
That’s a good case! Re-entering protect should be forbidden that must
not be allowed.
<3 that's good to know! It definately needs to be in the RFC. If you don't mind me asking: why is this the case?
Also, if I'm reading this correctly, a coroutine can mark itself as canceled, yet run to completion; however anyone await()'ing it, will get a CancellationException instead of the completed value?
If a coroutine is canceled, its return value will be ignored.
However, of course, it can still call return, and that will work
without any issues.
I considered issuing a warning for such behavior but later removed it,
since I don’t see it as particularly dangerous.
This point requires attention, because there’s a certain “flexibility”
here that can be confusing. However, the risk in this case is low.
I would find it surprising behaviour -- if you cancel a context in go, it may or may not complete, but you get back both the completion (if it completed) and/or the error. In C#, it throws an exception and it never completes. Languages have different ways to do it, but it should be documented in the RFC what the behaviour is and how to handle this case. Ergonomics matter as much as the feature existing.
As far as observability goes, it might be a good idea to issue a notice instead of a warning. Notice is often suppressed and rarely causes any issues, but in development, seeing that would at least let me know something was going on that I should investigate.
Allowing destructors to spawn feels extremely dangerous to me (but powerful). These typically -- but not always -- run between the return statement and the next line (typically best to visualize that > as the "}" since it runs in the original scope IIRC). That could make it 'feel like' methods/functions are hanging or never returning if a library abuses this by suspending or awaiting something.
Launching coroutines in destructors is indeed a relatively dangerous
operation, but for different reasons mainly related to who owns such
coroutines. However, I didn’t quite understand what danger you were
referring to?Asynchronous operations, as well as coroutine launching, are indeed
used in practice. The code executes properly, so I don’t quite see
what risks there could be, apart from potential resource leaks caused
by faulty coroutines.
I think we missed each other here. Consider the following code:
function test() {
$r = new AsyncResource();
return 42; // destructor suspends here
}
Would this delay the caller's return until the destructor's coroutine finished, or is it detached? If detached, can it interleave safely with subsequent code? This should be documented in the RFC so people can plan for it and use it appropriately (such as managing transactions or locks inside destructors).
async.zombie_coroutine_timeout says 2 seconds in the text, but 5 seconds in the php.ini section.
Thanks.What is defined as "application considered finished?" FrankenPHP workers, for instance, don’t "finish" — is there a way to reap zombies manually?
The Scheduler keeps track of the number of coroutines being executed.
When the number of active coroutines reaches zero, the Scheduler stops
execution. Zombie coroutines are not counted among those that keep the
execution running. If PHP is running in worker mode, then the worker
code must correctly keep the execution active. But even workers
sometimes need to shutdown.
I have some workers that haven't restarted since April. :) So, having a way to manually reap zombies (much like we do with OS-level code when running as PID 1) and track them, would be nice to have. At least, as part of the scheduler API.
it would be good to specify ordering and finally/onFinally execution here
Doesn’t the RFC define the order of onFinally handler execution?
onFinally handlers are executed after the coroutine or the Scope has completed.
onFinally is not directly related to dispose() in any way.When dispose() is called, coroutine cancellation begins. This process
may take some time. Only after the last coroutine has stopped will
onFinally be invoked. In other words, you should not attempt to link
the calls of these methods in any way.
This should be documented on the RFC, it still doesn't explain what the order of operations is though. This matters because if you are doing cleanup during disposal, you need to know what things will still be around (for reference, order of operations for GC is well documented and defined https://www.php.net/manual/en/features.gc.collecting-cycles.php which is what I'm expecting to see here).
Maybe something like Async\isEnabled() to know whether I should use fibers or not.
Good idea!,
considering that such a function actually exists at the C code level.Is this "same exception" mean this literally, or is it a clone? If it is the same, what prevents another code path from mutating the original exception before it gets to me?
It’s the exact same object that is, a reference to the same instance.
So if someone modifies it, those changes will, of course, take effect.
This should probably be documented in the RFC: "Exceptions and returned objects are shared objects; mutating them is undefined behavior if there are multiple awaiters."
Is there also a timeout on Phase 1 shutdown? Otherwise, if it is only an exception, then this could hang forever.
That’s true! A hang is indeed possible. I’m still not sure whether
it’s worth adding an auxiliary mechanism to handle such cases, because
that would effectively make PHP “smarter” than the programmer. I
believe that a language should not try to be smarter than the
programmer if the application runs in a certain way, then it’s
probably meant to be that way.
I think of it more as observability than smarts (esp if the timeout is configurable) ... otherwise, how would you even know if it is hanging on shutdown vs. doesn't even know it is supposed to be shutting down? I'm reminded of certain CLI tools that require me issuing a SIGTSTP (ctrl-z) to issue a SIGTERM or SIGKILL because SIGINT (ctrl-c) doesn't appear to work. If it is my program, is it that there is a bug with SIGINT handlers -- or is it hanging during shutdown? Having a timeout there would at least protect me from my customers/users getting stuck due to a bug, and I could always set the timeout to something infinite-ish (0? -1?) if that is the behaviour I want.
— Rob
So, IMHO, we should probably be using the american spelling here ...
Yes, it would be nice to have some way to validate the English language.
I’d definitely suggest allowing RFCs to be edited directly in Git.
This seems a bit contradictory and confuses things. When I await(), do I need to do it in a loop, or just once?
It might be a good idea to make a couple subtypes: Signal and Future. Coroutines become Future that only await once, while Signal is something that can be awaited many times.
I made a mistake in my previous response, and it requires clarification.
Classes that can be awaited multiple times are indeed possible.
These include TimeInterval, Channel, FileSystemEvent, as well as
I/O triggers.
All of these classes can be Awaitable.
This is done to allow bulk waiting on objects, regardless of how they
work internally.
So this is meant for functions like awaitXX, although such behavior
is also possible for the await() function itself:
$timeInterval = new Async\TimeInterval(1000);
while(true) {
await($timeInterval);
}
As for objects of type Future, it’s clear that in future RFCs there
will be a FutureInterface, which will be implemented by coroutines.
This seems a bit contradictory and confuses things. When I await(), do I need to do it in a loop, or just once?
It might be a good idea to make a couple subtypes: Signal and Future. Coroutines become Future that only await once, while Signal is something that can be awaited many times.I made a mistake in my previous response, and it requires clarification.
Classes that can be awaited multiple times are indeed possible.
These includeTimeInterval,Channel,FileSystemEvent, as well as
I/O triggers.All of these classes can be
Awaitable.
This is done to allow bulk waiting on objects, regardless of how they
work internally.
So this is meant for functions likeawaitXX, although such behavior
is also possible for theawait()function itself:$timeInterval = new Async\TimeInterval(1000); while(true) { await($timeInterval); }As for objects of type
Future, it’s clear that in future RFCs there
will be aFutureInterface, which will be implemented by coroutines.
Hi Edmond,
I've been meaning to review your RFC and implementation for some time, but for various reasons, I still haven't been able to give it a thorough read and review.
I noticed this portion of the discussion and wanted to drop a note now, rather than waiting until I was able to read the entire RFC.
Awaitables should always represent a single value. Awaiting multiple times should never result in a different value.
Async sets of values should use a different abstraction to represent a set. rxjs Observables (rxjs.dev) are on example. AMPHP has a pipeline library, https://github.com/amphp/pipeline, which defines a ConcurrentIterator interface. The latter IMO is more appropriate for PHP + fibers. I recommend having a look at how Future and ConcurrentIterator are used within AMPHP libraries.
I think you should consider additional time beyond only two more weeks for discussion of this RFC before bringing it to a vote. PHP 8.6 or 9 is some time away. This is definitely not an RFC to rush to voting.
Cheers,
Aaron Piotrowski
Hi
Awaitables should always represent a single value. Awaiting multiple times should never result in a different value.
Where did this rule come from?
In programming languages (except Rust), there is no explicit
restriction on the behavior of the await operation, nor a specific
requirement that it must always return the same value. However, from a
usability perspective, such a rule would make the code simpler. But...
On the other hand, if we restrict the behavior of await, we fail to
cover the full range of possible cases — and that’s also bad.
AMPHP has a pipeline library, https://github.com/amphp/pipeline,
That’s not quite the same.
The general case of interacting with Awaitable objects looks like this:
// Waiting for the first event from any object in the set.
// The objects in the set are of different types.
awaitAny(obj1, obj2, obj3); // or All ...
But, programming languages don’t always implement this general
case, and sometimes even try to avoid it altogether.
And working with a data stream is implemented differently through
await foreach.
There is another way to solve this problem (all Futures only) —
through a method that always returns a new Future. For example:
// $queue->whenReady() returns Future object
awaitAll($future, $queue->whenReady());
Downside: each time we create a new object in memory, while the loop
still remains.
At the moment, I don’t see any compelling reason to impose artificial
restrictions on behavior.
- Future objects are a special case of Awaitable objects,
- while Awaitable objects represent the general case.
I think you should consider additional time beyond only two more weeks for discussion of this RFC before bringing it to a vote.
PHP 8.6 or 9 is some time away. This is definitely not an RFC to rush to voting.
I have no objections.
Thank you, Ed
Hi
Awaitables should always represent a single value. Awaiting multiple times should never result in a different value.
Where did this rule come from?
I don’t think it’s a “rule” per se and why I suggested breaking it up into two different kinds of Awaitables. Invariants make code easier to reason about and work with. The more invariants you have, the easier it is to form, maintain, and refactor.
— Rob
Hi
I don’t think it’s a “rule” per se and why I suggested breaking it up into two different kinds of Awaitables.
Invariants make code easier to reason about and work with. The more invariants you have, the easier it is to form, maintain, and refactor.
So that is the rule: invariants make code easier to understand.
A more general principle is stated as follows: reducing complexity.
I think this point needs some thought.
If the await operation is allowed only for Future, it will make the
code more consistent which is a good thing.
Then it will be necessary to add a FutureInterface, and Awaitable
should be hidden from the UserLand namespace.
Hi
I don’t think it’s a “rule” per se and why I suggested breaking it up into two different kinds of Awaitables.
Invariants make code easier to reason about and work with. The more invariants you have, the easier it is to form, maintain, and refactor.So that is the rule: invariants make code easier to understand.
A more general principle is stated as follows: reducing complexity.I think this point needs some thought.
If the await operation is allowed only for Future, it will make the
code more consistent which is a good thing.Then it will be necessary to add a FutureInterface, and Awaitable
should be hidden from the UserLand namespace.
A simpler solution might be to keep things as they are, but have the non-idempotent constructs be generators of Awaitable instead of non-idempotent Awaitables. This would basically mean just changing some text in the RFC and some implementations that aren't documented in the RFC (as far as I can tell).
— Rob
A simpler solution might be to keep things as they are, but have the non-idempotent constructs be generators of Awaitable instead of non-idempotent Awaitables
So it’s the same as in C#?
The problem with this situation is that so far I haven’t been able to
find any cases proving that a non-Future object could cause serious
failures in the code.
At the same time, if a select case expression is introduced in the
future, it would make more sense for select to work with an awaitable
object rather than a Future.
That’s why I’m not yet sure it’s worth abandoning the general behavior
unless a convincing argument is found showing that it leads to real
problems.
-- Ed
A simpler solution might be to keep things as they are, but have the non-idempotent constructs be generators of Awaitable instead of non-idempotent Awaitables
So it’s the same as in C#?The problem with this situation is that so far I haven’t been able to
find any cases proving that a non-Future object could cause serious
failures in the code.At the same time, if a select case expression is introduced in the
future, it would make more sense for select to work with an awaitable
object rather than a Future.That’s why I’m not yet sure it’s worth abandoning the general behavior
unless a convincing argument is found showing that it leads to real
problems.-- Ed
The example I gave is probably a good one? If I'm writing framework-y code, how do I decide to await once, or in a loop? In other words, how do I detect whether an Awaitable is idempotent or will give a different result every time? If I'm wrong, I could end up in an infinite loop, or missing results. Further, how do I know whether the last value from an Awaitable is the last value? I think if you could illustrate that in the RFC or change the semantics, that'd be fine.
— Rob
A simpler solution might be to keep things as they are, but have the non-idempotent constructs be generators of Awaitable instead of non-idempotent Awaitables
So it’s the same as in C#?The problem with this situation is that so far I haven’t been able to
find any cases proving that a non-Future object could cause serious
failures in the code.At the same time, if a select case expression is introduced in the
future, it would make more sense for select to work with an awaitable
object rather than a Future.That’s why I’m not yet sure it’s worth abandoning the general behavior
unless a convincing argument is found showing that it leads to real
problems.-- Ed
The example I gave is probably a good one? If I'm writing framework-y code, how do I decide to await once, or in a loop? In other words, how do I detect whether an Awaitable is idempotent or will give a different result every time? If I'm wrong, I could end up in an infinite loop, or missing results. Further, how do I know whether the last value from an Awaitable is the last value? I think if you could illustrate that in the RFC or change the semantics, that'd be fine.
— Rob
Accidentally sent too early: but also, what if there are multiple awaiters for a non-idempotent Awaiter? How do we handle that?
— Rob
The example I gave is probably a good one? If I'm writing framework-y code, how do I decide to await once, or in a loop? In other words,
how do I detect whether an Awaitable is idempotent or will give a different result every time? If I'm wrong, I could end up in an infinite loop, or missing results.
Further, how do I know whether the last value from an Awaitable is the last value? I think if you could illustrate that in the RFC or change the semantics, that'd be fine.
If a function knows nothing about the object it’s awaiting, it’s
equally helpless not only in deciding whether to use while or not, but
also in determining how to handle the result.
As for the infinite loop issue, the situation depends on the
termination conditions. For example:
$queue = new Queue();
// Rust future-based
while(true) {
$future = $queue->next();
if($future === null) {
break;
}
await($future);
}
or
$queue = new Queue();
// Awaitable style
while($queue->isClosed() === false) {
await($queue);
}
In other words, a loop needs some method that limits its execution in
any case and it’s hard to make a mistake with that.
Accidentally sent too early: but also, what if there are multiple awaiters for a non-idempotent Awaiter? How do we handle that?
All of this completely depends on the implementation of the awaited object.
The Awaitable contract does not define when the event will occur or
whether it will be cached. it only guarantees that the object can be
awaited.
However, the exact moment when the object wakes the coroutine and what
type of data it provides are all outside the scope of the awaiting
contract.
In Rust, it’s common practice to use methods that create a new Future
(or NULL) when a certain action needs to be awaited, like:
while let Some(v) = rx.recv().await {
println!("Got: {}", v);
}
Multiple awaits usually appear as several different Future instances
that can be created by the same awaitable object.
However, the Rust approach doesn’t fundamentally change...
If the internal logic of an Awaitable object loses an event before a
Future is created, the behavior is effectively the same as if the
Future never existed.
The advantage of the Rust approach is that the programmer can clearly
see that a Future is being created (rx.recv() should return Future new
one or the same?). (Perhaps the code looks more compact)
But they still have to read the documentation to understand how this
Future completes, how it’s created, and what data it returns. Whether
the last message is cached or not, and so on.
In summary, a programmer must understand what kind of object they’re
actually working with. It’s unlikely that this can be avoided.
The example I gave is probably a good one? If I'm writing framework-y code, how do I decide to await once, or in a loop? In other words,
how do I detect whether an Awaitable is idempotent or will give a different result every time? If I'm wrong, I could end up in an infinite loop, or missing results.
Further, how do I know whether the last value from an Awaitable is the last value? I think if you could illustrate that in the RFC or change the semantics, that'd be fine.If a function knows nothing about the object it’s awaiting, it’s
equally helpless not only in deciding whether to use while or not, but
also in determining how to handle the result.As for the infinite loop issue, the situation depends on the
termination conditions. For example:$queue = new Queue(); // Rust future-based while(true) { $future = $queue->next(); if($future === null) { break; } await($future); }or
$queue = new Queue(); // Awaitable style while($queue->isClosed() === false) { await($queue); }In other words, a loop needs some method that limits its execution in
any case and it’s hard to make a mistake with that.Accidentally sent too early: but also, what if there are multiple awaiters for a non-idempotent Awaiter? How do we handle that?
All of this completely depends on the implementation of the awaited object.
The Awaitable contract does not define when the event will occur or
whether it will be cached. it only guarantees that the object can be
awaited.
However, the exact moment when the object wakes the coroutine and what
type of data it provides are all outside the scope of the awaiting
contract.In Rust, it’s common practice to use methods that create a new Future
(or NULL) when a certain action needs to be awaited, like:while let Some(v) = rx.recv().await { println!("Got: {}", v); }Multiple awaits usually appear as several different
Futureinstances
that can be created by the same awaitable object.
However, the Rust approach doesn’t fundamentally change...If the internal logic of an Awaitable object loses an event before a
Future is created, the behavior is effectively the same as if the
Future never existed.The advantage of the Rust approach is that the programmer can clearly
see that a Future is being created (rx.recv() should return Future new
one or the same?). (Perhaps the code looks more compact)
But they still have to read the documentation to understand how this
Future completes, how it’s created, and what data it returns. Whether
the last message is cached or not, and so on.In summary, a programmer must understand what kind of object they’re
actually working with. It’s unlikely that this can be avoided.
I think that might make sense for Rust/Go which generally don't rely heavily on frameworks, unlike PHP -- frameworks work from abstractions not concrete types. After some thinking about it the last day or so, here's the problems with the "multi-shot" vs. "single-shot" Awaitables:
- refactoring hazards
If you await a value, everything works, but then someone somewhere else awaits the same Awaitable that wasn't actually a "one-shot" Awaitable, so now everything breaks sometimes, and other times not -- depending on which one awaits first.
- memoization becomes an issue
function getOnce(Awaitable $response) {
static $cache = [];
$id = spl_object_id($response);
return $cache[$id] ??= await($response);
}
With a "multi-shot" Awaitable, this is not practical or even a good idea. You can't write general-purpose helpers, at all.
- static analysis
psalm/phpstan can't warn you that you are dealing with a "multi-shot" or "single-shot" Awaitable. The safest thing is to treat everything as "multi-shot" so you don't shoot yourself in the foot -- but there's no way to tell if you are intentionally getting the same object every time or it is a "single-shot" Awaitable.
- violation of algebraic laws with awaitAll/awaitAny
With "multi-shot" awaitables, awaitAll() becomes an infinite loop and does awaitAny() does/doesn't guarantee idempotency.
- violation of own invariants in the RFC
The RFC says that await will throw the SAME instance of exceptions, but with "multi-shot" Awaitables, will this in fact be the case? Could it throw an exception the first time but a result the next? Or maybe even different exceptions every time?
- common patterns aren't guaranteed anymore
A common case is to "peek then act" on a value, so now this would be a very subtle footgun:
if (await($response)) {
return await($response);
}
This is a pretty common pattern in TypeScript/JavaScript, where you don't want to go through the effort of keeping a variable that may not even be acted on. Not to mention, many codestyles outlaw the following (putting assignment in if-statements):
if ($val = await($response)) {
return $val;
}
- retries are broken
I can imagine something like this being in frameworks:
retry:
try {
return await($response, timeout(10));
} catch(CancellationException) {
logSomething() // for a few ms while we continue to wait
goto retry;
}
- select/case doesn't require multishot
I'm not sure what you mean by this. In Go, you await a channel, who's value is single-shot. In C#, you await an IEnumerable which return Tasks, which the value is single-shot. In kotlin, you receive via deferred, whose value is single-shot.
So, in other words, maybe the queue/stream/whatever is multi-shot, but the thing you pass to select() is single-shot.
- how will the scheduler handle backpressure?
I think you mentioned elsewhere that the scheduler is currently relatively rudimentary. From working on custom C# Task schedulers in the past, having multi-shot Awaitables will be terrible for scheduling. You'll have no way to handle backpressure and ensure fairness.
I'm not sure what your thought process is here, because in the last few emails you've gone from "maybe" to doubling-down on this (from my perspective), but I feel like this will be a footgun to both developers and the future of the language.
— Rob
If you await a value, everything works, but then someone somewhere else awaits the same Awaitable that wasn't actually a "one-shot" Awaitable,
so now everything breaks sometimes, and other times not -- depending on which one awaits first.
There is a function:
function my(mixed $i)
{
return $x + 5;
}
Someone called it like this somewhere: my("string"), and the code broke.
The question is: what should be done?
Do we really have to get rid of the mixed type just to prevent someone
from breaking something?
If a programmer explicitly violates the contracts in the code and then
blames the programming language for it. So the programmer is the one
at fault.
If someone uses a contract that promises non-idempotent behavior, it’s
logical to rely on that.
- memoization becomes an issue
The same question: why should the code work correctly if the
agreements are violated at the contract level?
There’s no element of randomness here.
- A person deliberately found the Awaitable interface and inserted it
into the code. - They deliberately wrote that function.
So what exactly is the problem with the interface itself?
With a "multi-shot" Awaitable, this is not practical or even a good idea. You can't write general-purpose helpers, at all.
In what way are generalized types a problem?
Why not just use the contracts correctly?
function getOnce(Awaitable $some) {
if($some instanceof FutureLike === false) {
return await($some);
}
static $cache = [];
$id = spl_object_id($some);
return $cache[$id] ??= await($some);
}
- static analysis
I really don’t see how static analysis could help here. What exactly
would it be checking?
- violation of algebraic laws with awaitAll/awaitAny
And the fact that awaitAll/awaitAny are designed to correctly handle
Awaitable objects?
I’m not aware of any such violations.
- violation of own invariants in the RFC
Can you show which statement of the RFC is being violated here?
What invariants?
- common patterns aren't guaranteed anymore
So JavaScript yes?
class NonIdempotentThenable {
constructor() {
this.count = 0;
}
then(resolve, reject) {
this.count++;
resolve(this.count);
}
}
async function demo() {
const obj = new NonIdempotentThenable();
console.log(await obj); // 1
console.log(await obj); // 2
console.log(await obj); // 3
if (await obj) {
console.log(await obj); // 5
}
}
demo();
Other cases generally have the same problem — a violation of the agreement.
If you await a value, everything works, but then someone somewhere else awaits the same Awaitable that wasn't actually a "one-shot" Awaitable,
so now everything breaks sometimes, and other times not -- depending on which one awaits first.There is a function:
function my(mixed $i) { return $x + 5; }Someone called it like this somewhere: my("string"), and the code broke.
The question is: what should be done?Do we really have to get rid of the mixed type just to prevent someone
from breaking something?
If a programmer explicitly violates the contracts in the code and then
blames the programming language for it. So the programmer is the one
at fault.
If someone uses a contract that promises non-idempotent behavior, it’s
logical to rely on that.
This isn't even the same example. We're not talking about type juggling, but an interface. Mixed is not an object nor an interface.
- memoization becomes an issue
The same question: why should the code work correctly if the
agreements are violated at the contract level?
There’s no element of randomness here.
- A person deliberately found the Awaitable interface and inserted it
into the code.- They deliberately wrote that function.
So what exactly is the problem with the interface itself?
With a "multi-shot" Awaitable, this is not practical or even a good idea. You can't write general-purpose helpers, at all.
In what way are generalized types a problem?
Why not just use the contracts correctly?function getOnce(Awaitable $some) { if($some instanceof FutureLike === false) { return await($some); } static $cache = []; $id = spl_object_id($some); return $cache[$id] ??= await($some); }
The "FutureLike" type is exactly what I'm arguing for!
- static analysis
I really don’t see how static analysis could help here. What exactly
would it be checking?
I can imagine getting something like the following
foreach($next = await($awaitable)) { }
error: usage of single-shot Awaitable in loop, this is an infinite loop
or:
$val = await($awaitable);
//in another file/function/5 lines later/etc
$val = await($awaitable);
error: multiple usages of multi-shot Awaitable; you may get a different result on each invocation of await()
or something like that.
- violation of algebraic laws with awaitAll/awaitAny
And the fact that awaitAll/awaitAny are designed to correctly handle
Awaitable objects?
I’m not aware of any such violations.
I had to go take a look at the implementation. It appears NOT to handle this case, at all. Meaning it treats all Awaitables passed to it as a single-shot Awaitable. (the code was easy to follow, but maybe I missed something). The RFC doesn't spell this out and needs some work here.
When I say "violations" I mean that, assuming $a and $b resolve instantly:
awaitAll([$a, $b]) !== [await($a), await($b)]
awaitAny([delay(10, fn() => await($a), await($b)]) !== await($b)
- violation of own invariants in the RFC
Can you show which statement of the RFC is being violated here?
What invariants?
sidenote: It would really help to include the full context of the emails you're replying to so I don't have to open our email conversation in another window to see exactly what I wrote.
This is my mistake though. The discussion about await() is all mixed up with coroutines so its hard to tell what the actual behaviour for await() is outside of the context of coroutines. The violation I thought of only applies to coroutines, the actual behaviour is "undefined" in the RFC.
- common patterns aren't guaranteed anymore
So JavaScript yes?
class NonIdempotentThenable { constructor() { this.count = 0; } then(resolve, reject) { this.count++; resolve(this.count); } } async function demo() { const obj = new NonIdempotentThenable(); console.log(await obj); // 1 console.log(await obj); // 2 console.log(await obj); // 3 if (await obj) { console.log(await obj); // 5 } } demo();Other cases generally have the same problem — a violation of the agreement.
But you're talking about this being the default. There's a reason this is an anti-pattern in Javascript and violation of the 'agreement' there.
how will the scheduler handle backpressure?
I don’t really understand the problem with the Scheduler, and even
less its relation to backpressure.
Dealing with backpressure is a matter of queue implementation.
The Awaitable contract has nothing to do with this situation.
It has everything to do with it! If a multi-shot Awaitable keeps emitting, this causes repeated wakeups to the same continuation. It has to schedule this every time and there isn't a way to say "don't produce until consumed". You can basically starve other tasks from executing. However, if you use iterables of single-shot Awaitables, you get backpressure "for free" and don't request the next future until the previous one is complete, otherwise, the buffering pressure lands in the awaitable (unbounded memory) or in the schedulers ready queue (which affects fairness). Further, when cancelling a multi-shot awaitable, should the scheduler drop pending emissions and what happens if it keeps re-enqueing it anyway? It makes the scheduler far more complicated than it needs to be!
I'm not sure what your thought process is here, because in the last few emails you've gone from "maybe"
to doubling-down on this (from my perspective), but I feel like this will be a footgun to both developers and the future of the language.Confident about what exactly?
And what exactly would be the footgun?
These are very general statements.It seems to me that people in this conference use the word “footgun”
far too often — and not always in the right context.
There’s a lack of rational boundaries here.
Not every programmer’s mistake is a footgun.
Especially that:if (await($response)) { return await($response); }
A footgun is any reasonable looking code that subtly breaks because the contract is weak, not because the programmer didn't RTFM.
— Rob
This isn't even the same example. We're not talking about type juggling, but an interface. Mixed is not an object nor an interface.
Why?
Type and Interface are contracts.
The "FutureLike" type is exactly what I'm arguing for!
I have nothing against this interface. My point is different:
- Should the await and awaitXX functions accept only Future?
- Should Awaitable be hidden from the PHP userland?
foreach($next = await($awaitable)) { }
What’s the problem here? The object will return a result.
If the result is iterable, it will go into the foreach loop. An
absolutely normal situation.
error: multiple usages of multi-shot Awaitable; you may get a different result on each invocation of await()
Why can’t you call await twice? What’s illegal about it?
If it’s a Future, you’ll get the previous result; if it’s an
Awaitable, you’ll get the second value.
But the correctness of the code here 100% depends on the programmer’s intent.
The RFC doesn't spell this out and needs some work here.
I don’t understand what exactly isn’t specified in the RFC?
When I say "violations" I mean that, assuming $a and $b resolve instantly:
awaitAll([$a, $b]) !== [await($a), await($b)]
...
This is a logical error. Сircular reasoning. The desired behavior is
being used here as proof of the undesired one. (recursion)
In the general case, these equations should not work — and that’s
normal if the code expects entities that are not Future, but, for
example, channels.
This is my mistake though. The discussion about await() is all mixed up with coroutines so its hard to tell what the actual behaviour for await() is outside of the context of coroutines.
The violation I thought of only applies to coroutines, the actual behaviour is "undefined" in the RFC.
await() does two things:
- Puts the coroutine into a waiting state.
- Wakes it up when the
Awaitableobject emits an event.
awaitAny / awaitAll do the same but for a list of objects or an iterator.
However, the result of await() depends on the object being awaited.
Thus, there are two separate contracts here:
- The waiting contract — how the waiting occurs.
- The result contract — how the result is obtained.
From a language design perspective, there should be an explicit
representation for these contracts.
Something similar appeared only in Swift (e.g., try await).
So, from the standpoint of “perfect design,” this is a simplification,
which makes it not ideal.
I can remove the Awaitable interface from the RFC and replace it
with FutureLike. It costs nothing to do so.
But before doing that, I want to be at least 95% sure that in 2–3
years we won’t have to bring it back or add workarounds.
It has everything to do with it! If a multi-shot Awaitable keeps emitting, this causes repeated wakeups to the same continuation. It has to schedule this every time and there isn't a way to say "don't > produce until consumed". You can basically starve other tasks from executing. However, if you use iterables of single-shot Awaitables, you get backpressure "for free" and don't request the next
future until the previous one is complete, otherwise, the buffering pressure lands in the awaitable (unbounded memory) or in the schedulers ready queue (which affects fairness). Further, when
cancelling a multi-shot awaitable, should the scheduler drop pending emissions and what happens if it keeps re-enqueing it anyway? It makes the scheduler far more complicated than it needs to > be!
If an Awaitable object isn’t being awaited, it can’t emit events, nor
can it interfere with the scheduler.
Unless the programmer explicitly creates new coroutines for every event by hand.
If an Awaitable has a lot of data and wakes up 1000 coroutines, then
the scheduler will ensure that all those coroutines get executed.
But until these 1000 coroutines finish processing the data, the
Awaitable cannot wake anyone else.
So there is a natural limit to the number of subscribers. it can only
be exceeded through explicitly incorrect handling of coroutines.
Because the scheduler’s algorithm is extremely simple, it has little
to no starvation issues.
Starvation problems usually arise in schedulers that use more
“intelligent” approaches.
A footgun is any reasonable looking code that subtly breaks because the contract is weak, not because the programmer didn't RTFM.
The code cannot be considered “reasonable,” because in this case the
programmer is clearly violating the contract.
And not because the contract is complex or confusing, but because they
completely ignored it. This doesn’t look like a footgun.
This isn't even the same example. We're not talking about type juggling, but an interface. Mixed is not an object nor an interface.
Why?
Type and Interface are contracts.The "FutureLike" type is exactly what I'm arguing for!
I have nothing against this interface. My point is different:
- Should the await and awaitXX functions accept only Future?
- Should Awaitable be hidden from the PHP userland?
foreach($next = await($awaitable)) { }
What’s the problem here? The object will return a result.
If the result is iterable, it will go into the foreach loop. An
absolutely normal situation.
Assuming an awaitable is idempotent (such as the result of a coroutine), this is an infinite loop. There’s a 99.9% chance that’s unintended, which is part of the point of static analysis.
error: multiple usages of multi-shot Awaitable; you may get a different result on each invocation of await()
Why can’t you call await twice? What’s illegal about it?
If it’s a Future, you’ll get the previous result; if it’s an
Awaitable, you’ll get the second value.
But the correctness of the code here 100% depends on the programmer’s intent.
There’s no world this would be intentional to get two separate values in two separate places, especially concurrently, except in very rare circumstances. You can’t guarantee ordering, and I can’t imagine wanting to interleave results across multiple code paths. Well, I can, but that feels like a very brittle system and a pita to maintain/debug.
The RFC doesn't spell this out and needs some work here.
I don’t understand what exactly isn’t specified in the RFC?When I say "violations" I mean that, assuming $a and $b resolve instantly:
awaitAll([$a, $b]) !== [await($a), await($b)]
...This is a logical error. Сircular reasoning. The desired behavior is
being used here as proof of the undesired one. (recursion)
In the general case, these equations should not work — and that’s
normal if the code expects entities that are not Future, but, for
example, channels.
Ok. If it’s circular reasoning, then what’s the definition of awaitAll? How do we explain it to junior devs, what guarantees can we make from it?
This is my mistake though. The discussion about await() is all mixed up with coroutines so its hard to tell what the actual behaviour for await() is outside of the context of coroutines.
The violation I thought of only applies to coroutines, the actual behaviour is "undefined" in the RFC.
await()does two things:
- Puts the coroutine into a waiting state.
- Wakes it up when the
Awaitableobject emits an event.
awaitAny/awaitAlldo the same but for a list of objects or an iterator.
The RFC says that for *coroutines, *but not Awaitable. It is undefined.
However, the result of
await()depends on the object being awaited.
Thus, there are two separate contracts here:
- The waiting contract — how the waiting occurs.
- The result contract — how the result is obtained.
From a language design perspective, there should be an explicit
representation for these contracts.
Something similar appeared only in Swift (e.g.,try await).So, from the standpoint of “perfect design,” this is a simplification,
which makes it not ideal.I can remove the
Awaitableinterface from theRFCand replace it
withFutureLike. It costs nothing to do so.
But before doing that, I want to be at least 95% sure that in 2–3
years we won’t have to bring it back or add workarounds.
Bringing it back is much much much easier than taking it away.
It has everything to do with it! If a multi-shot Awaitable keeps emitting, this causes repeated wakeups to the same continuation. It has to schedule this every time and there isn't a way to say "don't > produce until consumed". You can basically starve other tasks from executing. However, if you use iterables of single-shot Awaitables, you get backpressure "for free" and don't request the next
future until the previous one is complete, otherwise, the buffering pressure lands in the awaitable (unbounded memory) or in the schedulers ready queue (which affects fairness). Further, when
cancelling a multi-shot awaitable, should the scheduler drop pending emissions and what happens if it keeps re-enqueing it anyway? It makes the scheduler far more complicated than it needs to > be!If an Awaitable object isn’t being awaited, it can’t emit events, nor
can it interfere with the scheduler.
Unless the programmer explicitly creates new coroutines for every event by hand.
If an Awaitable has a lot of data and wakes up 1000 coroutines, then
the scheduler will ensure that all those coroutines get executed.
But until these 1000 coroutines finish processing the data, the
Awaitable cannot wake anyone else.
So there is a natural limit to the number of subscribers. it can only
be exceeded through explicitly incorrect handling of coroutines.Because the scheduler’s algorithm is extremely simple, it has little
to no starvation issues.
Starvation problems usually arise in schedulers that use more
“intelligent” approaches.
You’re making some assumptions:
- That the scheduler will always be cooperative.
- That the scheduler you implemented as a proof-of-concept will be the one shipped.
There’ll come a day where someone will want to write a preemptive coroutine scheduler, or even a multi-threaded scheduler. That might be in 20–30 years, or next year; we don’t know. But if they do, having Awaitables multi-shot will prevent that from being a realistic PR. The amount of complexity required for such an implementation would be huge.
A footgun is any reasonable looking code that subtly breaks because the contract is weak, not because the programmer didn't RTFM.
The code cannot be considered “reasonable,” because in this case the
programmer is clearly violating the contract.
And not because the contract is complex or confusing, but because they
completely ignored it. This doesn’t look like a footgun.
At this point, we’re debating philosophy, but according to the RFC, coroutines will be entirely safe to handle this way, but not ALL awaitables. That’s what makes it a footgun. Sometimes, it’s reasonable, and it works ... and in certain cases, it doesn’t work at all. Nobody can tell by simply reading the code in a code review unless they had been shot in the foot before.
— Rob
Hi
Assuming an awaitable is idempotent (such as the result of a coroutine), this is an infinite loop. There’s a 99.9% chance that’s unintended, which is part of the point of static analysis.
foreach(($next = await($awaitable)) as $value) { }
executed as:
- await($awaitable) - will return some value
- foreach got this value
- If the value has a valid interface, then foreach will iterate over it.
await($awaitable) -
It always returns only one value because under the hood it works like this:
function await(awaitable):
# 1) Create a waker for the current coroutine
waker = Waker(current_coroutine)
# 2) Register an event handler inside the awaitable
handler = function(result, error):
waker.set_result(result, error)
awaitable.remove_handler(handler) # detach
Scheduler.enqueue(waker.coroutine)
awaitable.add_handler(handler)
try:
# 3) Suspend the current coroutine until resumed
Scheduler.suspend(current_coroutine)
# 4) When the waker is triggered, return the stored result
return waker.get_result()
finally:
# 5) Destroy/cleanup the waker object
waker.dispose() # release buffers/slots
The RFC says that for coroutines, but not Awaitable. It is undefined.
Sorry I can’t understand the meaning of this.
Thanks, Ed
Hi
Assuming an awaitable is idempotent (such as the result of a coroutine), this is an infinite loop. There’s a 99.9% chance that’s unintended, which is part of the point of static analysis.
foreach(($next = await($awaitable)) as $value) { }executed as:
- await($awaitable) - will return some value
- foreach got this value
- If the value has a valid interface, then foreach will iterate over it.
await($awaitable) -
It always returns only one value because under the hood it works like this:
function await(awaitable): # 1) Create a waker for the current coroutine waker = Waker(current_coroutine) # 2) Register an event handler inside the awaitable handler = function(result, error): waker.set_result(result, error) awaitable.remove_handler(handler) # detach Scheduler.enqueue(waker.coroutine) awaitable.add_handler(handler) try: # 3) Suspend the current coroutine until resumed Scheduler.suspend(current_coroutine) # 4) When the waker is triggered, return the stored result return waker.get_result() finally: # 5) Destroy/cleanup the waker object waker.dispose() # release buffers/slotsThe RFC says that for coroutines, but not Awaitable. It is undefined.
Sorry I can’t understand the meaning of this.Thanks, Ed
Nowhere in the RFC does it explain how await() applies to the Awaitable interface, it only specifies it in the context of a coroutine.
— Rob
Nowhere in the RFC does it explain how await() applies to the Awaitable interface, it only specifies it in the context of a coroutine.
Which aspect of the behavior is not described here?
RFC:
The Awaitable interface is a contract that allows objects to be used
in the await expression.
The Awaitable interface does not impose limitations on the number of
state changes.
In the general case, objects implementing the Awaitable interface can
act as triggers — that is, they can change their state an unlimited
number of times. This means that multiple calls to await <Awaitable>
may produce different results.
Nowhere in the RFC does it explain how await() applies to the Awaitable interface, it only specifies it in the context of a coroutine.
Which aspect of the behavior is not described here?
RFC:
The Awaitable interface is a contract that allows objects to be used
in the await expression.
The Awaitable interface does not impose limitations on the number of
state changes.
In the general case, objects implementing the Awaitable interface can
act as triggers — that is, they can change their state an unlimited
number of times. This means that multiple calls to await <Awaitable>
may produce different results.
For example, it describes:
awaitsuspends the execution of the current coroutine until the awaited one returns a final result or completes with an exception.
But it doesn't explicitely say that it happens for Awaitable. For all we know, it doesn't suspend anything unless the value given to it is a coroutine. Since Awaitable doesn't necessarily have a "final result" or "a single exception" we know it isn't about Awaitable, but about coroutines.
— Rob
This isn't even the same example. We're not talking about type juggling, but an interface. Mixed is not an object nor an interface.
Why?
Type and Interface are contracts.The "FutureLike" type is exactly what I'm arguing for!
I have nothing against this interface. My point is different:
- Should the await and awaitXX functions accept only Future?
- Should Awaitable be hidden from the PHP userland?
foreach($next = await($awaitable)) { }
What’s the problem here? The object will return a result.
If the result is iterable, it will go into the foreach loop. An
absolutely normal situation.error: multiple usages of multi-shot Awaitable; you may get a different result on each invocation of await()
Why can’t you call await twice? What’s illegal about it?
If it’s a Future, you’ll get the previous result; if it’s an
Awaitable, you’ll get the second value.
But the correctness of the code here 100% depends on the programmer’s intent.
And that's exactly the problem. The correctness cannot be inferred without asking the programmer.
There is, obviously, no such thing as idiot-proof code, as the world will always create a better idiot. But we should still strive to be idiot-resistant. There are two general ways of doing that:
- Affordances - Basically, it should be hard to use wrong.
- Construction - It should be impossible to use wrong in the first place. (Ie, a compile error.)
The classic example is US electrical plugs vs EU plugs. They should be different shapes, because they're different voltages. If you plug an unsuspecting device into the wrong voltage, it goes boom. So the different plug interfaces make it patently obvious that you're doing something wrong because they don't fit together (construction), and if you use an adapter that is your signal that you need to worry about voltage conversion (affordance).
Using the same type for something that can be read only once vs something that should be read multiple times is equivalent to using the same plug for both 120v and 240v current. (Or USB-C using the same plug for 8 different speeds, 5 different power delivery levels, and sometimes no data at all. It sucks.) The developer needs to "just know" which one is intended, because the code doesn't tell them. That's a problem.
Instead, like Rob said, there should be a one-shot object that dies as soon as it is successfully read. That is guaranteed to be single-use, and reading it multiple times is detectable by a static analysis tool as "you clearly are doing this wrong, no question."
Then a multi-read object is... an Iterable that produces those single shot objects. I'm not sure if that even needs its own object that implements Iterable, or if it's OK to not define it and just say it's any iterable<Awaitable>. (So, a generator would work just as well.) I'm leaning toward the latter, but there may be uses for the former, not sure.
But having a "dual voltage" awaitable object is exactly the sort of footgun many of us are talking about. The contract is under-specified, so it's too easy for the developer to be "holding it wrong" and plug it into the wrong wall socket. Boom.
--Larry Garfield
how will the scheduler handle backpressure?
I don’t really understand the problem with the Scheduler, and even
less its relation to backpressure.
Dealing with backpressure is a matter of queue implementation.
The Awaitable contract has nothing to do with this situation.
I'm not sure what your thought process is here, because in the last few emails you've gone from "maybe"
to doubling-down on this (from my perspective), but I feel like this will be a footgun to both developers and the future of the language.
Confident about what exactly?
And what exactly would be the footgun?
These are very general statements.
It seems to me that people in this conference use the word “footgun”
far too often — and not always in the right context.
There’s a lack of rational boundaries here.
Not every programmer’s mistake is a footgun.
Especially that:
if (await($response)) {
return await($response);
}
Hi Edmond,
Thanks for putting in the work to start to bring async support to PHP. I have a few initial comments:
-
Is TrueAsync intended strictly to enable concurrency without parallelism (e.g. single-threaded cooperative multitasking)? Is there a future path in mind for parallelism (actual simultaneous execution)?
-
Following on from that question, I note that there is no way to annotate types as being safely sendable across concurrent execution contexts. PHP does have a thread-safe ZTS mode. And a long-term goal should be to allow for parallel (multiple concurrent execution) asynchronous tasks, especially if we want to expand asynchronous from just being a way to accomplish other single-threaded work while waiting on I/O.
In a concurrent environment, one has to consider the effects of multiple threads trying to read or write from the same value at once. (Strictly speaking, even in a single-threaded world, that's not necessarily safe to ignore; the stdclib or kernel code behind the scenes may be doing all sorts of shenanigans with I/O behind everyone's back, though I suspect an async-aware PHP could itself be a sufficient buffer between that and userspace code.)
With multithreaded code, it's generally not safe to pass around types unless you can reason about whether it's actually safe to do so. This could be trivially encoded in the type system with, say, a marker interface that declares that a type is safe to send across threads. But this needs to be opt-in, and the compiler needs to enforce it, or else you lose all pretense of safety.
It's tempting to handwave this now, but adding it in later might prove intractable due to the implied BC break. E.g. trivially, any call to spawn() might have its task scheduled on a different thread, but you can't do that safely unless you know all types involved are thread-safe. And while you could determine that at runtime, that's far more expensive and error-prone than making the compiler do it.
That said: the TrueAsync proposal isn't proposing async/await keywords. Perhaps it's sufficient to say that "TrueAsync" is single-thread only, and the multithreaded case will be handled with a completely different API later so BC isn't an issue. But either way, this should be at least mentioned in the Future Scope section.
- Rob Landers brought up the question about multiple-value Awaitables. I agree with him that it is a potential footgun. If an Awaitable is going to return multiple values, then it should implement have a different interface to indicate that contract.
Returning multiple values is fundamentally different than returning one value, and providing space for a contract change of that magnitude without reflecting in the type system is likely to cause significant problems as async interfaces evolve. Doing so would be like allowing a function that is defined to return an int to actually return an array of ints with no change to its type. That is clearly wrong. One should be able to look at a type definition and tell from its interfaces whether it is single-value or multiple-value.
(For example: Swift's standard library provides the AsyncSequence interface that one uses when wanting to asynchronously stream values. A concrete AsyncStream type provides an implementation of AsyncSequence that allows easily converting callback-based value streaming into an asynchronous sequence that can be iterated on with a for await loop. You don't await on an AsyncSequence type itself; instead you get an iterator and use the iterator to get values until the iterator returns null or throws an error. This is assisted with the syntactic sugar of a for await loop.)
Rob: The answer in Swift to what happens if multiple readers await a non-idempotent Awaiter (e.g., there are multiple readers for an AsyncSequence) is that as new values are made available, they go in turn to whatever readers are awaiting. So if you have five readers of an AsyncSequence, and the sequence emits ten values, they may all go to one reader, or each reader might get two values, or any combination thereof, depending on core count and how the scheduler schedules the tasks. So this can be used to spread workload across multiple threads, but if you need each reader to get all the values, you need to reach for other constructs.
I don't have a particular view on whether values should be replicated to multiple readers or not, but I do find it a bit annoying that Swift's standard library doesn't provide an easy way to get replicated values. So whichever way PHP goes, it should be clearly documented, and there be an easy way to get the other behavior.
- For cancellation, I'm confused on the resulting value. The section on cancellation suggests ("If a coroutine has already completed, nothing happens.") but does not explicitly state that an already-completed coroutine will have its return value preserved. The section on self-cancellation explicitly says that a self-cancelled coroutine throws away its return value, despite ultimately successfully completing. This seems counter-intuitive. Why should a self-cancellation be treated differently than an external cancellation? If the task completes through to the end and doesn't emit that it was cancelled, why should its return value be ignored in some cases and preserved in others?
I think self-cancellation is a weird edge case. It should probably be prohibited like self-await is. In general, I think a coroutine shouldn't ever have a handle to itself. Why would it need that? If a coroutine needs to abort early, it should throw or emit a partial result directly, rather than using the cancel mechanism.
-
exit/die called within a cooroutine is defined (to immediately terminate the app). For clarity, the case when exit/die is called outside of async code while there is a coroutine executing should also be explicitly documented. (Presumably this means that the async code is immediately terminated, but you don't state that, and Async\protect() would suggest that maybe it doesn't always apply.)
-
I think Async\protect() and forcibly cancelling coroutines beyond calling $coroutine->cancel() should be removed. Async\protect() is the answer to "but I need to make sure that this gets completed when a task is forcibly cancelled". But that just begs for a $scope->reallyCancelEvenProtectedCoroutines() and a Async\superProtect() to bypass that, and so on.
Yes, this means that a coroutine might never complete. But this is no different than non-async code calling a function that has an infinite loop and never returns.
-
Aleksander Machniak brought up possibly merging suspend and delay. As you note, they are semantically different (delay for a specified time interval, vs. cooperatively yield execution for an unspecified time), and I think they should stay separate. If we're bikeshedding, I might rename suspend() to yield(), though that might not be possible with yield being a language keyword. That could be worked around if yield() was, say, a static method on Coroutine, so you would call Coroutine::yield() to yield the current coroutine. (That could also apply to isCancelled, so a particular coroutine would never need to have a reference to itself to check if cancellation is requested.)
-
Also bikeshedding, but I would suggest renaming Coroutine to Task. Task is shorter and much easier to say and type, but more importantly, it encourages thinking about the effect (code that will eventually return a value), rather than how that effect is achieved. It also means the same abstraction can be used in a multithreaded environment.
-John
Hi Edmond,
Thanks for putting in the work to start to bring async support to PHP. I have a few initial comments:
Is TrueAsync intended strictly to enable concurrency without parallelism (e.g. single-threaded cooperative multitasking)? Is there a future path in mind for parallelism (actual simultaneous execution)?
Following on from that question, I note that there is no way to annotate types as being safely sendable across concurrent execution contexts. PHP does have a thread-safe ZTS mode. And a long-term goal should be to allow for parallel (multiple concurrent execution) asynchronous tasks, especially if we want to expand asynchronous from just being a way to accomplish other single-threaded work while waiting on I/O.
In a concurrent environment, one has to consider the effects of multiple threads trying to read or write from the same value at once. (Strictly speaking, even in a single-threaded world, that's not necessarily safe to ignore; the stdclib or kernel code behind the scenes may be doing all sorts of shenanigans with I/O behind everyone's back, though I suspect an async-aware PHP could itself be a sufficient buffer between that and userspace code.)
With multithreaded code, it's generally not safe to pass around types unless you can reason about whether it's actually safe to do so. This could be trivially encoded in the type system with, say, a marker interface that declares that a type is safe to send across threads. But this needs to be opt-in, and the compiler needs to enforce it, or else you lose all pretense of safety.
It's tempting to handwave this now, but adding it in later might prove intractable due to the implied BC break. E.g. trivially, any call to spawn() might have its task scheduled on a different thread, but you can't do that safely unless you know all types involved are thread-safe. And while you could determine that at runtime, that's far more expensive and error-prone than making the compiler do it.
That said: the TrueAsync proposal isn't proposing async/await keywords. Perhaps it's sufficient to say that "TrueAsync" is single-thread only, and the multithreaded case will be handled with a completely different API later so BC isn't an issue. But either way, this should be at least mentioned in the Future Scope section.
Off-topic a bit: but for the last several+ weeks, I’ve been reworking TSRM (moving it into Zend & making it a bit more frankenphp friendly) which could -- eventually -- allow for passing arbitrary types between threads without complex serialization and shenanigans.
— Rob
- Aleksander Machniak brought up possibly merging suspend and delay. As you note, they are semantically different (delay for a specified time interval, vs. cooperatively yield execution for an unspecified time), and I think they should stay separate. If we're bikeshedding, I might rename suspend() to yield(), though that might not be possible with yield being a language keyword. That could be worked around if yield() was, say, a static method on Coroutine, so you would call Coroutine::yield() to yield the current coroutine. (That could also apply to isCancelled, so a particular coroutine would never need to have a reference to itself to check if cancellation is requested.)
Yeah, my first reaction was "suspend() does not sound right, it should
be yield", but we indeed have yield already, so that might be confusing.
Why I wanted to get rid of delay()? Because "The delay function suspends
the execution of a coroutine". So, to me it implies suspend. Am I right
that it is essentially suspend() + usleep()?
--
Aleksander Machniak
Kolab Groupware Developer [https://kolab.org]
Roundcube Webmail Developer [https://roundcube.net]
PGP: 19359DC1 # Blog: https://kolabian.wordpress.com
Hi
- Is TrueAsync intended strictly to enable concurrency without parallelism (e.g. single-threaded cooperative multitasking)?
Is there a future path in mind for parallelism (actual simultaneous execution)?
In short, it mostly depends on the PHP core. TrueAsync itself could
support multithreading.
At the moment, achieving a fully multithreaded PHP seems extremely
complex and offers very little benefit.
In other words, the cost-to-benefit ratio is close to “irrational.”
If you’d like, I can elaborate on this idea, though probably not in this thread.
Following on from that question, I note that there is no way to annotate types as being safely sendable across concurrent execution contexts
I actually studied this topic about a year ago. All of this can be
done if one really wants to.
To put it simply, implementing a SharedObject is already possible now,
and it seems that it already exists in the parallel extension.
As for data types and deep changes to PHP, that’s not directly related
to TrueAsync. And even if it is, it can always be changed in the
future.
If someone decides to create a parallel version of PHP, they will
definitely face backward compatibility issues. I wouldn’t even worry
about that :)
That said: the TrueAsync proposal isn't proposing async/await keywords.
There is an RFC proposing new syntax, but it was unanimously decided
not to use it. For now. Maybe in PHP 9?
Perhaps it's sufficient to say that "TrueAsync" is single-thread only, and the multithreaded case will be handled with a completely different API later so BC isn't an issue. But either way, this should > be at least mentioned in the Future Scope section.
Yes, of course, that’s possible.
- Rob Landers brought up the question about multiple-value Awaitables.
I agree with him that it is a potential footgun. If an Awaitable is going to return multiple values, then it should implement have a different interface to indicate that contract.
It seems to me that this discussion has gotten a bit tangled:
- Functions like awaitXX are not part of this RFC, yet we’re
discussing them. Is that really necessary? - Awaitable objects do not return multiple values.
- The Awaitable interface is basic for others, just like in many
other programming languages. And for some reason, it’s not considered
a problem there.
So far, no real example has been found where this would actually be a problem.
For example: Swift's standard library provides the AsyncSequence
Comparing the Swift model directly with TrueAsync would be quite
difficult, and such a discussion would require studying Swift’s
implementation.
It’s not even worth comparing it to Rust, since Rust has its own
Future-based model, while Swift relies on code generation.
I haven’t looked deeply into this topic, so I can’t really add much more.
It would be good to fully figure things out with PHP first :)
The answer in Swift to what happens if multiple readers await a non-idempotent Awaiter
This is the general approach for the Event-Driven pattern, because
asynchronous execution under the hood is event-driven.
That’s exactly why the Awaitable exists. It acts as an EventProvider,
and await() is effectively a subscription to events.
Therefore, when await, awaitAll, or awaitAny is called, the code
receives the first event, not multiple events.
I don't have a particular view on whether values should be replicated to multiple readers or not,
but I do find it a bit annoying that Swift's standard library doesn't provide an easy way to get replicated values.
The current RFC does not prohibit this. It does not restrict how
objects produce their results or how many subscribers they send them
to. This behavior is part of the object’s own contract.
For cancellation, I'm confused on the resulting value.
Thank you, that’s a great catch!!!
I need to check this case.
For clarity, the case when exit/die is called outside of async code
In TrueAsync, there is no longer any non-asynchronous code. Everything
is a coroutine.
Actually, that’s not entirely accurate, but it’s fine to think of it that way.
I think Async\protect() and forcibly cancelling coroutines beyond calling $coroutine->cancel() should be removed.
If PHP were a language for spacecraft or airplanes, where extremely
high reliability is required, this might make sense.
However, the current model allows for more concise code and has
already proven itself well in other languages.
I actually have a stricter explanation for why the cancellable model
is more advantageous than any other.
This particular model fits well for read operations, which are usually
as frequent as, or even more frequent than, write operations.
Therefore, this approach simplifies asynchronous code while keeping
the overall risk low.
Yes, this means that a coroutine might never complete.
But this is no different than non-async code calling a function that has an infinite loop and never returns.
The cancellation (a cooperative cancellation model) mechanism does not
exist to interrupt infinite loops.
It is designed to provide a simple yet powerful way to cancel
coroutines that are running normally.
“Normally” means without an infinite loop.
Also bikeshedding, but I would suggest renaming Coroutine to Task.
It could be named that way too. Although I don’t think there’s much
difference here. But Task is a shorter word.
Thanks, Ed
Hi Edmond,
Hi
- Is TrueAsync intended strictly to enable concurrency without parallelism (e.g. single-threaded cooperative multitasking)?
Is there a future path in mind for parallelism (actual simultaneous execution)?In short, it mostly depends on the PHP core. TrueAsync itself could
support multithreading.
At the moment, achieving a fully multithreaded PHP seems extremely
complex and offers very little benefit.
In other words, the cost-to-benefit ratio is close to “irrational.”If you’d like, I can elaborate on this idea, though probably not in this thread.
To be clear, since I didn't explicitly state this, my point is that I would like to see whatever gets introduced by any RFC (as this one is) that is likely to be used as a stepping stone to full multithreaded async PHP (whether or not we think that such a destination is likely on any practical timescale) NOT get in the way of that destination. That is, I don't want us to go partway down a path, only to find later that we can't make it to the destination because of a bad or incomplete decision early on. Which is why I'm asking about stuff that might not be in scope for this RFC, because it will be in-scope for some future RFC, and if we wind up with a bad interface now, it makes that future RFC's job unnecessary harder. And since this isn't going to be shipping to any end-user for at least another year, we have plenty of time to make sure we get the interface right.
- Rob Landers brought up the question about multiple-value Awaitables.
I agree with him that it is a potential footgun. If an Awaitable is going to return multiple values, then it should implement have a different interface to indicate that contract.It seems to me that this discussion has gotten a bit tangled:
- Functions like awaitXX are not part of this RFC, yet we’re
discussing them. Is that really necessary?
I can't find where this awaitXX function is defined, so I can't comment on that. But I didn't bring it up, so I'm unclear why you are.
- Awaitable objects do not return multiple values.
So you say "Awaitable objects do not return multiple values." but the description of the Awaitable interface says it allows for that. Unless I'm misunderstanding something, I don't think that makes any sense. And also you seem to be saying that some objects are Awaitable, and also there's an Awaitable interface that means something different, which is definitely going to cause confusion.
- The Awaitable interface is basic for others, just like in many
other programming languages. And for some reason, it’s not considered
a problem there.
So far, no real example has been found where this would actually be a problem.
This may not be your intention, but to me it feels like you're just dismissing Rob and my concern, which is that single-value and multiple-value are two completely different beasts and PHP shouldn't allow you to just casually mix them without throwing a compile-time error. Rob has provided an example where this is a problem.
And Swift does recognize this as a problem, that's why if you want to iterate on the result of an asynchronous stream of values, you use an AsyncStream, and not a Task/Coroutine. (Granted: the stream's values might be sourced from within a Task, but that's an implementation detail. And why there's also a TaskGroup if what you want is to spawn a one async task per item in an iterable and iterate over the results as they complete.)
(btw, if you haven't read through the Swift Evolution proposals for structured concurrency, I highly recommend you do, if only to see what/how Swift has chosen to do. And then what they got wrong and the hoops they jumped through to try and remedy it afterwards, some of which many people here would find quite distasteful.)
For cancellation, I'm confused on the resulting value.
Thank you, that’s a great catch!!!
I need to check this case.For clarity, the case when exit/die is called outside of async code
In TrueAsync, there is no longer any non-asynchronous code. Everything
is a coroutine.
Actually, that’s not entirely accurate, but it’s fine to think of it that way.
Then, just saying, this is a place where people are going to be confused, and it needs to be stated clearly. The description in "exit and die keywords" section shouldn't be under cancellation policy, and shouldn't special-case being called within a coroutine. If exit/die still immediately terminate execution, then there should be no confusion about that.
I think Async\protect() and forcibly cancelling coroutines beyond calling $coroutine->cancel() should be removed.
If PHP were a language for spacecraft or airplanes, where extremely
high reliability is required, this might make sense.
However, the current model allows for more concise code and has
already proven itself well in other languages.
There may be a misunderstanding here. I'm not saying you should remove cooperative cancelation. That's perfectly fine and should stay. With cooperative cancellation, it's not necessary to assert you're in a critical section, you just ignore the cancellation signal and keep on processing. What I'm proposing that specifically Async\protect() and forcibly cancelling coroutines should be removed. (You'll note that in some multithreading APIs, forcibly terminating threads, while allowed, is very much discouraged because that can leave things in inconsistent states that causes future crashes or unpredictable behavior.)
I actually have a stricter explanation for why the cancellable model
is more advantageous than any other.
This particular model fits well for read operations, which are usually
as frequent as, or even more frequent than, write operations.
Therefore, this approach simplifies asynchronous code while keeping
the overall risk low.
The RFC should include a description of why cooperative cancellation has been chosen.
Thanks, Ed
-John
Hi,
it makes that future RFC's job unnecessary harder
If you want to make PHP multithreaded and plan to change the language
syntax while avoiding future issues, it would be reasonable to discuss
the changes using specific syntax examples.
Can you show me the cases you want to implement?
I can't find where this
awaitXXfunction is defined
It was in the third version of the RFC.
So you say "Awaitable objects do not return multiple values.
- multiple values:
$fs = new FileSystemEvents();
// At that moment, the user created file1 and then file2.
$event = await($fs);
// Now: $event contains [file1, file2] -- The coroutine captured two
events at once.
- signe value:
$fs = new FileSystemEvents();
// At that moment, the user created file1 and then file2.
$event = await($fs);
// Now: $event contains file1 -- ONLY ONE VALUE!
but to me it feels like you're just dismissing Rob and my concern
It’s hard three times over, because the expression foreach(await() as
) is considered valid only if await returns a value once.
And Swift does recognize this as a problem
I don’t know whether Swift actually recognizes it as a problem or not,
but it actively uses syntax to describe contracts.
And that’s a slightly different level of control compared to just
functions and interfaces.
Let me repeat the main conclusions:
- There’s no problem making
FutureLikethe base interface, which
would guarantee the idempotence of the data source. - The benefit is that calling
awaittwice would become more
predictable (although the data type would still be unknown). - However, we would lose an interface that describes the real world
as it is, and the programmer would lose some flexibility. - Yet, considering that non-idempotent
Awaitableobjects, while
allowed, are rarely used in high-level languages, this may not
actually be a problem.
Then, just saying, this is a place where people are going to be confused, and it needs to be stated clearly.
I don’t mind; I should take a look at this point.
btw, if you haven't read through the Swift Evolution proposals for structured concurrency,
Thanks for the idea. I haven’t read that discussion.
Is it https://forums.swift.org/t/se-0304-structured-concurrency/45314 ?
What I'm proposing that specifically Async\protect() and forcibly cancelling coroutines should be removed.
This RFC does not include forced coroutine termination, only
cooperative cancellation using exceptions.
If we remove the protect function, we’ll also have to remove
cooperative cancellation.
You'll note that in some multithreading APIs, forcibly terminating threads, while allowed
This feature was implemented in Swow using the VM interruption mechanism.
That’s why I wrote that such a solution is possible if someone needs
it, but it has nothing to do with either protect or cooperative
cancellation.
The RFC should include a description of why cooperative cancellation has been chosen.
If I add detailed explanations for every why in the RFC, it might
never get read :)
Thanks, Ed
Hi,
The RFC should include a description of why cooperative cancellation has been chosen.
If I add detailed explanations for every why in the RFC, it might
never get read :)Thanks, Ed
If someone opens a PR to fix a bug or an RFC that builds on this and there is question of whether the bug is actually a feature or if it should be done a different way — years from now — often the only thing to go off of is the reasoning in the RFC. See the recent Readonly Property Hooks discussion for an example, where it was different people interpreting the same near-decade-old RFC different ways. Spelling it out might feel long-winded, but it is already long… a few more pages won’t hurt.
— Rob
Hi
a few more pages won’t hurt
Yes, that makes sense.
But I’m more inclined to think that documentation is a better place
for it or that some comments should be moved to a separate section.
Hi
a few more pages won’t hurt
Yes, that makes sense.
But I’m more inclined to think that documentation is a better place
for it or that some comments should be moved to a separate section.
I once said something similar on my first RFC, this is about how it was explained to me:
The documentation team writes the documentation from the RFC. I’m sure you can contribute, but for a new feature, there’ll probably be some pushback on why it’s not in the RFC. Granted, you’d probably get some leeway because you’re the author of the RFC, but it’s not much different than some random person opening a PR explaining a feature and there is no guarantee you’d actually contribute to the documentation, so putting it in the RFC is the simplest solution.
— Rob
The documentation team writes the documentation from the RFC
Thanks, I’ll keep that in mind.
Hi,
Hi,
it makes that future RFC's job unnecessary harder
If you want to make PHP multithreaded and plan to change the language
syntax while avoiding future issues, it would be reasonable to discuss
the changes using specific syntax examples.
Can you show me the cases you want to implement?
I don't have any specific use-cases in mind. I'm just trying to be forward-looking here. But what is clear is that some people would like to see async/await in PHP. This RFC would ostensibly be a step towards that. All I'm trying to do is make sure that it doesn't put up roadblocks for future enhancements.
I can't find where this
awaitXXfunction is defined
It was in the third version of the RFC.So you say "Awaitable objects do not return multiple values.
- multiple values:
$fs = new FileSystemEvents(); // At that moment, the user created file1 and then file2. $event = await($fs); // Now: $event contains [file1, file2] -- The coroutine captured two events at once.
- signe value:
$fs = new FileSystemEvents(); // At that moment, the user created file1 and then file2. $event = await($fs); // Now: $event contains file1 -- ONLY ONE VALUE!but to me it feels like you're just dismissing Rob and my concern
It’s hard three times over, because the expression foreach(await() as
) is considered valid only if await returns a value once.
I think there are two bad designs here.
First: If you have something that streams results, then the interface to access should reflect streaming. And it should be an error (preferably thrown at compile time) if you try and treat a stream as a single value, or a single value as a stream. This is whether it's async or not. And we already have an interface one can implement if one has an iterable sequence of events, so that you can either call the iteration functions directly, or use foreach() which provides syntactic sugar over that.
Second: it should be emitting either a stream of files, or a stream of arrays of files. Returning list<File>|File is not a great pattern.
btw, if you haven't read through the Swift Evolution proposals for structured concurrency,
Thanks for the idea. I haven’t read that discussion.
Is it https://forums.swift.org/t/se-0304-structured-concurrency/45314 ?
That's the first review thread for Swift's Structured Concurrency proposal. The proposal itself is here, along with links to the three pitch and three review threads, and the async/await and actor proposals:
https://github.com/swiftlang/swift-evolution/blob/main/proposals/0304-structured-concurrency.md
(see also the async/await and actors proposals, linked from that proposal. Note that a nontrivial portion of the proposal covers actor isolation, which is obviously beyond the scope of this RFC, but whether or not a future multithreading async extension uses actor isolation as its safety concept, those concepts are still generally relevant.)
What I'm proposing that specifically Async\protect() and forcibly cancelling coroutines should be removed.
This RFC does not include forced coroutine termination, only
cooperative cancellation using exceptions.
If we remove the protect function, we’ll also have to remove
cooperative cancellation.
Why is the protect function needed, then? If you request cancellation, the coroutine is free to ignore the cancellation and run to completion. That's what makes it cooperative.
The RFC should include a description of why cooperative cancellation has been chosen.
If I add detailed explanations for every why in the RFC, it might
never get read :)
In addition to Rob's comments on expressing intent and a source for documentation: concurrency is a large, confusing, and hard topic. Anything that's unclear or underspecified is only going to cause problems in the long run. It's better to be as clear as possible so that when it eventually comes to voting, you have less chance of people voting No because of that reason.
Thanks, Ed
-John
Hi,
Hi,
it makes that future RFC's job unnecessary harder
If you want to make PHP multithreaded and plan to change the language
syntax while avoiding future issues, it would be reasonable to discuss
the changes using specific syntax examples.
Can you show me the cases you want to implement?I don't have any specific use-cases in mind. I'm just trying to be forward-looking here. But what is clear is that some people would like to see async/await in PHP. This RFC would ostensibly be a step towards that. All I'm trying to do is make sure that it doesn't put up roadblocks for future enhancements.
I can't find where this
awaitXXfunction is defined
It was in the third version of the RFC.So you say "Awaitable objects do not return multiple values.
- multiple values:
$fs = new FileSystemEvents(); // At that moment, the user created file1 and then file2. $event = await($fs); // Now: $event contains [file1, file2] -- The coroutine captured two events at once.
- signe value:
$fs = new FileSystemEvents(); // At that moment, the user created file1 and then file2. $event = await($fs); // Now: $event contains file1 -- ONLY ONE VALUE!but to me it feels like you're just dismissing Rob and my concern
It’s hard three times over, because the expression foreach(await() as
) is considered valid only if await returns a value once.I think there are two bad designs here.
First: If you have something that streams results, then the interface to access should reflect streaming. And it should be an error (preferably thrown at compile time) if you try and treat a stream as a single value, or a single value as a stream. This is whether it's async or not. And we already have an interface one can implement if one has an iterable sequence of events, so that you can either call the iteration functions directly, or use foreach() which provides syntactic sugar over that.
Second: it should be emitting either a stream of files, or a stream of arrays of files. Returning list<File>|File is not a great pattern.
btw, if you haven't read through the Swift Evolution proposals for structured concurrency,
Thanks for the idea. I haven’t read that discussion.
Is it https://forums.swift.org/t/se-0304-structured-concurrency/45314 ?That's the first review thread for Swift's Structured Concurrency proposal. The proposal itself is here, along with links to the three pitch and three review threads, and the async/await and actor proposals:
https://github.com/swiftlang/swift-evolution/blob/main/proposals/0304-structured-concurrency.md(see also the async/await and actors proposals, linked from that proposal. Note that a nontrivial portion of the proposal covers actor isolation, which is obviously beyond the scope of this RFC, but whether or not a future multithreading async extension uses actor isolation as its safety concept, those concepts are still generally relevant.)
What I'm proposing that specifically Async\protect() and forcibly cancelling coroutines should be removed.
This RFC does not include forced coroutine termination, only
cooperative cancellation using exceptions.
If we remove the protect function, we’ll also have to remove
cooperative cancellation.Why is the protect function needed, then? If you request cancellation, the coroutine is free to ignore the cancellation and run to completion. That's what makes it cooperative.
I assume it is because you can't control the whole stack. For example, if you're calling file_put_contents(), you don't want it to cancel if a cancellation is received from outside the coroutine, which might put your application in a bad state. This goes back to including the reasoning in the RFC though, because I can only assume.
— Rob
Hi
I don't have any specific use-cases in mind. I'm just trying to be forward-looking here. But what is clear is that some people would like to see async/await in PHP. This RFC would ostensibly be a
step towards that. All I'm trying to do is make sure that it doesn't put up roadblocks for future enhancements.
The main obstacles to multithreading are:
- The state of the PHP virtual machine, as well as how the bytecode is
stored (some elements cannot be transferred) - The memory manager
- The garbage collector
Of course, for a multithreaded version, the scheduler would have to be
almost completely rewritten, but this task doesn’t seem as demanding
as the ones above.
As for the RFC itself, it does not impose any additional limitations
beyond those already inherent to PHP.
If coroutines can be moved between threads, then all variables must be
thread-safe. How will that be guaranteed? The context of a Closure
must also be thread-safe, how will that be ensured?
All these questions are outside the scope of this RFC and relate to
the implementation of closures.
There is another risk here.
If PHP introduces async functionality that works within a single
thread, users will start writing code with closures, assuming it will
work correctly because everything runs in one thread.
The RFC does not regulate this at all, and that’s the problem.
Developers will begin writing code designed for single-threaded behavior.
If multithreading is later introduced, that code will inevitably break.
There’s much more that could be added here. The main point is that the
RFC is not designed around a memory model for parallelism, and such a
model should not be part of the RFC.
If you’re planning such changes for PHP, it would be wise to come up
with a way to handle them.
But to do that, we first need to understand which memory model you’ll choose.
Will it be Actors + SharedState, meaning special kinds of closures, or
something else?
Second: it should be emitting either a stream of files, or a stream of arrays of files. Returning list<File>|File is not a great pattern.
I wasn’t suggesting doing that, I just wanted to show the difference
between “multiple values” and “a single value.”
That's the first review thread for Swift's Structured Concurrency proposal.
The proposal itself is here, along with links to the three pitch and three review threads, and the async/await and actor proposals:
Thank you. I will look at it.
Why is the protect function needed, then? If you request cancellation, the coroutine is free to ignore the cancellation and run to completion. That's what makes it cooperative.
Because cancellation interrupts coroutine waiting.
If cancellation occurs deep within the runtime while it is writing a
buffer to a socket, it means that not all data will be written. That’s
exactly the case where protect is useful.
Thanks, Ed.
I’d like you to have the full information necessary to make an
informed decision.
There are two or more channel objects that need to be awaited. How
should this be implemented?
Currently
[$result1, $result2] = awaitAny($channel1, $channel2);
With Futures:
// Like Rust
[$result1, $result2] = awaitAny($channel1->recv(), $channel2->recv());
Rust:
tokio::select! {
msg = channel1.recv() => println!("Got from 1: {:?}", msg),
msg = channel2.recv() => println!("Got from 2: {:?}", msg),
}
Swift:
async let result1 = fetchData1()
async let result2 = fetchData2()
let (r1, r2) = await (result1, result2)
Although both Swift and Rust create a Future under the hood, that’s
part of the internal implementation, not the user-facing contract.
What I dislike about the Future example is that it creates an object
that isn’t actually needed.
What I like about it is that it clearly shows what action is being awaited.
When executing the awaitXX operation in a loop for such scenarios, the
programmer will pay for the visual clarity of the code with extra
operations to create a PHP object.
Are you sure this is what you really want?
I want to caution you against directly comparing PHP, Rust, and Swift.
What a compiler and zero-abstraction languages can do, PHP cannot.
And the approaches used by Rust or Swift cannot be directly applied to
a language without a compiler.
I have no objection to doing it the way you prefer, but keep in mind
that chasing what looks elegant often leads to poor decisions. It’s
better to think it over a few times.
I’d like you to have the full information necessary to make an
informed decision.There are two or more channel objects that need to be awaited. How
should this be implemented?Currently
[$result1, $result2] = awaitAny($channel1, $channel2);With Futures:
// Like Rust [$result1, $result2] = awaitAny($channel1->recv(), $channel2->recv());What I dislike about the Future example is that it creates an object
that isn’t actually needed.
What I like about it is that it clearly shows what action is being awaited.
Specifically:
What I dislike about the Future example is that it creates an object
that isn’t actually needed.
Why does it need to 'create' an object? Flyweights are a well known pattern and PHP knows the number of references to any specific object. Prevent them from being GC'd and have a pool of them for reuse. You don't need to actually create anything. That being said, that's an optimization for later.
As for which one I'd rather have, I'd rather have the one where my examples hold true:
assert(awaitAll($a, $b, $c) === [await($a), await($b), await($c)]);
It's clear what it does and that it is just a shorthand. When that fails, we have issues.
— Rob
I’d like you to have the full information necessary to make an
informed decision.There are two or more channel objects that need to be awaited. How
should this be implemented?Currently
[$result1, $result2] = awaitAny($channel1, $channel2);As for which one I'd rather have, I'd rather have the one where my examples hold true:
assert(awaitAll($a, $b, $c) === [await($a), await($b), await($c)]);
It's clear what it does and that it is just a shorthand. When that fails, we have issues.
— Rob
Just to add to this, in Swift, a Task (the analogue to this proposal's Coroutine) can emit, exactly once, a value or an error (exception). Attempting to read its value (or error) multiple times gets you the same value back. This is very handy in memoization patterns.
In this example, multiple readers can call getImage() with the same id and get a value, regardless of whether they're the first (which kicks off image loading), whether the image loading has completed, or whether the image load is still in progress. (Technically, one could omit the LoadingBox wrapper entirely, but that would cause the Task to persist after it's completed, and that's a more expensive structure than a simple enum.)
private enum LoadingBox<T : Sendable> {
case inProgress(Task<T, Never>) //Task<Success, Failure>
case ready(T)
var value: T {
get async {
switch self {
case .inProgress(let task):
return await task.value
case .ready(let value):
return value
}
}
}
}
private var imageCache: [ImageId : LoadingBox<Image>] = [:]
func getImage(_ id: ImageId) async -> Image {
if let cache = imageCache[id] {
return await cache.value
}
let task = Task { ...load the image... }
imageCache[id] = task
return await task.value
}
Ultimately, I would want to be able to write code such as this in PHP (with the generics, of course, but I'm not holding my breath).
If coroutines sometimes return single values and sometimes return multiple values, it becomes much harder to write a generic async cache/memoization system, because you can't indicate via the type system that a single value is expected.
-John
If coroutines sometimes return single values and sometimes return multiple values, it becomes much harder to write a generic async cache/memoization system, because you can't indicate via the > type system that a single value is expected.
There was never a proposal for coroutines to return different values.
The whole debate revolves around the await() function. But it seems
we’ve already decided what to do with it.
Hello.
It seems to me that the following solution balances the different
viewpoints better:
- Keep the base
Awaitableinterface, which defines an object that
can be awaited. - Introduce a
FutureLikeinterface that extendsAwaitable. - Modify the
await()function so that it only acceptsFutureLikeobjects. - Functions such as
awaitAllorawaitAnymay need to be renamed,
but that’s outside the scope of this RFC. - There will be no need to create a
Futurefor functions designed
to work with anAwaitableobject — this keeps theselect case
syntax clean.
Naming issue
I see a certain inconsistency between await and Awaitable.
It might be worth coming up with a better name for Awaitable.
I remember that “Observable” was once suggested.
Hello.
It seems to me that the following solution balances the different
viewpoints better:
- Keep the base
Awaitableinterface, which defines an object that
can be awaited.- Introduce a
FutureLikeinterface that extendsAwaitable.- Modify the
await()function so that it only acceptsFutureLikeobjects.- Functions such as
awaitAllorawaitAnymay need to be renamed,
but that’s outside the scope of this RFC.- There will be no need to create a
Futurefor functions designed
to work with anAwaitableobject — this keeps theselect case
syntax clean.Naming issue
I see a certain inconsistency between
awaitandAwaitable.
It might be worth coming up with a better name forAwaitable.
I remember that “Observable” was once suggested.
Why not keep the name Awaitable instead of creating a FutureLike, but just harden the interface? We can always loosen the interface in the future, or create a new interface for "streaming" awaitables (aka, multi-shot Awaitable). Then await() could change its signature from Async\await(Awaitable $awaitable): mixed to Async\await(Awaitable|StreamingAwaitable $awaitable) at some point in the future -- and it would be BC.
— Rob
Why not keep the name Awaitable instead of creating a FutureLike, but just harden the interface? We can always loosen the interface in the future, or create a new interface for "streaming"
awaitables (aka, multi-shot Awaitable). Then await() could change its signature from Async\await(Awaitable $awaitable): mixed to Async\await(Awaitable|StreamingAwaitable $awaitable) at some
point in the future -- and it would be BC.
The Awaitable interface is not typically associated with
Future/Promise. While the term Future is known to almost everyone. The
degree of recognition for Future is much higher.
The term StreamingAwaitable reflects a different meaning.
The fact that FutureLike extends the Awaitable interface reflects the
idea that Futures can be used in select-like operations just as
Awaitable objects can.
If someone prefers a different name instead of Awaitable, I have
nothing against it, but please provide a rationale.
This seems a bit contradictory and confuses things. When I await(), do I need to do it in a loop, or just once?
It might be a good idea to make a couple subtypes: Signal and Future. Coroutines become Future that only await once, while Signal is something that can be awaited many times.I made a mistake in my previous response, and it requires clarification.
Classes that can be awaited multiple times are indeed possible.
These includeTimeInterval,Channel,FileSystemEvent, as well as
I/O triggers.All of these classes can be
Awaitable.
This is done to allow bulk waiting on objects, regardless of how they
work internally.
So this is meant for functions likeawaitXX, although such behavior
is also possible for theawait()function itself:$timeInterval = new Async\TimeInterval(1000); while(true) { await($timeInterval); }As for objects of type
Future, it’s clear that in future RFCs there
will be aFutureInterface, which will be implemented by coroutines.
Hi Edmond,
I've been meaning to review your RFC and implementation for some time, but for various reasons, I still haven't been able to give it a thorough read and review.
I noticed this portion of the discussion and wanted to drop a note now, rather than waiting until I was able to read the entire RFC.
Awaitables should always represent a single value. Awaiting multiple times should never result in a different value.
Async sets of values should use a different abstraction to represent a set. rxjs Observables (rxjs.dev) are on example. AMPHP has a pipeline library, https://github.com/amphp/pipeline, which defines a ConcurrentIterator interface. The latter IMO is more appropriate for PHP + fibers. I recommend having a look at how Future and ConcurrentIterator are used within AMPHP libraries.
I think you should consider additional time beyond only two more weeks for discussion of this RFC before bringing it to a vote. PHP 8.6 or 9 is some time away. This is definitely not an RFC to rush to voting.
Cheers,
Aaron Piotrowski
The RFC is not easy to process. Here's some ideas.
- The glossary in "Overview" is good, but probably incomplete. The
examples there, with no description, do not help much and could be
removed, imo. - "Collable by design" and "Coroutine lifetime" sections should become
subsections of "Coroutine" or placed after it. - the "Scheduler and Reactor" section does not explain much over what's
in the glossary. - the "Critical section" section should not be a main section,
"Cancellation policy" probably either. - the "Basic usage" section in the "Suspension" section is useless.
- don't use "suspend keyword" and "await keyword", they are functions.
One question. Seems like we don't really need delay() function. Why not
add an argument to the suspend() function? I think it would make the
code easier to understand, considering seeing suspend(1000) versus
delay(1000).
--
Aleksander Machniak
Kolab Groupware Developer [https://kolab.org]
Roundcube Webmail Developer [https://roundcube.net]
PGP: 19359DC1 # Blog: https://kolabian.wordpress.com
Hello.
- The glossary in "Overview" is good, but probably incomplete. The
examples there, with no description, do not help much and could be
removed, imo.
Do I understand correctly that I should remove the examples without
descriptions?
Or would it be better to add descriptions to them?
Although the last examples might not be very illustrative or easy to grasp.
the "Scheduler and Reactor" section does not explain much over what's in the glossary.
What else do you think could be added?
The internal implementation doesn’t belong in the scope of the RFC, it
can change.
They don’t have any special API in the PHP userland.
The reactor can only be used directly at the C/C++ level, meaning
within a PHP extension.
It’s also intentionally impossible to directly affect the Scheduler’s behavior.
One question. Seems like we don't really need delay() function. Why not
add an argument to the suspend() function? I think it would make the
code easier to understand, considering seeing suspend(1000) versus
delay(1000).
That didn’t occur to me.
Interesting idea. You’re saying it would make the code more readable.
But wouldn’t it be confusing since the functions have slightly
different semantic purposes?
As a non-native English speaker, I don’t really feel the difference
between delay and suspend. They seem close in meaning. But is that the
case for others?
Thanks, Ed.
- The glossary in "Overview" is good, but probably incomplete. The
examples there, with no description, do not help much and could be
removed, imo.Do I understand correctly that I should remove the examples without
descriptions?
Examples in the Overview section aren't very helpful. I would remove
them from there, but maybe some of them need to appear later. I didn't
read it that carefully to suggest precise changes.
Or would it be better to add descriptions to them?
Although the last examples might not be very illustrative or easy to grasp.the "Scheduler and Reactor" section does not explain much over what's in the glossary.
What else do you think could be added?
The internal implementation doesn’t belong in the scope of the RFC, it
can change.
They don’t have any special API in the PHP userland.
The reactor can only be used directly at the C/C++ level, meaning
within a PHP extension.
It’s also intentionally impossible to directly affect the Scheduler’s behavior.
I think that it might be better to not mention them at all in the RFC.
Or better separate parts that describe userland and engine. The
structure of the Proposal is a bit chaotic.
--
Aleksander Machniak
Kolab Groupware Developer [https://kolab.org]
Roundcube Webmail Developer [https://roundcube.net]
PGP: 19359DC1 # Blog: https://kolabian.wordpress.com
Hi,
I think that it might be better to not mention them at all in the RFC.
Or better separate parts that describe userland and engine.
Different people have different opinions on this point.
But it seems there’s no harm in PHP developers knowing that these two
components exist under the hood.
Their description in the RFC isn’t part of the implementation details.
It’s more of a mention in the context of concurrency architecture.
The descriptions are given in the most abstract way possible and don’t
include any implementation details.
--- Ed
The volume of the discussion seems to have become too complex for
future processing.
Therefore, I tried to organize the proposals into specific tasks.
https://github.com/true-async/php-true-async-rfc/issues
I think this format will also be useful for those who want to see what
changes will be made and why they are being made.
Hello all.
Current work and discussion plan for this RFC
-
By the end of this week, the proposed changes at
https://github.com/true-async/php-true-async-rfc/issues
will be accepted if no objections are raised. -
After that, the RFC document will be updated, and a new 2-week
discussion period will begin. -
After the new changes are accepted, the RFC will be updated again,
and the process will repeat.
The discussion will be extended as long as necessary, including at the
request of the participants.
With best regards,
Ed
Hello all.
Current work and discussion plan for this RFC
- By the end of this week, the proposed changes at
https://github.com/true-async/php-true-async-rfc/issues
will be accepted if no objections are raised.
After that, the RFC document will be updated, and a new 2-week
discussion period will begin.After the new changes are accepted, the RFC will be updated again,
and the process will repeat.The discussion will be extended as long as necessary, including at the
request of the participants.
With best regards,
Ed
I am not sure how feasible this is, but would there be a way to split the "async toggle" of IO operations off to its own PR/RFC? To me, that is by far the most important part of this RFC as that's the biggest blocker for wider async adoption, but I'm not sure how many layers are needed above it to make it possible to toggle in a safe fashion.
--Larry Garfield
On Mon, Oct 27, 2025 at 6:05 PM Larry Garfield larry@garfieldtech.com
wrote:Hello all.
Current work and discussion plan for this RFC
- By the end of this week, the proposed changes at
https://github.com/true-async/php-true-async-rfc/issues
will be accepted if no objections are raised.
After that, the RFC document will be updated, and a new 2-week
discussion period will begin.After the new changes are accepted, the RFC will be updated again,
and the process will repeat.The discussion will be extended as long as necessary, including at the
request of the participants.
With best regards,
EdI am not sure how feasible this is, but would there be a way to split the
"async toggle" of IO operations off to its own PR/RFC? To me, that is by
far the most important part of this RFC as that's the biggest blocker for
wider async adoption, but I'm not sure how many layers are needed above it
to make it possible to toggle in a safe fashion.Hi!
Can you clarify what you mean by "async toggle"?
Is it the actual implementation that would use async constructs if the
current context is a coroutine for each implementation of IO functions?
Yes, for that, it would be nice to have separate PRs, even multiple ones
for easier review. But maybe you mean something else...--
Alex
I am not sure how feasible this is, but would there be a way to split the "async toggle" of IO operations off to its own PR/RFC? To me, that is by far the most important part of this RFC as that's the biggest blocker for wider async adoption, but I'm not sure how many layers are needed above it to make it possible to toggle in a safe fashion.
Hi!
Can you clarify what you mean by "async toggle"?
Is it the actual implementation that would use async constructs if the current context is a coroutine for each implementation of IO functions?
Yes, for that, it would be nice to have separate PRs, even multiple ones for easier review. But maybe you mean something else...--
Alex
The most important feature of this RFC, IMO, is that when async is "active", all IO operations become non-blocking and automatically suspend an active coroutine, so that other coroutines can act. That means you can write file_get_contents() or whatever, and in non-async land it will block as normal, but in async land it will suspend and let other coroutines run, then pick up again when ready, without requiring any code changes.
(At least that's how I understand that part of the RFC.)
That is huge, and easily the most important feature. Really, if we had that in core it would be possible to do most of the rest in user-space with the existing Fibers, I suspect. But I don't know how feasible it is to separate that part out, in large part because it would mean exposing some kind of way to toggle if async is "active" (for some definition of active). But if that is possible/feasible, that would be a much narrower, more easily reviewable, and still highly useful RFC that could be iterated on in both user space and core.
I do not know how coupled that is to the new Fiber-incompatible loop, which is the biggest problem.
--Larry Garfield
Hello
I am not sure how feasible this is, but would there be a way to split the "async toggle" of IO operations off to its own PR/RFC?
To me, that is by far the most important part of this RFC as that's the biggest blocker for wider async adoption,
but I'm not sure how many layers are needed above it to make it possible to toggle in a safe fashion.
Why is this considered the main blocker? (Or maybe I didn’t quite
catch the meaning.)
After all, this is precisely the part of the RFC that doesn’t actually
change the behavior from the user’s perspective inside a coroutine.
As for the PRs. There’s no doubt there will be several. I hope the PHP
core team will help with the code separation process and advise on the
best way to do it, since it’s not a trivial task.
As for the toggle switch. If a developer doesn’t use spawn, then why
toggle anything at all?
And even if they do, there’s still no difference — the code won’t
start executing “differently.”
Best Regards, Ed
I am not sure how feasible this is, but would there be a way to split the "async toggle" of IO operations off to its own PR/RFC?
To me, that is by far the most important part of this RFC as that's the biggest blocker for wider async adoption,
but I'm not sure how many layers are needed above it to make it possible to toggle in a safe fashion.
Ah, I think I understand what you mean.
Yes, from an implementation standpoint, non-blocking behavior is
provided by the Scheduler API and Reactor API. These two APIs are
always available anywhere — in the core, extensions, and so on.
However, this isn’t part of the RFC itself, as it belongs to
implementation details. Moreover, do you remember the first version of
the RFC? It had a function that explicitly started the Scheduler. So
indeed, in that first version PHP could be in two states: synchronous
and asynchronous.
What’s the difference in this RFC?
PHP is always in an asynchronous state. Even when executing
index.php, you can think of it as running inside a coroutine. There’s
just a single coroutine, so no switching occurs (although an extension
could, for example, create another coroutine — which is perfectly
valid).
You keep writing your code exactly as before. You don’t need to think
about PHP being “asynchronous” now. As long as you’re writing code
inside a single coroutine, you’re still writing synchronous code.
-- Ed
I am not sure how feasible this is, but would there be a way to split the "async toggle" of IO operations off to its own PR/RFC?
To me, that is by far the most important part of this RFC as that's the biggest blocker for wider async adoption,
but I'm not sure how many layers are needed above it to make it possible to toggle in a safe fashion.Ah, I think I understand what you mean.
Yes, from an implementation standpoint, non-blocking behavior is
provided by the Scheduler API and Reactor API. These two APIs are
always available anywhere — in the core, extensions, and so on.However, this isn’t part of the RFC itself, as it belongs to
implementation details. Moreover, do you remember the first version of
the RFC? It had a function that explicitly started the Scheduler. So
indeed, in that first version PHP could be in two states: synchronous
and asynchronous.What’s the difference in this RFC?
PHP is always in an asynchronous state. Even when executing
index.php, you can think of it as running inside a coroutine. There’s
just a single coroutine, so no switching occurs (although an extension
could, for example, create another coroutine — which is perfectly
valid).You keep writing your code exactly as before. You don’t need to think
about PHP being “asynchronous” now. As long as you’re writing code
inside a single coroutine, you’re still writing synchronous code.-- Ed
As far as I understand, the only thing missing from php-src that would allow everything to be async is a Fiber scheduler. We already have tons of Fiber libraries out there ... but just no unified scheduler. If we had one of those, then making Fiber-aware i/o functions is "trivial" compared to implementing this async RFC. If we’re going to merge just a scheduler/reactor, it might make more sense to implement a Fiber scheduler than a completely new way of doing async.
— Rob
Hello!
There are posts online, specifically in this discussion, from the
Swoole owner and the Swow author.
These are technically well-written messages that clearly and precisely
explain why Fiber was a premature solution.
It has been four or five years since then, and it is surprising that
this is still not clear to everyone.
Is it possible to implement a Reactor and Scheduler in PHP? Of course.
Programming is a colorful world filled with magical creatures such as
griffin-foxes, turtle-ravens, and octocats.
The main thing is to stop in time :)
As far as I understand, the only thing missing from php-src that would
allow everything to be async is a Fiber scheduler. We already have tons of
Fiber libraries out there ...
I'm not sure if Fibers were particular success. They are quite hard to use
and you need an extra library like amp so I think it would be useful to
give users a solution that is available in the core. I think this would
give it a bit more trust as it will also get the security guarantees
(standard core support for handling security issues). It will take time and
there will be multiple RFC's to get there and the implementation will also
require thorough review. So splitting that to smaller pieces is quite
important.
That said I'm planning introduction of better API for polling (almost done
and soon to be announced) that will be using special polling handles that
will be available for streams, sockets, curl and possible other extensions.
Streams should then get special notification callbacks that would be called
before IO and most likely provide the polling handle as a parameter so this
could be used by user space in their own reactor / scheduler. I plan to
also provide a similar internal API so async can integrate cleanly into
this without overwriting stream polling function as it's the case in the
current implementation. Similar API should be also provided for curl (Joe
already created part of it), sockets and possible other exts. In other
words, the user space should be able to gain this functionality and the
async core extension would be more core internal user of it.
The above is just for sockets and there will be needed further work for
file IO. I'm preparing new PHP IO internal API that will be initially
mainly for copying but will allow extension for other operations including
file operations and will use io_uring on Linux. I'm still not sure how to
best expose it to user space as the ring entry completion don't have usual
polling flow so it does not exactly fit into those callbacks. We could
possible do some sort pipe and thread that would process the completion
queue but maybe there is a better solution. This is still TBD but usually
file IO is not the main IO blocker so this can come a bit later.
We will also need to think about DNS resolving that might be even trickier
and might require IO thread pools - those might be needed as a backup
solution for platform that don't support io_uring anyway.
I think those API's will be also prerequisites for this async to implement
this functionality as I'm not fond of adding some hacks and functionality
duplication (that would be quite a pain for maintenance).
Kind regards,
Jakub
Hi,
Good day, everyone. I hope you're doing well.
I’m happy to present the fourth version of the RFC. It wasn’t just me
who worked on it — members of the PHP community contributed as well.
Many thanks to everyone for your input!
I just re-read it again with all the feedback provided and I think it
should get further stripped as it seems problematic to get an agreement on
all of this and properly discuss it on ML all the details.
I think this (or more v5) should strip the following:
- exposing Scope and all operations in it. It means it should allow using
only the default scope in this version. That should significantly reduce
the size of the RFC as it removes structured concurrency and other parts
related to scopes (including the reduction of error handling logic). - timer functions could also be removed even though it will make it less
usable but the point is to make it as small as possible and those are not
absolutely essential parts. - critical section should be stripped as well
- nginx unit example should be removed as it might be confusing - I
understand why it was added but it might be more confusing than useful - drop php.ini setting and just use default for now
The idea is to make this as small as possible so this might be possible to
discuss and get actually more people to read the RFC (this is just too
long). This will also allow to concentrate on specific pieces like for
example deciding whether to use FutureLike or Awaitable.
We just had a chat about this work during our PHP Foundation and the
agreement seems to be that this should be reduced and come in smaller
pieces to be able to better figure out what's actually useful for PHP.
Another point was also to make clear that this proposal is not meant to
introduce a new "right" way to use PHP but it's actually useful everywhere.
We actually discussed this with Edmond privately and agreed that this is
useful for FPM as well and he even created an example proving it. So we
should just try to make it clearer in the RFC as there was some confusion
in the discussion.
Kind regards,
Jakub
Hello.
I think this (or more v5) should strip the following
- exposing Scope and all operations in it. It means it should allow using only the default scope in this version. That should significantly reduce the size of the RFC as it removes structured
concurrency and other parts related to scopes (including the reduction of error handling logic).- timer functions could also be removed even though it will make it less usable but the point is to make it as small as possible and those are not absolutely essential parts.
- critical section should be stripped as well
- nginx unit example should be removed as it might be confusing - I understand why it was added but it might be more confusing than useful
- drop php.ini setting and just use default for now
I suppose that’s exactly what we’ll do.
However, I won’t completely remove Scope. I’ll move it to a separate
document in the WIKI, so it can be easily referenced later.
Thank you, Ed