PHP True Async RFC Stage 5

11 months ago by Deleu — view source — reply

unread

Hi Edmond!

Hi

1.5 RFC:
https://wiki.php.net/rfc/true_async

Here’s the fifth version of the RFC with the updates made after the
1.4 discussion.

Starting from 2025-11-03, there will be a two-week discussion period.

Changelog:

Added FutureLike interface methods: cancel(), isCompleted(),
isCancelled()

Renamed Coroutine::isFinished() to Coroutine::isCompleted()

Clarified exit/die behavior: always triggers Graceful Shutdown mode
regardless of where called

Added rationale for “Cancellable by design” policy: explains why
default cancellability reduces code complexity for read-heavy PHP
workloads

RFC structure improvements: reorganized Cancellation section with
proper subsections hierarchy

Moved “Coroutine lifetime” as subsection under Coroutine section

Extended glossary with Awaitable, Suspension, Graceful Shutdown, and
Deadlock terms

Introduced FutureLike interface with single-assignment semantics and
changed await() signature to accept FutureLike instead of Awaitable
for type safety

Split RFC: Moved Scope and structured concurrency functionality to
separate Scope RFC. Base RFC now focuses on core async primitives
(coroutines, await, cancellation)

I decided not to wait until Monday and made the changes today. If
anyone has read version 1.4 and has comments on it, they’re still
relevant.

The Scope API has been moved to a separate RFC:
https://wiki.php.net/rfc/true_async_scope

Best Regards, Ed

Thanks for the RFC update! I've been trying to read and understand this RFC
since its earlier versions and I can definitely feel it getting easier to
digest, which I can only assume it's a good thing for RFC voters - it's
easier to vote No because something is too complex / too hard to understand.

One minor question: is this section
https://wiki.php.net/rfc/true_async#awaiting_a_result_with_cancellation
named wrongly? I'm not sure how this snippet of code relates to
cancellation.

Onto more important things. In regards to the change of Awaitable vs
FutureLike, my understanding of the discussion is that on the
implementation side it was asked about whether an Awaitable object should
be awaited in a loop (consumed until completion) or if it should be awaited
only once, which also raised the question about idempotency and whether an
object is awaitable more than once. While the change makes the type-system
somewhat more explicit in regards to how await() is meant for a single-shot
Awaitable object (named FutureLike), it does mean the implementation of
things like awaitAll(Awaitable[] $awaitables) is no longer a simple loop to
await every item in the array.

My questions are:

What's the difference between Multishot Awaitables and Generators?
If await() is hardened to FutureLike only, doesn't this mean that
multishot awaitables are not really capable of being awaited anymore?
Doesn't this mean that the Awaitable interface becomes out-of-sync with the
await() function and it stops making sense?
Shouldn't FutureLike be Awaitable and what has been described as
Multishot awaitables should actually be generators / array of / list of
Awaitables?

In regards to Cancellable by Design. In the current state of PHP, we can
assume that if a function used to throw an Exception and a future version
stops throwing said exception it is not considered a BC Break. Of course,
throwing a different exception not part of the hierarchy is a BC break.
But when an exception signals "I cannot handle this" and a future version
becomes capable of handling it, that in essence is a feature/enhancement
and not treated as a BC break. With "Cancellation by Design" we are
expected that every coroutine be cancellable and that we must write try /
catch to design for cancellation. This seems to open up a different
development flow where catch blocks can be fundamentally part of the BC
promise. One possible alternative would be e.g. await(Awaitable $awaitable,
Closure $cancellation) where the cancellation of a coroutine would trigger
the specified Closure. Now, I don't want to dive too much into the
trade-offs of these options, what I want is to spark the idea that there
may be multiple ways to design cancellable coroutines. When I consider
that, plus the added fact that the RFC is very dense, extensive and hard to
digest, wouldn't it be best to postpone coroutine cancellations altogether?
The RFC itself states:

[...] read operations (database queries, API calls, file reads) are
typically as frequent as—or even more frequent than—write operations. Since
read operations generally don't modify state, they're inherently safe to
cancel without risking data corruption.

which not only I agree with, but also want us to focus on this very fact.
What if the first version of Async PHP provides userland with the ability
to trigger coroutines for reading purposes only? We will not forbid/prevent
anybody from shooting themselves if they want to, but we can still clearly
state the design principle that Async PHP is meant to spawn coroutines
cancellable by design. There will not be any coroutine markers in the
future nor there will be assumptions that coroutines written for the 1st
version of Async PHP must not be cancellable. The assumption is that the
first version of Async PHP should be treated as a way to perform read
operations only and a future RFC / enhancement will bring cancellation
capabilities (be it Closure, Try / Catch, what have you). My reasoning is
that this would further reduce the scope of the RFC while still introducing
real life useful async components to PHP, even if at a limited capacity. It
gives voters less concepts to understand, digest, agree on and approve and
it extends the opportunity to focus on specific deep aspects of Async PHP
in chunks and throughout different stages.

--
Marco Deleu

11 months ago by Edmond Dantes — view source — reply

unread

Hi

One minor question: is this section https://wiki.php.net/rfc/true_async#awaiting_a_result_with_cancellation named wrongly? I'm not sure how this snippet of code relates to cancellation.
Yes, second parameter is a CancellationToken. (spawn('sleep', 2)).

What's the difference between Multishot Awaitables and Generators?
There’s nothing in common between them.
It’s better to think of asynchronous objects as components of an
EventDriven pattern.

doesn't this mean that multishot awaitables are not really capable of being awaited anymore?
Exactly. It’s not needed for now.

Shouldn't FutureLike be Awaitable and what has been described as Multishot awaitables should actually be generators / array of / list of Awaitables?
FutureLike is a child interface of Awaitables.

we can assume that if a function used to throw an Exception and a future version stops throwing said exception it is not considered a BC Break.
This RFC does not change the behavior of existing functions. For
example, sleep works the same as before.
PHP functions that previously did not throw a CancellationError do not
throw it in this RFC either.
However, when you use functions (await example) that do throw these
exceptions, you handle them the same way as always. In that sense,
there’s no new paradigm.

What if the first version of Async PHP provides userland with the ability to trigger coroutines for reading purposes only?

Sorry, but I couldn’t understand why this needs to be done.
There’s nothing terrible about canceling write operations — no
“shooting yourself in the foot” either. A program can terminate at any
moment; that’s perfectly normal.

The concept of Cancellation by Design isn’t about guns — it’s about
reducing code. That’s all.
In the Swift language, for example, a different concept is used:
cancellation is always written explicitly in the code, like this:

func fetchData() async throws -> String {
    for i in 1...5 {
        try Task.checkCancellation()
        print("Fetching chunk \(i)...")
        try await Task.sleep(nanoseconds: 500_000_000)
    }
    return "Data loaded"
}

The only question is about how much code you write and the likelihood
of errors. The more code you write, the higher the chance of making
one.

Write safety is a different topic; it concerns how code is implemented
at the lowest level, meaning the code that calls OS functions.
Cancelling a coroutine cannot interrupt a kernel-level write operation
— it can only interrupt waiting for the write, and what to do next is
decided by the user-level code.

The cancellation design was implemented as early as 2015 in Python
(and in other languages as well) and has worked perfectly since then.
For languages like PHP, it’s a convenient and, most importantly,
familiar mechanism. The real problem is that an exception can
accidentally be caught without exiting the coroutine. For example:

catch (\Throwable $e) {
    Logger::log();
}
continue;

11 months ago by Dennis Birkholz — view source — reply

unread

Hi Edmond,

Am 30.10.25 um 9:19 AM schrieb Edmond Dantes:

Hi

1.5 RFC:
https://wiki.php.net/rfc/true_async

first of all thank you for investing so much time and effort into
improving PHP.

The True Async RFC changed a lot in the past iterations and removed a
lot of related but tangential topics. I really appreciate your
willingness to adapt to get the best possible outcome.

What I now see is as far as I understand essentially a Fiber 2.0 RFC so
I wonder if it would not be better to improve the available Fibers
instead of creating an incompatible second mechanism.

The Coroutine class is essentially a Fiber class that can be cancelled
and restricted by a cancellation awaitable.
The Fiber class could get startWithTimeout($timeout, ...$args) and
resumeWithTimeout($timeout, mixed $value = null) methods as well as a
cancel() method.
It wouldn't be a Fiber in the pure compsci way any more but I am willing
to accept that if it prevents us from having two ways for (semi)
cooperative multitasking.

The part about how the Awaitable and FutureLike interfaces work is very
unclear to me. They are there but they do not describe how they could be
used in a truly multitasking fashion. That is somehow open to the
Scheduler/Reactor which are not described to reduce the complexity.
The RFC as it is allows to await(new Coroutine()) which is syntactical
sugar for $fiber = new Fiber(); $fiber->start(); while (!$fiber->isTerminated()) $fiber->resume(); So a followup RFC would
need introduce this additional mechanism into these interfaces.

Also I do not really understand why the "cancellation" is an awaitable.
If the provided awaitable is itself some infinitely blocking Coroutine
(e.g. while (true) {}), how can the scheduler run the actual Coroutine
and the "cancellation" awaitable to determine whether the Coroutine
should be cancelled or not? As long as there is no multithreading, this
does not make sense for me.

In addition, what happens if a Coroutine is suspended and is restarted
again. Is the cancellation awaitable restarted? Or just continued?

I am really skeptical if the current RFC is the right way to go,
establishing a Coroutine and Awaitable and FutureLike interfaces in
competition to the existing Fiber.

I would rather see a step-by-step plan with gradual improvements like this:

Propose some changes to Fiber so it can be interrupted after a timer
expired and it can be cancelled.
Add a unified polling mechanism for all kinds of IO events (timeouts
and signals included) like Jakub's "Polling API".
Enhance the Fiber class so it can expose a PollHandle/Pollable that
it is currently waiting on, either as a property of the Fiber
(Fiber::$pollHandle) or as a Fiber::suspendPolling(PollHandle $pollHandle, mixed $value = null) method.
Now internal IO methods can be changed to start a pollable Fiber
instead of blocking the execution if they are started in a specific way
(e.g. by a then introduced spawn() call).
With all that in place, userland can now create their own
Scheduler/Rector. The Core could also include a simple default
implementation used the PollContext/PollWatcher in addition with a
scheduling policy for other Fibers.

Kind regards
Dennis

11 months ago by Edmond Dantes — view source — reply

unread

Hello Dennis.

With all that in place, userland can now create their own Scheduler/Rector.

I once wrote about why such solutions are unsuccessful — and clearly
bad from PHP’s point of view.
But since I don’t remember where that text was, I’ll try to express it again.

Programming languages have levels of abstraction, which researchers
have been trying to quantify mathematically since the 1970s.
PHP is a high-level programming language with a relatively high degree
of abstraction, thanks to its memory management, built-in runtime, and
abstraction over the operating system.

Suppose someone created an RFC proposing to add assembly inserts into PHP.
Would you vote in favor of this RFC?
If not, why?

Most software — almost all of it — from operating systems to browsers,
is built on the principles of multilayered architecture,
where abstractions are separated into distinct layers.
This is done to reduce and control interdependencies, which directly
affects what are probably the three most important parameters in
programming:
the cost of code, the cost of debugging, and the cost of refactoring.

For this architecture to work, developers try to follow the Strict
Layering rule (known by other terms in different contexts),
which states that code from a higher-level layer must not interact
with a lower-level layer while skipping the intermediate one.
Although this rule is almost always violated, adhering to it is
justified in most cases.

Assembly inserts in PHP violate the Strict Layering rule and give
the programmer the ability to completely break the language’s
operation, since they belong to the lowest layer.

A Fiber in PHP is a context that stores a pointer to the C stack, CPU
registers, and part of the VM state, combined with a generator.
Fibers cannot be used as coroutines because this approach is
inefficient in terms of performance and memory.
The reason is that a Fiber is an extremely low-level primitive — only
slightly higher than assembly.

Let’s recall what a full-fledged abstraction of asynchrony looks like
in any proper programming language:
https://editor.plantuml.com/uml/TLBDRjGm4BxFKunw0QJTvSu1LQfQgH9L4HLmTftPpQYE7SrCkXigtXqx8MxOYfF7zZVVZyUNQaviw0Be4yVUYUimS2GRUy97-iKa0COM2B-HyvPasmW_KyIh96cm3CMxr5004FBcuY4ZBwvFvFDTAgXeT39yVyEF91ykq6azUm74LLCbd41r1x__eNxmBJL389bGTOSlPxZPxOpwMvyBNkSOXbzIwWjgAWh9Exm9wOXfZxJ4WDUms-tdolS9tT6nuUt7UwH21ijDHbLl1HUJyNv48TUCw6kqIlkcOMGA3QQ8aKusom1a5aBXGsl5NOL3hJ9rD4b1NpKmy9xyw0FjuDPG1-qfDYk05fKvXuiD2kdGaOArrE6nfLZJ2lL9JASGfKztGBcXc3gpjamOtlw3DeKi_kCErPpH1lBYdpQJyjNNxoXqO3KItQtUPXu3AHxPMex8zb_bnMmTH9SYvrLnpu6m8VN2VJdOW76NXMPjvKDqGV6P7Tu_xE1d2UxYF5LCtWy5oJOFacdryrPUBdCrTE4F

Therefore, Fibers violate the Strict Layering principle, and PHP
has no way to prevent this — unlike Rust, for example, where you can
hide parts of the implementation within a crate (package).
Even in C++, there is no such violation — the programmer works with
the coroutine abstraction.

It is also important to understand the difference between PHP and
C++/Rust: PHP is a single-runtime language.
When you have multiple runtime libraries, for example asynchronous
ones, a segmentation problem arises:
you cannot simply use code written for runtime A in runtime B, because
the runtimes are incompatible!

What made PHP popular?
What made Go popular?
That’s right — the built-in runtime!
Just write code.

Just a week or two ago, I read an article from the Python community
discussing the problems caused by the segmentation of asynchronous
libraries.
And let me remind you, that Python has had built-in language-level
support for asynchrony since 2015.

So...
I’m convinced that PHP should remain a high-level language with a
built-in runtime (at least as long as it’s interpreted).
Attempts to create a “backdoor” in the language to let libraries
implement what should be written in C bring no benefit to PHP users.

After all, to use asynchrony, it’s not enough to just add spawn or coroutines.
Libraries and frameworks must also be adapted, and all of that takes time.
Go added more abstractions to the language to make writing business
logic easier.
Python will soon implement JIT.
People will choose the tool that “just works” with minimal effort and
they won’t care what it’s called.

Best regards,
Ed

11 months ago by Dennis Birkholz — view source — reply

unread

Hi Edmond,

thank you for your reply.

Am 01.11.25 um 8:32 AM schrieb Edmond Dantes:

A Fiber in PHP is a context that stores a pointer to the C stack, CPU
registers, and part of the VM state, combined with a generator.
Fibers cannot be used as coroutines because this approach is
inefficient in terms of performance and memory.
The reason is that a Fiber is an extremely low-level primitive — only
slightly higher than assembly.

Therefore, Fibers violate the Strict Layering principle ...

From the standpoint of PHP language user, I have a completely different
view on Fibers vs. Corotines.
They look very similar from the outside and if we talk about
abstractions, that is the point that matters as the inner workings are
hidden.

I really belief we should avoid fragmentation and enhance/adjust Fibers
to meet the memory and performance requirements of a Coroutine.

Thanks
Dennis

11 months ago by Edmond Dantes — view source — reply

unread

From the standpoint of PHP language user, I have a completely different view on Fibers vs. Corotines.
That’s sad.

They look very similar from the outside
Coroutines and Fibers have completely different behavior. I hope
you’re not comparing them just by appearance?

I really belief we should avoid fragmentation and enhance/adjust Fibers to meet the memory and performance requirements of a Coroutine.
But the problem has already happened, and it’s not directly related to this RFC.
Of course, there’s a possibility to bridge the two worlds by calling
PHP functions from C, but as I’ve said before: just because something
can be done doesn’t mean it should be done.

If the provided awaitable is itself some infinitely blocking Coroutine (e.g. while (true) {}),

If you have a coroutine with an infinite loop, it means other
coroutines will never get control.
(more about it by searching for the keyword: “concurrency")

The RFC contains an example that isn’t very elegant from a semantic
point of view, but is completely correct in terms of logic:

// Await task 1, but no longer than 5 seconds.
await($task1, spawn(sleep(...), 5));

And here’s another piece of code (Async\Signal is not present in the
RFC, but it’s entirely possible.):

// Await task 1 until a signal occurs.
await($task1, new Async\Signal(SIG_TERM));

In addition, what happens if a Coroutine is suspended and is restarted again.

The Await function waits for the coroutine to complete.
The suspended state does not affect the waiting process.
The wait is interrupted for two reasons: an unhandled exception or the
coroutine’s completion.

All of this is described in the RFC: https://wiki.php.net/rfc/true_async#await

Ed

11 months ago by Larry Garfield — view source — reply

unread

So...
I’m convinced that PHP should remain a high-level language with a
built-in runtime (at least as long as it’s interpreted).
Attempts to create a “backdoor” in the language to let libraries
implement what should be written in C bring no benefit to PHP users.

In concept, I agree. Which is part of why I want to see an Async RFC that goes even higher than the current one, not lower. :-)

But that is separate from the question of whether it's possible to build on Fibers, rather than effectively deprecate them in practice if not in name.

--Larry Garfield

11 months ago by Edmond Dantes — view source — reply

unread

Hi

But that is separate from the question of whether it's possible to build on Fibers, rather than effectively deprecate them in practice if not in name.

Originally, Fiber was proposed with a Scheduler, but the Scheduler was
refused.
To allow Fiber switching without a Scheduler, they were made
symmetric (so that the switching code could do it manually).
This, in turn, creates a problem when trying to add a Scheduler later.

To create coroutines, you need to write the "switching code".
But to write the switching code, Fibers must be allowed to switch arbitrarily.
And for Fibers to switch arbitrarily, backward compatibility must be broken.

What could be reused? The context-switching code and the observer
component handlers — these were reused.
The issue isn’t that Fiber behavior can’t be changed, but that it
should be hidden as an internal component.
There should be no access to it from PHP code.

The experience with Fiber shows that language features like asynchrony
must be designed as a whole from the start — thoughtfully and
consistently.
You can’t make a language "a little" asynchronous today and a bit
"more" tomorrow:

Critical components must be designed in advance to understand
how they interact.
They must be placed within the same layer of abstraction.
Use cases must be thought out in advance.

Best regards
Ed

11 months ago by Dennis Birkholz — view source — reply

unread

Hi Edmond,

could you please clarify these two questions? Thanks.

Am 31.10.25 um 11:59 PM schrieb Dennis Birkholz:

Also I do not really understand why the "cancellation" is an
awaitable. If the provided awaitable is itself some infinitely
blocking Coroutine (e.g. while (true) {}), how can the scheduler run
the actual Coroutine and the "cancellation" awaitable to determine
whether the Coroutine should be cancelled or not? As long as there is
no multithreading, this does not make sense for me.

In addition, what happens if a Coroutine is suspended and is restarted
again. Is the cancellation awaitable restarted? Or just continued?

Kind regards
Dennis

11 months ago by luisvscbarros@gmail.com — view source — reply

unread

Hi Edmond,

First of all, sorry for my bad English, and thanks a lot for the huge
amount of work you’ve put into this proposal.
You researched, wrote the RFC, implemented it, and answered tons of
questions. Really impressive.

I have one suggestion and two small questions.

Suggestion

Maybe keep the base |Awaitable| internal and expose two userland
interfaces that match the two cases described in the RFC:

|// Single state change, idempotent read, same result on each await
interface Future extends Awaitable {} // Multiple state changes, each
await may observe a new state interface Streamable extends Awaitable {} |

This makes the single-shot vs multi-shot difference explicit and easier
for tools and libraries to reason about.
Later on, it could even be extended with something like:

|interface Retryable extends Awaitable {}|

Questions (self-cancellation)

What happens here?

|use function Async\spawn; use function Async\suspend; $coroutine =
spawn(function() use (&$coroutine) { $coroutine->cancel(new
\Async\CancellationError("Self-cancelled")); echo "Before suspend\n";
suspend(); echo "After suspend\n"; // should this run? return
"completed"; }); await($coroutine); |

Can a cancelled coroutine suspend?
And if a function that yields is called after the cancel, should that
suspension still happen?

And what about this one?

|use function Async\spawn; $coroutine2 = spawn(function() use
(&$coroutine2) { $coroutine2->cancel(new
\Async\CancellationError("Self-cancelled")); echo "Before exception\n";
throw new \RuntimeException("boom after cancel"); }); await($coroutine2); |

Again, thanks for the work, especially on such an important feature for
PHP’s future.
Hope to see it in php-src soon.

Best,
Luís Vinícius

10 months ago by Edmond Dantes — view source — reply

unread

Hello all.

Today marks two weeks since the RFC was published.

I need to apply a few minor fixes that Luis pointed out.

If anyone else is working on comments for the RFC, please let me know.
If there are no objections, we can start the vote on Monday.

Best regards, Ed

10 months ago by tim@bastelstu.be — view source — reply

unread

Hi

Given your planned timeline of voting, I wanted to chime in here before
my vacation. I'll likely only see the reply on Monday morning.

If anyone else is working on comments for the RFC, please let me know.
If there are no objections, we can start the vote on Monday.

It appears that you believe that the RFC and the proposal finally
settled. I frankly lost track of what has been discussed in all the
different discussion threads related to various Async RFCs.

What I am missing from the RFC text is some kind of "Executive Summary"
to make it clear what is and what is not actually being proposed.
The RFC starts of with goals and a glossary that primarily explains by
means of an example.

This makes it hard for me to see what I am actually voting for (and what
I am not), especially after the many changes to refine the RFC. I would
suggest to add a “full stub” (as suggested in the RFC template
https://wiki.php.net/rfc/template#proposal) at the start and also to
shortly explain what is proposed and what is left untouched (e.g. the
RFC already mentions that file_get_contents is not proposed to change,
but that is easy to miss without carefully reading everything) before
diving into the details for each of the functions.

With regard to the relationship with fibers, it is not clear to me why
e.g. Fiber::suspend() could map to Async\suspend() and why the Async
event loop couldn't call ->resume() on suspended Fibers. Elaborating a
little more would be helpful I think.

Also, please make sure to add the “Abstain” option to the vote
(https://wiki.php.net/rfc/rfc_vote_abstain).

Best regards
Tim Düsterhus

10 months ago by Edmond Dantes — view source — reply

unread

Hello, Tim.

What I am missing from the RFC text is some kind of "Executive Summary"
to make it clear what is and what is not actually being proposed

I’ll try to look through the text again to see what exactly might be
unclear. Although to be honest, I also don’t really understand what
exactly is unclear.

This makes it hard for me to see what I am actually voting for
I suggest we do it this way. I’ll write a summary here, and if needed
we can add some of these phrases to the RFC.
I have an idea to format the summary as questions and answers. That
will probably make it easier to understand.

I would suggest to add a “full stub”

Also, please make sure to add the “Abstain” option to the vote
I will do it, thanks

With regard to the relationship with fibers
I will also take these questions into account in the next message.

Thanks, Ed

10 months ago by Edmond Dantes — view source — reply

unread

Hello.

Mini Faq: https://github.com/true-async/php-true-async-rfc/blob/main/faq.md
Text here:

Executive Summary: What This RFC Proposes

This RFC proposes adding built-in concurrency support to PHP
through two core components:

1. Coroutines via `spawn()`

Launch any PHP function as a lightweight coroutine that can be
suspended and resumed:

use function Async\spawn;
use function Async\await;

$coroutine = spawn(file_get_contents(...), 'https://php.net');
$result = await($coroutine);

2. Non-blocking I/O Functions

50+ existing PHP functions automatically become non-blocking when
used inside coroutines:

Database: PDO MySQL, MySQLi operations
Network: CURL, sockets, streams, DNS lookups
Files: file_get_contents(), fread(), fwrite()
Process: exec(), shell_exec(), proc_open()
Timers: sleep(), usleep()

Key principle: From the developer's perspective, these functions
work identically to their synchronous versions.
The difference is that they suspend only the current coroutine instead
of blocking the entire PHP process.

See full list: https://github.com/true-async/php-async#adapted-php-functions

What This RFC Does NOT Propose

No changes to existing synchronous behavior - code without
coroutines works exactly as before
No new syntax keywords - uses function calls (spawn(),
await(), suspend())
No changes to Fiber API - Fibers and True Async are mutually
exclusive by design
No structured concurrency primitives - covered in separate
Scope RFC

General Questions

Q: What is the main goal of this RFC?

A: The RFC aims to provide a standardized way to write concurrent
code in PHP without requiring developers
to rewrite existing synchronous code. The key value proposition is
that existing code works exactly the same
inside a coroutine without modifications, unlike explicit async/await models.

Q: How is this different from Fibers?

A: Fibers and True Async serve fundamentally different purposes
and cannot coexist:

Fibers:

Low-level symmetric execution contexts
Programmer explicitly controls switching ($fiber->resume(),
Fiber::suspend())
Direct access to execution stack management
Suitable for building custom scheduling solutions

True Async:

High-level asymmetric coroutines
Automatic switching managed by the scheduler
Transparent to the developer
Designed for business logic, not infrastructure

Why they can't work together:

Resource conflicts: Both manage the same low-level resources
(execution context, CPU stack) in incompatible ways
Architectural incompatibility: Mixing symmetric (Fibers) and
asymmetric (coroutines) models creates unpredictable behavior
Abstraction level: Fibers expose low-level primitives in a
high-level language, violating the "Strict Layering" principle

Why not map Fiber::suspend() to Async\suspend()?

This would create a leaky abstraction:

Fibers require explicit scheduling decisions (who to resume? when?)
True Async scheduler makes these decisions automatically
Mixing both models would break scheduler guarantees and lead to race
conditions
The execution models are fundamentally incompatible (symmetric vs asymmetric)

If you need Fibers' explicit control, use Fibers.
If you want automatic concurrency for I/O-bound applications, use True Async.
Attempting to unify them would result in a solution that's neither
simple nor safe.

Q: Isn't this just Fibers 2.0?

A: No. While both deal with execution contexts:

Fibers require explicit switching and manual control
True Async provides automatic scheduling and high-level primitives
They solve different problems at different abstraction levels
They are mutually exclusive by design

Q: Can I use this with FPM?

A: Yes! True Async works in all execution modes including FPM.
The reactor activates within the context of
php_request_startup/php_request_shutdown(),
requiring no SAPI modifications.

Q: What about `exit` and `die`?

A: They always trigger Graceful Shutdown mode:

All coroutines in globalScope are cancelled
Application continues execution without restrictions to shut down naturally
This allows proper cleanup operations

Q: Do I need to rewrite my existing code?

A: No. The main value of this implementation is that existing
synchronous code works inside coroutines without modification.
You can gradually adopt async features where beneficial.

Q: How does this RFC affect I/O functions?

A: From the coroutine's perspective, I/O functions do not change
their behavior
they work exactly as they always have.
However, functions that previously blocked the entire PHP process now
only suspend the current coroutine,
allowing other coroutines to continue executing.

Best regards, Ed

10 months ago by Jakub Zelenka — view source — reply

unread

Hi,

Hello all.

Today marks two weeks since the RFC was published.

I need to apply a few minor fixes that Luis pointed out.

If anyone else is working on comments for the RFC, please let me know.
If there are no objections, we can start the vote on Monday.

I thought about it and I think we should have this with implementation
otherwise the whole thing is kind of pointless because we might not get it
merged if some internals disagreement happens. It will be also impossible
to get anything bigger than what is proposed now reviewed (read that we
won't be likely able to merge the complete implementation in one go). In
other words if this passes, it will just means that the API is ok but there
is nothing actionable. In addition, I think some people might be even
voting against it if there is no implementation that they can review.

It means there should be a PR implementing exactly what is in this RFC
(minimal stripped version of your current implementation) IMO.

Cheers

Jakub

10 months ago by Edmond Dantes — view source — reply

unread

Hello, Jakub.

It means there should be a PR implementing exactly what is in this RFC (minimal stripped version of your current implementation) IMO.

This RFC includes not only the API, but also the Scheduler, Reactor,
and non-blocking versions of PHP functions. Removing Scope from the
public classes doesn’t really change that.
Are you saying that some other implementation is needed as well?

Best regards, Ed

10 months ago by Jakub Zelenka — view source — reply

unread

Hi,

Hello, Jakub.

It means there should be a PR implementing exactly what is in this RFC
(minimal stripped version of your current implementation) IMO.

This RFC includes not only the API, but also the Scheduler, Reactor,
and non-blocking versions of PHP functions. Removing Scope from the
public classes doesn’t really change that.
Are you saying that some other implementation is needed as well?

I think it would be good to see the implementation that can cover the
currently proposed API and try to strip it as much as possible so it
doesn't contain much more than that. We saw that PR for the async API was
already quite big and we didn't really get any agreement there partially
also because there was no user of that and it was not possible to have any
tests for it (without writing them in C). So what I'm thinking is that if
some minimal version that implements just this (e.g. reactor can be just
dummy because there is no io atm. and other things can be stripped too),
then the voters would get better idea what they are dealing with and could
even try it out.

Cheers

Jakub

10 months ago by Edmond Dantes — view source — reply

unread

Hello Jakub.

I think it would be good to see the implementation that can cover the currently proposed API and try to strip it as much as possible so it doesn't contain much more than that. We saw that PR for the async API was already quite
big and we didn't really get any agreement there partially also because there was no user of that and it was not possible to have any tests for it (without writing them in C).
So what I'm thinking is that if some minimal version that implements just this (e.g. reactor can be just dummy because there is no io atm. and other things can be stripped too), then the voters would get better idea what they are > dealing with and could even try it out.

I understand what you mean.

** Regarding simplifying the code.**
Any “simplification” essentially comes down to removing stub files and
the C classes that implement the PHP classes. This is a relatively
small part of the project.
For example, removing Scope from the C code doesn’t make much sense,
because it turned out (even unintentionally) to be a very convenient
structure for tracking a group of coroutines.
In other words, no major changes to the code are expected before the
PR review begins.

** Reactor. **
Since the reactor uses libUV and we currently do not plan to provide a
pure-C implementation, we agreed to move it into a separate library.

** Testing. **
Some functions that do not involve I/O can be covered by unit tests.
This is a small portion of the API.
However, covering all the remaining code that performs I/O with unit
tests is not practical. We could try to emulate the OS and so on… but
I think you understand that the result is not worth the effort.
This means we cannot avoid integration tests.

P.S. Personally, I would prefer to agree on the PR first and the RFC
afterwards. As I mentioned earlier, the code is more important than
the RFC, because it defines the real relationships and logic, while
the RFC only “describes” them. But we live in a world that follows its
own rules.

--- Ed

10 months ago by Jakub Zelenka — view source — reply

unread

Hi,

Hello Jakub.

I think it would be good to see the implementation that can cover the
currently proposed API and try to strip it as much as possible so it
doesn't contain much more than that. We saw that PR for the async API was
already quite
big and we didn't really get any agreement there partially also because
there was no user of that and it was not possible to have any tests for it
(without writing them in C).
So what I'm thinking is that if some minimal version that implements
just this (e.g. reactor can be just dummy because there is no io atm. and
other things can be stripped too), then the voters would get better idea
what they are > dealing with and could even try it out.

I understand what you mean.

** Regarding simplifying the code.**
Any “simplification” essentially comes down to removing stub files and
the C classes that implement the PHP classes. This is a relatively
small part of the project.

It's 1.6k lines so it might help a little bit

For example, removing Scope from the C code doesn’t make much sense,
because it turned out (even unintentionally) to be a very convenient
structure for tracking a group of coroutines.
In other words, no major changes to the code are expected before the
PR review begins.

I don't think you can create PR with the whole project. It's not gonna get
reviewed and merged. It might not even open in GH. So you will need to come
up with a way how to split to small pieces and I think this is the first
self contained bit that should be offered in minimal form.

** Reactor. **
Since the reactor uses libUV and we currently do not plan to provide a
pure-C implementation, we agreed to move it into a separate library.

Why do you need reactor for this specific part of proposal? The thing is
that there shouldn't be any IO so you reduce scheduler code as well and
make it simpler and more reviewable.

Kind regards,

Jakub

10 months ago by Edmond Dantes — view source — reply

unread

Hello

It's 1.6k lines so it might help a little bit
Yeah :)

Why do you need reactor for this specific part of proposal? The thing is that there shouldn't be any IO so you reduce scheduler code as well and make it simpler and more reviewable.

So you are suggesting removing all I/O from the RFC. On the one hand,
that sounds appealing. It immediately eliminates a bunch of tests. But
there is a trap we could all fall into.
By separating the reactor and the scheduler, as well as the rules of
how they work together, we might accidentally introduce an error into
the document simply because the documents would be split.
(interface drift)

The fact that the I/O rules and coroutine rules are part of a single
document developed together is actually an advantage, just as the
existence of separate RFCs for await and Scope is. And this does not
prevent splitting the implementation code into many small parts.

On the other hand, STREAM has corner cases, especially in error
situations, that would be appropriate to discuss and formalize in a
separate RFC. Ideally, this should probably be done right before
accepting the STREAM PR. However, such a description does not carry
significant risks.

This requires some thought.

10 months ago by Jakub Zelenka — view source — reply

unread

Hello

It's 1.6k lines so it might help a little bit
Yeah :)

Why do you need reactor for this specific part of proposal? The thing is
that there shouldn't be any IO so you reduce scheduler code as well and
make it simpler and more reviewable.

So you are suggesting removing all I/O from the RFC. On the one hand,
that sounds appealing. It immediately eliminates a bunch of tests. But
there is a trap we could all fall into.
By separating the reactor and the scheduler, as well as the rules of
how they work together, we might accidentally introduce an error into
the document simply because the documents would be split.
(interface drift)

I don't see it that way. You have already implementation showing that this
is workable for the proposed user interface so it's just addition into
that. I'm not sure how it could even impact user interface that is being
proposed? If you are talking about internal interface, then it doesn't
matter, because this can be changed (you don't have to keep BC there
especially for such a new internal interface like this).

The fact that the I/O rules and coroutine rules are part of a single
document developed together is actually an advantage, just as the
existence of separate RFCs for await and Scope is.

But are I/O rules really part of this document? There are just a few
mentioning of I/O and that seems more like a leftover from previous version
to me. I don't think it needs to keep reactor in there and mention I/O in
other parts. It should just clearly put it to the future scope to make
clear that this is something that is part of the plan.

Kind regards,

Jakub

10 months ago by Edmond Dantes — view source — reply

unread

Hello all.

I’ve updated the RFC by adding a brief summary at the very beginning
and adjusting the voting section.

As far as I understand, opening the vote requires creating a separate
page on the Wiki. For some reason, I couldn’t find clear instructions
for this in the documentation, which is a bit surprising.

Regarding the timeline, there is no need to rush. Someone may still
want to share their thoughts and simply hasn’t had the chance yet.
Therefore, we can extend the period until next Thursday. In the
meantime, I may also find a few more issues in the document.

Best regards, Ed

10 months ago by Rowan Tommins [IMSoP] — view source — reply

unread

As far as I understand, opening the vote requires creating a separate
page on the Wiki. For some reason, I couldn’t find clear instructions
for this in the documentation, which is a bit surprising.

The voting stage is described in step 6 of the How-To here: https://wiki.php.net/rfc/howto

There's no separate page, you just edit the attributes on the voting widget tag. You can see how it looks on the two RFCs that are currently "in voting" on the RFC index: https://wiki.php.net/rfc

Rowan Tommins
[IMSoP]

10 months ago by Rob Landers — view source — reply

unread

Hello all.

Today marks two weeks since the RFC was published.

I need to apply a few minor fixes that Luis pointed out.

If anyone else is working on comments for the RFC, please let me know.
If there are no objections, we can start the vote on Monday.

Best regards, Ed

I have concerns about the clarity of when suspension occurs in this RFC.

The RFC states as a core goal:

"Code that was originally written and intended to run outside of a Coroutine must work EXACTLY THE SAME inside a Coroutine without modifications."

And:

"A PHP developer should not have to think about how Coroutine switch and should not need to manage their switching—except in special cases where they consciously choose to intervene in this logic."

However, the RFC doesn’t clearly define what these "special cases" are or provide guidance on when developers need to intervene.

Specific questions:

CPU-bound operations: If I have a tight loop processing data in memory (no I/O), will it monopolise the coroutine scheduler? Do I need to manually insert suspend() calls? How do I know when and where?
The RFC suggests that existing PHP functions won’t automatically be non-blocking. So which will? Is there a way to identify suspension points at the language/API level?
Performance implications: Without knowing where suspensions occur, how do developers avoid either:

Starving other coroutines (too few suspensions)
Degrading performance (too many suspensions)

With explicit async/await ("coloured functions"), developers know exactly where suspension can occur. This RFC’s implicit model seems convenient, but without clear rules about suspension points, I’m unclear how developers can write correct concurrent code or reason about performance.

Could the RFC clarify the rules for when automatic suspension occurs versus when manual suspend() calls are required? Is this RFC following Go’s model where suspension timing is an implementation detail developers shouldn’t rely on? If so, that should be stated explicitly. Keep in mind that Go didn’t start that way and took nearly a decade to get there. Earlier versions of Go explicitly stated where suspensions were.

— Rob

10 months ago by Edmond Dantes — view source — reply

unread

Hello all.

Some of these questions sound familiar. Let’s try to sort them out.

If I have a tight loop processing data in memory (no I/O), will it monopolise the coroutine scheduler? Do I need to manually insert suspend() calls? How do I know when and where?

A coroutine must yield control on its own. If it keeps it through an
endless loop, then it will be the only one running.

The RFC suggests that existing PHP functions won’t automatically be non-blocking. So which will? Is there a way to identify suspension points at the language/API level?

The RFC does not say that. PHP I/O functions automatically become
non-blocking relative to the whole process. In other words, an I/O
function calls suspend() on its own when needed. The programmer writes
code exactly as before, under the illusion that operations are
executed one after another.

Performance implications: Without knowing where suspensions occur, how do developers avoid either:

In most cases, this is not the developer’s concern. Situations where
performance is critical should be handled with dedicated tools.
A PHP developer should not have to drop down to the C level. Properly
designed abstractions must provide the required performance.

I can already anticipate the question: but a developer could write
something like “for i < 10000 suspend” or something similar.
The answer is this: a developer must know how to use abstractions. As
always. Everywhere. In any area of programming.
It’s just as important as respecting proper layering in code.

Provided that PHP does not try to play the role of a C-level language
(there have already been such attempts, and they keep resurfacing),
and does not try to play a web server or a database system.
For most web scenarios, the current approach is more than sufficient.
This has been proven by Swoole, which has been on the market for many
years. Therefore, performance questions are outside the scope of this
RFC.

As for the concurrency model, let me remind you that Go has true
multitasking. Goroutines in Go, although they have a “synthetic”
stack. But you already know all this well.
This RFC and its implementation describe coroutines in a single
thread. That is very far from what Go provides.

There is no preemptive multitasking in this RFC because it is
completely unrelated.
This RFC and its implementation provide cooperative concurrency in a
single thread, where coroutine code yields control on its own.
Why was this model chosen? The simple answer is: because it is the
only one that can realistically be implemented within a finite
timeframe.

Any more questions?

-- Ed

10 months ago by Rob Landers — view source — reply

unread

Hello all.

Some of these questions sound familiar. Let’s try to sort them out.

If I have a tight loop processing data in memory (no I/O), will it monopolise the coroutine scheduler? Do I need to manually insert suspend() calls? How do I know when and where?

A coroutine must yield control on its own. If it keeps it through an
endless loop, then it will be the only one running.

The RFC suggests that existing PHP functions won’t automatically be non-blocking. So which will? Is there a way to identify suspension points at the language/API level?

The RFC does not say that. PHP I/O functions automatically become
non-blocking relative to the whole process. In other words, an I/O
function calls suspend() on its own when needed. The programmer writes
code exactly as before, under the illusion that operations are
executed one after another.

Performance implications: Without knowing where suspensions occur, how do developers avoid either:

In most cases, this is not the developer’s concern. Situations where
performance is critical should be handled with dedicated tools.
A PHP developer should not have to drop down to the C level. Properly
designed abstractions must provide the required performance.

I can already anticipate the question: but a developer could write
something like “for i < 10000 suspend” or something similar.
The answer is this: a developer must know how to use abstractions. As
always. Everywhere. In any area of programming.
It’s just as important as respecting proper layering in code.

Provided that PHP does not try to play the role of a C-level language
(there have already been such attempts, and they keep resurfacing),
and does not try to play a web server or a database system.
For most web scenarios, the current approach is more than sufficient.
This has been proven by Swoole, which has been on the market for many
years. Therefore, performance questions are outside the scope of this
RFC.

As for the concurrency model, let me remind you that Go has true
multitasking. Goroutines in Go, although they have a “synthetic”
stack. But you already know all this well.
This RFC and its implementation describe coroutines in a single
thread. That is very far from what Go provides.

There is no preemptive multitasking in this RFC because it is
completely unrelated.
This RFC and its implementation provide cooperative concurrency in a
single thread, where coroutine code yields control on its own.
Why was this model chosen? The simple answer is: because it is the
only one that can realistically be implemented within a finite
timeframe.

Any more questions?

-- Ed

Hey Ed,

I feel like we’re talking past each other a bit. I’m not questioning the choice of a cooperative scheduler, that’s totally fine, I’m trying to understand what the actual suspension points are. Every other language/runtime with cooperative concurrency spells this out, because without it developers can’t reason about performance or why a deadlock is happening.

Based on the conversation so far, I’d imagine the list to look something like:

network i/o (streams/sockets/dns/curl/etc)
sleeps
subprocess waits? proc_open and friends?
extensions with a reactor implementation
awaiting FutureLike
suspend()

If that’s the intended model, it’d help to have that spelled out directly; it makes it immediately clear which functions can or will suspend and prevents surprises.

I also think the RFC needs at least minimal wording about scheduler guarantees, even if the details are implementation-specific. For example, is the scheduler run-to-suspend? FIFO or round-robin wakeup? And non-preemptive behaviour only appears here in the thread. It isn’t mentioned in the RFC itself. That’s important for people writing long, CPU-bound loops, since nothing will interrupt them unless they explicitly yield.

Lastly, cancellation during a syscall is still unclear. If a coroutine is cancelled while something like fwrite() or a DB write is in progress, what should happen? Does fwrite() still return the number of bytes written? Does it throw? For write-operations in particular, this affects whether applications can maintain a consistent state.

Clarifying these points would really help people understand how to reason about concurrency with this API.

— Rob

10 months ago by Edmond Dantes — view source — reply

unread

Hello.

Based on the conversation so far, I’d imagine the list to look something like:

Yes, that’s absolutely correct. When a programmer uses an operation
that would normally block the entire thread, control is handed over to
the Scheduler instead.
The suspend function is called inside all of these operations.

If that’s the intended model, it’d help to have that spelled out directly; it makes it immediately clear which functions can or will suspend and prevents surprises.

In the Async implementation, it will be specified which functions are supported.

I also think the RFC needs at least minimal wording about scheduler guarantees, even if the details are implementation-specific.
The Scheduler guarantees that a coroutine will be invoked if it is in the queue.

For example, is the scheduler run-to-suspend? FIFO or round-robin wakeup? And non-preemptive behaviour only appears here in the thread. It isn’t mentioned in the RFC itself.

In Go, for example, when it was still cooperative, these details were
also not part of any public contract. The only guarantee Go provided
was that a coroutine would not be interrupted arbitrarily. The same
applies to this RFC: coroutines are interrupted only at designated
suspension points.
However, neither Go nor any other language exposes the internal
details of the Scheduler as part of a public contract, because those
details may change without notice.

That’s important for people writing long, CPU-bound loops, since nothing will interrupt them unless they explicitly yield.
Hypothetically, in the future it may become possible to interrupt
loops, just like Go eventually did. This would likely require an
additional RFC. PHP does have the ability to interrupt a loop at any
point, but most likely only for terminating execution.
This RFC does nothing of the sort.

Lastly, cancellation during a syscall is still unclear. If a coroutine is cancelled while something like fwrite() or a DB write is in progress, what should happen?
Does fwrite() still return the number of bytes written? Does it throw? For write-operations in particular, this affects whether applications can maintain a consistent state.

If the write operation is interrupted, the function will return an
error according to its contract. In this case, it will return false.

Clarifying these points would really help people understand how to reason about concurrency with this API.

This is described in the document.
There is, of course, a nuance regarding extended error descriptions,
but at the moment no such changes are planned.

10 months ago by Edmond Dantes — view source — reply

unread

As for:

For example, is the scheduler run-to-suspend?
And non-preemptive behaviour only appears here in the thread. It isn’t mentioned in the RFC itself.

There is no direct statement in the RFC that cooperative multitasking
is implemented.
I think this text was removed, and that needs to be fixed. But on the
other hand, there is a clear description of the contract expressed in
different words:

RFC: "A coroutine can stop itself passing control to the scheduler.
However, it cannot be stopped externally."

which essentially means the same thing.

This is exactly what constitutes the public contract between PHP and
the developer.

I included a list of I/O functions for demonstration purposes, but
this list is not part of the RFC.
It is part of the implementation. This means that not all I/O
functions can or should be adapted immediately.

10 months ago by Morgan — view source — reply

unread

RFC: "A coroutine can stop itself passing control to the scheduler.
However, it cannot be stopped externally."

which essentially means the same thing.

Oh ... there's an ambiguity in that line. "I'm now going to pass control
to the scheduler ... no, no I won't; that's exactly what they'll be
expecting me to do."

I suggest rewriting the sentence as:

"A coroutine can stop itself and pass control to the scheduler."
or
"A coroutine can stop itself, passing control to the scheduler."

10 months ago by Edmond Dantes — view source — reply

unread

Hello, Morgan

I suggest rewriting the sentence as:
"A coroutine can stop itself and pass control to the scheduler."
Thanks.
I would also add that a coroutine cannot be stopped from the outside.
Will this be ok?

10 months ago by Rob Landers — view source — reply

unread

Hello.

Based on the conversation so far, I’d imagine the list to look something like:

Yes, that’s absolutely correct. When a programmer uses an operation
that would normally block the entire thread, control is handed over to
the Scheduler instead.
The suspend function is called inside all of these operations.

I think that "normally" is doing a lot of work here. fwrite() can block, but often doesn’t. file_get_contents() is usually instant for local files but can take seconds on NFS or with an HTTP URL. An array_map() always blocks the thread but should never suspend.

Without very clear rules, it becomes impossible to reason about what’ll suspend and what won’t.

If that’s the intended model, it’d help to have that spelled out directly; it makes it immediately clear which functions can or will suspend and prevents surprises.

In the Async implementation, it will be specified which functions are supported.

This is exactly the kind of thing that needs to be in the RFC itself. Relying on "the implementation will document it" creates an unstable contract.

Even something simple like:

if it can perform network IO
if it can perform file/stream IO
if it can sleep or wait on timers
if it awaits a FutureLike
if it calls suspend()

This would then create a stable baseline and require an RFC to change the rules, forcing people to think through BC breakages and ecosystem impact.

I also think the RFC needs at least minimal wording about scheduler guarantees, even if the details are implementation-specific.
The Scheduler guarantees that a coroutine will be invoked if it is in the queue.

That’s not quite enough. The order really matters. Different schedulers produce different observable results.

For example:

function step(string $name, string $msg) {
echo "$name: $msg\n";
suspend();
}

spawn(function() { step("A", "1"); step("A", "2"); step("A", "3"); });
spawn(function() { step("B", "1"); step("B", "2"); step("B", "3"); });
spawn(function() { step("C", "1"); step("C", "2"); step("C", "3"); });

Under different scheduling strategies you get different, but stable patterns.

Consider FIFO or round-robin, run-to-suspend:

A: 1
B: 1
C: 1
A: 2
B: 2
Cl: 2
A: 3
B: 3
C: 3

But with a stack-like or LIFO strategy, running-to-suspend:

A: 1
B: 1
C: 1
C: 2
C: 3
B: 2
B: 3
A: 2
A: 3

Both are valid, but are important to know which one is implemented, and if someone wants to replace the scheduler, they also need to ensure they guarantee this behaviour.

For example, is the scheduler run-to-suspend? FIFO or round-robin wakeup? And non-preemptive behaviour only appears here in the thread. It isn’t mentioned in the RFC itself.

In Go, for example, when it was still cooperative, these details were
also not part of any public contract. The only guarantee Go provided
was that a coroutine would not be interrupted arbitrarily. The same
applies to this RFC: coroutines are interrupted only at designated
suspension points.
However, neither Go nor any other language exposes the internal
details of the Scheduler as part of a public contract, because those
details may change without notice.

Go did document these details during its cooperative era, including exactly where goroutines might yield. Unfortunately, I can’t find a link to documentation that old. I did come across the old design docs that might shed some light on how things worked back then: https://go.dev/wiki/DesignDocuments

The key point is that Go made cooperative scheduling predictable enough that developers could write performant code without guessing.

That’s important for people writing long, CPU-bound loops, since nothing will interrupt them unless they explicitly yield.
Hypothetically, in the future it may become possible to interrupt
loops, just like Go eventually did. This would likely require an
additional RFC. PHP does have the ability to interrupt a loop at any
point, but most likely only for terminating execution.
This RFC does nothing of the sort.

My concern isn’t the lack of loop preemption. My concern is that the RFC never says CPU loops don’t yield. If it isn’t stated explicitly, it won’t be documented, and users will discover it the hard way. That’s exactly the sort of footgun we should avoid at the language level.

Lastly, cancellation during a syscall is still unclear. If a coroutine is cancelled while something like fwrite() or a DB write is in progress, what should happen?
Does fwrite() still return the number of bytes written? Does it throw? For write-operations in particular, this affects whether applications can maintain a consistent state.

If the write operation is interrupted, the function will return an
error according to its contract. In this case, it will return false.

fwrite() almost never returns false, it returns "bytes written OR false". Partial successful writes are normal and extremely common. So, cancellation does change the behaviour unless this is spelled out very carefully so calling code can recover appropriately.

Clarifying these points would really help people understand how to reason about concurrency with this API.

This is described in the document.

I may be missing something, but I don’t see this spelled out anywhere in the RFC.

There is, of course, a nuance regarding extended error descriptions,
but at the moment no such changes are planned.

That’s fine, but then do you expect the RFC to pass as-is? Right now, without suspension rules, scheduler guarantees, defined syscall-cancellation semantics, it’s tough to evaluate the correctness and performance implications. Leaving some of the most important aspects as an "implementation detail" seems like asking for trouble.

— Rob

10 months ago by Jakub Zelenka — view source — reply

unread

Hi,

Hello.

Based on the conversation so far, I’d imagine the list to look something
like:

Yes, that’s absolutely correct. When a programmer uses an operation
that would normally block the entire thread, control is handed over to
the Scheduler instead.
The suspend function is called inside all of these operations.

I think that "normally" is doing a lot of work here. fwrite() can block,
but often doesn’t. file_get_contents() is usually instant for local files
but can take seconds on NFS or with an HTTP URL. An array_map() always
blocks the thread but should never suspend.

Without very clear rules, it becomes impossible to reason about what’ll
suspend and what won’t.

If that’s the intended model, it’d help to have that spelled out
directly; it makes it immediately clear which functions can or will suspend
and prevents surprises.

In the Async implementation, it will be specified which functions are
supported.

This is exactly the kind of thing that needs to be in the RFC itself.
Relying on "the implementation will document it" creates an unstable
contract.

Even something simple like:

if it can perform network IO

if it can perform file/stream IO

if it can sleep or wait on timers

None of the above is part is this RFC so why is this being discussed. Any
of the changes to stream layer and extensions will require special RFC and
mainly clean implementation. We will need to carefully consider where the
suspension is going to be done.

I think if there are parts of the RFC that mention IO, it should be removed
here. I think this RFC should also remove any mention of reactor as it's
irrelevant for this.

Kind regards,

Jakub

10 months ago by Rob Landers — view source — reply

unread

Hi,

__

Hello.

Based on the conversation so far, I’d imagine the list to look something like:

Yes, that’s absolutely correct. When a programmer uses an operation
that would normally block the entire thread, control is handed over to
the Scheduler instead.
The suspend function is called inside all of these operations.

I think that "normally" is doing a lot of work here. fwrite() can block, but often doesn’t. file_get_contents() is usually instant for local files but can take seconds on NFS or with an HTTP URL. An array_map() always blocks the thread but should never suspend.

Without very clear rules, it becomes impossible to reason about what’ll suspend and what won’t.

If that’s the intended model, it’d help to have that spelled out directly; it makes it immediately clear which functions can or will suspend and prevents surprises.

In the Async implementation, it will be specified which functions are supported.

This is exactly the kind of thing that needs to be in the RFC itself. Relying on "the implementation will document it" creates an unstable contract.

Even something simple like:

if it can perform network IO

if it can perform file/stream IO

if it can sleep or wait on timers

None of the above is part is this RFC so why is this being discussed. Any of the changes to stream layer and extensions will require special RFC and mainly clean implementation. We will need to carefully consider where the suspension is going to be done.

My point is that it should be a part of the RFC.

— Rob

10 months ago by Jakub Zelenka — view source — reply

unread

Hi,

Hello.

Based on the conversation so far, I’d imagine the list to look something
like:

Yes, that’s absolutely correct. When a programmer uses an operation
that would normally block the entire thread, control is handed over to
the Scheduler instead.
The suspend function is called inside all of these operations.

I think that "normally" is doing a lot of work here. fwrite() can block,
but often doesn’t. file_get_contents() is usually instant for local files
but can take seconds on NFS or with an HTTP URL. An array_map() always
blocks the thread but should never suspend.

Without very clear rules, it becomes impossible to reason about what’ll
suspend and what won’t.

If that’s the intended model, it’d help to have that spelled out
directly; it makes it immediately clear which functions can or will suspend
and prevents surprises.

In the Async implementation, it will be specified which functions are
supported.

This is exactly the kind of thing that needs to be in the RFC itself.
Relying on "the implementation will document it" creates an unstable
contract.

Even something simple like:

if it can perform network IO

if it can perform file/stream IO

if it can sleep or wait on timers

None of the above is part is this RFC so why is this being discussed. Any
of the changes to stream layer and extensions will require special RFC and
mainly clean implementation. We will need to carefully consider where the
suspension is going to be done.

My point is that it should be a part of the RFC.

But this is hard to know exactly. Also there will be always 3rd extensions
that can block so we will need to do it piece by piece. You can just take
it that ideally everything that can block would be suspendable . The first
candidate is surely stream internall poll that is used for stream IO in
various places and could handles most suspensions including in mysqlnd.
Then curl and sockets would be probably added. There are various other bits
already present in Edmonds PoC but we will need to consider them one by one.

In other words, we can't really know that until we have some base pieces
merged (this RFC) and there is acceptable implementation that can be merged
for those parts.

Kind regards,

Jakub

10 months ago by Rob Landers — view source — reply

unread

__

Hi,

__

Hello.

Based on the conversation so far, I’d imagine the list to look something like:

Yes, that’s absolutely correct. When a programmer uses an operation
that would normally block the entire thread, control is handed over to
the Scheduler instead.
The suspend function is called inside all of these operations.

I think that "normally" is doing a lot of work here. fwrite() can block, but often doesn’t. file_get_contents() is usually instant for local files but can take seconds on NFS or with an HTTP URL. An array_map() always blocks the thread but should never suspend.

Without very clear rules, it becomes impossible to reason about what’ll suspend and what won’t.

If that’s the intended model, it’d help to have that spelled out directly; it makes it immediately clear which functions can or will suspend and prevents surprises.

In the Async implementation, it will be specified which functions are supported.

This is exactly the kind of thing that needs to be in the RFC itself. Relying on "the implementation will document it" creates an unstable contract.

Even something simple like:

if it can perform network IO

if it can perform file/stream IO

if it can sleep or wait on timers

None of the above is part is this RFC so why is this being discussed. Any of the changes to stream layer and extensions will require special RFC and mainly clean implementation. We will need to carefully consider where the suspension is going to be done.

My point is that it should be a part of the RFC.

But this is hard to know exactly. Also there will be always 3rd extensions that can block so we will need to do it piece by piece. You can just take it that ideally everything that can block would be suspendable . The first candidate is surely stream internall poll that is used for stream IO in various places and could handles most suspensions including in mysqlnd. Then curl and sockets would be probably added. There are various other bits already present in Edmonds PoC but we will need to consider them one by one.

In other words, we can't really know that until we have some base pieces merged (this RFC) and there is acceptable implementation that can be merged for those parts.

Kind regards,

Jakub

I guess my main thing is that this RFC should only cover coroutine machinery: it should not promise "transparent async" or "code that works exactly the same" *OR *if it wants to make those claims, it should actually demonstrate how instead of hand-waving everything as an "implementation detail" when none of those claims can actually be validated without those details.

— Rob

10 months ago by Edmond Dantes — view source — reply

unread

I guess my main thing is that this RFC should only cover coroutine machinery: it should not promise "transparent async" or "code

It’s like trying to dig a hole with half a shovel :)

that works exactly the same" OR if it wants to make those claims, it should actually demonstrate
how instead of hand-waving everything as an "implementation detail" when none of those claims can actually be validated without those details.

All of my claims are backed by tests :)

10 months ago by Jakub Zelenka — view source — reply

unread

I guess my main thing is that this RFC should only cover coroutine
machinery: it should not promise "transparent async" or "code

It’s like trying to dig a hole with half a shovel :)

I think Rob has got a point that you don't really need to give such
promises that lack on details.

that works exactly the same" OR if it wants to make those claims, it
should actually demonstrate
how instead of hand-waving everything as an "implementation detail" when
none of those claims can actually be validated without those details.

All of my claims are backed by tests :)

The problem is that no tests are really provided with the RFC in the PR
that can be easily checked. I think it's kind of a problem of all RFC's
that don't have implementation (this one have but it's really hard to
extract). The people can only guess your intention but cannot really verify
them as they could if there was implementation.

Kind regards,

Jakub

10 months ago by Edmond Dantes — view source — reply

unread

Hello.

I think Rob has got a point that you don't really need to give such promises that lack on details.
The problem is that the phrase from the RFC is being interpreted in a
distorted way, and meaning is being attributed to it that it never
had. Not to mention the fact that real-world facts are being ignored.

The problem is that no tests are really provided with the RFC in the PR that can be easily checked.
I think it's kind of a problem of all RFC's that don't have implementation (this one have but it's really hard to extract).
The people can only guess your intention but cannot really verify them as they could if there was implementation.

This RFC has more than 300 tests that you can review. They’re also
sorted into folders, have descriptions, and are quite easy to read.
There are also two Dockerfiles for this RFC that let you run PHP 8.6
and try all the features with a single command.
But wait, this is only the beginning of the fun.

The funniest part is that this RFC has effectively existed in the real
world for many years (7 or 10?), and there are entire teams of
developers who have been using PHP according to this RFC for a long
time.
And even if this project had no code at all, you can at any moment
take Swoole or Swow, try PHP on steroids with real code, and see how
this RFC affects existing code.

I would understand if this RFC were proposing something very new,
something that had never existed in PHP before. But the irony is that
it proposes to standardize what has already been in PHP for many years
but has never been made a standard. And instead of discussing
implementation details, we’re discussing that 2 + 2 ≠ 5.

10 months ago by Jakub Zelenka — view source — reply

unread

Hi,

Hello.

I think Rob has got a point that you don't really need to give such
promises that lack on details.
The problem is that the phrase from the RFC is being interpreted in a
distorted way, and meaning is being attributed to it that it never
had. Not to mention the fact that real-world facts are being ignored.

Well I can see why this phrases are confusing for some users. The thing is
that saying that something will behave exactly the same omits the fact that
it introduces the new suspension points that can have side effects. This
might be clear to you but I can see why it's not clear to others. I think
this is really more wording issue and you should make it clearer or not to
claim it at all.

I think what could help is to also add a section comparing the coloring
approach and why you think this is a better for PHP.

The problem is that no tests are really provided with the RFC in the PR
that can be easily checked.
I think it's kind of a problem of all RFC's that don't have
implementation (this one have but it's really hard to extract).
The people can only guess your intention but cannot really verify them
as they could if there was implementation.

This RFC has more than 300 tests that you can review. They’re also
sorted into folders, have descriptions, and are quite easy to read.
There are also two Dockerfiles for this RFC that let you run PHP 8.6
and try all the features with a single command.

But that's in your php-src branch + extension code so it's very hard for
everyone to find and even harder to try. My argument was that this should
be extracted in the minimal form to the PR so people can see what's being
proposed and don't need to guess based on the RFC content.

Kind regards,

Jakub

10 months ago by Rob Landers — view source — reply

unread

I guess my main thing is that this RFC should only cover coroutine machinery: it should not promise "transparent async" or "code

It’s like trying to dig a hole with half a shovel :)

that works exactly the same" OR if it wants to make those claims, it should actually demonstrate
how instead of hand-waving everything as an "implementation detail" when none of those claims can actually be validated without those details.

All of my claims are backed by tests :)

I will leave with some final advice. The problem with tests is that they only validate the current implementation, which isn’t guaranteed to be the final implementation. I would recommend reviewing your tests and matching up each of them to where you mention that behavior or define it in the RFC. If the tests are implementation-specific, then it needs to be defined in the RFC. For example, you say that the scheduler is 100% an implementation detail, but your outputs in the tests rely on a specific ordered queue. You should at least define the ordering the queue should be processed in the RFC (LIFO vs FIFO) so that even if the implementation changes, the tests still pass.

That’s one example, you can review my previous comments to discover other examples, such as defining the rules of suspension points.

I wish you the best,

— Rob

10 months ago by Edmond Dantes — view source — reply

unread

I would recommend reviewing your tests and matching up each of them to where you mention that behavior or define it in the RFC. If the tests are implementation-specific, then it needs to be defined in the RFC.
Yes, in some tests I break best practices and rely on implementation
details to verify behavior. This makes the tests fragile.

For example, you say that the scheduler is 100% an implementation detail, but your outputs in the tests rely on a specific ordered queue.
Yes, I’m a lazy programmer :)

I wish you the best,
Thank you.

Best regards, Ed

10 months ago by Larry Garfield — view source — reply

unread

I guess my main thing is that this RFC should only cover coroutine machinery: it should not promise "transparent async" or "code

It’s like trying to dig a hole with half a shovel :)

that works exactly the same" OR if it wants to make those claims, it should actually demonstrate
how instead of hand-waving everything as an "implementation detail" when none of those claims can actually be validated without those details.

All of my claims are backed by tests :)

I will leave with some final advice. The problem with tests is that
they only validate the current implementation, which isn’t guaranteed
to be the final implementation. I would recommend reviewing your tests
and matching up each of them to where you mention that behavior or
define it in the RFC. If the tests are implementation-specific, then it
needs to be defined in the RFC. For example, you say that the scheduler
is 100% an implementation detail, but your outputs in the tests rely on
a specific ordered queue. You should at least define the ordering the
queue should be processed in the RFC (LIFO vs FIFO) so that even if the
implementation changes, the tests still pass.

That’s one example, you can review my previous comments to discover
other examples, such as defining the rules of suspension points.

I wish you the best,

A few thoughts to add:

First: People, please don't include me on the reply line. I just got 30 messages doubled. Once or twice, fine, but somewhere in that thread someone should have trimmed the To field back to just the list. Or use reply-list, just once. If nothing else, please remove me from the name list. %3C/rant>

Second: I think the key point here is one that Kevlin Henny has raised before in presentations: If you have any sort of concurrency, shared mutable state is a problem. Shared immutable state is no problem. Unshared mutable state is no problem. Unshared immutable state is no problem. Shared mutable state is where the problem is, and if you eliminate that, you eliminate virtually all race conditions. (See also: the entire design of Rust). Naturally, easier said than done.

While I presume the people who frequent this list are disproportionately the sort that already try to avoid shared mutable state on principle (I do), we are not a representative sample. So "the code runs exactly the same, modulo any shared mutable state, there be dragons" (Edmond's point) is true, but given the state of the PHP ecosystem also means "There's therefore an unknown but probably very not-small number of dragons out there, just waiting for us" (Rob and John's point, among others).

"This hasn't been a problem in practice for Swoole", I will take at face value as true (I have no data on the matter); the degree to which that is indicative of the rest of the ecosystem is what is highly debatable; Swoole et al represent a tiny fraction of the ecosystem, and is used by people that know they're using it. Or they're using code written by top-notch developers who have been preparing for this sort of scenario for a decade (Laminas, Symfony, etc.). The degree to which we can apply that lack of concern to the billions of other lines of PHP code out there is an open question.

So the debate seems to be between "let's assume it's safe and make the nicer-to-work-with option" and "let's assume it's unsafe and so we need another option." But Javascript-style colored functions would suck, for well-documented reasons. I recall a very unpleasant experience where I had a perfectly fine sync function in JS that I needed to modify to include an IO call, which required rewriting 5 functions and touching a dozen more. That's what I think we all want to NOT have in PHP.

Third: So my question would be, is there a practical middle-ground? Is there a syntax and semantics we could define that would cover most edge cases existing code may have involving shared mutable state automatically, and have some manual way of handling the rest that doesn't require updating 17 functions, some of which may not even be my code? Would that be "safe enough"? Is there a "safe enough"?

I am still on team "structured concurrency only or bust" myself, though I admit that doesn't fully resolve the issue. But the particular question here is how to indicate where suspension may occur; the RFC right now says "any IO operation, implicitly, hope that's OK." Which is both very convenient and potentially dangerous. But the status quo today of "no IO operations, which means you have to write your own IO libraries from the socket level up, GFL" is, obviously, not a great time.

Just completely spitballing, so possibly awful idea but maybe it will make someone else think of a better one: Colored functions to opt-in to suspension but do NOT propagate that dependency up. Something like, if you prefix a function call with await or async, then you could suspend there; you're explicitly telling the engine you're OK with suspending here if it wants to. If there's no other coroutines active it behaves exactly (and in this case, actually exactly) as it would today, so there's no cost to add it. It would also work on non-IO functions, so long-running CPU-bound processes could opt-in to "hey, I'm taking a long time, if you want to jump in here that's safe." However, you're NOT limited to using that keyword inside a marked function. You can use it on any call to file_get_contents(). However, a function can also declare itself sync, which will override that opt-in for the entire call stack down. Basically it disables the suspend operation. So if you have a tricky bit of global-using code, you can opt it out of switching without having to rewrite the world.

In code, it would look something like:

function log(string $m) {
// We're explicitly allowing the scheduler to swap out here,
// but it doesn't have to.
async file_put_contents('log.txt', $m, FILE_APPEND);
}

function first() {
// Do work.

// This call may end up switching to another coroutine for a while, or not, we don't care.
// Importantly, we don't need to "color" first() for it to work.
log('Did work');
}

sync function second() {
// Nothing inside here will swap to another coroutine, even though log()
// contains an async call inside it. The code behaves as if that
// keyword is just not there.
log('starting work.');
// Do work.
log ('finished work.");
}

This would mean that IO suspension becomes opt-in by default, so most IO-heavy libraries are not going to benefit from any async until they're updated. But that seems it may be safer than just assuming they're safe or else "oops, you're screwed on 8.6 until you add some new 8.6-specific syntax, sorry." (Being able to write code that runs correctly in two, ideally several, versions is crucially important for BC.)

As I said, this is just me spitballing so there may be a very good reason why it can't work; I'm just trying to brainstorm a way out of the devil's dichotomy of "you have to rewrite all of your code to have no shared mutable state, even though you don't know what shared means, sorry" vs "your code may work, probably, but we're not sure, there could be subtle bugs that are hard to find, sorry." Neither of which seems like good options.

--Larry Garfield

10 months ago by Edmond Dantes — view source — reply

unread

Hello Larry

Regarding the phrase about “problems with mutable shared state”.

In this context, when I say “problem,” I mean something that
significantly worsens development. “Not a problem” does not mean that
errors never occur. It means that resolving them takes an acceptable
amount of time. This phenomenon has reasonable explanations:

Developers usually know where they use global state. And even if
they don’t, they know that this is the first thing to check. That
helps a lot.
Bugs involving shared memory are usually easy to reproduce (which
is rare in asynchronous programming). This drastically reduces
debugging time.
In a single-threaded environment, shared-state bugs are much
simpler. For example, the probability of corrupting data structures is
relatively low.
In PHP projects, shared state is typically used in a small set of
scenarios, and these scenarios are usually simple, which also
minimizes the time needed to fix issues.

For example: some global variable stores a SessionId. And then it
turns out that an action was recorded not for the user who performed
it, but for another one with a different session. In most cases, the
bug report alone already points you directly to where you need to
look.

My point is not that you should ignore shared-memory bugs, but that
you shouldn’t fear them or treat them as something huge.

I also want to remind that in programming there are no “bad”
approaches or technologies. Shared mutable state is a very useful
thing and should be used. For caching, for storing session data, for
access-rights memoization, and so on. You don’t need to avoid shared
state. You need good tools, patterns, and proper practices. As for
language-level abstraction, features like local $var, effects, and
context will help make the code safer.
Monads would also fit PHP well, as they allow expressing parallelism
better than procedural code.

That's what I think we all want to NOT have in PHP.
Exactly. PHP exists to save money, not to spend it. Recently there was
an article about Python describing the problems caused by having
multiple runtimes.

Third: So my question would be, is there a practical middle-ground?
Is there a syntax and semantics we could define that would cover most edge cases existing code may have involving shared mutable state automatically

Based on the current state of programming languages, there is no good
solution. Only radical ones like Actors, Arenas, and so on. All of
them rely on complete memory isolation. But that’s not the point. The
point is that PHP still has legacy code, and you cannot just throw it
away.

the RFC right now says "any IO operation, implicitly, hope that's OK."

Go does the same thing, and it’s a very convenient mechanism.
There is a small percentage of cases where this breaks the logic.
Here is one of them:

$start = microtime(true);
$data = file_get_contents('data.txt');

$elapsed = microtime(true) - $start;

if ($elapsed > 0.001) {
    echo "Reading took too long\n";
}

The code breaks only in two cases:

When there is usage of mutable shared memory/shared resource
When there is logic based on a timer

Yes, such code will break inside a coroutine. But should we be afraid
of that? I don’t think so. The likelihood of this kind of logic is
very low. And if needed, this case can be explicitly documented.

Besides this case, special attention is also needed for:

libraries that use sockets
libraries that use threads
libraries that use processes

But this does not mean there will necessarily be major or serious
problems adapting the code. PHP already has experience with Swoole,
and Swoole works exactly the same way as the code in this RFC. For
example, adapting the RabbitMQ library for Swoole required changing
about 10–20 lines of code, roughly 2–3 days of work. Most of the time
was spent on testing.

Regarding the hybrid function-tainting model, it is designed for
compilers or static analysis. For PHP, it could be implemented like
this:

#NoAsync
function myFun() {
    asyncFun(); // Error <---
}

function asyncFun() {}

This is a solvable task for static analysis.

10 months ago by Rowan Tommins [IMSoP] — view source — reply

unread

Regarding the hybrid function-tainting model, it is designed for
compilers or static analysis. For PHP, it could be implemented like
this:
#NoAsync
function myFun() {
     asyncFun(); // Error <---
}

function asyncFun() {}
This is a solvable task for static analysis.

As I understand it, Larry's suggestion was not to prevent marked code
from calling into an async function, but to force the called function
into sync mode.

Imagine you have code like this running PHP 8.4, using a third-party SDK
library:

function myFun() {
$input = someThingWithSharedState();
$data = SomeApiSdk\someFunction($input);
someOtherThingWithSharedState($data);
}

Then in 9.0, we get True Async, and the author of the SDK starts using
async I/O functions somewhere deep in the library.

Without any special marking, other functions can run between the
statements in this function, completely unknown to the author of myFun()

If you mark it as "NoAsync" in the sense you've described, a static
analyser can point out the problem, but you still have no solution,
short of forking the SDK and removing all async code.

Larry's "sync" marker would be a change to run-time behaviour: when
myFun() begins, the scheduler switches into "linear mode", until the end
of that function. In that mode, when a suspend() point is reached, the
scheduler immediately resumes that coroutine rather than choosing a
different one.

If this was marked by an attribute, you could add it to code that
supports both PHP 8.x and 9.x, and the code would run the same in both
versions.

Aside: I definitely think that if/when True Async is added, we should
call the resulting version PHP 9.0 not 8.5 or 8.6 or whatever. As well
as potentially requiring users to review code for dangerous cases, it
would be a huge new feature worthy of the branding of a major version.

--
Rowan Tommins
[IMSoP]

10 months ago by Edmond Dantes — view source — reply

unread

Hello.

Larry's "sync" marker would be a change to run-time behaviour: when
myFun() begins, the scheduler switches into "linear mode", until the end
of that function. In that mode, when a suspend() point is reached, the
scheduler immediately resumes that coroutine rather than choosing a
different one.

This is very similar to the async\protect mechanism, but with a
restriction that forbids switching inside IO functions. It is possible
to implement this, but other coroutines will experience serious
starvation. In other words, it can be done, but it should not be done.

Concurrency works well when the programmer follows one rule. Every
coroutine must have many suspension points, and the time between these
points must not exceed a certain value. This is a typical scenario for
web applications. It is not universal. There are completely different
scenarios. For example, a PHP linter.

For a PHP linter, parallel execution will be more useful. Async
functions can also help when combined with threads. The architecture
will be as follows.

A job manager that monitors execution where one coroutine is one task.
A pool of threads where there are almost no coroutines.

The job manager creates a coroutine that sends a task to a separate
thread and handles it.
It can also perform IO. In the separate thread, coroutines are not
really needed because the code mostly performs heavy computations.
Of course, TrueAsync will provide the programmer with a channel for
interacting with a coroutine in another thread, and that interaction
will also be asynchronous.
This approach is 100 percent memory-safe because it fully controls
which object is passed through the channel.

In other words, by understanding the strong side of coroutines and
having tools like a thread pool, a programmer can build PHP
applications without true parallelism that will not be inferior to Go.
The only downside is that you have to manually split the code into an
IO stream and a job worker. This adds a bit of development time, but
not critically.

Best regards, Ed

10 months ago by Edmond Dantes — view source — reply

unread

Hello.

An array_map() always blocks the thread but should never suspend.
This function is not related to this discussion or to the RFC.

Without very clear rules, it becomes impossible to reason about what’ll suspend and what won’t.
As I mentioned earlier, this RFC clearly defines the rules for
integrating functions. The functions themselves will be documented.

That’s not quite enough. The order really matters. Different schedulers produce different observable results.
No modern language guarantees a fixed execution order of coroutines.
Go, Kotlin, Python, JavaScript, C#, Rust. All only guarantee order
when explicit synchronization is used.
Everything else is an implementation detail of the scheduler, and user
code must not rely on it.
(Because concurrency naturally has many valid execution paths, not one.
Forcing a single fixed order is impossible without adding heavy
synchronization everywhere, which destroys performance and breaks the
concurrency model.)

Unfortunately, I can’t find a link to documentation that old.
no one can

I may be missing something, but I don’t see this spelled out anywhere in the RFC.
What exactly were you unable to find?

Right now, without suspension rules, scheduler guarantees, defined syscall-cancellation semantics, it’s tough to evaluate the correctness and performance implications.
Leaving some of the most important aspects as an "implementation detail" seems like asking for trouble.

I’m sorry, but it’s difficult for me to understand what this is referring to.

10 months ago by John Bafford — view source — reply

unread

Hello.

An array_map() always blocks the thread but should never suspend.
This function is not related to this discussion or to the RFC.

function writeData() {
return array_map(function($elt) {
[$path, $content] = $elt;
return [$path, file_put_contents($path, $content)]; //POSSIBLE SUSPENSION POINT
}, $this->data);
}

Now this array_map function potentially suspends. As of course does this writeData(). And whatever calls this writeData(). And so on up the stack.

Any function that calls any other function might have a hidden suspension point. And as Rob pointed out, any hook might also have a hidden suspension point, so you can't even trust code that doesn't look like it calls functions. And even if there are no hooks, __get(), __set(), etc are also there to ruin your day.

To provide an explicit example for this, code that fits this pattern is going to be problematic

Why is this considered a problem if this behavior is part of the
language’s contract?

Because this RFC changes the contract out from every line of php code ever written. Code that used to be strictly synchronous now has async suspension points it didn't ask for.

$this->data can be changed out from under writeData(), which leads to unexpected behavior.

So the developer must intentionally create two different coroutines.
Intentionally pass them the same object.
Intentionally write this code.
And the behavior is called “unexpected”? :)

Yes.

To repeat, a core premise of this RFC is:

• From a PHP developer's perspective, the main value of this implementation is that they DO NOT NEED to change existing code (or if changes are required, they should be minimal) to enable concurrency. Unlike explicit async models, this approach lets developers reuse existing synchronous code inside coroutines without modification.
• Code that was originally written and intended to run outside of a Coroutine must work EXACTLY THE SAME inside a Coroutine without modifications.

With this, the RFC implies that I should be able to take my synchronous PHP code and run it in a coroutine with other synchronous PHP code, and it will all just work. But obviously that won't work.

I understand that a different interpretation of this wording is, "well, that code does exactly what it did before, just that with coroutines, that happens to be broken". I could maybe squint and grunt disapprovingly about it being "technically correct", except "has a suspension point" is definitely not exactly how it worked before.

The changes described in the RFC refer to the algorithm for handling
I/O functions in blocking mode. And of course these words assume that
we haven’t lost our minds and understand that you cannot write
completely different message sequences to the same socket at the same
time. In practice, changes are of course sometimes necessary, but
throughout my entire experience working with coroutines, I should note
that I have never once run into the example you mentioned. Even when
adapting older projects. And do you know why? Because the first thing
we refactored in the old code was the places with shared variables.

Well, but that's not what my example was doing. My example was taking a set of (filename, content) pairs and writing them to individual files. But it was doing it in an async-unsafe way (because that has never before been a consideration), and so its data source can be corrupted while it is executing.

The async problems in my example might be glaringly obvious (for people skilled in the art), but many other async issues are much more subtle, such as Rob described in his follow-on email to mine. For people who have never had the pleasure of working with async code before, "the world changed out from under me" is not a scenario you're accustomed to thinking about.

But in PHP, colored functions are inconvenient. Overloading I/O
functions does not lead to serious errors that make developers suffer;
on the contrary, it saves time and gives the language more
flexibility.

Of course colored functions are inconvenient. But it's necessary or else you open to a whole class of easily avoidable problems. Those problems are much more inconvenient than colored functions.

Code that was written to be synchronous should remain synchronous unless it is explicitly modified to be asynchronous.

A developer should strive to minimize asynchronous code in a project.
The less of it there is, the better. Asynchronous code is evil. An
anti-pattern. A high-complexity zone. But if a developer chooses to
use asynchronous code, they shouldn’t act like they’re three years old
and seeing a computer for the first time. Definitely not. This
technology requires steady, capable hands :)

What this says to me is, "Here's a foot-gun. Please use it responsibly."

Now, I definitely want to have advanced features available for when they're needed. But that said, It would be great if we can avoid introducing new foot-guns, especially when we have the knowledge and experience from other languages to draw on and do better. And when we do introduce new foot-guns, it's better if we can make them strategically-targeted sniper rifles instead of blunderbusses.

If we allow for hidden async behavior, the entire system becomes impossible to reason about. Some library I'm using can cause an async race condition without me asking for it, and I can't know it doesn't unless I audit it and its interaction with my code.

And we know from practice that enough people won't use async responsibly that it will make the language look bad. What will happen is a junior dev will decide (or be told) that some code is performing poorly and they will see a forum post that says "use coroutines" or "use async" and they will cargo-cult their way to a hidden problem because they didn't take into consideration the full problem space. Not to say that can't happen with explicit suspensions, but the ceremony of declaring you have a possible suspension at least gives a pointer of, "this is where there's a suspension; what happens if it does?".

-John

10 months ago by Michael Morris — view source — reply

unread

I've been reading this discussion from the peanut gallery. I don't have
enough knowledge to really add anything to the conversation, but I do want
to take a moment to thank everyone involved. Of the changes to PHP since 7,
this promises to be some of the most significant. The care and time taken
to get this right is greatly appreciated. To all participants of the
thread, thank you.

10 months ago by Deleu — view source — reply

unread

I've been reading this discussion from the peanut gallery. I don't have
enough knowledge to really add anything to the conversation, but I do want
to take a moment to thank everyone involved. Of the changes to PHP since 7,
this promises to be some of the most significant. The care and time taken
to get this right is greatly appreciated. To all participants of the
thread, thank you.

As another bystander of this RFC, I can’t help but feel that too much
effort is being put into an RFC where the author is eager to take it to a
vote that is likely to fail. There doesn’t seem to be anybody else clearly
understanding the entire RFC and there are a lot of room for confusion for
voters, PHP users and core maintainers.

Maybe that’s inherent to a subject matter that indeed could be the biggest
change to PHP since 7, but given RFC voting history, all signs point to a
rejection vote. I don’t know how much has been discussed off list and/or
how many voters are involved off list.

At a very minimal, Async PHP would be interesting. A door to a new universe
seems more fitting to describe its impact, but I don’t know how this RFC
can be helped into a more likely successful outcome - or if my limited
pessimistic view of its results are even valid concerns.

10 months ago by Edmond Dantes — view source — reply

unread

Hello

Maybe that’s inherent to a subject matter that indeed could be the biggest change to PHP since 7,
but given RFC voting history, all signs point to a rejection vote.

I think so too.

I don’t know how much has been discussed off list and/or how many voters are involved off list.

There were almost no discussions apart from the one on INTERNALS. I
mean, there were no discussions that I’m aware of.

I don’t know what to do about a situation where PHP developers are
surprised to learn that their language has supported transparent
asynchrony as a core paradigm for several years already.
And instead of discussing the important details of the RFC, all the
effort goes into “basic questions” that aren’t worth discussing at
all.

It would be wonderful if there were a dedicated person whom everyone
would listen to carefully and who could explain the basic questions
privately. Most likely, this approach would solve many problems.

Best Regards, Ed

10 months ago by Rob Landers — view source — reply

unread

Hello
[snip]
I don’t know what to do about a situation where PHP developers are
surprised to learn that their language has supported transparent
asynchrony as a core paradigm for several years already.
And instead of discussing the important details of the RFC, all the
effort goes into “basic questions” that aren’t worth discussing at
all.

It would be wonderful if there were a dedicated person whom everyone
would listen to carefully and who could explain the basic questions
privately. Most likely, this approach would solve many problems.

You are that person in this context.

RFC authors are the subject-matter experts for their proposals, and the way the process succeeds is by the author being willing to explain, restate, clarify, and anchor concepts for people coming from different backgrounds and different levels of familiarity. That sometimes means answering questions that feel “basic”, or repeating an explanation with more context, or linking back to previous messages so readers can follow the chain of reasoning.

Speaking personally: I’ve put several hours into each of my responses because I want the RFC to succeed, and I think many others on the list are doing the same. When people ask questions or point out tension between claims, it’s not to nitpick: it’s because the RFC makes several strong promises, and people want to understand the implications for their frameworks, their libraries, and their day-to-day work.

That’s exactly what the “C” in RFC is for.

If something has been explained earlier, linking to that explanation is extremely helpful. If something hasn’t been explained yet, then the question isn’t trivial, it’s a signal that the document might need clearer wording so that future readers won’t have the same confusion. It’s like a bug report: just because one user reported the bug doesn’t mean only one person is suffering from it.

Everyone here is trying to understand the proposal on its own terms. Clear, patient communication from the author is what makes that possible.

— Rob

10 months ago by Edmond Dantes — view source — reply

unread

Hello.

RFC authors are the subject-matter experts for their proposals, and the way the process succeeds is by the author being willing to explain, restate, clarify, and anchor concepts for people coming from different backgrounds and
different levels of familiarity.

For the discussion to be constructive, there has to be a shared desire
to reach a common goal.
It sometimes feels like I am the only one who needs asynchrony :)

PHP does not have many options for implementing asynchrony.
Why?

Colored functions require the language to support two runtimes. For
PHP this is an anti-pattern. A single unified runtime is the key
feature of the language.
Any code with global state inside coroutines will not behave as
expected. This is not a property of this RFC but of any asynchronous
code. I can add this sentence, but last time I was asked to remove
wording that sounded alarming.

True Async, Swoole, and Swow follow the same idea. Different
implementations, but the same logic. AMPHP and React differ only in
that they do not intercept PHP functions, but they still use the same
technology, the same algorithms, and the same approach.

TrueAsync was originally written with code from AMPHP, so you can see
microtasks that AMPHP borrowed from JavaScript. Later, it adopted some
techniques from Swoole.
In other words, PHP has had a de facto coroutine approach for many
years, but no one has ever standardized it. There is an approach and
there are implementations, but there is no RFC.
Do you understand why this RFC is written the way it is?

Why does the RFC state that code does not need to be changed?
Because in AMPHP and React you cannot simply take your code and run it
inside a coroutine.
So you must replace all PHP functions with special ones.
This does not mean that a developer should forget about memory or
corner cases. There is no silver bullet in programming.
Do we really need to add a disclaimer under every statement that there
are always exceptions? In programming, every statement has exceptions.

What about colored functions. I recently explored a hybrid approach.
Its logic is relatively simple. The compiler allows writing async next
to a function, and then it automatically infects other functions. At
the same time there is a special word noasync that prevents the
compiler from automatic “infection.” Something similar can be
implemented in PHP using attributes or static analysis. This would
make the code more predictable while keeping it flexible. This RFC
does not conflict with that approach at all, and it can be implemented
in the future if there is a desire to make the language stricter.

10 months ago by Jakub Zelenka — view source — reply

unread

Hi,

Hello

Maybe that’s inherent to a subject matter that indeed could be the
biggest change to PHP since 7,
but given RFC voting history, all signs point to a rejection vote.

I think so too.

It looks like it but it's hard to say because many people have not had
chance to review / comment on this. I think it needs more time and the RFC
text needs more work to explain the basic difference. I think it also need
to have a good implementation for what is being proposed to have any chance
because there are often voters that will vote down just because there is no
implementation that can be reviewed. Basically everything that can be
addressed should be addressed before the vote so it's just decisions
between the actual approach.

I don’t know how much has been discussed off list and/or how many voters
are involved off list.

There were almost no discussions apart from the one on INTERNALS. I
mean, there were no discussions that I’m aware of.

I don’t know what to do about a situation where PHP developers are
surprised to learn that their language has supported transparent
asynchrony as a core paradigm for several years already.
And instead of discussing the important details of the RFC, all the
effort goes into “basic questions” that aren’t worth discussing at
all.

I think this is more that some people clearly prefer coloring and want to
point out the potential issues with this approach. This might be quite hard
to overcome and depends how many voters have such preference. Personally I
agree that coloring would be bad for PHP as I was dealing with "coloring
migration" in Python and it wasn't nice. That's why I think it might help
to have dedicated section about coloring (more comparison with the current
approach) in the RFC.

Kind regards,

Jakub

10 months ago by Edmond Dantes — view source — reply

unread

I think it needs more time and the RFC text needs more work to explain the basic difference.

I do not mind, but no one is writing anything.
(El coronel no tiene quien le escriba :))

I think it also need to have a good implementation for what is being proposed to have any chance because there are often voters that will vote down just because there is no implementation that can be reviewed.
Basically everything that can be addressed should be addressed before the vote so it's just decisions between the actual approach.

Perhaps, but no one has said that the current implementation is bad or
needs to be replaced.
I mean that so far no one except you has said this.
It seems to me that no one except a few people has actually looked at
the implementation. I am almost certain of that.
There has never been a conversation like “Yes, this RFC is fine, but
it needs a different implementation.”

I think this is more that some people clearly prefer coloring and want to point out the potential issues with this approach.
If someone in this discussion had actually written about the real
issue of not having colored functions, it would have been great and
extremely useful. No one did.

Most people simply have not encountered these problems. Colored
functions sometimes help localize errors related to global state. Not
always, but sometimes.
But there is also the opposite situation. The tragedy of the Python
async migration which slows down the adoption of async in Python.

Speaking more seriously, colored functions are more useful for
low-level languages. For high-level languages, the absence of colored
functions and full commitment to asynchronous code is actually more
beneficial. Languages that follow this path, like Go, have succeeded.
Why is that? Because high-level languages can hide contracts.
Low-level languages have the opposite requirement: they need to know
every contract explicitly. This happens because increasing abstraction
relies on hiding details.

That's why I think it might help to have dedicated section about coloring (more comparison with the current approach) in the RFC.
In the early versions there was similar text, but I removed it so it
wouldn’t overload the RFC. But in short, the absence of colored
functions simply saves time.

Best Regards, Ed

10 months ago by Deleu — view source — reply

unread

This thread is a clear, explicit and direct representation that there are
people following all the hard work that you’ve put into this RFC. I dare
say that nobody doubts your intentions, effort and initiative to tackle
such a complex and hard task.

The next step is to recognize that anybody participating in an RFC
discussion is a person taking time out of their lives, their family, their
work, their hobbies and volunteering their time similar to you. Everyone
has PHP best interest at heart even if what we think is best for PHP
differs.

Hello

Maybe that’s inherent to a subject matter that indeed could be the
biggest change to PHP since 7,
but given RFC voting history, all signs point to a rejection vote.

I think so too.

I don’t know how much has been discussed off list and/or how many voters
are involved off list.

There were almost no discussions apart from the one on INTERNALS. I
mean, there were no discussions that I’m aware of.

I don’t know what to do about a situation where PHP developers are
surprised to learn that their language has supported transparent
asynchrony as a core paradigm for several years already.

I suppose you’re referring to Swoole, AMPHP, ReactPHP, etc. While there’s
no denying they’re an important part of the ecosystem and living proof
there’s room for asynchronous PHP, your positioning here seems to assume
that everybody participating in your RFC should know, understand and use
one of these projects and have that baseline knowledge. From my personal
experience, context and little bubble, I would guess that less than 10% of
PHP engineers worldwide has any firsthand experience with any of these
tools combined. Your RFC can bridge that gap, but to do so you need to be
able to address why the other 90% hasn’t used any of these existing
approaches. Your target audience is exactly the opposite of what you expect.

And instead of discussing the important details of the RFC, all the
effort goes into “basic questions” that aren’t worth discussing at
all.

This is perhaps the most likely reason for the RFC to fail a vote. Let’s
break down voting in an abstract format. Some common reasons voters may
choose to vote NO are:

they don’t like the change and don’t want it on PHP
they disagree with the approach
the change increases PHP maintenance burden for a subjective benefit
the RFC is unclear

There’s not much you can do about 1. And for every NO you get, you need 2
YES to overcome it.

My personal guess is that 4 would drive a lot of NO and the reason is a
combination of factors. It’s a very complex and dense RFC and not every
voter will have any experience with the existing async tools provided by
the community. When trying to take time out of their personal lives to go
through such a complex and extensive RFC, basic questions will arise. They
will attempt to address it before going deeper and deeper and will be met
with a dismissive response from the author “because this is too basic and
we shouldn’t focus on it”. They will make the easy choice to not continue
to volunteer their time into something so hard to wrap your head around
since it’s too challenging to even get past the basics. A negative result
is not really a surprise at this point.

It would be wonderful if there were a dedicated person whom everyone

would listen to carefully and who could explain the basic questions
privately. Most likely, this approach would solve many problems.

Best Regards, Ed

Short of forking PHP, that seems like wishful thinking. PHP is driven by
committee and there doesn’t seem to be any way around that. But there are
approaches you can try.

One of them is pairing it up with someone better at docs, communication,
etc. Take a look at Larry’s RFCs. Majority of it he doesn’t write any C
code, but he makes up for it 10 times by producing clearly readable,
understandable and decision points communicated. Voters may still disagree
at the end, but it always feels like they understand what they’re voting
on/for.

I don’t know if you’re good with a camera, but going in a podcast with
someone like Brandt, Nuno Maduro or any other PHP podcast that tries to
breakdown internals for easy consumption could also help your cause.

Ultimately, I think your vast expertise on the matter is both a blessing
and a curse. A blessing because without it maybe the RFC would have never
even existed - at least no other RFC on the matter has reached so far. A
curse because nobody else seems to be able to understand what you’re
saying, what you’re doing and what PHP will be like if your RFC is approved.

As someone writing PHP for 15 years, I’m scared of bringing any async
community tools to any project I work because I don’t know what would
break. Your RFC states that you want all my code to keep working which is
great, but the moment I make use of something async it could inadvertently
affect something and I don’t know what/why/how. I’m dumb and don’t know how
async works and I don’t want PHP to allow me to shoot myself in the head
like that. Is there anything you can do about it?

I wish you all the best luck on the RFC and your next steps. I thank you
for all your time so far and I’m eager to see what come out of it.

10 months ago by Edmond Dantes — view source — reply

unread

why the other 90% hasn’t used any of these existing approaches

Five or more years ago, when the question of improving performance
came up and Swoole was being considered, I rejected it. Why? Because
it wasn’t part of PHP. That was the reason.
If Swoole became the PHP standard, our company would most likely switch to it.
Because support from the PHP community is a long-term guarantee, which
means the time and money invested will not be wasted.

Your target audience is exactly the opposite of what you expect.
If my audience consists of people who need new knowledge, that’s a
different story!

A negative result is not really a surprise at this point.
I don’t see any way to make this RFC clearer.
I doubt the problem is in the text.

They will attempt to address it before going deeper and deeper and will be met with a dismissive response from the author “because this is too basic and we shouldn’t focus on it”.
The problem is not a lack of willingness to discuss the basics, but
that the discussion should move toward clarifying the details. And
that is exactly what isn’t happening.
As you correctly noted, no one wants to waste their time for nothing.
I also don’t want to spend my time on a conversation with no result.

I don’t know if you’re good with a camera, but going in a podcast with someone like Brandt, Nuno Maduro or any other PHP podcast that tries to breakdown internals for easy consumption could also help your cause.
No problem with the camera. But I cannot speak English fluently.

A curse because nobody else seems to be able to understand what you’re saying, what you’re doing and what PHP will be like if your RFC is approved.
I have a small presentation that I prepared for TrueAsync, and I could
adapt it to the questions of this RFC.
But...
I know that the PHP community has specialists in asynchronous PHP who
are better than me.
All the experts from Swoole, Swow, AMPHP, React, and so on. That’s not
two or three developers.
In the previous discussion, Daniil Gentili wrote an amazingly professional post.

I’m scared of bringing any async community tools to any project I work because I don’t know what would break.
That was my first reaction several years ago.

Your RFC states that you want all my code to keep working which is great, but the moment I make use of something async it could inadvertently affect something and I don’t know what/why/how.
I’m dumb and don’t know how async works and I don’t want PHP to allow me to shoot myself in the head like that. Is there anything you can do about it?
In that case it’s better to choose a programming language like Elixir, Erlang.
PHP will not provide the required level of memory safety, although it
may be possible to add a special type of static variables to it.

I wish you all the best luck on the RFC and your next steps. I thank you for all your time so far and I’m eager to see what come out of it.
Thank you.

--- Ed

10 months ago by Michael Morris — view source — reply

unread

100% agree with Deleu.

Edmond, part of the problem you're facing is you aren't just introducing a
new feature - you're introducing a paradigm shift. I did an RFC like that
once - the introduction of runtime assertions into Drupal - and the
discussion around the change took far, far longer than the code - about 6
months. The initial reaction was that such are unneeded in Drupal, after
all there's thousands of unit tests. I had to explain that unit tests are
for known configurations and that runtime assertions are for the
unknown, among other arguments for and against. The RFC was eventually
accepted because it provided a means to significantly speed up Drupal by
stepping over its development token sanity checks in prod, but it was a
stormy journey.

Coming back to your proposal, While I'm very aware of the async pattern
from its use in JavaScript and C#, I don't quite get where it will be
useful in PHP. My experience with PHP is that it responds to HTTP requests
and that stepping over a function that hasn't finished its output isn't
helpful because you aren't really saving any time in that request thread as
it has very defined start and end points. But we write PHP scripts the way
we do because of how the language is structured. Many apps spend a lot of
time setting up for the request - Drupal for example has to boot its
kernel and part of that is the reading in of an enormous dependency
injection map generated from the configuration yaml files of the active
modules - that file alone can get into tens of thousands of lines of code.
All of that processing work to just be thrown away once the echo back to
the server occurs.

It would be nice to have a coordinator process that does all this setup
work and stays alive handling requests by handing them off to workers.
That's how the various C# apps I've seen work. But that sort of setup would
require asynchronous code. So I can see a future use case for all of this.
But you don't need to convince me - I don't have a vote.

Just, don't give up and don't get frustrated. It's hard. You can still read
the Drupal discussion on runtime assertions. I got rather testy at times
after explaining the same thing for the 12th time. But I didn't take it
personal and when my emotions needed checking I'd step away for a few hours.

Everyone, It might help to do an RFC just on the principle of adding
asynchronous code to 9 without commitment to any particular
implementation. Unlike the lofty goal of adding unicode to the language
which never panned out and caused us to need to step over 6, Edmond's RFC
seems far enough along to prove that it is a doable change for the language
in some form. Such an RFC would get Edmond off the clock and relieve some
stress of feeling like "I've got to get this approved before the window to
submit closes." I know what that window feels like because the PHP runtime
assertions addition to Drupal 8 was dead last and only accepted because it
provided a solution to an issue blocking release. It would also get the
rest of us off the clock as well. This proposal needs very careful
consideration as it has wide ranging implications. But it's also the sort
of proposal that can't walk in on a minor release, and major release
windows are rare.

Again, thank you everyone.

This thread is a clear, explicit and direct representation that there are
people following all the hard work that you’ve put into this RFC. I dare
say that nobody doubts your intentions, effort and initiative to tackle
such a complex and hard task.

The next step is to recognize that anybody participating in an RFC
discussion is a person taking time out of their lives, their family, their
work, their hobbies and volunteering their time similar to you. Everyone
has PHP best interest at heart even if what we think is best for PHP
differs.

Hello

Maybe that’s inherent to a subject matter that indeed could be the
biggest change to PHP since 7,
but given RFC voting history, all signs point to a rejection vote.

I think so too.

I don’t know how much has been discussed off list and/or how many
voters are involved off list.

There were almost no discussions apart from the one on INTERNALS. I
mean, there were no discussions that I’m aware of.

I don’t know what to do about a situation where PHP developers are
surprised to learn that their language has supported transparent
asynchrony as a core paradigm for several years already.

I suppose you’re referring to Swoole, AMPHP, ReactPHP, etc. While there’s
no denying they’re an important part of the ecosystem and living proof
there’s room for asynchronous PHP, your positioning here seems to assume
that everybody participating in your RFC should know, understand and use
one of these projects and have that baseline knowledge. From my personal
experience, context and little bubble, I would guess that less than 10% of
PHP engineers worldwide has any firsthand experience with any of these
tools combined. Your RFC can bridge that gap, but to do so you need to be
able to address why the other 90% hasn’t used any of these existing
approaches. Your target audience is exactly the opposite of what you expect.

And instead of discussing the important details of the RFC, all the
effort goes into “basic questions” that aren’t worth discussing at
all.

This is perhaps the most likely reason for the RFC to fail a vote. Let’s
break down voting in an abstract format. Some common reasons voters may
choose to vote NO are:

they don’t like the change and don’t want it on PHP

they disagree with the approach

the change increases PHP maintenance burden for a subjective benefit

the RFC is unclear

There’s not much you can do about 1. And for every NO you get, you need 2
YES to overcome it.

My personal guess is that 4 would drive a lot of NO and the reason is a
combination of factors. It’s a very complex and dense RFC and not every
voter will have any experience with the existing async tools provided by
the community. When trying to take time out of their personal lives to go
through such a complex and extensive RFC, basic questions will arise. They
will attempt to address it before going deeper and deeper and will be met
with a dismissive response from the author “because this is too basic and
we shouldn’t focus on it”. They will make the easy choice to not continue
to volunteer their time into something so hard to wrap your head around
since it’s too challenging to even get past the basics. A negative result
is not really a surprise at this point.

It would be wonderful if there were a dedicated person whom everyone

would listen to carefully and who could explain the basic questions
privately. Most likely, this approach would solve many problems.

Best Regards, Ed

Short of forking PHP, that seems like wishful thinking. PHP is driven by
committee and there doesn’t seem to be any way around that. But there are
approaches you can try.

One of them is pairing it up with someone better at docs, communication,
etc. Take a look at Larry’s RFCs. Majority of it he doesn’t write any C
code, but he makes up for it 10 times by producing clearly readable,
understandable and decision points communicated. Voters may still disagree
at the end, but it always feels like they understand what they’re voting
on/for.

I don’t know if you’re good with a camera, but going in a podcast with
someone like Brandt, Nuno Maduro or any other PHP podcast that tries to
breakdown internals for easy consumption could also help your cause.

Ultimately, I think your vast expertise on the matter is both a blessing
and a curse. A blessing because without it maybe the RFC would have never
even existed - at least no other RFC on the matter has reached so far. A
curse because nobody else seems to be able to understand what you’re
saying, what you’re doing and what PHP will be like if your RFC is approved.

As someone writing PHP for 15 years, I’m scared of bringing any async
community tools to any project I work because I don’t know what would
break. Your RFC states that you want all my code to keep working which is
great, but the moment I make use of something async it could inadvertently
affect something and I don’t know what/why/how. I’m dumb and don’t know how
async works and I don’t want PHP to allow me to shoot myself in the head
like that. Is there anything you can do about it?

I wish you all the best luck on the RFC and your next steps. I thank you
for all your time so far and I’m eager to see what come out of it.

10 months ago by Edmond Dantes — view source — reply

unread

Hello, Michael Morris

Coming back to your proposal, While I'm very aware of the async pattern from its use in JavaScript and C#, I don't quite get where it will be useful in PHP.
My experience with PHP is that it responds to HTTP requests and that stepping over a function that hasn't finished its output isn't helpful because you aren't really saving any time in that request thread as it has very defined start > and end points. But we write PHP scripts the way we do because of how the language is structured.

In the PHP community, there were developers who started using PHP for
stateful applications. It seems that Workerman was the first. Then
Swoole appeared, which turned PHP into the same thing Node.js turned
JavaScript into. Swoole introduced several key features:

A built-in HTTP server
Coroutines
Workers
The ability to use standard PHP functions directly inside coroutines

This made it possible to use a single PHP process to handle multiple
requests, increasing CPU utilization, reducing memory usage, and
lowering the number of processes. In certain scenarios, Swoole + JIT
PHP performs no worse than Go. This gave companies with legacy PHP
projects the ability to avoid rewriting everything in Go. Remember the
hype around Node.js a few years ago? “Node.js is better than PHP, it
outruns PHP.” JavaScript itself didn’t become faster, but its async
model allowed developers to build more responsive applications with
low latency. The secret of that performance boost is exactly the
combination of stateful architecture and coroutines. This formula
became the holy grail of modern backend applications.

Swoole went even further and created an entire software ecosystem that
not only made PHP asynchronous but also provided a full set of
standard tools for modern backend development. Swoole could have given
the language a huge push forward, especially if Symfony and Laravel
had adopted it. PHP had a very good chance to compete with Go in the
API market, offering simpler development and a large set of ready-made
solutions. This is what an alternate universe of PHP could have looked
like: a PHP that people would finally stop considering a second-class
language.
...
It seems I got a bit too carried away dreaming :) I’ll go turn off my
imagination now…

In other words, it is the absence of asynchrony in the language that
is the problem seriously slowing down further development. PHP has a
theoretical growth limit in its current form, and it has not yet been
reached.

Just, don't give up and don't get frustrated. It's hard. You can still read the Drupal discussion on runtime assertions.
I got rather testy at times after explaining the same thing for the 12th time. But I didn't take it personal and when my emotions needed checking I'd step away for a few hours.

Thank you for the support.

--- Ed

10 months ago by Larry Garfield — view source — reply

unread

It would be nice to have a coordinator process that does all this setup
work and stays alive handling requests by handing them off to workers.
That's how the various C# apps I've seen work. But that sort of setup
would require asynchronous code. So I can see a future use case for all
of this. But you don't need to convince me - I don't have a vote.

Alternatively, this is exactly what FrankenPHP solves. Bootstrap once, run many times. It doesn't require async to work, though it doesn't conflict with it either. Personally I think FrankenPHP offers a more compelling way forward for taking PHP to the next level as it requires less retooling than async, but I am quite open to having both.

Just, don't give up and don't get frustrated. It's hard. You can still
read the Drupal discussion on runtime assertions. I got rather testy at
times after explaining the same thing for the 12th time. But I didn't
take it personal and when my emotions needed checking I'd step away for
a few hours.

Oh dear, I remember those discussions. :-) And yes, the biggest thing I learned from Drupal 8 is that no one reads the <censored> plan, and you have to explain it to everyone individually, personally, several times, even though you'd expect smart people to be able to read the damned blog post that outlined the whole plan, but they of course did not.

--Larry Garfield

PS: Please remember to bottom-post, not top post.

10 months ago by Bart Vanhoutte — view source — reply

unread

Hello everybody,

Long time lurker, first time poster here. I hope I've respected all
the rules as this is my first time posting on a mailing list,
apologies in advance if I haven't.

As someone who has been following this RFC since inception and as a
daily user of both sync PHP (Drupal) and async PHP (ReactPHP) I have a
couple of things I'd like to say. I'll leave out the technical things
for now and will keep it to the subject of governance.

It seems to me that the process that usually takes place when an RFC
is proposed is not compatible with an RFC of this size. This is not an
RFC to add a function that does something with arrays (all due
respect), the changes in the language and ecosystem in general are
profound. As this language is mainly developed by volunteers, I
understand that not everybody can allocate the time and mental
fortitude it takes to process an RFC like this. Hell, it took me quite
some time to rewire my brain from thinking synchronously to thinking
asynchronously while first developing applications using ReactPHP, I
can only imagine how daunting an RFC like this can be if you've rarely
or never developed an application this way, even if you are someone
who's contributed to the PHP language in the past.
What is clear to me is that currently, the amount of people who have
the right to vote and that have enough knowledge on this subject to be
able to judge whether or not this RFC should pass or fail is too low.
Moreover, even though there are quite a few people in the community
who have the knowledge required because they either develop or work
with aforementioned libraries or extensions, (almost) none of them
seem to be involved in discussing this RFC. For an RFC that can
drastically change the way we develop applications I would expect more
experts on this matter to be involved. Ideally, PHP core developers,
library developers & maintainers, IDE developers, ..., would develop
software using this branch to at least get some feel for the paradigm
and this RFC in general.

I think Edmond has done a fantastic job developing this RFC but I
think an RFC of this size requires more resources, even if it just
means giving other PHP core developers the means to explore this topic
further. I feel like there is a big role to be played here by the PHP
Foundation to facilitate the development of a feature like this. I
hope this somehow gets picked up by them and that a working group or
something like that can be created that includes PHP core developers,
current async library maintainers, FIG-members, ..., so this RFC and
further related RFCs can be developed in a more structured (time &
money) way.

Best Regards,
Bart Vanhoutte

Op zo 16 nov 2025 om 21:28 schreef Larry Garfield larry@garfieldtech.com:

It would be nice to have a coordinator process that does all this setup
work and stays alive handling requests by handing them off to workers.
That's how the various C# apps I've seen work. But that sort of setup
would require asynchronous code. So I can see a future use case for all
of this. But you don't need to convince me - I don't have a vote.

Alternatively, this is exactly what FrankenPHP solves. Bootstrap once, run many times. It doesn't require async to work, though it doesn't conflict with it either. Personally I think FrankenPHP offers a more compelling way forward for taking PHP to the next level as it requires less retooling than async, but I am quite open to having both.

Just, don't give up and don't get frustrated. It's hard. You can still
read the Drupal discussion on runtime assertions. I got rather testy at
times after explaining the same thing for the 12th time. But I didn't
take it personal and when my emotions needed checking I'd step away for
a few hours.

Oh dear, I remember those discussions. :-) And yes, the biggest thing I learned from Drupal 8 is that no one reads the <censored> plan, and you have to explain it to everyone individually, personally, several times, even though you'd expect smart people to be able to read the damned blog post that outlined the whole plan, but they of course did not.

--Larry Garfield

PS: Please remember to bottom-post, not top post.

10 months ago by Edmond Dantes — view source — reply

unread

Hello Bart.
I am ready to agree with every word.

Participation from a working group, framework representatives, and the
ability to adapt libraries in advance would remove the concerns that
are currently causing fear. This is probably the only effective
process for developing and introducing such a feature.

Best Regards, Ed

10 months ago by Markus Podar — view source — reply

unread

I've been reading this discussion from the peanut gallery. I don't have
enough knowledge to really add anything to the conversation, but I do want
to take a moment to thank everyone involved. Of the changes to PHP since 7,
this promises to be some of the most significant. The care and time taken
to get this right is greatly appreciated. To all participants of the
thread, thank you.

I'll add myself to the Thank You-list. Thanks everyone, but especially
Edmond, for having the courage and bravery to tackle such a big and complex
topic. Your patience is virtue and I think you're showing a lot of it; I
hope it will be enough ;)

I for one hope we'll get this version of async, to me it checks everything
I'd expect and need from PHP and this way of backwards compatibility or
basically compatibility with most existing code is truly intriguing.

Again, thanks to everyone.

Markus

10 months ago by Joseph Leedy — view source — reply

unread

I've been reading this discussion from the peanut gallery. I don't have
enough knowledge to really add anything to the conversation, but I do want
to take a moment to thank everyone involved. Of the changes to PHP since 7,
this promises to be some of the most significant. The care and time taken
to get this right is greatly appreciated. To all participants of the
thread, thank you.

I too would like to join the thank you train. I appreciate the time, thought and detail that you've put into this proposal, Ed. I myself don't quite understand async and I've struggled to work with it in JavaScript (looking at you, fetch and Promise). That said, I'm glad you added the summary to the RFC, which breaks down your goals into easy to understand pieces.

A huge thank you to all of the participants in this discussion as well as the reviewers. I know that not everyone agrees on what the final implementation will look like, but I hope that you all can find a common ground and move forward with this monumental new feature. (Ideally something close to what has been proposed by the RFC in its entirety.)

Best Regards,
Joseph "Seph" Leedy

10 months ago by Edmond Dantes — view source — reply

unread

Now this array_map function potentially suspends. As of course does this writeData(). And whatever calls this writeData(). And so on up the stack.

And the most important thing is that the execution flow for this
function will not change.
Do you understand this?
This is described very clearly in the RFC.

Because this RFC changes the contract out from every line of php code ever written. Code that used to be strictly synchronous now has async suspension points it didn't ask for.

It’s not the RFC that changes the contract. It’s asynchronous
programming that changes the contract. And that lies outside the scope
of this RFC, because once a developer starts using asynchronous
programming, they must accept the global contracts of asynchronous
execution.
And this RFC does not deny that.

To repeat, a core premise of this RFC is:
Sorry, but I’m not going to repeat points that were ignored. My goal
is to help you understand the text of this RFC. I have no intention of
banging my head against a wall.

With this, the RFC implies that I should be able to take my synchronous PHP code and run it in a coroutine with other synchronous PHP code, and it will all just work. But obviously that won't work.
For most cases that’s exactly how it works.
In human language, there is no definition that cannot be twisted and
misinterpreted. Why elevate this idea to an absolute while ignoring
common sense? I have no idea. What’s the purpose of this discussion?
It seemed to me that the purpose of the discussion was a professional
conversation, not playing with meanings.

I understand that a different interpretation of this wording is, "well, that code does exactly what it did before, just that with coroutines, that happens to be broken".
I have no idea what you’re talking about.

Of course colored functions are inconvenient. But it's necessary or else you open to a whole class of easily avoidable problems. Those problems are much more inconvenient than colored functions.
I already discussed this topic back in March, and I can briefly
summarize: the absence of colored functions in PHP is the only viable
way to implement async.

Code that was written to be synchronous should remain synchronous unless it is explicitly modified to be asynchronous.
Practice has proven the opposite.

What this says to me is, "Here's a foot-gun. Please use it responsibly."
A footgun implies something hidden or implicit.
Asynchronous programming requires the developer to handle memory
correctly explicitly. This is equally true for all programming
languages.
It does not relate specifically to this RFC.

especially when we have the knowledge and experience from other languages to draw on and do better

How do you plan to draw on the experience of other languages if you’re
ignoring the experience of asynchronous PHP that has existed for many
years? If you truly wanted to rely on such experience, there wouldn’t
be this discussion.

10 months ago by John Bafford — view source — reply

unread

Hi Rob, Edmond,

I have concerns about the clarity of when suspension occurs in this RFC.

The RFC states as a core goal:

"Code that was originally written and intended to run outside of a Coroutine must work EXACTLY THE SAME inside a Coroutine without modifications."

And:

"A PHP developer should not have to think about how Coroutine switch and should not need to manage their switching—except in special cases where they consciously choose to intervene in this logic."

[...]

With explicit async/await ("coloured functions"), developers know exactly where suspension can occur. This RFC’s implicit model seems convenient, but without clear rules about suspension points, I’m unclear how developers can write correct concurrent code or reason about performance.

Could the RFC clarify the rules for when automatic suspension occurs versus when manual suspend() calls are required? Is this RFC following Go’s model where suspension timing is an implementation detail developers shouldn’t rely on? If so, that should be stated explicitly. Keep in mind that Go didn’t start that way and took nearly a decade to get there. Earlier versions of Go explicitly stated where suspensions were.

— Rob

To provide an explicit example for this, code that fits this pattern is going to be problematic:

function writeData() {
$count = count($this->data);
for($x = 0; $x < $count; $x++) {
[$path, $content] = $this->data[$x];
file_put_contents($path, $content);
}
$this->data = [];
}

While there are better ways to write this function, in normal PHP code, there's no problem here. But if file_put_contents() can block and cause a different coroutine to run, $this->data can be changed out from under writeData(), which leads to unexpected behavior. (e.g. $this->data changes length, and now writeData() no longer covers all of it; or it runs past the end of the array and errors; or doesn't see there's a change and loses it when it clears the data).

Now, yes, the programmer would have to do something to cause there to be two coroutines running in the first place. But if this code was correct when "originally written and intended to run outside of a Coroutine", and with no changes is incorrect when run inside a coroutine, one can only say that it is working "exactly the same" with coroutines by ignoring that it is now wrong.

Suspension points, whether explicit or hidden, allow for the entire rest of the world to change out from under the caller. The only way for non-async-aware code to operate safely is for suspension to be explicit (which, of course, means the code now must be async-aware). There is no way in general for code written without coroutines or async suspensions in mind to work correctly if it can be suspended.

-John

10 months ago by Edmond Dantes — view source — reply

unread

To provide an explicit example for this, code that fits this pattern is going to be problematic

Why is this considered a problem if this behavior is part of the
language’s contract?
Exactly the same way as in Go for example, this is also part of the
contract between the language and the programmer.

$this->data can be changed out from under writeData(), which leads to unexpected behavior.

So the developer must intentionally create two different coroutines.
Intentionally pass them the same object.
Intentionally write this code.
And the behavior is called “unexpected”? :)

that it is working "exactly the same" with coroutines by ignoring that it is now wrong
I understand that a word written by one person can be interpreted
however another person feels like.
Language is not a reliable carrier of information, so people must take
context into account to extract useful information with minimal
distortion.

The changes described in the RFC refer to the algorithm for handling
I/O functions in blocking mode. And of course these words assume that
we haven’t lost our minds and understand that you cannot write
completely different message sequences to the same socket at the same
time. In practice, changes are of course sometimes necessary, but
throughout my entire experience working with coroutines, I should note
that I have never once run into the example you mentioned. Even when
adapting older projects. And do you know why? Because the first thing
we refactored in the old code was the places with shared variables.

There is no way in general for code written without coroutines or async suspensions in mind to work correctly if it can be suspended.

Agreed.

A developer must understand that potentially any function can
interrupt execution. This is a consequence of transparent asynchrony.
It is both its strength and its weakness. I will repeat it again: not
some specific function, but almost ANY function. Because under
transparent asynchrony you can use suspend() inside any function. This
does not negate the fact that documentation should list all functions
that switch context, but a certain coding style encourages this way of
thinking.

Modern programming languages strive for clarity and cleanliness. In
other words, colored functions provide code clarity and prevent errors
caused by misunderstanding what a function does. Critics of colored
functions criticize them precisely for what is actually their
strength, not their weakness. Color is an advantage.

But in PHP, colored functions are inconvenient. Overloading I/O
functions does not lead to serious errors that make developers suffer;
on the contrary, it saves time and gives the language more
flexibility.

I can explain why. The amount of code that works with sockets in PHP
is generally several times smaller than the code that works with
databases. In other words, the modules where such errors could occur
are simply not that many. They do exist. library clients... but
compared to all other code, there are far fewer of them. And as it
turns out, refactoring them for coroutines requires very few changes.
Especially if the code was already well-written with best practices in
mind, then it will most likely work excellently with coroutines with
minimal adjustments.

How did we refactor old code for coroutines?

We took the modules that had global state. There were not many of them.
We used a Context, which is essential, and moved the global state
into the context.

I don’t remember exactly how many thousands of lines of code there
were, but definitely more than 20,000.
But why anyone would intentionally pass the same object to different
coroutines and then complain that the code broke. I have no idea who
would need that. :)

A developer should strive to minimize asynchronous code in a project.
The less of it there is, the better. Asynchronous code is evil. An
anti-pattern. A high-complexity zone. But if a developer chooses to
use asynchronous code, they shouldn’t act like they’re three years old
and seeing a computer for the first time. Definitely not. This
technology requires steady, capable hands :)

Best Regards, Ed.

10 months ago by Rob Landers — view source — reply

unread

To provide an explicit example for this, code that fits this pattern is going to be problematic

Why is this considered a problem if this behavior is part of the
language’s contract?
Exactly the same way as in Go for example, this is also part of the
contract between the language and the programmer.

One of the stated goals of the RFC:

Code that was originally written and intended to run outside of a Coroutine must work EXACTLY THE SAME inside a Coroutine without modifications.

The examples you give here seem to contradict that. You are now saying that developers must refactor shared state, must avoid passing objects to multiple coroutines, and must adopt a certain programming style to avoid breaking existing code. That’s the opposite of "works exactly the same without modification".

$this->data can be changed out from under writeData(), which leads to unexpected behavior.

So the developer must intentionally create two different coroutines.
Intentionally pass them the same object.
Intentionally write this code.
And the behavior is called “unexpected”? :)

The original claim of the RFC is that code not written with coroutines in mind should still behave the same inside them. If any function can suspend at arbitrary points, the ordinary synchronous assumptions, including read/modify/write patterns on properties, no longer hold. Whether that pattern is good style or not doesn’t change the fact that the behaviour is different once asynchrony is introduced.

A developer must understand that potentially any function can
interrupt execution. This is a consequence of transparent asynchrony.

This is also a direct tension with another major goal:

A PHP developer should not have to think about how Coroutine switch and should not need to manage their switching—except in special cases where they consciously choose to intervene in this logic.

If any function can suspend, then developers MUST reason about all the usual concurrency hazards: torn writes, interleaving, race conditions, and the entire class of bugs that coloured function models prevent. That absolutely counts as "thinking about coroutine switching".

It is both its strength and its weakness. I will repeat it again: not
some specific function, but almost ANY function. Because under
transparent asynchrony you can use suspend() inside any function. This
does not negate the fact that documentation should list all functions
that switch context, but a certain coding style encourages this way of
thinking.

These statements also seem to go against another goal of the RFC:

A PHP developer should not have to think about how Coroutine switch and should not need to manage their switching—except in special cases where they consciously choose to intervene in this logic.

How did we refactor old code for coroutines?

We took the modules that had global state. There were not many of them.

We used a Context, which is essential, and moved the global state
into the context.
[snip]
But why anyone would intentionally pass the same object to different
coroutines and then complain that the code broke. I have no idea who
would need that. :)

Existing PHP codes does this today without issue. Many libraries, parsers, database clients, stream decorators, in-memory caches, DTOs, middleware chains ... are built around shared mutable objects. That style is extremely common in PHP, and today it’s perfectly fine to share these things.

Saying "just refactor all your shared-state-code" seems to contradict the goals given in the RFC.

A developer should strive to minimize asynchronous code in a project.
The less of it there is, the better. Asynchronous code is evil. An
anti-pattern. A high-complexity zone. But if a developer chooses to
use asynchronous code, they shouldn’t act like they’re three years old
and seeing a computer for the first time. Definitely not. This
technology requires steady, capable hands :)

Best Regards, Ed.

Right now, today, PHP has almost zero async code in the ecosystem. If the position of the RFC is that transparent asynchrony is inherently dangerous, requires careful discipline, breaks common patterns, and requires refactoring shared state, then it isn’t clear how the central value proposition "existing code works unchanged" is meant to hold.

This is why the semantics need to be written down explicitly, not left to implication or the experience of those who already work with coroutines.

— Rob

10 months ago by Edmond Dantes — view source — reply

unread

The examples you give here seem to contradict that. You are now saying that developers must refactor shared state, must avoid passing objects to multiple coroutines,
and must adopt a certain programming style to avoid breaking existing code. That’s the opposite of "works exactly the same without modification".

I provided an explanation in my earlier messages.

Right now, today, PHP has almost zero async code in the ecosystem.

That is not accurate. PHP already has a significant amount of
async-style code in the ecosystem: Amphp, ReactPHP, Swoole, Swow, and
multiple async HTTP clients, database drivers, and event-loop
libraries. The ecosystem is not “almost zero”; it’s simply fragmented
across several implementations.

If the position of the RFC is that transparent asynchrony

My comment was about programming languages in general.

10 months ago by Rob Landers — view source — reply

unread

Hi Rob, Edmond,

I have concerns about the clarity of when suspension occurs in this RFC.

The RFC states as a core goal:

"Code that was originally written and intended to run outside of a Coroutine must work EXACTLY THE SAME inside a Coroutine without modifications."

And:

"A PHP developer should not have to think about how Coroutine switch and should not need to manage their switching—except in special cases where they consciously choose to intervene in this logic."

[...]

With explicit async/await ("coloured functions"), developers know exactly where suspension can occur. This RFC’s implicit model seems convenient, but without clear rules about suspension points, I’m unclear how developers can write correct concurrent code or reason about performance.

Could the RFC clarify the rules for when automatic suspension occurs versus when manual suspend() calls are required? Is this RFC following Go’s model where suspension timing is an implementation detail developers shouldn’t rely on? If so, that should be stated explicitly. Keep in mind that Go didn’t start that way and took nearly a decade to get there. Earlier versions of Go explicitly stated where suspensions were.

— Rob

To provide an explicit example for this, code that fits this pattern is going to be problematic:

function writeData() {
$count = count($this->data);
for($x = 0; $x < $count; $x++) {
[$path, $content] = $this->data[$x];
file_put_contents($path, $content);
}
$this->data = [];
}

While there are better ways to write this function, in normal PHP code, there's no problem here. But if file_put_contents() can block and cause a different coroutine to run, $this->data can be changed out from under writeData(), which leads to unexpected behavior. (e.g. $this->data changes length, and now writeData() no longer covers all of it; or it runs past the end of the array and errors; or doesn't see there's a change and loses it when it clears the data).

Now, yes, the programmer would have to do something to cause there to be two coroutines running in the first place. But if this code was correct when "originally written and intended to run outside of a Coroutine", and with no changes is incorrect when run inside a coroutine, one can only say that it is working "exactly the same" with coroutines by ignoring that it is now wrong.

Suspension points, whether explicit or hidden, allow for the entire rest of the world to change out from under the caller. The only way for non-async-aware code to operate safely is for suspension to be explicit (which, of course, means the code now must be async-aware). There is no way in general for code written without coroutines or async suspensions in mind to work correctly if it can be suspended.

-John

I should have put all these emails combined into a single email ... but here we are.

John’s example captures the core issue, and I want to take a moment and expand on it from a different angle. My concern with implicit suspensions isn’t theoretical. It’s exactly why nearly every modern language abandoned this model.

Transparent, implicit suspension means that any line of code can become an interleaving point. That makes a large class of patterns, which are perfectly safe in synchronous PHP today, unsafe the moment they run inside a coroutine. A few concrete examples:

With property hooks and implicit suspension, event this becomes unsafe:

$this->counter++;

A suspension can happen between the read and the write. Another coroutine can mutate the counter in between. The programmer did nothing wrong; it's just a hazard introduced by invisible suspension.

And consider this can break invariants:

$this->balance -= $amount;
$this->ledger->writeEntry($this->id, -$amount);

If the first line suspends, the balance can be changed somewhere else before the ledger entry is written (which breaks an invariant that the balance is a reflection of the ledger). With transparent async, it's suddenly a race condition.

Then you can have time pass invisibly:

if(!$cache->has($key)) {
$cache->set($key, $value);
}

If has() suspends, anything can happen to that cache key before the set. The invariant becomes incorrect.

Implicit suspension allows any function to be re-entered before it returns. That can lead to partially updated objects, state machines appearing to skip states, "method called twice before return" bugs, double writes, and re-entrant callbacks being invoked with inconsistent state.

The bugs are extremely challenging to debug because the programmer never actually wrote any async code.

I’ve had the "pleasure" of working on Fiber frameworks that use raw fibers (no async/await you get from React/Amp, though I’ve worked with those pretty extensively as well). These are the bugs you run into all the time, where you sometimes have to literally put a suspension in a seemingly random place to fix a bug.

Implicit async blurs one of the most important boundaries in software design: "this code cannot be interrupted" vs "this code can be interrupted".

JavaScript moved from implicit async -> promises -> async/await
Python moved from callbacks/greenlets -> async/await
Ruby moved from fibers -> explicit schedulers
Go eventually added true preemption

Even the creators of Fibers eventually wrote async/await on top of them, because implicit async is broken and coloured functions close off entire classes of bugs and make reasoning possible again.

I understand the desire for "transparent async" but once a language allows suspension at arbitrary points, the language can no longer promise invariants, atomic sequences, non-reentrancy, predictable control flow, or even correctness, in-general.

— Rob

10 months ago by Edmond Dantes — view source — reply

unread

With property hooks and implicit suspension, event this becomes unsafe:
A suspension can happen between the read and the write. Another coroutine can mutate the counter in between. The programmer did nothing wrong; it's just a hazard introduced by invisible suspension.

The risk of a variable being modified by different coroutines does not
depend on the transparency model. This effect is possible in both
implementations.
Even if a setter triggers a suspension, it does not affect the logical
execution flow. Therefore, no danger arises.

The difference between the transparent model and the explicit one lies
in other aspects. It seems this discussion took place in March of this
year.

PHP True Async RFC Stage 5

Executive Summary: What This RFC Proposes

1. Coroutines via spawn()

2. Non-blocking I/O Functions

What This RFC Does NOT Propose

General Questions

Q: What is the main goal of this RFC?

Q: How is this different from Fibers?

Q: Isn't this just Fibers 2.0?

Q: Can I use this with FPM?

Q: What about exit and die?

Q: Do I need to rewrite my existing code?

Q: How does this RFC affect I/O functions?

1. Coroutines via `spawn()`

Q: What about `exit` and `die`?