PHP True Async RFC Stage 3

Hello, Rowan!

I don't have time to dig into the details of this draft at the moment,
but will say that a vote in two weeks is not at all realistic.
I just used the Flow from the Wiki. :)

My only other comment at a very quick glance is that I see the Context
section is still included, and still has most of its complexity.
Almost every section of the RFC has a "Motivation" subsection, which
explains why a particular tool is needed and what problem it solves.
I removed everything from the RFC that is not directly related to
asynchrony or not essential at this stage.

Yesterday I also received feedback that the RFC contains too many changes,
so I can briefly summarize what’s new in this version compared to the
previous one:

The Scope class can no longer be used in an await expression. A separate
method is now provided for waiting on a Scope, intended only for special
use cases.
2.

Scope is now responsible solely for coroutine lifetime and error
handling.
3.

A TaskGroup class has been added for working with groups of tasks. It is
used explicitly with the expression spawn with $taskGroup. Together with
Scope, TaskGroup supports structured concurrency, offering different
strategies for tracking hanging coroutines.

These are the key changes.

Secondary changes include:

Syntax refinement
2.

Edge case handling
3.

Ability to extend spawn with logic
4.

Added combinator functions: all, any, anyOf, ignoreErrors
5.

A special function Async\protect() has been added for critical sections
that cannot be cancelled.
6.

Improved examples

Contextual expressions like with $context and async blocks were removed
from the RFC as they are out of its scope.

The next important step is to determine which coroutine model to choose.
There are three models in total:

Go – no Scope
2.

Java Loom – Scope is always passed explicitly
3.

The current RFC – Scope is passed both explicitly and implicitly

The choice of model affects both the syntax and the API. Therefore, if the
current model is not accepted, there is no point in further developing this
RFC — a new one should be created instead.

This is the key question I want to clarify at this stage.

--

Ed

The link in Patches and Tests is currently giving a 404 Not Found.
Can you update the link to your proof of concept implementation?

Thank you for pointing that out. I’ve corrected the link.

https://github.com/EdmondDantes/php-src/tree/async/ext/async

Good day, Deleu!

What happens if the coroutine didn't finish execution? does
disposeSafely() means that it will wait until completion to safely clear
it up or does it mean it will attempt to dispose and throw an exception if
it fails to do so?

Cooperative cancellation implies that a coroutine must always complete
voluntarily. Compare this behavior to a process: the OS can terminate a
process at will. In languages that support the concept of virtual
processes, this is also possible. However, in the case of coroutines,
attempting to interrupt execution at any point can lead to application
failures that are difficult to debug.
In one of the earlier drafts of this RFC, I had the idea of using an
execution time limit during the cancellation phase, but I found cases where
such logic is unacceptable. Overall, attempts to complicate the
cancellation mechanism do not reduce the number of errors caused by
oversight.

If a programmer swallows the cancellation exception in an infinite loop,
this situation should essentially lead either to a resource leak or to the
termination of the entire application.

There’s an idea to reduce the likelihood of such errors using a special try
{} cancellation {} block, but at this stage I decided not to overcomplicate
the already complex logic. However, yes, I want to highlight that errors
involving CancellationException are among the most painful. This holds true
for all languages that have a similar mechanism.

My first impression here was a little odd. Wouldn't it make more sense to
hide the syntax behind the TaskGroup class?

Using special syntax has several advantages over functions. The most
important one is clarity. You can define a method with the same name in any
class — who knows what it actually does. But a distinct expression provides
a 100% guarantee that you’re looking at coroutine creation. It also makes
static code analysis easier, as well as reading the code, since the
programmer gets used to a single consistent form.

Could this example be moved to a later block *after *spawn use () has
been introduced?
Would it be possible to elaborate further an example that could not be
easily replaced by spawn use()? In other words, what is unique about
Context that makes it indispensable as opposed to spawn use()?

Context is a storage accessible from any function operating within a Scope.
For example, it can store the current user's session ID, which can be
retrieved using a special function or service.

This is described in more detail in the Context section, along with an
example.

Context helps in using code that previously relied on static variables, but
now there’s a need to support a concurrent web server. The modern
programming community considers the context pattern to be dangerous — and
that’s true. However, when used correctly, it helps reduce the coupling
between components.

I may be misunderstanding, but it feels like any() is another way of
$taskGroup->race()? and all() is another way of $taskGroup->await()?
Would it make sense to elaborate further their difference in this
example? Do you think the first RFC perhaps could propose one or the other
and keep one of them for future scope?

You’re right — the any function essentially does the same thing as the
race() method in terms of behavior. But the difference lies elsewhere:

TaskGroup is a collection of coroutines, and when you use race(), it
operates on that collection.
2.

any() can work with an iterator, and not just coroutines, but also
Futures (when they become available). In principle, it’s possible to make
TaskGroup an iterator, which would then allow writing any(TaskGroup).
This is a good idea from a consistency standpoint.

isn't this just standard PHP with extra steps? I'm assuming await here
blocks and spawn creates a coroutine. Wouldn't every use of await spawn
be effectively the same as just using PHP as-is today in blocking mode?

Exactly. In this case, the code has little practical value. I might be able
to improve the example to make it reflect something more realistic.

What is the output of this statement? Assuming it takes longer than 2
seconds, do we get null back? an exception?

This behavior is discussed in the corresponding section. If the timeout
expires, an exception will be thrown. As for the syntax, a decision was
recently made to move it to a separate RFC, so I believe this will still be
discussed.

If users need to limit the duration of a coroutine maybe they can either
cancel it or safely dispose?

Users have the ability to create so-called Responsibility Points. This is
code that explicitly creates a $scope object and controls what happens when
the $scope needs to be disposed of. This is an important aspect discussed
in the RFC.

I think you wanted to use implode instead of array_merge here.

Thanks!

It feels to me that the inner function processJob is supposed to await

This RFC has an important distinction that makes it unique compared to
other similar models. It allows passing the Scope implicitly between
coroutines, but does not allow the user to freely await all tasks within
it. Why does this matter? When Scope is used implicitly, a programmer might
accidentally create a coroutine and forget about it. If the programmer
awaits all tasks in the Scope without restrictions, it creates a risk of
infinite waiting — and such a situation won’t be detected automatically.
That’s why awaiting a Scope is not a typical operation for everyday
programming, but rather a special case meant for detecting erroneous
coroutines.
In most cases, it’s better to use TaskGroup.

I'm not trying to be pedantic, but it is a rather complex RFC. This seems
to be the first time it uses stop a coroutine. Is it different from
cancel() and disposeSafely()? How so?

There is no stop() method. But if you mean the methods disposeSafly() and
cancel() — then yes. And yes, it’s a bit complex, but in most cases you
don’t need to worry about it. It’s enough to ensure that $scope is released
at the right moment.

These methods implement two different strategies for terminating a Scope.
disposeSafely() does not cancel coroutines but marks them as zombies. At
the same time, it issues a warning indicating an error was detected.
The dispose() method cancels the coroutines and also issues an error.
The cancel() method does not issue any error.

This number of methods is needed to separate behaviors where PHP should
emit a warning and where it shouldn't.
Error messages related to coroutines are very important — they allow the
application to continue running, while still letting you know that there is
a problem.

If a programmer uses the cancel() method, it means they’re saying: I don’t
care what’s happening there — just cancel everything.
If they use the dispose() method, they’re saying: I know everything should
be finished at this point, but if it’s not — just cancel it and let me
know.

No other language has such a built-in tracking mechanism — only as
additional tools. But for PHP, this is more important, because PHP is a
language for fast feature delivery.

This seems like a void statement. Whether the namespace gets used in a
way that respects this statement or not, it doesn't mean that PHP won't
break BC in future versions when the language itself defines a symbol that
has been defined in userland.

Do you mean there's a risk that another developer might also use the same
namespace?

Seems like a good candidate for future scope and/or left for userland
since the SpawnStrategy interface already exposes the necessary
capabilities?

Of course, that can be done. I don’t think it’s a particularly critical
point.

This explains a bit more the "Cooperative cancellation" example early in
the RFC, but it does seem rather awkward to suspend the main in order to
start a coroutine. Shouldn't await be used for that instead? Would making
suspend a fatal error inside the main flow somehow worse for
implementation?

suspend is just one way to yield control to the Scheduler. It can be
convenient in long loops, for example, when the programmer wants to
explicitly indicate that the coroutine can pause at that point.

In the examples, it’s used only to indicate the moment of switching to
another coroutine. In real scenarios, it will be used quite rarely.

Thank you very much for your feedback and corrections!

--

Ed