PHP True Async - Externals

1 year ago by Edmond Dantes — view source — reply

unread

Good day, everyone.

I would like to request permission to create an RFC dedicated to the
asynchronous module for PHP.

Although the source code development is still in progress, a significant
part of it has already been written, and the initial tests are working
successfully. Therefore, I could start the documentation process if you
consider this topic relevant and interesting for the community.
A brief overview of what this RFC will cover:

True asynchronous support for PHP core functions without additional
"tricks."
Scheduler and Reactor components, providing a C-API for PHP
extensions to enable non-blocking I/O in plugins or core code.
User-Land API that mirrors the C-API for PHP developers.
Basic primitives for concurrent programming, such as Channel and
Iterator.
Built-in integration with the LibUV library.

The library is based on Fiber, extending and enhancing it to provide PHP
developers with a full set of tools for concurrent applications.

Have a great day!

Edmond

1 year ago by Jakub Zelenka — view source — reply

unread

Hi

Good day, everyone.

I would like to request permission to create an RFC dedicated to the
asynchronous module for PHP.

Someone will surely grant you karma soon. RFC makes sense.

Although the source code development is still in progress, a significant
part of it has already been written, and the initial tests are working
successfully. Therefore, I could start the documentation process if you
consider this topic relevant and interesting for the community.
A brief overview of what this RFC will cover:

True asynchronous support for PHP core functions without
additional "tricks."

Scheduler and Reactor components, providing a C-API for PHP
extensions to enable non-blocking I/O in plugins or core code.

User-Land API that mirrors the C-API for PHP developers.

Basic primitives for concurrent programming, such as Channel and
Iterator.

Built-in integration with the LibUV library.

The library is based on Fiber, extending and enhancing it to provide
PHP developers with a full set of tools for concurrent applications.

Looks interesting. I understand that's not ready for review yet but quickly
checked the code anyway. Is there some particular reason why this is not an
extension? Think it might actually make sense to split it to two extensions

one for the core main stuff (that could be enabled by default) and then
libuv optional part - similar to pdo for example. I haven't checked too
deeply the code though so I might be missing things. Feel free to create a
draft PR even if it's not ready. It might help to get some initial
pre-reviews... :)

Regards

Jakub

1 year ago by Jakub Zelenka — view source — reply

unread

A brief overview of what this RFC will cover:

True asynchronous support for PHP core functions without
additional "tricks."

Scheduler and Reactor components, providing a C-API for PHP
extensions to enable non-blocking I/O in plugins or core code.

User-Land API that mirrors the C-API for PHP developers.

Basic primitives for concurrent programming, such as Channel and
Iterator.

Built-in integration with the LibUV library.

The library is based on Fiber, extending and enhancing it to provide
PHP developers with a full set of tools for concurrent applications.

Looks interesting. I understand that's not ready for review yet but
quickly checked the code anyway. Is there some particular reason why this
is not an extension? Think it might actually make sense to split it to two
extensions - one for the core main stuff (that could be enabled by default)
and then libuv optional part - similar to pdo for example. I haven't
checked too deeply the code though so I might be missing things. Feel free
to create a draft PR even if it's not ready. It might help to get some
initial pre-reviews... :)

In terms of RFC, it would be good to also compare it with alternatives.
From a quick look it slightly reminded me swow (
https://github.com/swow/swow ) but I didn't really look that deeply into it
and I also don't know that much about swow so might be wrong.

1 year ago by Christoph M. Becker — view source — reply

unread

I would like to request permission to create an RFC dedicated to the
asynchronous module for PHP.
Have you already registered a new Wiki user? If so, what's your
username? Otherwise head over to https://wiki.php.net/?do=register.

Christoph

1 year ago by Bilge — view source — reply

unread

Good day, everyone.

I would like to request permission to create an RFC dedicated to the
asynchronous module for PHP.

Although the source code development is still in progress, a
significant part of it has already been written, and the initial tests
are working successfully. Therefore, I could start the documentation
process if you consider this topic relevant and interesting for the
community.
  A brief overview of what this RFC will cover:
True asynchronous support for PHP core functions without
additional "tricks."

Scheduler and Reactor components, providing a C-API for PHP
extensions to enable non-blocking I/O in plugins or core code.

User-Land API that mirrors the C-API for PHP developers.

Basic primitives for concurrent programming, such as Channel and
Iterator.

Built-in integration with the LibUV library.

The library is based on Fiber, extending and enhancing it to provide
PHP developers with a full set of tools for concurrent applications.

Have a great day!

Edmond

Is it really necessary to have all these Async\launchScheduler();
calls? Why can't the scheduler always be running, the same as it is in
JavaScript or any other async language? Even (userland) Revolt does not
require the event loop to be manually started.

Cheers,
Bilge

1 year ago by Edmond Dantes — view source — reply

unread

Hello Bilge!

Is it really necessary to have all these Async\launchScheduler();
calls? Why can't the scheduler always be running, the same as it is in
JavaScript or any other async language? Even (userland) Revolt does not
require the event loop to be manually started.

Short answer: This implementation is simpler — not so much in terms of
code size but in terms of reducing сonfusion. And we achieve a consistent
state when asynchronous code runs within a Fiber, rather than having a
situation where we are in a Fiber here but not there.

For example, consider how the Fiber switching mechanism works in PHP. If
FiberB is launched from FiberA, FiberB can only return to FiberA.
Consequently, if the Scheduler component is placed inside a Fiber, the code
must track the "direction" of switching.

Running the Scheduler in the "zero" Fiber results in the simplest possible
switching logic. It also defines a clear point where exceptions can be
thrown.

One might argue that no other language does it this way. But then, no other
language has Fiber as a low-level tool (except C). From this perspective,
explicit activation of the Scheduler aligns with Fiber as a low-level
construct.

Can this be avoided? Yes, but such alternative approaches often rely
on implicit
behavior. And implicit behavior tends to break things.

Of course, we could start the code inside a Fiber to initialize the
Scheduler and hide the implementation, but again, that's not always
necessary or useful for everyone.

Ed.

1 year ago by Edmond Dantes — view source — reply

unread

I forgot to mention this. There is an important limitation in how this
solution behaves in the context of PHP.

Calls to PHP functions that normally block execution in the zero-Fiber do
not change their behavior. This solution has both advantages and
disadvantages. The advantage is that it does not break backward
compatibility in any way. Calling sleep(10) at the beginning of <?php will
cause the process to go into a wait state.

True Async changes the behavior of functions only if they are running
inside a Fiber with an active Scheduler.

This is yet another consequence of this architectural decision, and some
people might not like it. However, at this point, I don’t see a better
solution that would be optimally balanced between legacy support, explicit
behavior, and implementation simplicity.

Have a nice day!

Ed.

1 year ago by Bilge — view source — reply

unread

I forgot to mention this. There is an important limitation in how
this solution behaves in the context of PHP.

Calls to PHP functions that normally block execution in the zero-Fiber
do not change their behavior. This solution has both advantages and
disadvantages. The advantage is that it does not break backward
compatibility in any way. Calling |sleep(10)| at the beginning of
|<?php| will cause the process to go into a wait state.

True Async changes the behavior of functions only if they are
running inside a Fiber with an active Scheduler.

This is yet another consequence of this architectural decision, and
some people might not like it. However, at this point, I don’t see a
better solution that would be optimally balanced between legacy
support, explicit behavior, and implementation simplicity.

Have a nice day!

Ed.

There should be no perceptible difference between a blocking sleep(10)
and an async sleep(10), so what backwards compatibility are you
referring to?

Bilge

1 year ago by Edmond Dantes — view source — reply

unread

There should be no perceptible difference between a blocking sleep(10)
and an async sleep(10), so what backwards compatibility are you referring
to?

For example, the behavior of the code below will not change. The code will
execute sequentially, and context switching will only occur when resume/
suspend is called.

However, when the Scheduler is activated, this behavior changes. Now,
calling sleep() inside a Fiber will lead to a context switch.

If the activation of the Scheduler is implicit, previously written code may
not work as the developer expects.

<?php

$fiber = new Fiber(function (): void {
echo "Start fiber\n";
sleep(1);
Fiber::suspend("Paused");
echo "Resume fiber\n";
});

$result = $fiber->start();

echo "Fiber suspended with: $result\n";

sleep(10);

$fiber->resume();

echo "Fiber finished\n";

1 year ago by Rob Landers — view source — reply

unread

There should be no perceptible difference between a blocking sleep(10) and an async sleep(10), so what backwards compatibility are you referring to?

For example, the behavior of the code below will not change. The code will execute sequentially, and context switching will only occur when resume/suspend is called.

However, when the Scheduler is activated, this behavior changes. Now, calling sleep() inside a Fiber will lead to a context switch.

If the activation of the Scheduler is implicit, previously written code may not work as the developer expects.

<?php

$fiber = new Fiber(function (): void {
echo "Start fiber\n";
sleep(1);
Fiber::suspend("Paused");
echo "Resume fiber\n";
});

$result = $fiber->start();

echo "Fiber suspended with: $result\n";

sleep(10);

$fiber->resume();

echo "Fiber finished\n";

I think what bilge was trying to point out is that there should be absolutely no change on existing software with or without the scheduler running (for software not using fibers).

— Rob

1 year ago by Edmond Dantes — view source — reply

unread

I think what bilge was trying to point out is that there should be
absolutely no change on existing software with or without the scheduler
running (for software not using fibers).

I thought the same. But where would you hide Fibers? They are part of the
language.

The existence of Fibers is an unpleasant issue in the context of language
design. Essentially, they break the integrity of the implementation by
creating a "backdoor" for user-land.
So... various ways can be devised to disguise this fact, but they will all
have drawbacks.

Of course, you can create the Scheduler before starting the script and run
the script inside a Fiber. But I'm sure there will be someone who will say
they don't like this approach.

Ed.

1 year ago by Christoph M. Becker — view source — reply

unread

Have you already registered a new Wiki user? If so, what's your
username?

Yes, I registered today. My login is: edmond.

RFC karma granted. Best of luck!

Christoph

1 year ago by Edmond Dantes — view source — reply

unread

Hello, Jakub.
(It seems that simply clicking "Reply" does not work for the conference)

*> *I understand that's not ready for review yet but quickly
You could say that the code is in a state of "architectural stability,"
so it can be reviewed and criticized. (Unfortunately, only the Windows
build is currently working.)
In the near future, I plan to start documenting the C-API, where I will
describe the implementation details in depth, but feel free to ask any
questions about this component.

Link: https://github.com/EdmondDantes/php-src/tree/async/async

Is there some particular reason why this is not an extension?

There are two reasons: one objective and one relatively subjective.

The objective reason is that there are critical changes that simply
cannot be done in an extension. This includes modifying the logic of
functions like php_select/php_poll2.

The relatively subjective reason is that it was extremely difficult for me
to design the architecture within the constraints of an extension because,
at the start of the project, I had no clear idea of how to implement it at
all... :)

Theoretically, the project could be split into two separate components:
core modifications and an external EXT component. Now that I have a working
codebase, it would be much easier to do.

And yes, you've raised a very important point. In CGI applications, this
component is likely to be of little use (although in some cases, it might
still be beneficial even for CGI). So keeping the entire codebase in the
PHP core is not a good approach. I’m sure many people will point that out
:) And I agree. But for now, the component is built using configuration
options in buildconf, which is sufficient as a starting point.

If your question is whether core modifications could be avoided, the answer
is no. The whole purpose of this project is to make PHP more suitable for
LongRunning Apps.

and then libuv optional part - similar to pdo for example.

That's exactly how it has already been implemented! LibUV is not tightly
coupled to the code. Moreover, it can be replaced on the fly with any other
implementation—for example, Swoole can serve as the backend for the
reactor, or RUST TOKIO, or perhaps someone might want to create something
else.

In essence, the Reactor is just an API and can be defined via an extension.
LibUV is included as an embedded piece of code that can be compiled
directly with the project if a specific flag is set, but of course, it can
also be provided as an extension. (The reason why LibUV is provided as the
"default solution" is that it is written in C and is cross-platform. This
is important for PHP. )

I plan to do the same with the Scheduler component in the future.

Thanks for questions!

пт, 14 февр. 2025 г. в 15:05, Jakub Zelenka bukka@php.net:

1 year ago by Jakub Zelenka — view source — reply

unread

Hi,

Hello, Jakub.
(It seems that simply clicking "Reply" does not work for the conference)

*> *I understand that's not ready for review yet but quickly
You could say that the code is in a state of "architectural stability,"
so it can be reviewed and criticized. (Unfortunately, only the Windows
build is currently working.)
In the near future, I plan to start documenting the C-API, where I will
describe the implementation details in depth, but feel free to ask any
questions about this component.

Link: https://github.com/EdmondDantes/php-src/tree/async/async

Is there some particular reason why this is not an extension?

There are two reasons: one objective and one relatively subjective.

The objective reason is that there are critical changes that simply
cannot be done in an extension. This includes modifying the logic of
functions like php_select/php_poll2.

Yeah we are actually planning better API for this supporting epoll and
kqueu which is long over due. There are other use cases for this so
hopefully it will get prioritised soon.

The relatively subjective reason is that it was extremely difficult for me
to design the architecture within the constraints of an extension because,
at the start of the project, I had no clear idea of how to implement it at
all... :)

Theoretically, the project could be split into two separate components:
core modifications and an external EXT component. Now that I have a working
codebase, it would be much easier to do.

You could still achieve this in extensions. But if you really wanted to
have some main part, then it should rather go to main/ like streams and
other similar stuff.

And yes, you've raised a very important point. In CGI applications, this
component is likely to be of little use (although in some cases, it might
still be beneficial even for CGI). So keeping the entire codebase in the
PHP core is not a good approach. I’m sure many people will point that out
:) And I agree. But for now, the component is built using configuration
options in buildconf, which is sufficient as a starting point.

If your question is whether core modifications could be avoided, the
answer is no. The whole purpose of this project is to make PHP more
suitable for LongRunning Apps.

and then libuv optional part - similar to pdo for example.

That's exactly how it has already been implemented! LibUV is not
tightly coupled to the code. Moreover, it can be replaced on the fly with
any other implementation—for example, Swoole can serve as the backend for
the reactor, or RUST TOKIO, or perhaps someone might want to create
something else.

In essence, the Reactor is just an API and can be defined via an extension.
LibUV is included as an embedded piece of code that can be compiled
directly with the project if a specific flag is set, but of course, it can
also be provided as an extension. (The reason why LibUV is provided as the
"default solution" is that it is written in C and is cross-platform. This
is important for PHP. )

Yes and that's why it would better to the ext directory.

I plan to do the same with the Scheduler component in the future.

Thanks for questions!

I have got some additional feedback from our discussion that we just had
with the PHP foundation devs. It was that it looks like an async/await
implementation with fiber switching which is something that was explicitly
decided not to be done in fibers RFC - https://wiki.php.net/rfc/fibers (read
FAQ). The reasoning was that this should be done in user space. So you
should properly describe in the RFC what this offers compare to user space
variants (revolt/react/amp). There were also some concerns that it would
makes things harder to debug in internal implementation so this should be
also addressed in the RFC. It should be noted that Fibers RFC did not
preclude introduction of such extensions to the core but you will likely
need describe it in the RFC and show that there are good reasons to include.

Hope it helps!

Kind regards,

Jakub

1 year ago by Edmond Dantes — view source — reply

unread

Yeah we are actually planning better API for this supporting epoll and
kqueu which is long over due. There are other use cases for this so
hopefully it will get prioritised soon.
Oh, it would be absolutely great if PHP used a better abstraction for I/O
waiting!

You could still achieve this in extensions. But if you really wanted to
have some main part, then it should rather go to main/ like streams and
other similar stuff.
Of course, this will need some thought.

I have got some additional feedback from our discussion that we just had

with the PHP foundation devs. It was that it looks like an async/await
implementation with fiber switching which is something that was explicitly
decided not to be done in fibers RFC - https://wiki.php.net/rfc/fibers (read
FAQ). The reasoning was that this should be done in user space. So you
should properly describe in the RFC what this offers compare to user space
variants (revolt/react/amp). There were also some concerns that it would
makes things harder to debug in internal implementation so this should be
also addressed in the RFC. It should be noted that Fibers RFC did not
preclude introduction of such extensions to the core but you will likely
need describe it in the RFC and show that there are good reasons to include.

Unfortunately, I don’t know much about what happened during the adoption of
Fibers, but I remember that the situation was not ideal. I don’t know why
that decision was made.

For me, as a developer, the ultimate criterion is the code itself. If a
solution makes the code better, simpler, and reduces the number of lines,
then it’s a good solution. If a solution forces developers to repeatedly
write the same boilerplate code, including entire MySQL/PostgreSQL drivers,
then I can’t consider it a rational decision from a development
perspective—though, of course, there could be other reasons, such as
financial constraints or lack of resources. Or look at Swoole—it has to
override dozens of PHP functions. Even worse, it recently started literally
copying entire extensions just for a few lines of code... This does not
look like a "good solution."

AMPHP and Revolt are talented projects, but they are not part of the
language itself. This means that PHP extensions cannot call user-mode code
and rely on it as a standard, unlike how Rust extensions do. Of course,
from C, you can call any user-mode function, but that doesn't seem like the
right approach. Especially since PHP is a high-level language, which
imposes a responsibility for consistency in its paradigm.

Regarding debugging—over the past three years, there have been no debugging
issues with AMPHP. Consequently, there shouldn’t be any, since this project
does not modify the Fiber mechanism itself. It only extends it to make the
tool more complete and fully functional.

And this is precisely the main reason why I’m writing these lines. As a PHP
developer, I see that the language is not truly complete when it comes to
concurrent programming, and I am genuinely surprised that this has not been
addressed yet. How did this happen?

Have a nice day!

Ed.

1 year ago by Rob Landers — view source — reply

unread

Yeah we are actually planning better API for this supporting epoll and kqueu which is long over due. There are other use cases for this so hopefully it will get prioritised soon.
Oh, it would be absolutely great if PHP used a better abstraction for I/O waiting!

You could still achieve this in extensions. But if you really wanted to have some main part, then it should rather go to main/ like streams and other similar stuff.
Of course, this will need some thought.

I have got some additional feedback from our discussion that we just had with the PHP foundation devs. It was that it looks like an async/await implementation with fiber switching which is something that was explicitly decided not to be done in fibers RFC - https://wiki.php.net/rfc/fibers (read FAQ). The reasoning was that this should be done in user space. So you should properly describe in the RFC what this offers compare to user space variants (revolt/react/amp). There were also some concerns that it would makes things harder to debug in internal implementation so this should be also addressed in the RFC. It should be noted that Fibers RFC did not preclude introduction of such extensions to the core but you will likely need describe it in the RFC and show that there are good reasons to include.

Unfortunately, I don’t know much about what happened during the adoption of Fibers, but I remember that the situation was not ideal. I don’t know why that decision was made.

For me, as a developer, the ultimate criterion is the code itself. If a solution makes the code better, simpler, and reduces the number of lines, then it’s a good solution. If a solution forces developers to repeatedly write the same boilerplate code, including entire MySQL/PostgreSQL drivers, then I can’t consider it a rational decision from a development perspective—though, of course, there could be other reasons, such as financial constraints or lack of resources. Or look at Swoole—it has to override dozens of PHP functions. Even worse, it recently started literally copying entire extensions just for a few lines of code... This does not look like a "good solution."

AMPHP and Revolt are talented projects, but they are not part of the language itself. This means that PHP extensions cannot call user-mode code and rely on it as a standard, unlike how Rust extensions do. Of course, from C, you can call any user-mode function, but that doesn't seem like the right approach. Especially since PHP is a high-level language, which imposes a responsibility for consistency in its paradigm.

Regarding debugging—over the past three years, there have been no debugging issues with AMPHP. Consequently, there shouldn’t be any, since this project does not modify the Fiber mechanism itself. It only extends it to make the tool more complete and fully functional.

And this is precisely the main reason why I’m writing these lines. As a PHP developer, I see that the language is not truly complete when it comes to concurrent programming, and I am genuinely surprised that this has not been addressed yet. How did this happen?

Have a nice day!

Ed.

I'm reminded of Rowan's comment here: Why did fibers get added to php core over something more fleshed out like swoole? - Externals https://externals.io/message/121291#121315 (https://externals.io/message/121291#121315)

Sometimes the answer to "why doesn't X do Y?" is just "because nobody's stepped forward to implement it yet"; sometimes it's "nobody's worked out how to do it without breaking Z"; in which case, feel free to volunteer that time, or solve that issue. But, yes, sometimes it's "because we had a long and tiring debate, and ended up with a compromise that nobody really likes"; or "because the lack of official leadership and a relatively high turnover of contributors makes us pretty bad at longer-term planning".

It pretty much sums it up (also, you can always search the mailing list archives for previous discussions on these topics).

— Rob

1 year ago by Rob Landers — view source — reply

unread

Hello, Jakub.
(It seems that simply clicking "Reply" does not work for the conference)
**
*> *I understand that's not ready for review yet but quickly
You could say that the code is in a state of "architectural stability," so it can be reviewed and criticized. (Unfortunately, only the Windows build is currently working.)
In the near future, I plan to start documenting the C-API, where I will describe the implementation details in depth, but feel free to ask any questions about this component.

Link: https://github.com/EdmondDantes/php-src/tree/async/async

Is there some particular reason why this is not an extension?

There are two reasons: one objective and one relatively subjective.

The objective reason is that there are critical changes that simply cannot be done in an extension. This includes modifying the logic of functions like php_select/php_poll2.

The relatively subjective reason is that it was extremely difficult for me to design the architecture within the constraints of an extension because, at the start of the project, I had no clear idea of how to implement it at all... :)

Theoretically, the project could be split into two separate components: core modifications and an external EXT component. Now that I have a working codebase, it would be much easier to do.

And yes, you've raised a very important point. In CGI applications, this component is likely to be of little use (although in some cases, it might still be beneficial even for CGI). So keeping the entire codebase in the PHP core is not a good approach. I’m sure many people will point that out :) And I agree. But for now, the component is built using configuration options in buildconf, which is sufficient as a starting point.

If your question is whether core modifications could be avoided, the answer is no. The whole purpose of this project is to make PHP more suitable for LongRunning Apps.

and then libuv optional part - similar to pdo for example.

That's exactly how it has already been implemented! LibUV is not tightly coupled to the code. Moreover, it can be replaced on the fly with any other implementation—for example, Swoole can serve as the backend for the reactor, or RUST TOKIO, or perhaps someone might want to create something else.

In essence, the Reactor is just an API and can be defined via an extension.
LibUV is included as an embedded piece of code that can be compiled directly with the project if a specific flag is set, but of course, it can also be provided as an extension. (The reason why LibUV is provided as the "default solution" is that it is written in C and is cross-platform. This is important for PHP. )

I plan to do the same with the Scheduler component in the future.

Thanks for questions!

пт, 14 февр. 2025 г. в 15:05, Jakub Zelenka bukka@php.net:

Hello,

I'm actually curious why you chose libuv? This may no longer be true today, but I remember distinctly not choosing libuv on a previous project due to some compatibility concerns. It was a php project + custom extensions, and the issue was directly related to other official/pecl extensions. We didn't have this issue with libev, fwiw. It was a long time ago though, so maybe the issues no longer exist, and I don't remember the details.

— Rob

1 year ago by Rob Landers — view source — reply

unread

Hello, Jakub.
(It seems that simply clicking "Reply" does not work for the conference)
**
*> *I understand that's not ready for review yet but quickly
You could say that the code is in a state of "architectural stability," so it can be reviewed and criticized. (Unfortunately, only the Windows build is currently working.)
In the near future, I plan to start documenting the C-API, where I will describe the implementation details in depth, but feel free to ask any questions about this component.

Link: https://github.com/EdmondDantes/php-src/tree/async/async

Just looking through the PHP stubs, and here are some notes:

Channels:

It is probably best to have channels be non-buffered by default (like in Go); buffered channels can hide architectural issues in async code.
Why do we need "finishing" producing and consuming? Won't "closing" the channel be enough, or does closing a channel prevent consumers from reading pending items? Personally, it should be an exception (or at least a warning) if a channel gets GC'd with pending data.
Looking at the public api, it seems that it is very hard to use correctly. It would probably be a good idea to pair it down and come up with an api/ideomatic usage that makes it hard to use incorrectly.

Callback:

Can we name this Async\Closure or something similar? Just to keep it inline with the current \Closure
Speaking of, you probably meant to use \Closure as the $callback property instead of mixed?

Futures:

The constructor for a Future is public, but it looks like it would be weird to use.

Module:

It would be nice to see this simply use Futures as much as possible. For example, returning a Future instead of FiberHandles.
Is onSignal OS signals or application signals? (I didn't look at the C code) If the latter, it is probably better to use something other than integer. If the former, this is usually handled by extensions (such as pcntl) and comes with a ton of caveats. It might be better to let those extensions handle that.

Notifier:

Exposing $reserved -- which is accessible via reflection -- is probably a bad idea.
Should this be implementing Stringable interface? Also, this will automatically cast to a string if accidentally passed to a function that takes a string -- unless you are using strict mode. https://3v4l.org/3AqcW This is probably not desired.
Instead of Terminate(), should it be Close(), to be consistent with the rest of the library?
It would be nice to get a list of callbacks on the Notifier as well.

Walker:

It would be nice to be able to get a future from the Walker instead of polling "isFinished".
IMHO, it would be better to implement this around Futures (see amphp's extensive library around this). Here are some examples from my own code:
timeouts: race two futures, one which times out and the other which performs a long-running task. If the timeout wins, then I cancel the long-running task and proceed as an error. If the long-running task wins, I cancel the timeout.
wait for all to complete: there are a couple of variations on this theme. But sometimes we want to wait for all to complete, errors or not; sometimes we want to stop on the first error; and sometimes we want to just get the first success (see timeouts).

All in all, this is a good start. It would probably be a good idea to take a deeper look at other Channel implementations and borrow heavily from them. The current implementation seems to leak a lot of internal information that most programs should not use. As a producer, all I care about is that the channel is open; not that anyone is listening or not. There may be nobody listening right now; but because it is async, it is my job as a programmer to make sure that someone is listening, eventually. Same with consumers, they don't need to know whether someone is actually producing, just that the channel is open so they can consume. It may be a good idea to create a way to use it with match:

$result = match (Async\select($channel)) {
Async\EOF => "channel closed",
default => $channel->receive(),
}

Where Async\select() just pauses until there is a value and returns "null" or "EOF" or something when it has a value available or becomes closed. Also, keep in mind this is pretty valid:

$channel->send($data);
$channel->close();

A consumer should read $data before seeing the channel as closed. I presume that this is what "finishProducing()" is for, but it requires synchronization to work properly and the entire idea of channels is to not need synchronization at all.

— Rob

1 year ago by Edmond Dantes — view source — reply

unread

Hello, Rob!

It is probably best to have channels be non-buffered by default (like in

Go); buffered channels can hide architectural issues in async code.

Great point! If a channel has a default capacity greater than 1, it leads
to implicit behavior that the user must be aware of. If something results
in implicit behavior, it's a bad practice. I'll fix it!

Why do we need "finishing" producing and consuming? Won't "closing" the
channel be enough, or does closing a channel prevent consumers from reading
pending items?

Because additional methods allow us to express explicit semantics while
also performing extra checks. Although, in essence, it's the same as the
close() method.
I'm not 100% sure whether this implementation is the right choice because
it has more elements compared to Go. However, the close() method has an
ambiguity regarding whether we want to terminate the Consumer or the
Producer. And ambiguity leads to errors.

Personally, it should be an exception (or at least a warning) if a
channel gets GC'd with pending data.

Yes, we need to think about what would be better—an exception or a warning.

The GC considers whether the channel was explicitly closed or not.
However, if someone tries to close consumption while there is still data, a
warning is mandatory because this is, once again, explicit semantics. If
a programmer wants to close a channel with data, they must first call
discardData() and only then close it.

So yes, explicit operations are much better.

Looking at the public api, it seems that it is very hard to use
correctly. It would probably be a good idea to pair it down and come up
with an api/ideomatic usage that makes it hard to use incorrectly.

What exactly is difficult to use?

Can we name this Async\Closure or something similar? Just to keep it
inline with the current \Closure

It's hard for me to judge this. On one hand, the name Callback clearly
reflects what this object does.

Speaking of, you probably meant to use \Closure as the $callback
property instead of mixed?

Unfortunately, PHP does not support the callable type for properties.

The constructor for a Future is public, but it looks like it would be
weird to use.

I'm currently working on this module and decided to base it on the
implementation in AMPHP. So, the Future class will just be a wrapper around
FutureState. And yes, its constructor can remain public, allowing Future to
be used as DeferredFuture.

It would be nice to see this simply use Futures as much as possible. For
example, returning a Future instead of FiberHandles.

Yes, this will be implemented a bit later, after the Future module is
fully ready. Most likely, this week.

Is onSignal OS signals or application signals? (I didn't look at the C
code) If the latter, it is probably better to use something other than
integer. If the former, this is usually handled by extensions (such as
pcntl) and comes with a ton of caveats. It might be better to let those
extensions handle that.

Yes, these are OS signals, but they are essentially cross-platform. Can int
be replaced with something else, like an enum? Of course.

Exposing $reserved -- which is accessible via reflection -- is probably
a bad idea.

Yes, I’ve thought about this. It really doesn’t seem like the best idea.

I might end up making the Notifier object final, which would eliminate this
issue.

Should this be implementing Stringable interface? Also, this will
automatically cast to a string if accidentally passed to a function that
takes a string -- unless you are using strict mode. https://3v4l.org/3AqcW This
is probably not desired.

Yes, all Notifier classes have a string representation for analysis
purposes.

Do you think it would be better to use a separate function instead of
__toString to avoid implicit behavior? If it helps prevent unintended
behavior, then you’re probably right.

Instead of Terminate(), should it be Close(), to be consistent with the
rest of the library?

Yes, maybe close() is better.

It would be nice to get a list of callbacks on the Notifier as well.

I 100% agree. I'll add this method now before I forget.

It would be nice to be able to get a future from the Walker instead of
polling "isFinished".
Of course.

IMHO, it would be better to implement this around Futures (see amphp's
extensive library around this). Here are some examples from my own code:

Sorry, but for some reason, I don't see the code. But just in case, I can
say in advance that Walker was essentially made thanks to AMPHP—I
borrowed part of the implementation from there :)

timeouts: race two futures, one which times out and the other which
performs a long-running task. If the timeout wins, then I cancel the
long-running task and proceed as an error. If the long-running task wins, I
cancel the timeout.

Why do you need Walker for this?

wait for all to complete: there are a couple of variations on this
theme. But sometimes we want to wait for all to complete, errors or not;
sometimes we want to stop on the first error; and sometimes we want to just
get the first success (see timeouts).
This looks more like AwaitAll() / AwaitAny(). These functions will also
be available once I finish Future.

$result = match (Async\select($channel)) {
Async\EOF => "channel closed",
default => $channel->receive(),
}

Why?

foreach ($channel as $data) {
// ...
}

just use it :)

Also, keep in mind this is pretty valid
That's exactly how it's implemented now. The channel does not treat NULL
as the end of transmission—the programmer must close it explicitly, or the
GC will do it.

Thanks for the great comments! They will help make this product better.

1 year ago by Edmond Dantes — view source — reply

unread

I'm actually curious why you chose libuv?

I relied on the usual set of selection criteria: relevance, support, and
documentation.
Hope there won’t be any serious issues with this component. But if there
are, we can implement a Plan B: copy the necessary Win32 code from libUV,
adapt it for PHP, and use something else for Unix-like systems. Another
option would be to implement native support for IO-URING.

Ed.