[RFC] Fiber support (again)

This has already been discussed in github

Fiber::resume()
https://github.com/fiberphp/fiber-ext/issues/7

Fiber::throw()
https://github.com/fiberphp/fiber-ext/issues/6

Keywords - async/await
https://github.com/fiberphp/fiber-ext/issues/10

I believe that current API is normal for the Pecl ext, but to become
semantics of PHP language, the API needs to be improved

The language should offer a sane API, not the absolute minimum required
to
work for these things.

The Ruby's Fiber do offer a live? method but does not have a getStatus
method.
The Lua's coroutine only offer a status method.

So do we really need to offer three additional helper method? Or what is
your
advice about these API?

What's the downside?

• What about throwing exceptions into a fiber?

Currently does not support throw exception into the fiber. User land
code
could check
the value of Fiber::yield and throw exception themselves. The Ruby's
Fiber
and Lua's
coroutine also does not support such api as well.

And throw the exception where? That means async code with fibers can't
really handle errors?

Actually you can transfer any thing to Fiber by the resume method. And you
can
check the return value of Fiber::yield to handle error.

Fiber is designed as a primitive, low level, and lightweight feature. User
land
code seldom not need to use them directly in your normal code.
So the following is not a big problem,

$a = Fiber::yield(...);
if ($a === false) {
throw new Exception(...);
}

And both the Ruby and Lua does not offer such API as well.

We have started building a PoC library on top of Fibers, see
https://github.com/amphp/green-thread/blob/7bd3470e7986169372d5e9c39500f3652091b512/src/functions.php
.

We'd like to avoid the additional await() function and rather directly
couple Fibers with promises in the PHP core.

Using await (the keyword) instead of Fiber::yield() feels way better
and avoids the need for any wrapping libraries for what should be a core
feature.

async can be added to create a new Fiber and Fiber can implement Promise.

• Using Fiber::resume() to initialize the fiber and resume feels
  awkward. Separate methods again would be better here, perhaps
  Fiber::init(...$args) and Fiber::resume($send).

All Fiber created with a suspended status. So make resume to do both the
init and resume
do make sense.

It does't make sense to me. Reading the example in the README and
understanding why the first resume() takes two arguments instead of one
took me quite some minutes.

This Ruby's Fiber and Lua's coroutine using one resume API to init and
resume
the coroutine. I do not think a dedicate is really required.

The generator cannot be init by it's send method. And if you want to
implement
coroutine feature(without stack) by it, you have to write code

function run() {
if ($this->beforeFirstYield) {
$this->beforeFirstYield = false;
return $this->coroutine->current();
} else {
$retval = $this->coroutine->send($this->sendValue);
$this->sendValue = null;
return $retval;
}
}

It is verbose.

That's why we directly prime coroutines on creation, see
https://github.com/amphp/amp/blob/4a742beb59615f36ed998e2dc210e36576e44c44/lib/Coroutine.php#L36-L52

See https://nikic.github.io/2012/12/22/Cooperative-
multitasking-using-coroutines-in-PHP.html

Please see Ruby Fiber API https://ruby-doc.org/core-2.2.0/Fiber.html.

• What happens if the sub1() function in the RFC is invoked outside of
  a
  fiber?

You will get a Fatal Error like
Fatal error: Uncaught Error: Cannot call Fiber::yield out of Fiber

• I think a keyword here would be beneficial, even if it has a minor BC
  impact. Fibers could then be written like generators. await or emit
  as
  a keyword perhaps? This would be a less verbose API, feel less magical
  (a
  static method call that actually pauses execution feels out of place),
  and
  would allow Fibers to be returned from methods, named functions, etc
  with
  less boilerplate.

Wishing this to be accepted by the community in the PHP 7.3, so no
keyword
is accepted.
And if the community cannot accept, the Fiber can be still distributed
as
a standalone
extension. So we cannot depend on a new keyword.

Right, if it's a standalone extension it can't use a new keyword, but as
a
language feature it totally can.

In my opinion, using a keyword or call a method is just a coding style
problem.
Introducing a new keyword does not offer any benefit by makes a minor BC.

It makes code way easier to write IMO.

Both Ruby's Fiber and Lua's coroutine does not required a dedicate keyword.

Looking at the current README, there are two issues that must be
completely
solved IMO before accepting this:

Each Fiber has a separate 4k stack. You can use the fiber.stack_size ini
option to change the default stack size. You can also use the second
argument of Fiber::__construct to set the stack size on fly.

Resizing of the stack should happen automatically, just like generators
resize automatically.

This size is the init stack size. It means when a Fiber created, it will
get a
dedicate stack of 4k size. When the fiber use all the stack space, zend vm
will
allocate additional space for feature call frame, automatically.

The default size is 4k means that every fiber requires at least 4k memory
to use
as there own stack. But user can change this by php.ini and the construct
argument
to reduce the memory footprint.

That's good to hear! Any reason why these need to be configureable? They're
not for generators either. IMO a sane default would probably be enough.

Fiber::yield cannot be used in internal callback

This also seems problematic and will make fibers quite less useful,
especially as these yields can happen anywhere down the stack.

This do be a problem. But not so much big problem.

You cannot use Fiber::yield like

$f = new Fiber(function () {
array_map(function ($i) {
Fiber::yield($i);
}, [1, 2, 3]);
});
$f->resume();
$f->resume();

Because when zend execute the array_map, it will push a new frame onto the
c stack.
When zend execute Fiber::yield, it only backup it's php stack, and it's c
stack
will be overwrites.

However, you can use Fiber::yield like

$f = new Fiber(function () {
Fiber::yield(1); // will cause by resume, by an internal callback
});

ExtEventLoop::onRead($fd, function () { // this is an internal call
$f->resume();
});

The issue here is not using it directly within array_map, but rather
somewhere down the stack, which a user can never know.

This might result in subtle BC breaks. It means that a Fiber::yield() can
never be added to code without breaking BC. I guess we need to rewrite all
functions that accept user-defined callbacks for it to work.

IMO PHP's main() function should be a Fiber, too, which makes it impossible
to use Fiber::yield() outside a Fiber and enables top-level await().

Regards, Niklas

Hi, all,

I have updated the RFC https://wiki.php.net/rfc/fiber

changes list:

• introduce the throw(Exception $exceptin) API
• record issues discussed

• How do you determine when a fiber has returned? Looking at the source,
  it appears Fiber::status() must be used, comparing against constants.
  Separate methods similar to Generator would be better IMO. e.g.:
  Fiber::alive(), Fiber::suspended(), Fiber::running()

Offering methods like Fiber::alive, Fiber::running makes no difference to
check the Fiber::status() return value. This is just a style issue. And as
a language feature,
Fiber only offer the essential API and let other works to the user land.

The language should offer a sane API, not the absolute minimum required to
work for these things.

The Ruby's Fiber do offer a live? method but does not have a getStatus method.
The Lua's coroutine only offer a status method.

So do we really need to offer three additional helper method? Or what is your
advice about these API?

• What about throwing exceptions into a fiber?

Currently does not support throw exception into the fiber. User land code
could check
the value of Fiber::yield and throw exception themselves. The Ruby's Fiber
and Lua's
coroutine also does not support such api as well.

And throw the exception where? That means async code with fibers can't
really handle errors?

Actually you can transfer any thing to Fiber by the resume method. And you can
check the return value of Fiber::yield to handle error.

Fiber is designed as a primitive, low level, and lightweight feature. User land
code seldom not need to use them directly in your normal code.
So the following is not a big problem,

$a = Fiber::yield(...);
if ($a === false) {
throw new Exception(...);
}

And both the Ruby and Lua does not offer such API as well.

• Using Fiber::resume() to initialize the fiber and resume feels
  awkward. Separate methods again would be better here, perhaps
  Fiber::init(...$args) and Fiber::resume($send).

All Fiber created with a suspended status. So make resume to do both the
init and resume
do make sense.

It does't make sense to me. Reading the example in the README and
understanding why the first resume() takes two arguments instead of one
took me quite some minutes.

This Ruby's Fiber and Lua's coroutine using one resume API to init and resume
the coroutine. I do not think a dedicate is really required.

The generator cannot be init by it's send method. And if you want to implement
coroutine feature(without stack) by it, you have to write code

function run() {
if ($this->beforeFirstYield) {
$this->beforeFirstYield = false;
return $this->coroutine->current();
} else {
$retval = $this->coroutine->send($this->sendValue);
$this->sendValue = null;
return $retval;
}
}

It is verbose.

See https://nikic.github.io/2012/12/22/Cooperative-multitasking-using-coroutines-in-PHP.html

Please see Ruby Fiber API https://ruby-doc.org/core-2.2.0/Fiber.html.

• What happens if the sub1() function in the RFC is invoked outside of a
  fiber?

You will get a Fatal Error like
Fatal error: Uncaught Error: Cannot call Fiber::yield out of Fiber

• I think a keyword here would be beneficial, even if it has a minor BC
  impact. Fibers could then be written like generators. await or emit as
  a keyword perhaps? This would be a less verbose API, feel less magical (a
  static method call that actually pauses execution feels out of place), and
  would allow Fibers to be returned from methods, named functions, etc with
  less boilerplate.

Wishing this to be accepted by the community in the PHP 7.3, so no keyword
is accepted.
And if the community cannot accept, the Fiber can be still distributed as
a standalone
extension. So we cannot depend on a new keyword.

Right, if it's a standalone extension it can't use a new keyword, but as a
language feature it totally can.

In my opinion, using a keyword or call a method is just a coding style problem.
Introducing a new keyword does not offer any benefit by makes a minor BC.

Both Ruby's Fiber and Lua's coroutine does not required a dedicate keyword.

Looking at the current README, there are two issues that must be completely
solved IMO before accepting this:

Each Fiber has a separate 4k stack. You can use the fiber.stack_size ini
option to change the default stack size. You can also use the second
argument of Fiber::__construct to set the stack size on fly.

Resizing of the stack should happen automatically, just like generators
resize automatically.

This size is the init stack size. It means when a Fiber created, it will get a
dedicate stack of 4k size. When the fiber use all the stack space, zend vm will
allocate additional space for feature call frame, automatically.

The default size is 4k means that every fiber requires at least 4k memory to use
as there own stack. But user can change this by php.ini and the construct argument
to reduce the memory footprint.

Fiber::yield cannot be used in internal callback

This also seems problematic and will make fibers quite less useful,
especially as these yields can happen anywhere down the stack.

This do be a problem. But not so much big problem.

You cannot use Fiber::yield like

$f = new Fiber(function () {
array_map(function ($i) {
Fiber::yield($i);
}, [1, 2, 3]);
});
$f->resume();
$f->resume();

Because when zend execute the array_map, it will push a new frame onto the c stack.
When zend execute Fiber::yield, it only backup it's php stack, and it's c stack
will be overwrites.

However, you can use Fiber::yield like

$f = new Fiber(function () {
Fiber::yield(1); // will cause by resume, by an internal callback
});

ExtEventLoop::onRead($fd, function () { // this is an internal call
$f->resume();
});

Regards, Niklas

• Using Fiber::resume() to initialize the fiber and resume feels
  awkward. Separate methods again would be better here, perhaps
  Fiber::init(...$args) and Fiber::resume($send).

All Fiber created with a suspended status. So make resume to do both the
init and resume
do make sense.

It does't make sense to me. Reading the example in the README and
understanding why the first resume() takes two arguments instead of one
took me quite some minutes.

This Ruby's Fiber and Lua's coroutine using one resume API to init and
resume
the coroutine. I do not think a dedicate is really required.

I agree, two methods are probably not required.

But I also agree with Aaron that the verb "resume" is confusing/misleading
here - it implies something has already been running, which isn't the case
when you initially create the Fiber.

I'd suggest changing the name to start() - you can "start" something
whether it's starting for the first time or starting again after a pause,
but to "resume" something, it must have been already running, so it's a bit
confusing.

• How do you determine when a fiber has returned? Looking at the
  source,
  it appears Fiber::status() must be used, comparing against constants.
  Separate methods similar to Generator would be better IMO. e.g.:
  Fiber::alive(), Fiber::suspended(), Fiber::running()

Offering methods like Fiber::alive, Fiber::running makes no difference
to
check the Fiber::status() return value. This is just a style issue. And
as
a language feature,
Fiber only offer the essential API and let other works to the user land.

The language should offer a sane API, not the absolute minimum required
to
work for these things.

The Ruby's Fiber do offer a live? method but does not have a getStatus
method.
The Lua's coroutine only offer a status method.

So do we really need to offer three additional helper method? Or what is
your
advice about these API?

• What about throwing exceptions into a fiber?

Currently does not support throw exception into the fiber. User land
code
could check
the value of Fiber::yield and throw exception themselves. The Ruby's
Fiber
and Lua's
coroutine also does not support such api as well.

And throw the exception where? That means async code with fibers can't
really handle errors?

Actually you can transfer any thing to Fiber by the resume method. And you
can
check the return value of Fiber::yield to handle error.

Fiber is designed as a primitive, low level, and lightweight feature. User
land
code seldom not need to use them directly in your normal code.
So the following is not a big problem,

$a = Fiber::yield(...);
if ($a === false) {
throw new Exception(...);
}

And both the Ruby and Lua does not offer such API as well.

• Using Fiber::resume() to initialize the fiber and resume feels
  awkward. Separate methods again would be better here, perhaps
  Fiber::init(...$args) and Fiber::resume($send).

All Fiber created with a suspended status. So make resume to do both the
init and resume
do make sense.

It does't make sense to me. Reading the example in the README and
understanding why the first resume() takes two arguments instead of one
took me quite some minutes.

This Ruby's Fiber and Lua's coroutine using one resume API to init and
resume
the coroutine. I do not think a dedicate is really required.

The generator cannot be init by it's send method. And if you want to
implement
coroutine feature(without stack) by it, you have to write code

function run() {
if ($this->beforeFirstYield) {
$this->beforeFirstYield = false;
return $this->coroutine->current();
} else {
$retval = $this->coroutine->send($this->sendValue);
$this->sendValue = null;
return $retval;
}
}

It is verbose.

See https://nikic.github.io/2012/12/22/Cooperative-
multitasking-using-coroutines-in-PHP.html

Please see Ruby Fiber API https://ruby-doc.org/core-2.2.0/Fiber.html.

• What happens if the sub1() function in the RFC is invoked outside of
  a
  fiber?

You will get a Fatal Error like
Fatal error: Uncaught Error: Cannot call Fiber::yield out of Fiber

• I think a keyword here would be beneficial, even if it has a minor BC
  impact. Fibers could then be written like generators. await or emit
  as
  a keyword perhaps? This would be a less verbose API, feel less magical
  (a
  static method call that actually pauses execution feels out of place),
  and
  would allow Fibers to be returned from methods, named functions, etc
  with
  less boilerplate.

Wishing this to be accepted by the community in the PHP 7.3, so no
keyword
is accepted.
And if the community cannot accept, the Fiber can be still distributed
as
a standalone
extension. So we cannot depend on a new keyword.

Right, if it's a standalone extension it can't use a new keyword, but as
a
language feature it totally can.

In my opinion, using a keyword or call a method is just a coding style
problem.
Introducing a new keyword does not offer any benefit by makes a minor BC.

Both Ruby's Fiber and Lua's coroutine does not required a dedicate keyword.

Looking at the current README, there are two issues that must be
completely
solved IMO before accepting this:

Each Fiber has a separate 4k stack. You can use the fiber.stack_size ini
option to change the default stack size. You can also use the second
argument of Fiber::__construct to set the stack size on fly.

Resizing of the stack should happen automatically, just like generators
resize automatically.

This size is the init stack size. It means when a Fiber created, it will
get a
dedicate stack of 4k size. When the fiber use all the stack space, zend vm
will
allocate additional space for feature call frame, automatically.

The default size is 4k means that every fiber requires at least 4k memory
to use
as there own stack. But user can change this by php.ini and the construct
argument
to reduce the memory footprint.

Fiber::yield cannot be used in internal callback

This also seems problematic and will make fibers quite less useful,
especially as these yields can happen anywhere down the stack.

This do be a problem. But not so much big problem.

You cannot use Fiber::yield like

$f = new Fiber(function () {
array_map(function ($i) {
Fiber::yield($i);
}, [1, 2, 3]);
});
$f->resume();
$f->resume();

Because when zend execute the array_map, it will push a new frame onto the
c stack.
When zend execute Fiber::yield, it only backup it's php stack, and it's c
stack
will be overwrites.

However, you can use Fiber::yield like

$f = new Fiber(function () {
Fiber::yield(1); // will cause by resume, by an internal callback
});

ExtEventLoop::onRead($fd, function () { // this is an internal call
$f->resume();
});

Regards, Niklas