generator/yield operator feature request

17 years ago by moo.tinys — view source — reply

unread

this topic may be discussed before, but i'd bring it up again.

closure was introduced in PHP 5.3, function with its context can now
be stored in an callable object(or a handler in string type, whatever)
i think it's easy and read to introduce generator and yield operator
in php

let's see what was in Iterator:

every class/interface can be prototyped in php userland, not
internal, although many of the them was builtin written in c so make
life easier
pro: "simple" to implement in ZendEngine, see note 2 aboive
con: really ugly to code Iterator/Filter in userland, hard to
understand how they worked, code has to be split in rewind/next
functions
no changes is required in ZendEngine, except powering up "foreach"
to support iterator, so ppl can iterate over it much more easier.
pro: "simple" to implement in ZendEngine, see note 2 aboive
con: -

let's see what will yeild/generator looks like:

require saving function calling context (not call stack)
con(was): there was no way to save it, we had to implement it first
con(now): no con, implemented already in 5.3
just 1 function, yield at any where in the body
pro: much more readable than Iterator
con: "yield" is now reserved keyword
example:
function foo($space_and_new_line = false)
{
yield "hello";
if ($space_and_new_line) {
yield " ";
}
yield "world";
if ($space_and_new_line) {
yield PHP_EOL;
}
}

$g = foo(true); // function foo() is not called here
foreach ($g as $i) { // function foo() IS called here
echo $i;
}

function filter($g, $regex)
{
foreach ($g as $baa) {
if (preg_match($regex, $i)) {
return true;
} // if
} // foreach
}

$g = foo(true);
foreach (filter($g, "^\w+$") as $i) {
echo $i; // now letters only
}

$g = foo(true);
echo $g(); // same as: echo $g->value(); $g->next(); this is not in
python, neither in php iterator
echo $g();
echo $g();
(btw, i think iterator should be able to be used in this way)

I don't think you can reimplement it into Iterator in any simpler
form, except that you can reuse those classes already implemented.

is it perl way? no. because yield/generator make code much more
readable, easier to understood, not just "easier" to write, it's the
php way

==========
for those who don't know what yield/generator is, you can see python
document or the following brief (correct me if i'm wrong)
. functions (named or unamed/closure) with yield in its body, is a
generator function that with context saved, not a normal. it's similar
as closure context but with "yield point" saved
. when you call a generator function, it returns an generator object
. the interface to generator object is exactly like Iterator
interface, that can be design to have rewind/next/valid methods, etc
. generator is an OuterIterator
. you can call $generator->next() (and other methods) explicitly, or
call foreach which call iterator/generator methods implictly
. the only difference between iterator and generator is how they
implement it inside. when php execute "yield", the generator is
execution is returned but its execution context is saved, next time
the generator is iterated, php execute from the last yield point with
all context restored.

17 years ago by Etienne Kneuss — view source — reply

unread

Hello,

this topic may be discussed before, but i'd bring it up again.

closure was introduced in PHP 5.3, function with its context can now
be stored in an callable object(or a handler in string type, whatever)
i think it's easy and read to introduce generator and yield operator
in php

let's see what was in Iterator:

every class/interface can be prototyped in php userland, not
internal, although many of the them was builtin written in c so make
life easier
pro: "simple" to implement in ZendEngine, see note 2 aboive
con: really ugly to code Iterator/Filter in userland, hard to
understand how they worked, code has to be split in rewind/next
functions

no changes is required in ZendEngine, except powering up "foreach"
to support iterator, so ppl can iterate over it much more easier.

On the contrary, It will most likely require a lot of changes to
provide structures that are able to store their execution context to
be able to resume it later. I.e.

will the error be thrown?
what will the error_reporting value be after the execution of the function ?

error_reporting is a setting that is shared, and here you'd like to
resume the execution using the old context (error_reporting = 0) to
continue the function, affecting the outer context, which will stick
after the function ends? Or do you plan to keep both contexts
completely separated and create inconsistencies?

pro: "simple" to implement in ZendEngine, see note 2 aboive
con: -

let's see what will yeild/generator looks like:

require saving function calling context (not call stack)
con(was): there was no way to save it, we had to implement it first
con(now): no con, implemented already in 5.3

How is that implemented already in 5.3 ?

just 1 function, yield at any where in the body
pro: much more readable than Iterator
con: "yield" is now reserved keyword
example:
function foo($space_and_new_line = false)
{
yield "hello";
if ($space_and_new_line) {
yield " ";
}
yield "world";
if ($space_and_new_line) {
yield PHP_EOL;
}
}

$g = foo(true); // function foo() is not called here
foreach ($g as $i) { // function foo() IS called here
echo $i;
}

function filter($g, $regex)
{
foreach ($g as $baa) {
if (preg_match($regex, $i)) {
return true;
} // if
} // foreach
}

$g = foo(true);
foreach (filter($g, "^\w+$") as $i) {
echo $i; // now letters only
}

That example looks wrong(and I hope it is):

$i is not defined in the scope of filter()
filter() returns true or null. You can't iterate on such values.

Also, could you provide an example where this construct is actually useful ?

The example you provided doesn't demonstrate anything, as it can be
implemented in a far easier way using:

function foo($s) {
return $s ? array('hello', ' ', 'world', PHP_EOL) : array('hello', 'world');
}

foreach(foo() as $word) {
...
}

additionally, SPL's filterIterator and arrayiterator can be used to
implement a filtered iteration on the array.

$g = foo(true);
echo $g(); // same as: echo $g->value(); $g->next(); this is not in
python, neither in php iterator
echo $g();
echo $g();

This syntax is definitely not self-explanatory, and will make things
really hard to understand.

(btw, i think iterator should be able to be used in this way)

I don't think you can reimplement it into Iterator in any simpler
form, except that you can reuse those classes already implemented.

is it perl way? no. because yield/generator make code much more
readable, easier to understood, not just "easier" to write, it's the
php way

==========
for those who don't know what yield/generator is, you can see python
document or the following brief (correct me if i'm wrong)
. functions (named or unamed/closure) with yield in its body, is a
generator function that with context saved, not a normal. it's similar
as closure context but with "yield point" saved
. when you call a generator function, it returns an generator object
. the interface to generator object is exactly like Iterator
interface, that can be design to have rewind/next/valid methods, etc
. generator is an OuterIterator
. you can call $generator->next() (and other methods) explicitly, or
call foreach which call iterator/generator methods implictly
. the only difference between iterator and generator is how they
implement it inside. when php execute "yield", the generator is
execution is returned but its execution context is saved, next time
the generator is iterated, php execute from the last yield point with
all context restored.

I'm skeptic at what value this construct would really add to PHP. And
my guess is, it will not be worth the trouble implementing it.

Regards,

--

--
Etienne Kneuss
http://www.colder.ch

Men never do evil so completely and cheerfully as
when they do it from a religious conviction.
-- Pascal

17 years ago by moo.tinys — view source — reply

unread

On the contrary, It will most likely require a lot of changes to
provide structures that are able to store their execution context to
be able to resume it later. I.e.

function foo() {
error_reporting(0);
yield "foo";
trigger_error("foo", E_NOTICE);
yield "bar";
}
$e = false;
error_reporting(E_ALL);
foreach (foo() as $b) {
if (!$e) { $e = true; error_reporting(E_ALL); }
}

will the error be thrown?

what will the error_reporting value be after the execution of the function ?

error_reporting is a setting that is shared, and here you'd like to
resume the execution using the old context (error_reporting = 0) to
continue the function, affecting the outer context, which will stick
after the function ends? Or do you plan to keep both contexts
completely separated and create inconsistencies?

pro: "simple" to implement in ZendEngine, see note 2 aboive
con: -

let's see what will yeild/generator looks like:

require saving function calling context (not call stack)
con(was): there was no way to save it, we had to implement it first
con(now): no con, implemented already in 5.3

How is that implemented already in 5.3 ?
i haven't dig deep into how closure is implemented, but it's a big
step forward, yield/generator function is another minor step base on
or smilar as closure

just 1 function, yield at any where in the body
pro: much more readable than Iterator
con: "yield" is now reserved keyword
example:
function foo($space_and_new_line = false)
{
yield "hello";
if ($space_and_new_line) {
yield " ";
}
yield "world";
if ($space_and_new_line) {
yield PHP_EOL;
}
}

$g = foo(true); // function foo() is not called here
foreach ($g as $i) { // function foo() IS called here
echo $i;
}

function filter($g, $regex)
{
foreach ($g as $baa) {
foreach ($g as $i) { // <- correction, and sorry, here's the right version
if (preg_match($regex, $i)) {
return true;
yield $i; // <- correction
} // if
} // foreach
}

$g = foo(true);
foreach (filter($g, "^\w+$") as $i) {
echo $i; // now letters only
}

That example looks wrong(and I hope it is):

$i is not defined in the scope of filter()

filter() returns true or null. You can't iterate on such values.

Also, could you provide an example where this construct is actually useful ?
every Iterator class can be rewritten into more simpler form in
generator function
every generator function can also be rewritten into more
simpler/complexer form in Iterator class (most of the time, it's more
complexer)

for more complex code, i have to write it in pseudo code
function fetch_items()
{
foreach (array(some builtin constant items here) as $item) {
yield $item;
}
foreach (get_version1_urls() as $v1url) {
$content = file_get_contents($v1url);
foreach (parse_version1_content($content) as $item) {
yield $item;
}
}
foreach (get_version2_urls() as $v2url) {
$content = file_get_contents($v2url);
foreach (parse_version2_content($content) as $item) {
yield $item;
}
}
}

yes, as i said you can rewrite it in iterator

iterator class is a stateful iteratable implemented in class, while
generator function is a stateful iteratable implemented in function
(stateful version of function)
if iterator class looks like a state machine that split code into
multiple function/methods, generator function looks like thread base
deisgn that can yield at some point which is sure more easier to write
and understood
you're skilled programmer like i am, understanding/writing iterator
class is no big deal, but generator can make life much better for vast
of php guys

The example you provided doesn't demonstrate anything, as it can be
implemented in a far easier way using:

function foo($s) {
return $s ? array('hello', ' ', 'world', PHP_EOL) : array('hello', 'world');
}
this is not in iterator class "framework"

foreach(foo() as $word) {
...
}

additionally, SPL's filterIterator and arrayiterator can be used to
implement a filtered iteration on the array.
they're already implemented. yes, my example can be rewritten by
reusing already implemented iterator class, but why not think about a
case when ppl have to implemented their own iterator class? or have we
all possible iterator class implemented already?

$g = foo(true);
echo $g(); // same as: echo $g->value(); $g->next(); this is not in
python, neither in php iterator
echo $g();
echo $g();

This syntax is definitely not self-explanatory, and will make things
really hard to understand.
well.. i'm not sure if this is useful either, just a thought when i
bring this topic up :)

I'm skeptic at what value this construct would really add to PHP. And
my guess is, it will not be worth the trouble implementing it.
thanks for your time