List comprehensions and generator expressions for PHP

Heya,

So, what do you think? Do we want something like this in PHP?

I think it looks great, especially the generator functions would be
useful to avoid having to create a full blown iterator for simple stuff.

Cheers

--
Jordi Boggiano
@seldaek - http://nelm.io/jordi

Hi,

Whats the difference to the (already existing) function array_map() (except
the syntax and one more new keyword)?

$firstNames = array_map(function($user){return $user->firstname;},
$users);

Don't want to rewrite every example you gave, but you probably see my point.

Regards,
Sebastian

2012/6/28 Nikita Popov [email protected]

Hi internals!

Python and several other languages include support for list
comprehensions and generator expressions and I'd like to see something
similar in PHP too.

I created a hacky proof of concept implementation here:
https://github.com/nikic/php-src/tree/addListComprehensions. It's
really dirty, but it implements both features in about ~150 lines of
code.

Currently I'm using the following syntax:

$firstNames = [foreach ($users as $user) yield $user->firstName];

This code is roughly equivalent to writing:

$firstNames = [];
foreach ($users as $user) {
$firstNames[] = $user->firstName;
}

You may notice that this particular list comprehension provides the
same functionality as array_column(), just in a little more
generalized way. E.g. you could use all of the following without
having special functions for them all:

$firstNames = [foreach ($users as $user) yield $user->firstName];

$firstNames = [foreach ($users as $user) yield $user->getFirstName()];

$firstNames = [foreach ($users as $user) yield $user['firstName']];

It's also possible to explicitly specify a key:

$firstNames = [foreach ($users as $user) yield $user->id =>
$user->firstName];

It is also possible to filter elements using list comprehensions:

$underageUsers = [foreach ($users as $user) if ($user->age < 18)
yield $user];
// or just the names
$underageUserNames = [foreach ($users as $user) if ($user->age <

18. yield $user->firstName];

It is also possible to nest multiple foreach loops:

$aList = ['A', 'B'];
$bList = [1, 2];
$combinations = [foreach ($aList as $a) foreach ($bList as $b)
yield [$a, $b]];
// gives: [ ['A', 1], ['A', 2], ['B', 1], ['B', 2] ]

All the above are list comprehensions (or in PHP rather array
comprehensions), i.e. they create an array as the result.

If this is not needed it is also possible to compute the values lazily
using generator expressions, which use () instead of [].

$firstNames = (foreach ($users as $user) yield $user->firstName);

In this case $firstNames will no longer be an array of first names,
but instead will be a generator producing first names.

This is handy if you only need to iterate the resulting "list" only
once as it saves you holding the whole list in memory.

Also it allows you to work with infinite lists easily:

function *naturalNumbers() {
for ($i = 0; ; ++$i) {
yield $i;
}
}

// all natural numbers
$numbers = naturalNumbers();
// only the odd ones
$oddNumbers = (foreach ($numbers as $n) if ($n % 2) yield $n);
// ...

(At this point I wonder whether one should include support for
for-loops in list comprehensions. So the naturalNumbers() function
could be replaced with (for ($i = 0;; ++$i) yield $i), etc)

So, what do you think? Do we want something like this in PHP?

Nikita

-----Original Message-----
From: Nikita Popov [mailto:[email protected]]
Sent: 28 June 2012 11:49
To: PHP internals
Subject: [PHP-DEV] List comprehensions and generator
expressions for PHP

Hi internals!

Python and several other languages include support for list
comprehensions and generator expressions and I'd like to see
something
similar in PHP too.

I created a hacky proof of concept implementation here:
https://github.com/nikic/php-src/tree/addListComprehensions. It's
really dirty, but it implements both features in about ~150 lines of
code.

Currently I'm using the following syntax:

$firstNames = [foreach ($users as $user) yield

$user->firstName];

This code is roughly equivalent to writing:

$firstNames = [];
foreach ($users as $user) {
    $firstNames[] = $user->firstName;
}

You may notice that this particular list comprehension provides the
same functionality as array_column(), just in a little more
generalized way. E.g. you could use all of the following without
having special functions for them all:

$firstNames = [foreach ($users as $user) yield

$user->firstName];

$firstNames = [foreach ($users as $user) yield 

$user->getFirstName()];

$firstNames = [foreach ($users as $user) yield 

$user['firstName']];

It's also possible to explicitly specify a key:

$firstNames = [foreach ($users as $user) yield $user->id =>

$user->firstName];

It is also possible to filter elements using list comprehensions:

$underageUsers = [foreach ($users as $user) if ($user->age < 18)

yield $user];
// or just the names
$underageUserNames = [foreach ($users as $user) if ($user->age <

18. yield $user->firstName];

It is also possible to nest multiple foreach loops:

$aList = ['A', 'B'];
$bList = [1, 2];
$combinations = [foreach ($aList as $a) foreach ($bList as $b)

yield [$a, $b]];
// gives: [ ['A', 1], ['A', 2], ['B', 1], ['B', 2] ]

All the above are list comprehensions (or in PHP rather array
comprehensions), i.e. they create an array as the result.

If this is not needed it is also possible to compute the values
lazily
using generator expressions, which use () instead of [].

$firstNames = (foreach ($users as $user) yield

$user->firstName);

In this case $firstNames will no longer be an array of first names,
but instead will be a generator producing first names.

This is handy if you only need to iterate the resulting "list" only
once as it saves you holding the whole list in memory.

Also it allows you to work with infinite lists easily:

function *naturalNumbers() {
    for ($i = 0; ; ++$i) {
        yield $i;
    }
}

// all natural numbers
$numbers = naturalNumbers();
// only the odd ones
$oddNumbers = (foreach ($numbers as $n) if ($n % 2) yield $n);
// ...

(At this point I wonder whether one should include support for
for-loops in list comprehensions. So the naturalNumbers() function
could be replaced with (for ($i = 0;; ++$i) yield $i), etc)

So, what do you think? Do we want something like this in PHP?

No mention of yielding keys in the comprehensions. Presume that would
work?

I would definitely like to see generators in, the amount of
boilerplate
code required for the current Iterator interface is not nice.

Also have been playing with your addGeneratorsSupport branch, trying
to see
If can get any benefits from using asynchronous operations.

The simplest example I could think of using Memcached
(assuming libmemcached fetch() does return as soon as it recieves an
item)

Eg.

function *getMulti(Memcached $memcached, array $keys)
{
if ($memcached->getDelayed($keys)) {
while ($item = $memcached->fetch())
yield $item['key'] => $item['value'];
}
}

foreach(getMulti($memcached, ['foo', 'bar', ... ]) as $key => $value)
{
doSomeWork();
}

So doSomeWork would be called as soon as a single value is available,
rather
than having to wait til all values have returned.

Should in theory work right?

Jared