Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:104644
User-Agent: Cyrus-JMAP/3.1.5-925-g644bf8c-fmstable-20190228v5
Message-ID: <c04bc5a3-d83f-4ab0-9623-628fa56dd971@www.fastmail.com>
In-Reply-To: 
 <VI1PR07MB535779365912DD51C731C9EDE24F0@VI1PR07MB5357.eurprd07.prod.outlook.com>
References: <a70baca7-f0af-49de-9cf9-5421e42e0a0c@www.fastmail.com>
 <VI1PR07MB535779365912DD51C731C9EDE24F0@VI1PR07MB5357.eurprd07.prod.outlook.com>
Date: Mon, 11 Mar 2019 00:29:59 -0400
To: internals@lists.php.net
Content-Type: text/plain
Subject: Re: [PHP-DEV] RFC Draft: Comprehensions
From: larry@garfieldtech.com ("Larry Garfield")

On Sun, Mar 10, 2019, at 7:16 PM, Bob Weinand wrote:
> > Am 10.03.2019 um 22:44 schrieb Larry Garfield <larry@garfieldtech.com>:
> > 
> > Hello, peoples.  I know it's been discussed once or twice before on the list, many years ago, but not recently.  I therefore feel OK putting forth the following draft proposal for Comprehensions, aka "compact generators", in PHP:
> > 
> > https://wiki.php.net/rfc/comprehensions
> > 
> > Sara Golemon has written a preliminary patch that is partially complete (see the RFC link for details, it's impressively simple), but unfortunately doesn't have the bandwidth to complete it at this time.  I am therefore looking for collaborators with more knowledge of internals than I (which is almost everyone) to help finish it up.
> > 
> > The syntax proposed is also subject to revision if a terser but still lexer-friendly alternative can be proposed.
> > 
> > At the moment I'm not calling this Proposed yet, as I don't feel comfortable doing so until someone else is on board to finish coding it.  That said, if someone wants to weigh in on the concept for now (hopefully supportively) that's also fine.
> > 
> > Anyone excited enough to help finish the job?
> > 
> > (This is my first RFC collaboration, so if you're going to smack me for goofing something please be gentle about it.)
> > 
> > -- 
> >  Larry Garfield
> >  larry@garfieldtech.com
> 
> Hey Larry,
> First, I like it. (the idea/intention behind of providing a compact way 
> for list comprehensions)
> 
> Though in general, I think this would be better off implemented with a 
> more flexible short closure; it feels like a special case of these. 
> I.e. that the comprehension syntax can be simply desugared to an 
> immediately invoked short closure. At that point also, it can be a 
> compiler only patch, if we have short closures with the appropriate 
> binding behavior.
> I guess it should be pretty easy to build that on top of my existing 
> short closures patch.
> 
> Regarding the syntax, I'd not conflate for with foreach, the 
> distinction is built into PHP and as such should remain.
> We also might have trivial argument-less short closures written as "{ 
> stmt }" simply, having the comprehension written as $gen = 
> {foreach($list as $x) if($x % 2) yield $x * 2;}(); (note the trailing 
> "()" function call - if we desire to, we might make the semicolon at 
> the end of that single-statement closure optional.)
> 
> Bob

Replying to a couple of people at once here:

1) I am 100% in favor of adopting short closures, the less syntax ceremony in it the better.  Bob, if there's a logistical/non-code way I can help make that happen let me know, because we wants it, precious.

2) That said, I'm not sure it would fully obviate the need for a generator-specific syntax.  Even if the examples from Bob and Wes are shorter than what I had envisioned for a short-closure version (nice!), they're still notably longer than a purpose-built generator version.  Too, and I acknowledge this is a highly subjective statement, they have a lot of syntax salad.  ";)();" at the end is a superb thing to mistype more often than you get it right.  If there were some way to reduce that then I might be convinced but I'd have to see it first.  (And that means we need short closures still, which we don't have yet.)

3) I'd still be in favor of adding a type check to comprehensions, too; I don't know how or if that would be possible if they were "just" a fancy use of short-closures.

4) Regarding the specific syntax proposed, it's honestly more verbose than I'd like as is.  Sara and I tried putting the expression first, as in Python, but that made the lexer very unhappy since "[$foo ... ]" is already legal syntax for an array of one iterable.  The yield keyword is there to help the parser (and reader) tell where the expression begins since, being after the conditional, it needs some kind of separator.  If anything, I'd be interested in considering entirely different syntaxes borrowed from other languages (see the linked Wikipedia page) that would involve even fewer keywords, not using longer keywords.  That's why, for instance, it uses for instead of foreach.

5) Regarding Javascript comprehensions, well drat.  I didn't realize it had been withdrawn.  That's disappointing.  I'll adjust the RFC at some point soonish (it's after midnight here right now) to not mention them.

6) array_map/array_filter order reversed:  Bruce, I think you're right.  It took a while for me to grok that it is.  Which IMO is even more reason we want some more comprehensible and compact syntax for that use case, as it's surprisingly common. :-)

7) Bruce, there is an example of multiple foreach()s.  See the "Iterate a 2D array" section in the long code block.

8) To George: yes, since comprehensions as envisioned would work on any iterable, and generators are iterables, and comprehensions are generators, passing the generator produced by one comprehension to another comprehension is 100% legit.  Strings are not currently iterable, though, so they wouldn't work.  Making strings iterable is an entirely separate discussion so let's not go there, but were that to ever happen then they would "just work" in comprehensions as well.

9) To Alex: Because the syntax feels more akin to Generators at the user-level, that's how I described them.  You're writing code that yields items one by one.  That said, if under the hood it fits the code better to have it nominally return an \Iterator rather than \Generator I don't much care either way.  That's an implementation detail that I leave to whoever I manage to bribe to finish the code. :-)

--Larry Garfield