Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:104857
User-Agent: Cyrus-JMAP/3.1.5-976-g376b1f3-fmstable-20190314v3
Mime-Version: 1.0
Message-ID: <cd2093de-ac85-41da-825c-dd248a135a00@www.fastmail.com>
In-Reply-To: <301552a9-6f01-3177-0e57-3d9aac54cc0c@gmail.com>
References: <a70baca7-f0af-49de-9cf9-5421e42e0a0c@www.fastmail.com>
 <CAF+90c9nrLQGfxoJWi72xatxz21qvqc5jyRnnHpcgivumhSXeg@mail.gmail.com>
 <85417a73-a391-c9ad-49aa-5b88c65b0900@gmail.com>
 <5c8ea43d.1c69fb81.e50f1.7d18SMTPIN_ADDED_MISSING@mx.google.com>
 <d21ae93f-d275-f526-7e2a-abe9a3a14397@gmail.com>
 <CAMuFt-5WQm13WkkKWO3MnzdqQ4ruKM8=8panK10SvCfyeVuDXg@mail.gmail.com>
 <0d34b3b1-866e-4bab-9a8f-3c70f1fc9721@www.fastmail.com>
 <c526a0b3-012c-a582-8933-619629f44c7e@gmail.com>
 <CALKiJKpXOfrVPRueU-BLBXHwNimBbsJZebY=u42gFLa3mdV7Lg@mail.gmail.com>
 <752c83ab-0e6f-d8f5-ff15-e1f75b5d04b5@gmail.com>
 <301552a9-6f01-3177-0e57-3d9aac54cc0c@gmail.com>
Date: Thu, 21 Mar 2019 11:22:50 -0400
To: internals@lists.php.net
Content-Type: text/plain
Subject: Re: [PHP-DEV] RFC Draft: Comprehensions
From: larry@garfieldtech.com ("Larry Garfield")

On Wed, Mar 20, 2019, at 5:44 PM, Rowan Collins wrote:
> On 20/03/2019 20:39, Stanislav Malyshev wrote:
> > Hi!
> >
> >> It's not that you can't make an array into a generator, but you can't make
> >> an eagerly-evaluated expression into a lazily-evaluated one.
> > Not sure what you mean here.
> 
> 
> I mean that, given a syntax that lazily-evaluates something, you can 
> "fast-forward" the result to make it eagerly-evaluated; given a syntax 
> that eagerly-evaluates something, you cannot do the opposite. Therefore, 
> a lazy-evaluating syntax is more powerful, in that it can do everything 
> an eager-evaluating one can do *and more*.

This is really the crux of the array-vs-generator question for comprehensions.

No one disputes that both arrays and generators have their good use cases.  No one disputes that there are cases where one is favorable over another.  I argue that in *most* cases where you'd be using a comprehension it wouldn't actually matter in practice which one it is.  Which one is more prevalent in practice depends who the developer is; I personally use generators a ton because I find them easier to work with syntactically when processing complex data structures regardless of their size, but others have their own styles.

It comes down to conceptual simplicity.  A greedy-evaluated list can be trivially composed from a lazy-evaluated list and a "fast-forward" operator.  Both of those are useful in their own right, and can be composed to produce a greedy evaluated list.  However, given a greedy-evaluated list there is no way to produce a lazy-evaluated list at all.  Thus we will need a lazy-evaluated list either way (and that's my goal with comprehensions).  

Given a lazy-evaluated list and a fast-forward operator... an alternate greedy-evaluated list is a nice-to-have, unless it creates more syntactic complexity and/or confusion.  As an unrealistic Pythonic example, were we to do:

{foreach $foo as $bar yield $bar*2} // gives a generator

[foreach $foo as $bar yield $bar*2] // gives an array

Then we're introducing confusion for users who have to learn two different very-similar-looking syntaxes, and we're laying claim to both []-wrapped expressions and {}-wrapped expressions to mean something.

OTOH, if we just have the one syntax:

[foreach $foo as $bar yield $bar*2] // gives a generator

And include a nicer "fast-forward" operator than interator_to_array(), then we automatically get:

...[foreach $foo as $bar yield $bar*2] // turns the generator into an array on the fly

Which is both easier to learn (two independently useful primitives, composed), easier to understand (because the latter is not as visually similar), and doesn't lay claim to two different potentially useful pieces of block syntax, leaving the other open for later use by some other idea later.

(And if we don't introduce a new fast-forward operation, then iterator_to_array() is already still there and while a bit more verbose than we'd like it certainly doesn't the job just as effectively.)

Also, I reiterate that in the majority of use cases I see where a comprehension would be used (not all, but most) the result would end up in a foreach() loop, or returned as an iterable and thus whether it's an array or generator won't actually matter.

So really, "I don't use generators much" is not a meaningful argument.  A lower-overhead implementation (both for the user and the parser) is the argument, and on that front "generator only" wins, with "both" a weak compromise position that doesn't add much.  "Array only" is by far the weakest result.

Now can we go back to bikeshedding the actual syntax style to use, per my earlier email? :-)

--Larry Garfield