[RFC] Auto-capture multi-line closures and short functions take 2

On Wed, Mar 24, 2021 at 5:20 PM Larry Garfield larry@garfieldtech.com
wrote:

Greetings, Internalians.

Some months back I proposed an RFC for short functions.

https://wiki.php.net/rfc/short-functions

After some discussion, I put it on hold to ensure that it was compatible
with the other discussion floating about around an alternate syntax for
multi-line closures that did auto-capture. It's important that the
syntax
of both proposals is consistent.

Nuno Maduro and i have talked recently and worked out the syntax that is
consistent between both proposals. That takes 'fn' off the table for
short-functions, for reasons discussed in both RFCs.

To that end, I offer:

1. The updated short-functions RFC:
   https://wiki.php.net/rfc/short-functions

2. A new RFC, code by Nuno, for auto-capturing multi-statement closures:
   https://wiki.php.net/rfc/auto-capture-closure

These are separate RFCs and at this time we plan to send them to separate
votes, but I list them together to demonstrate that they have been
developed in a mutually-compatible way.

Further details in the auto-capture-closure RFC.

Could you please create two separate threads for these RFCs?

Nikita

I guess my one question would be why we didn't support auto-capture when we
first implemented anonymous functions, and if there was a reason, why does
that no longer apply?

--
Chase Peeler
chasepeeler@gmail.com

My suggestion would be to allow use(&$count) on fn(), which allows capturing
certain variables by reference rather than by value.

It might be rather confusing for the use block to sometimes mean
"capture exactly these variables", and sometimes mean "capture all
variables, but modify how you capture these ones", based on if it
follows "function" or "fn".

In other words, it's not obvious at a glance that these two closures
have completely different effects, and why:

$increment = 1;
$count = 0;
$a = function() use (&$count) { $count += $increment; }
$b = fn() use (&$count) { $count += $increment; }

I think that extensive experience in other programming languages has shown
that auto-capture does not hinder readability of code, and I don't see any
particular reason why the effects in PHP would be different.

The difference between PHP and a lot of those other languages is that
variables are not imported automatically from other scopes.

In PHP, the following global function and closure would not use the same
definition of $foo:

$foo = 42;
function echoFoo { echo $foo; }
$echoFoo = fn() { echo $foo; }

That's not automatically a bad thing, but it is an extra concern that
PHP has which other languages don't.

Regards,

--
Rowan Tommins
[IMSoP]

On Wed, Mar 24, 2021 at 5:20 PM Larry Garfield larry@garfieldtech.com
wrote:

Greetings, Internalians.

Some months back I proposed an RFC for short functions.

https://wiki.php.net/rfc/short-functions

After some discussion, I put it on hold to ensure that it was compatible
with the other discussion floating about around an alternate syntax for
multi-line closures that did auto-capture. It's important that the
syntax
of both proposals is consistent.

Nuno Maduro and i have talked recently and worked out the syntax that is
consistent between both proposals. That takes 'fn' off the table for
short-functions, for reasons discussed in both RFCs.

To that end, I offer:

1. The updated short-functions RFC:
   https://wiki.php.net/rfc/short-functions

2. A new RFC, code by Nuno, for auto-capturing multi-statement closures:
   https://wiki.php.net/rfc/auto-capture-closure

These are separate RFCs and at this time we plan to send them to separate
votes, but I list them together to demonstrate that they have been
developed in a mutually-compatible way.

Further details in the auto-capture-closure RFC.

Hey Larry,

I'm generally in favor of supporting auto-capture for multi-line closures.
I think that extensive experience in other programming languages has shown
that auto-capture does not hinder readability of code, and I don't see any
particular reason why the effects in PHP would be different. I also think
there is value in having a consistent syntax for closures -- currently, you
have two options, and may need to switch between them as you add or remove
code.

There are some caveats though, which this RFC should address:

1. Auto-capture in arrow functions works by-value. This is sufficient for
   the purpose of single-line arrow functions, where you would be hard-pressed
   to perform any meaningful state mutation. For multi-line closures, it
   becomes much more likely that you want to modify a value from the outer
   scope. It should be possible to do this without switching to the function()
   notation.

I do believe that by-value capture is the correct default, and should
remain as such, but there should be an opt-in for by-reference capture. My
suggestion would be to allow use(&$count) on fn(), which allows capturing
certain variables by reference rather than by value. An alternative that
has previously been discussed is to allow changing the default capture mode
using something like use(&), but I believe it would be better if variables
captured by reference had to be spelled out explicitly.

2. For much the same reason, capture analysis for arrow functions is very
   primitive. It basically finds all variables that are used inside the
   closure body and tries to import them, silently failing if they are not
   available in the outer scope. For multi-line closures, this would commonly
   end up importing too many variables. For example:

   fn() {
   $tmp = foo();
   bar($tmp);
   return $tmp;
   }

Here, there is no need to import $tmp from the outer scope, but a naive
implementation (which I assume you're using) will still try to capture it.
Now, this has two effects:

a) Performance, as we're trying to capture unnecessary variables. This may
be negligible, but it would be good to at least quantify. I would rather
there not be recommendations along the line of "you should use function()
for performance-critical code, because it's faster than fn()".

b) Subtle side-effects, visible in debugging functionality, or through
destructor effects (the fact that a variable is captured may be
observable). I think it nothing else, the RFC should at least make clear
that this behavior is explicitly unspecified, and a future implementation
may no longer capture variables where any path from entry to read passes a
write.

Regards,
Nikita

Hey Nikita,

Concerning point 1: Agreed. Going to update both the pull request and the
RFC accordingly.

Concerning point 2: Developing some sort of static analysis that can infer
what needs to be captured may be problematic. First, that won't be
consistent with the way auto-capture currently works in PHP (one-line short
closures). Second, what needs to be captured may be determined at runtime
(thinking about eval, variables names computed at runtime, etc). Of course,
I am willing to hear more thoughts on this.

Regards,
Nuno.

Hi,

Thank you for the feedback on this thread so far. Programmers are
opinionated, therefore you will find developers that see themselves using
this RFC and others who don't. Just like "Fibers", "Type-hints", or any
other feature in the language really.

The reason why we believe the vast majority of PHP Developers are going to
appreciate this RFC is because multi-line short closures (aka
Auto-capturing multi-statement closures) *are more simple, shorter to
write, and prettier to read *— and the community love these changes as
proven on "property promotions", "one-line short closures", etc. [1]

Besides, one advantage of this RFC is that it is consistent with the
current syntax of the language and with the short-functions RFC[2]. For
example, by proposing that "fn" keyword indicates a function will
auto-capture variables, by value. And, the "=>" sigil always means
"evaluates to the expression on the right" in all circumstances - one-line
short closures, named functions, arrays, and match() already follow these
patterns. [3]

On the other hand "use (*)" has no usages / or current meaning in the
language.

Thanks,
Nuno.

[1] Pretty much "Brent Roose" have said here:
https://externals.io/email/112010/source
[2] https://wiki.php.net/rfc/short-functions
[3] Check "Auto-capture multi-statement closures" -
https://wiki.php.net/rfc/auto-capture-closure.

Actually, there is a way to declare a local variable. Use the long function syntax instead as there is no proposal to deprecate it.

That's not quite what I meant. I meant that you can't say "capture by default, but this variable is definitely local".

Yes I can see you want a way to declare a local in an auto-capturing fn(), but respectfully I do not see why using the function() syntax is not a viable alternate solution when and if you want to explicitly not capture a variable.

I said "want" because you will never need to as you can always use a variable name that is not in the outer scope.

And yes I know, someone may come along later and add a named variable in the outer scope that matches your implicit local variable's name and potentially break your code, but again if that is a concern why can't you just use function() instead of fn()? And wouldn't they actually be breaking their code (the outer code) if they did this?

Respectfully, you seem to be calling out a problem for which there is a tailor-designed solution explicitly recommended in the RFC. How can function() not address your stated concern?

I'm guessing maybe "$foo = null;" at the top of the function would stop it auto-capturing, but the semantics of what exactly gets captured aren't spelled out in the RFC. That really needs to be added IMO.

Instead of using array_filter() with a side-effect free closure, I use a for loop because it is easier to reason about visually.

Is that a bad thing? In many cases it's probably more efficient, and easier to read.

Depends on what your priorities are. If you priorities are coding in the absolute most efficient manner and/or with the most clarity, no it is probably not a bad thing.

But if your priorities are coding in the most robust manner where your code is free from side-effects by design — in a functional style — and the performance difference is insignificant, then yes it is a bad thing.

And at least one of the pair of RFCs spell out this motivation to allow for more functional style explicitly.

It is not clear to me from reading the PEP how a "with" statement obviates the benefit of variable auto-capture? Maybe because I don't "think" in Python, so from reading their examples I cannot see how this relates?

I didn't mean to say that the "with" statement would replace closures in general, but it would make them unnecessary for some specific use cases.

Respectfully again, to argue a use-case could be addressed in a different way than proposed by the RFC when that different way uses a hypothetical language feature that AFAIK does not have an RFC currently under consideration rings hollow to me.

Minimally I would expect you to at least first float the hypothetical on the list to see if there is appetite for it rather than just using is idea to snipe at an RFC that many people appear to support.

You mentioned:

prime examples are database transactions and using throw-aware buffering handlers.

Actually, I did not provide that example. Other people did.

What I stated was I had other use-cases that concerned me more than transactions, which I already explained in my email.

However, in no way do I mean to argue against their value for other developers who mentioned them.

If your current code looks something like this:

wrap_in_buffer(function($transaction) use ($to, $library, $thread, $author, $title, $library_name, $top_post) {
doSomething($to, $library, $thread);
andThenSomethingElse($author, $title, $library_name, $top_post);
});

That's a perfect use case for a Context Manager:

with ( wrap_in_buffer() ) {
doSomething($to, $library, $thread);
andThenSomethingElse($author, $title, $library_name, $top_post);
}

There's no callback, so no need to capture anything, and the implementation of all the try-catch-finally logic would be baked into the language, so the implementation of wrap_in_buffer() would also be much simpler.

But thanks for clarifying with examples, because I did ask for that.

That said, "with" is still hypothetical. Are you planning to champion its inclusion into PHP with a discussion and then follow up RFC?

-Mike

I feel like some of these examples would be better covered by command
object pattern and data-transaction objects, instead of FP idioms.
wrap_in_buffer pretty clearly so.

2021-03-25 10:28 GMT+01:00, Rowan Tommins rowan.collins@gmail.com:

Actually, there is a way to declare a local variable. Use the long
function syntax instead as there is no proposal to deprecate it.

That's not quite what I meant. I meant that you can't say "capture by
default, but this variable is definitely local".

I'm guessing maybe "$foo = null;" at the top of the function would stop
it auto-capturing, but the semantics of what exactly gets captured
aren't spelled out in the RFC. That really needs to be added IMO.

Instead of using array_filter() with a side-effect free closure, I use a
for loop because it is easier to reason about visually.

Is that a bad thing? In many cases it's probably more efficient, and
easier to read.

It is not clear to me from reading the PEP how a "with" statement obviates
the benefit of variable auto-capture? Maybe because I don't "think" in
Python, so from reading their examples I cannot see how this relates?

I didn't mean to say that the "with" statement would replace closures in
general, but it would make them unnecessary for some specific use
cases. You mentioned:

prime examples are database transactions and using throw-aware
buffering handlers.

If your current code looks something like this:

wrap_in_buffer(function($transaction) use ($to, $library, $thread,
$author, $title, $library_name, $top_post) {
doSomething($to, $library, $thread);
andThenSomethingElse($author, $title, $library_name, $top_post);
});

That's a perfect use case for a Context Manager:

with ( wrap_in_buffer() ) {
doSomething($to, $library, $thread);
andThenSomethingElse($author, $title, $library_name, $top_post);
}

There's no callback, so no need to capture anything, and the
implementation of all the try-catch-finally logic would be baked into
the language, so the implementation of wrap_in_buffer() would also be
much simpler.

Regards,

--
Rowan Tommins
[IMSoP]

--

To unsubscribe, visit: https://www.php.net/unsub.php

That's not quite what I meant. I meant that you can't say "capture by
default, but this variable is definitely local".

I think if there's one argument against, this would be it, but IMHO it
is a weakness in PHP as a whole.

The solution would be adding JS-like let / const statements. Which would
be a benefit to other things too.

On Thu, Mar 25, 2021 at 10:38 AM Olle Härstedt olleharstedt@gmail.com
wrote:

2021-03-25 15:23 GMT+01:00, Christian Schneider cschneid@cschneid.com:

Am 25.03.2021 um 14:29 schrieb Mark Randall marandall@php.net:

That's not quite what I meant. I meant that you can't say "capture by
default, but this variable is definitely local".

I think if there's one argument against, this would be it, but IMHO it
is
a weakness in PHP as a whole.

I'm not sure if I misunderstand what you're saying but to me it is one of
the greatest things about PHP that everything is local by default (minus
a
narrow set of well-known and easily enough recognizable things).

The solution would be adding JS-like let / const statements. Which would
be a benefit to other things too.

I disagree that this is the solution. I think JS had to add var and later
let because of the unfortunate decision to have C-like scoping rules.
Making scoping in PHP more complex to be able to repeat this mistake in
some
form seems ill-advised to me.

• Chris

You could make the change that all variables are now block-scoped,
not function-scoped. That would make auto-capture less dangerous, but
I don't know how much code it would break.

The BC implications would be huge, not to mention that it wouldn't be a
good idea anyway (at least in my opinion). The following would no longer
work:

if($check){
$match = true;
} else {
$match = false;
}

//stuff

if($match){...}

Yes, that could be done in a way that works with block scoping, but the
fact that PHP has never been blocked scoped means there would be a LOT of
code that would break if it was.

--

To unsubscribe, visit: https://www.php.net/unsub.php

--
Chase Peeler
chasepeeler@gmail.com

Hi,

Concerning the comments about what's exactly "auto-captured" by the scope
of a multi-line short closure, we will be just reusing the "auto-capture"
feature that already exists in one-line short closures. Therefore, this RFC
doesn't have plans on changing the way "auto-capture" already works in PHP.