Hi!

• I am a little confused about the OOP interaction. How does a function
  become a public method of the class?
  To clarify: the "public method" ist just the internal representation of
  the lambda function and has nothing to do with the semantics of
  calling the lambda itself. The "method" only means that the lambda
  function defined inside another method can access the class members and
  "public" only means that the lambda function can still be called from
  outside the class.

If one knew how to access it, which it seems is not possible/feasible for
user-space code.

No, that's not what I meant. The engine uses the following internal trick:

a) Upon copmilation, my patch simply adds the lambdas as normal
functions to the function table with an automatically generated
unique (!) name. If it happens to be defined within a class method,
the function will be added as a public final method to that class.

b) That added function is not directly callable due to checks of a flag
in the internal structure of that function.

c) At the place of the function definition the compiler leaves an
opcode "grab function $generatedname and make a closure out of it".
This opcode then looks up the generated lambda function, copies the
function structure, saves the bound variables in that structure and
returns the copied structure as a resource.

d) Normally, when a function is called, the name is looked up in the
function table. The function structure that is retrieved from there
is then used to execute the function. Since a lambda resource is
already a function structure, there is no necessity to look up
anything in the function table but the function structure can be
directly passed on to the executor.

Please note step d): The closure functionality only changes the lookup
of the function - so instead of getting the function structure from a
hash table lookup I get the function structure by retrieving it from the
resource. But after the lookup of a class method there are checks for
the access protection of that method. So these access protection checks
also apply to closures that were called. If a lambda function was not
declared public, it could not be used outside of the class it was
defined in. Perhaps this makes it clearer?

I see. It would be great if you could update the RFC with this information so
that it's clearer.

Done: http://wiki.php.net/rfc/closures

Two other questions that just occurred to me:

1. What is the interaction with namespaces, if any? Are lambdas as
   implemented here ignorant of namespace, or do they take the namespace where
   they are lexically defined?

My patch itself is namespace-ignorant, but the net result is not:

a) The generated internal function names do not contain the current
namespace name, but since namespace names in function names are only
used for lookup if you want to call the function. And calling
lambdas by name (!) directly doesn't work anyway (is not supposed
to work) so this poses no problem.

b) The code inside the closure is namespace-aware because the
information of which namespace is used is added at compile time.
Either the name lookup is done entirely at compile time or the
current compiler namespace is automatically added to all runtime
lookup calls (this is already the case with current code). So
the information which namespace a function resides in is currently
irrelevant at runtime when calling other functions.

For (b) let me make two examples:

Suppose you have the following code:

namespace Foo;

function baz () {
return "Hello World!\n";
}

function bar () {
return function () {
echo baz ();
};
}

and in another file:

$lambda = Foo::bar ();
$lambda ();

This will - as expected - print "Hello World!\n".

The reason is that the compiler upon arriving at the baz() function call
inside the closure already looks up the function in the function table
directly (it knows the current namespace) - and simply creates a series
of opcodes that will call the function with the name "Foo::baz" (the
lookup is already done at compile time).

Consider this other code:

foo-bar.php:

namespace Foo;

function bar () {
return function () {
echo baz ();
};
}

foo-baz.php:

namespace Foo;

function baz () {
return "Hello World!\n";
}

baz.php:

function baz () {
return "PHP!\n";
}

test1.php:

require 'foo-bar.php';
require 'foo-baz.php';

$lambda = Foo::bar ();
$lambda ();

test2.php:

require 'foo-bar.php';
require 'baz.php';

$lambda = Foo::bar ();
$lambda ();

Running test1.php yields "Hello World!" whereas running test2.php yields
"PHP!". Why is this? Because when the compiler reaches the baz ()
function call in the closure, it cannot find the function so it cannot
determine whether it's a function in global or in namespace scope. So it
will simply add a series of opcodes that say "try Foo::bar and if that
does not exist try bar". Here again, Foo:: is added by the compiler to
the opcode because the compiler has the necessary information which
namespace the call is currently in.

The runtime execution engine NEVER looks at the function as to which
namespace the function belongs to, it ONLY looks at the function calling
opcodes that the compiler already correctly (!) generates.

Please note that I have only described what the PHP engine does with
namespaces until now ANYWAY, the closures DO NOT CHANGE ANYTHING related
to this. That's why my patch doesn't have to care about namespaces at
all but the net-result closures will.

2. What happens with the following code?

class Foo {
private $a;
}

$f = new Foo();

$b = 5;

$f->myfunc = function($c) {
lexical $b;
print $a; // This generates an error, no?
print $b; // This prints 5, right?
print $c; // Should print whatever $c is.
}

$f->myfunc(3);

Or is the above a parse error entirely?

Well, first of all, it's a parse error because you forgot the ; after
the } of the closure. ;-)

Then: Closures only have access to the scope in which they are defined
in. Whether you assign that closure to a class property or not does not
change the semantics of the closure itself. The closure definition is
simply the following part: function ($c) { ... }. You then use the part
to store the closure as a class property. The closure you define is
entirely oblivious of the class since it is not defined inside a class
method. Consider this code:

function foo () {
echo $a;
}

$f->myfunc = 'foo';

Well, of course, first of all $a will not work anyway because $a has
never been the object member access method of PHP. But even if you use
$this->a, that won't work with foo either. For the exact same reasons it
will not work with closures.

Your comments inside the closure are correct though: $a gives an error
(notice: undefined variable), $b gives 5 (unless $b is modified later
on) and $c gives the current value of the parameter $c.

But: As I already said last time, $f->myfunc(3) won't work. That will
give "no such method". For the same reason as $f->myfunc = 'foo';
$f->myfunc(); will not work - because -> can only be used to call
METHODS, not dynamic functions stored in properties.

Regards,
Christian

I'm not Johannes and I didn't review the proposal in detail yet, but I
think we have enough for 5.3 right now. I'd think we better concentrate
on tying the loose ends and rolling beta out and then moving towards the
release than adding more and more features and never releasing it. 5.3
is not the final release until the end of times, there will be 5.4 etc.
and 6, so there will be ample opportunity to add stuff. And 5.3 has
enough stuff to be released, there's no rush to add more new things,
especially radically new ones. My opinion is that we better take some
time with it and not tie it to 5.3.

Amen to that.

• Steph

--
Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com

I'm not Johannes and I didn't review the proposal in detail yet, but I
think we have enough for 5.3 right now. I'd think we better concentrate on
tying the loose ends and rolling beta out and then moving towards the
release than adding more and more features and never releasing it. 5.3 is
not the final release until the end of times, there will be 5.4 etc. and 6,
so there will be ample opportunity to add stuff. And 5.3 has enough stuff to
be released, there's no rush to add more new things, especially radically
new ones. My opinion is that we better take some time with it and not tie it
to 5.3.

although I really want to have this functionality right now, I agree with Stas.
5.3 has a lot of new things already. this should go to HEAD, and,
hopefully, it will attract more people to actually try php-6

--
Alexey Zakhlestin
http://blog.milkfarmsoft.com/

Hello Stanislav,

nicely put but not in agreement with the PHP world. First we cannot add
a new feature like this in a mini release as it comes with an API change.
And second PHP is not anywhere close so we'd have to do it in a PHP 5.4
and personally I would like to avoid it.

marcus

Tuesday, June 17, 2008, 9:19:56 PM, you wrote:

Hi!

Johannes, what's your take on this one for 5.3?

I'm not Johannes and I didn't review the proposal in detail yet, but I
think we have enough for 5.3 right now. I'd think we better concentrate
on tying the loose ends and rolling beta out and then moving towards the
release than adding more and more features and never releasing it. 5.3
is not the final release until the end of times, there will be 5.4 etc.
and 6, so there will be ample opportunity to add stuff. And 5.3 has
enough stuff to be released, there's no rush to add more new things,
especially radically new ones. My opinion is that we better take some
time with it and not tie it to 5.3.

Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com

Best regards,
Marcus

Hi Markus, hi Stas,

Am Dienstag, den 17.06.2008, 12:19 -0700 schrieb Stanislav Malyshev:
[...]

I'm not Johannes and I didn't review the proposal in detail yet, but I
think we have enough for 5.3 right now. I'd think we better concentrate
on tying the loose ends and rolling beta out and then moving towards the
release than adding more and more features and never releasing it. 5.3
is not the final release until the end of times, there will be 5.4 etc.
and 6, so there will be ample opportunity to add stuff. And 5.3 has
enough stuff to be released, there's no rush to add more new things,
especially radically new ones. My opinion is that we better take some
time with it and not tie it to 5.3.

I would like to see 5.3 released first before we add really cool
features like this. I'm really +1 for closures but I please for 5.4 and

cu, Lars

The Lua interpreter handles this by resolving variable references
as they're made; "someVariable1" is looked up in the closure's
scope and not found, so the interpreter steps out one scope and
looks for it
You may get into a problem here - creator's scope may not exist when
you execute the closure, and using caller's scope would be very
unexpected - usually closures are intended to capture part of
creating environment, not calling environment. It would also impose
serious penalty if you just use undefined variable - you'd have to
go through whole stack up to the top.

This lookup happens at the time the closure is first declared, and the
value is stored for later use by the closure; the calling scope
doesn't need to exist anymore. The problem with going to the top of
the stack is an issue, though; the Lua interpreter's idea of "scope"
is rather different from PHPs, and it's not nearly the same penalty
there.

-- Gwynne, Daughter of the Code
"This whole world is an asylum for the incurable."

Hi,

[I'm going to collect here a bit:]

Stanislav Malyshev wrote:

lexical in the proposal binds to creator's scope, not caller's scope, as
I understood. Anyway, binding to caller's immediate scope doesn't seem
that useful since you could just pass it as a parameter when calling.

Correct and I completely agree.

Chris Stockton wrote:

I am curious if is_callable will be able to detect these?

Yes, as is call_user_func able to call them. (But as I was verifying
that I saw that there was a tiny bug in the code that makes sure the
internal names are not used directly, I will fix that.)

Richard Quadling wrote:

[JS Example]
I'm not sure I would say that there is a reference used there.

I'm no expert, but x, f() and n are all working like normal variables with x
having to look outside of f().

Unless I'm getting confused with "pass by reference".

Yes, shure, ok, it's not a reference in the classical sense BUT the
effect is the same: A change INSIDE the closure changes the variable
outside. The only useful way of doing that in PHP without rewriting the
complete engine is using references - and such things are already done
via references - namely global variables: If you import a variable via
$global, in reality you are creating a reference to the actual global
variable - global $foo is actually more or less the same as $foo =&
$_GLOBALS['foo'].

Regards,
Christian

Which brings me to another point - how bad would it be if closure's
lifetime would be limited to parent's lifetime? Of course, this would limit
some tricks, but would allow some other - like this direct access to
parent's scope.

that would be seriously bad, because it will eliminate possibility of
lambda-generating functions

--
Alexey Zakhlestin
http://blog.milkfarmsoft.com/

See below:

-----Original Message-----
From: Christian Seiler [mailto:chris_se@gmx.net]
Sent: Wednesday, June 18, 2008 1:14 PM
To: php-dev List
Subject: Re: [PHP-DEV] [PATCH] [RFC] Closures and lambda functions in PHP

Frankly, I don't really see a problem with using references. It fits
into what's already there in PHP and it assures that closures have the
necessary properties to make them useful.

I think you are right that there isn't really a good alternative as the "parent" scope does not necessarily exist anymore. Your solution is likely the best.

• Please check eval(). I assume it will bind to global scope but
  let's just make sure what happens esp. when it's called from within
  a method...

Hmm, closures inside eval() will bind variables to the scope in which
eval() was called. But closures defined inside eval will NOT be class
methods, even if eval() is called within a class.

But I do find that behaviour consistent with what PHP currently does
with normal functions and variables: If eval()'d or include()'d inside a
function, variables will the "global scope" of eval() or the included
file will actually be the local function scope whereas defined functions
inside will automatically become global functions.

Of course, this behaviour should be documented but I don't see a reason
to try and change it.

I agree. It behaves as I would expect I just wanted to make sure you verify that because I didn't have the opportunity to do so. You'd actually have to work very hard for it not to behave in that way :) Let's just make sure we have unit tests for both cases just so we have a good regression on this one.

• In PHP 5, object storage is resources done right. I don't think
  we should be using the resource infrastructure for this
  implementation and would prefer to use the object one. It's better.
  I suggest to take a look at it.

Hmm, seems like a good idea. If nobody objects in the next few days,
I'll rewrite my patch to use objects instead of resources. What class
name do you suggest?

Great. I think Closure is probably a good name.
[Btw, if we want to get fancy we could even have a __toString() method on those which would print out information about the Closure. But this is not a must, just something which eventually could be nice for debugging purposes...]

PS: Somebody made me aware of a segfault in my code when destroying the
closure variable while still inside the closure. I'll fix that.

:)

Thanks,
Andi

First, a comment from haskell-land:
http://www.haskell.org/pipermail/haskell-cafe/2008-June/044533.html
http://www.haskell.org/pipermail/haskell-cafe/2008-June/thread.html#44379

Frankly, I don't really see a problem with using references. It fits
into what's already there in PHP and it assures that closures have the
necessary properties to make them useful.

References are necessary, but an easy way to obtain copies of variables from
the lexical context would be really nice.

I have been introduced to functional programming through Haskell, where values
are immutable, so a reference is basically the same as a copy. I like this
behaviour because it makes closures distinctly non-dangerous by default.

Getting the same behaviour out of PHP should not be as difficult as this:
for ($i = 0; $i < 10; $i++) {
$loopIndex = $i;
$arr[$i] = function () { lexical $loopIndex; return $loopIndex; };
unset ($loopIndex);
}
This is not only quite a hassle (making beer much cheaper than water, so to
speak), I also believe it to be error-prone. A lot of programmers are going
to forget that unset().

I would prefer something like this:
for ($i = 0; $i < 10; $i++) {
$arr[$i] = function () { lexical_copy $i; return $i; };
}

An alternative would be to let lexical behavie like function parameters:

• copies by default
  lexical $x;
• objects referenced by default
  lexical $obj;
• other references optional
  lexical &$y;

Of course this would make lexical behave quite differently from global in this
regard, decreasing consistency, but f*ck global, nobody should use that
anyway. Better to have nice lexical closures.

Gesundheit
Wag

--
Be careful about reading health books. You may die of a misprint.

• Mark Twain

Hi!

Hmm, seems like a good idea. If nobody objects in the next few days,
I'll rewrite my patch to use objects instead of resources. What class
name do you suggest?

While we are at it maybe even having special standard handler
(__invoke?) that could be also used by objects created by reflection and
maybe later of some other purposes. I.e. if we do $foo($bar, $baz) and
$foo is an object and it defines __invoke, then we call it (in which
case if $foo is Closure it does its thing) otherwise we get an error
"object $foo is not callable". Of course, this goes also for
is_callable, etc.
What do you think?

Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com

Hi,

Rodrigo Saboya wrote:

function ($x, $y) ($a, $b, $c) {};

This looks better

function ($x, $y) [$a, $b, $c] {};

I think this looks even better:

function ($x, $y) use ($a, $b, &$c) {};

(one could use this syntax even for traditional functions to
use variable copies/references from global scope - just an idea).

my 2,
L.

No it won't.

Dmitry.

Lars Strojny wrote:

Hi Dmitry, hi Christian,

Am Freitag, den 20.06.2008, 15:12 +0400 schrieb Dmitry Stogov:

$func = function ($x, $y) use $a, $b, $c {
}

Will lexical scoping work with normal ("named") functions too?

function foo($x, $y) use $a, $b, $c {
}

cu, Lars

Hello Christian,

Thursday, June 26, 2008, 6:23:53 PM, you wrote:

Hi Dmitry,

I'm fine if you'll improve my patch (It's mainly yours :)

I updated my closures RFC: http://wiki.php.net/rfc/closures

I have based my new version of the patch on yours (Dmitry), but I made
some changes to that:

• Objects instead of resources are used, two new files
  zend_closures.[ch] are added where the new Closure class
  is defined. Currently, it contains a dummy __toString method
  that in future may be extended to provide enhanced debugging info,
  also further additional cool stuff could be added to such a
  class later on. But I prefer to only add the basic closure
  functionality at first - you can always extend it once it's there.

• I have not added any __invoke() magic to normal objects. This is
  mainly due to the simple reason that adding that would not help
  a closure implementation at all. Closures need some engine internal
  magic (use a dynamically created op_array instead of looking one up,
  setting the correct class scope and setting the correct EG(this). And
  as I said: I want to stick with the closure basics for now.

That said, I do like the possibility of invoking objects directly, so
I suggest someone created an additional proposal for that?

• I've added a patch for PHP HEAD (PHP 6.0). This is due to the fact
  that Dmitry's variant of my patch has far less intersections with
  the unicode functionality than my original patch, so it was quite
  straight-forward to do so.

• Lexical vars are now copied instead of referenced by default. Using
  & in front of the var, the behaviour may be changed. I added that in
  order to demonstrate that both was possible and that a simply change
  of grammar suffices. In my eyes this is the main issue where a
  discussion has to take place (i.e. copy or reference by default?
  possibility to change default via syntax? which lexical syntax?)
  before the proposal can be accepted.

• I provided patches for both lexical $var and use ($var) syntaxes.

• I provided a patch variant that only stores $this if $this is
  explicitely used inside a closure (or a nested closure of that
  closure). This works since it is possible to detect whether $this
  is used at compile time. For this, I have added a this_used flag
  to the op_array structure.

• I added tests (Zend/tests/closures_*.phpt) that ensure the correct
  behaviour of closures.

[Note that I created my own local SVN repos for developing these patches
because I was fed up with CVS's inability to local diffs and locally
mark files as added to include them in the diffs. Just to explain the
format of the patch.]

Anyway, feel free to discuss.

In my eyes, the following questions should be answered:

• Do you want closures in PHP?

I have not seen a single negative reaction to my proposal, so I
assume the answer to that is yes. ;-)

yes

• Which syntax should be used for lexical variables? Should references
  or copies be created by default?

This is far trickier.

First of all: There must *always* be the _possiblity_ to create
references, else you can't call it closures.

Second: I prefer the 'lexical' keyword, but I could live with the
use solution [function () use ($...)].

'use' becasue no new keyword has to be introduced which would brake stuff
no matter what the keyword will be.

Third: There are several arguments against default referencing:

  * (use syntax) As they look similar to parameters and normal
                 parameters aren't passed by ref, this could be quite
                 odd.
  * Loop index WTFs (see proposal)
  * Speed? (You always read that refs are slower in PHP than normal
    variable copies. Is that actually true?)
  * While it is possible to simply add an & in front of the variable
    name to switch to refs in "no refs default" mode, there is no
    obvious syntax to use copies in "refs default" mode other than
    unsetting the variable in the parent scope immediately after
    closure creation.

I like the new ability to reference if wanted. But then I don't like
references at all. Along with the fact that in PHP objects are always
references I slightly tend to not want reference functionality. Since it
is handled in the parser you could submit with either version and check
during evaluation periode if people disagree with your choice - or the
list choice if that's what decides.

Fourth: There are several arguments for default referencing:

  * (lexical syntax) global also creates a reference, why shouldn't
                     lexical?
  * Other languages *appear* to exhibit a very similar behaviour to
    PHP if PHP created references. This is due to the fact that
    other languages have a different concept of scope as PHP
    does.

Although the list of against arguments appears to be longer, I do
prefer using references by default nevertheless. But that's just
my personal opinion.

• Are you OK with the change that $this is only stored when needed?

I don't see a problem. Dmitry seems to be very touchy (;-)) about
changing op_arrays but in this case it's only a flag so I don't
see a problem for opcode caches (in contrast to a HashTable where
the opcode cache must actually add code to duplicate that table).

I see it dangerous.... eval comes to mind. And also, why create something
special when the normal way of doing things that is done everywhere else
would work too.

• Do you want closures in PHP 5.3?

Since the majority of core developers appear to be against it, I
presume the answer is no.

I am still in favor it and against a 5.4.

I will provide a revised patch that incorporates the results of the
following discussion for 5_3 and HEAD once consensus or at least a
majority regarding the remaining issues is reached. I will also rewrite
the proposal to reflect the discussion results and adjust the tests.
After that, I hope that someone will commit at least the HEAD version.

Comments on the first patch version:

• coooool, I am the listed author: Zend/zend_closures.c/h

• you shouldn't be having a __destruct. Can you prevent that?

• please drop __toString, with the new behavior only stuff that has
  something to say in a string context should have a __toString

• a tiny optimization:
  +ZEND_API zend_closure *zend_get_closure(zval obj TSRMLS_DC) / {{{ */
  +{

•   zend_class_entry *ce = Z_OBJCE_P(obj);
  

•   if (instanceof_function(ce, zend_ce_closure TSRMLS_CC)) {
  

•           zend_closure *closure = (zend_closure *)zend_object_store_get_object(obj);
  

•           if (closure->initialized) return closure;
  

•   }
  

•   return NULL;
  

+}

• a faster way would be to:
  a) add a new type (probably not so good though)
  b) add a new ce_flag
  +#define ZEND_ACC_CLOSURE ...
  +ZEND_API zend_closure *zend_get_closure(zval obj TSRMLS_DC) / {{{ */
  +{

•   zend_class_entry *ce = Z_OBJCE_P(obj);
  

•   if ((ce->ce_flags & ZEND_ACC_CLOSURE) != 0) {
  

•           zend_closure *closure = (zend_closure *)zend_object_store_get_object(obj);
  

•           if (closure->initialized) return closure;
  

•   }
  

•   return NULL;
  

+}

• maybe even inline this function?

• very high quality work! thanks a lot! very well thought out and nicely
  tested even

Best regards,
Marcus

• I added tests (Zend/tests/closures_*.phpt) that ensure the correct
  behaviour of closures.

I'd like to propose an additional test to ensure closures can all themselves:
<?php
$i = 3;

$lambda = function ($lambda) use ($i) {
if ($i==0) return;
echo $i--."\n";
$lambda($lambda);
};

$lambda($lambda);
echo "$i\n";
?>
Expected output:
3
2
1
3

I see exactly one problem with the patch, which is that the above script
shouldn't work without "use (&$i)".
I find it counterintuitive that the creation of the lambda creates a copy of
$i, but all invocations of $lambda use a reference to the same $i.
For n calls to $lambda, there are only 2 copies of $i (one global, one static
in $lambda) where I would expect n+1 copies.

Gesundheit
Wag

--
Sieh dich, nimm Sorge.

Hey Christian,

Nice job!

More below:

-----Original Message-----
From: Christian Seiler [mailto:chris_se@gmx.net]
Sent: Thursday, June 26, 2008 9:24 AM
To: Dmitry Stogov
Cc: php-dev List; Andi Gutmans; Stas Malyshev
Subject: Re: [PHP-DEV] [PATCH] [RFC] Closures and lambda functions in PHP

-snip-

• Lexical vars are now copied instead of referenced by default. Using
  & in front of the var, the behaviour may be changed. I added that in
  order to demonstrate that both was possible and that a simply change
  of grammar suffices. In my eyes this is the main issue where a
  discussion has to take place (i.e. copy or reference by default?
  possibility to change default via syntax? which lexical syntax?)
  before the proposal can be accepted.

I think doing this mixed approach is the right one, i.e. $var1, &$var2. It delivers choice and adds clarity as it's explicit about value vs. reference.

• I provided patches for both lexical $var and use ($var) syntaxes.

I lean towards the use(...) syntax.

• I provided a patch variant that only stores $this if $this is
  explicitely used inside a closure (or a nested closure of that
  closure). This works since it is possible to detect whether $this
  is used at compile time. For this, I have added a this_used flag
  to the op_array structure.

Safest not to take shortcuts. You get yourself into trouble with things which will stop working. -1 on this optimization.

• I added tests (Zend/tests/closures_*.phpt) that ensure the correct
  behaviour of closures.

Excellent.

Anyway, feel free to discuss.

In my eyes, the following questions should be answered:

• Do you want closures in PHP?

I think most people here feel it's useful or are at least indifferent.

• Do you want closures in PHP 5.3?

I really think we need to get the release process for 5.3 going so I suggest to do it for the following minor version and commit it to PHP 6.

Andi

On Thursday 26 June 2008 11:23:53 am Christian Seiler wrote:

Hi Dmitry,

I'm fine if you'll improve my patch (It's mainly yours :)

I updated my closures RFC: http://wiki.php.net/rfc/closures
In my eyes, the following questions should be answered:

• Do you want closures in PHP?

  I have not seen a single negative reaction to my proposal, so I
  assume the answer to that is yes. ;-)

Yea. :-)

• Which syntax should be used for lexical variables? Should references
  or copies be created by default?

  This is far trickier.

  First of all: There must always be the possiblity to create
  references, else you can't call it closures.

  Second: I prefer the 'lexical' keyword, but I could live with the
  use solution [function () use ($...)].

  Third: There are several arguments against default referencing:

  • (use syntax) As they look similar to parameters and normal
    parameters aren't passed by ref, this could be quite
    odd.
  • Loop index WTFs (see proposal)
  • Speed? (You always read that refs are slower in PHP than normal
    variable copies. Is that actually true?)
  • While it is possible to simply add an & in front of the variable
    name to switch to refs in "no refs default" mode, there is no
    obvious syntax to use copies in "refs default" mode other than
    unsetting the variable in the parent scope immediately after
    closure creation.

  Fourth: There are several arguments for default referencing:

  • (lexical syntax) global also creates a reference, why shouldn't
    lexical?
  • Other languages appear to exhibit a very similar behaviour to
    PHP if PHP created references. This is due to the fact that
    other languages have a different concept of scope as PHP
    does.

  Although the list of against arguments appears to be longer, I do
  prefer using references by default nevertheless. But that's just
  my personal opinion.

I see these two issues as related, actually. Consider:

$foo = function($a, &$b) {
global $c;
lexical $d;
// ...
}

Since they look the same, you'd expect them to behave the same. However,
global will import by reference and lexical by value. Hilarity ensues, and
not the good kind. Naturally changing the behavior of global in this case is
out of the question, and as you point out defaulting to reference and having
an extra flag (a la &) to force it to value is unprecedented. I think most
seem to agree that being able to pass by value or by reference at the
developer's discretion is necessary.

However, something in the function signature itself would naturally follow the
behavior of function arguments:

$foo = function($a, &$b) lexical ($d, &$e) {
global $c;
// ...
}

Here, $d and $e behave "as you'd expect them to", with the same visual parsing
semantics as $a and $b. That to me is a much lower wtf factor than having
global and lexical keywords that look alike but behave differently. I would
therefore favor the signature-based syntax.

As an aside, I did use "lexical" in the second example above deliberately.
Personally I don't think re-using "use" here is wise, as that will make it
seem namespace related when in fact it is not. I suspect the instances of
function or constant names of "lexical" will be pretty minimal (although I
admit to having no evidence to back up that suspicion.)

(I would much much rather have a closures implementation that used "use" than
not one at all, mind you; I will still jump for joy if it lands using "use",
I just think it would be better using "lexical".)

• Are you OK with the change that $this is only stored when needed?

Ignoring the compiler-level concerns, about which I know nothing useful,
doesn't this introduce a bit of wtf? "All lexically-imported variables must
be explicit or they don't exist... oh yeah, except for $this because it's
special."

To which the question is: If you are able to magically determine if $this is
used and optimize accordingly, why can't you for anything else?

To which the response is, I think: "Performance". That's still something of
an inconsistency, however. Would it make the engine code any cleaner/messier
if $this was required to be declared explicitly like everything else?

• Do you want closures in PHP 5.3?

As a PHP developer I'd love to have closures in 2 years when I'm able to use
PHP 5.3 instead of 5 years when I'm able to use PHP 5.4 or PHP 6. :-) I do
understand the need to draw a line somewhere and justbloodyshipit(tm),
however, so if that's the decision I can accept that.

Regards,
Christian

You so rock. :-)

--
Larry Garfield
larry@garfieldtech.com

Hi Christian,

I reworked your patch a little bit.

1. Fixed ref/noref issues

2. Explicit $this passing

$a = function foo() use ($this) {}

3. Some code reorganization to encapsulate closure implementation.

Thanks. Dmitry.

Christian Seiler wrote:

Hi Dmitry,

I'm fine if you'll improve my patch (It's mainly yours :)

I updated my closures RFC: http://wiki.php.net/rfc/closures

I have based my new version of the patch on yours (Dmitry), but I made
some changes to that:

• Objects instead of resources are used, two new files
  zend_closures.[ch] are added where the new Closure class
  is defined. Currently, it contains a dummy __toString method
  that in future may be extended to provide enhanced debugging info,
  also further additional cool stuff could be added to such a
  class later on. But I prefer to only add the basic closure
  functionality at first - you can always extend it once it's there.

• I have not added any __invoke() magic to normal objects. This is
  mainly due to the simple reason that adding that would not help
  a closure implementation at all. Closures need some engine internal
  magic (use a dynamically created op_array instead of looking one up,
  setting the correct class scope and setting the correct EG(this). And
  as I said: I want to stick with the closure basics for now.

  That said, I do like the possibility of invoking objects directly, so
  I suggest someone created an additional proposal for that?

• I've added a patch for PHP HEAD (PHP 6.0). This is due to the fact
  that Dmitry's variant of my patch has far less intersections with
  the unicode functionality than my original patch, so it was quite
  straight-forward to do so.

• Lexical vars are now copied instead of referenced by default. Using
  & in front of the var, the behaviour may be changed. I added that in
  order to demonstrate that both was possible and that a simply change
  of grammar suffices. In my eyes this is the main issue where a
  discussion has to take place (i.e. copy or reference by default?
  possibility to change default via syntax? which lexical syntax?)
  before the proposal can be accepted.

• I provided patches for both lexical $var and use ($var) syntaxes.

• I provided a patch variant that only stores $this if $this is
  explicitely used inside a closure (or a nested closure of that
  closure). This works since it is possible to detect whether $this
  is used at compile time. For this, I have added a this_used flag
  to the op_array structure.

• I added tests (Zend/tests/closures_*.phpt) that ensure the correct
  behaviour of closures.

[Note that I created my own local SVN repos for developing these patches
because I was fed up with CVS's inability to local diffs and locally
mark files as added to include them in the diffs. Just to explain the
format of the patch.]

Anyway, feel free to discuss.

In my eyes, the following questions should be answered:

• Do you want closures in PHP?

  I have not seen a single negative reaction to my proposal, so I
  assume the answer to that is yes. ;-)

• Which syntax should be used for lexical variables? Should references
  or copies be created by default?

  This is far trickier.

  First of all: There must always be the possiblity to create
  references, else you can't call it closures.

  Second: I prefer the 'lexical' keyword, but I could live with the
  use solution [function () use ($...)].

  Third: There are several arguments against default referencing:

  • (use syntax) As they look similar to parameters and normal
    parameters aren't passed by ref, this could be quite
    odd.
  • Loop index WTFs (see proposal)
  • Speed? (You always read that refs are slower in PHP than normal
    variable copies. Is that actually true?)
  • While it is possible to simply add an & in front of the variable
    name to switch to refs in "no refs default" mode, there is no
    obvious syntax to use copies in "refs default" mode other than
    unsetting the variable in the parent scope immediately after
    closure creation.

  Fourth: There are several arguments for default referencing:

  • (lexical syntax) global also creates a reference, why shouldn't
    lexical?
  • Other languages appear to exhibit a very similar behaviour to
    PHP if PHP created references. This is due to the fact that
    other languages have a different concept of scope as PHP
    does.

  Although the list of against arguments appears to be longer, I do
  prefer using references by default nevertheless. But that's just
  my personal opinion.

• Are you OK with the change that $this is only stored when needed?

  I don't see a problem. Dmitry seems to be very touchy (;-)) about
  changing op_arrays but in this case it's only a flag so I don't
  see a problem for opcode caches (in contrast to a HashTable where
  the opcode cache must actually add code to duplicate that table).

• Do you want closures in PHP 5.3?

  Since the majority of core developers appear to be against it, I
  presume the answer is no.

I will provide a revised patch that incorporates the results of the
following discussion for 5_3 and HEAD once consensus or at least a
majority regarding the remaining issues is reached. I will also rewrite
the proposal to reflect the discussion results and adjust the tests.
After that, I hope that someone will commit at least the HEAD version.

Regards,
Christian

Hi Christian,

thanks again for your (and Dmitry's) great work on making closures a
part of PHP.

Am Donnerstag, den 26.06.2008, 18:23 +0200 schrieb Christian Seiler:

• I have not added any __invoke() magic to normal objects. This is
  mainly due to the simple reason that adding that would not help
  a closure implementation at all. Closures need some engine internal
  magic (use a dynamically created op_array instead of looking one up,
  setting the correct class scope and setting the correct EG(this). And
  as I said: I want to stick with the closure basics for now.

I understand that you want to keep your proposal basic. However I have
the feeling that we need a complete implementation to make closures
really beneficial for our users. I would consider the following features
to be central for a feature complete implementation:

  * Class::__invoke() to allow functors[1]. The class "Closure" in
    your proposal should also implement that method to make
    `method_exists()` and ext/reflection behave.
  * Change the behaviour how method calls are resolved. Method calls
    on invokable objects (closures or functors) should work.

cu, Lars

[1] http://en.wikipedia.org/wiki/Functor
[2] Example for a closure assigned to a property:
class View();
{
public $escape;
}

$view = new View();
$view->escape = function($string) {
return htmlentities($string, ENT_QUOTES, 'UTF-8');
}

echo $view->escape("<script>alert(1)</script>");