Reading the prior discussion, I think either $_SCOPE['x'] or the
lexical $x syntax is fine for accessing local variables in the
enclosing scope. But closures also should also support $this and
static:: when the closure is defined in a method.

While I realise that the possibilities of static scope in PHP, are
intriguing, I really think it's a whole different discussion. Related,
yes. Interesting, yes. But let's keep it separate, to avoid derailing
the whole discussion.

While we are at it, what's wrong with knowing the name? I can see why
closure can be fun when you can dynamically use outer-scope variables.
But when you can't, what exactly prevents one from just doing the function?

If the code block, which needs to create the function, could be run
more than once, it would try to define the function twice, leading to
a fatal error. Yes, I could guard it with is_function_defined(), but
that wouldn't be pretty. Technically speaking, you're right, but I
think aesthetics does have some weight. Or should. It's also my main
gripe against create_function() in the first place.

First, about 100% on the first encounter for any user ever seeing
closures in any other language. Second, all the confusion possible, like

So how big a part of PHP's userbase is that? I'm guessing, it's small.

closures in any other language. Second, all the confusion possible, like
"oh, closures! cool! let me do this and that! what, I can't use
variables?! Are you kidding me?! WTF is it useful for?!"

True, but the people who will anguish over lack of closures, are
already tearing their hair out over create_function().

I'm not sure I understand what you mean by "static scope" and "dynamic
scope", ...

Static scope is a synonym for lexical scope.

scope", but anyway thing that looks like closure but works like regular
function definition that isn't - is not a very good idea, IMO.

Yes, that's the primary problem with the patch. I don't think it's as
big an issue, as you do and given the alternatives, I'm willing to
make that sacrifice. How's other peoples take on this point?

--
troels

Christian Seiler wrote:

Hi,

I was following this thread and came upon Jeff's posting on how closures
could be implemented in PHP.

Hi,

typo alert:

Index: Zend/zend_vm_def.h

RCS file: /repository/ZendEngine2/zend_vm_def.h,v
retrieving revision 1.59.2.29.2.48.2.25
diff -u -r1.59.2.29.2.48.2.25 zend_vm_def.h
--- Zend/zend_vm_def.h 14 Dec 2007 14:14:50 -0000 1.59.2.29.2.48.2.25
+++ Zend/zend_vm_def.h 22 Dec 2007 13:52:44 -0000
@@ -1,4 +1,4 @@
-/*
+f/*

Very impressive patch, I'll be interested to try it out when I get a chance.

Greg

One question about the names you generate for the function table in
combination with opcode caches.

Let's assume I have APC installed, and do the following:

foo.php:
$foo = function() {
echo "foo";
}

bar.php:
include('foo.php');

All works fine, and cached versions of both files would be created.

However, if I now change bar.php to this:
$bar = function() {
echo "bar";
}
include('foo.php');

wouldn't I get a "cannot redefine function __compiled_lamda_0" fatal,
if foo.php was loaded from the opcode cache?

While this is a constructed example, it could easily occur with
conditional includes with environments that use opcode caches.

Or am I completely wrong? :p

• David

Am 22.12.2007 um 16:08 schrieb Christian Seiler:

Hi,

I was following this thread and came upon Jeff's posting on how
closures
could be implemented in PHP.

Since I would find the feature to be EXTREMELY useful, I decided to
actually implement it more or less the way Jeff proposed. So, here's
the
patch (against PHP_5_3, I can write one against HEAD if you whish):

http://www.christian-seiler.de/temp/closures-php-5-3.patch

I started with Wez's patch for adding anonymous functions that aren't
closures. I changed it to make sure no shift/reduce or reduce/reduce
error occur in the grammar. Then I started implementing the actual
closure stuff. It was fun because I learned quite a lot about how PHP
actually works.

I had the following main goals while developing the patch:

1. Don't reinvent the wheel.
2. Don't break anything unless absolutely necessary.
3. Keep it simple.

Jeff proposed a new type of zval that holds additional information
about
the function that is to be called. Adding a new type of zval would
need
changes throughout the ENTIRE PHP source and probably also throughout
quite a few scripts. But fortunately, PHP already posesses a zval that
supports the storage of arbitrary data while being very lightweight:
Resources. So I simply added a new resource type that stores zend
functions. The $var = function () {}; will now make $var a resource
(of
the type "anonymous function".

Anonymous functions are ALWAYS defined at compile time, no matter
where
they occur. They are simply named __compiled_lamda_1234 and added to
the
global function table. But instead of simply returning the string
'__compiled_lambda_1234', I introduced a new opcode that will create
references to the correct local variables that are referenced inside
the
function.

For example, if you have:

$func = function () {
echo "Hello World\n";
};

This will result in an anonymous function called '__compiled_lambda_0'
that is added to the function table at compile time. The opcode for
the
assignment to $func will be something like:

    1  ZEND_DECLARE_ANON_FUNC     ~0      '__compiled_lambda_0'
    2  ASSIGN                             !0, ~0

The ZEND_DECLARE_ANON_FUNC opcode handler does the following:

It creates a new zend_function, copies the contents of the entire
structure of the function table entry corresponding to
'__compiled_lamda_0' into that new structure, increments the refcount,
registeres it as a resource and returns that resource so it can be
assigned to the variable.

Now, have a look at a real closure:

$string = "Hello World!\n";
$func = function () {
lexical $string;
echo $string;
};

This will result in the same opcode as above. But here, three
additional
things happen:

1. The compiler sees the keyword 'lexical' and stores the information,
   that a variable called 'string' should be used inside the closure.

2. The opcode handler sees that a variable named 'string' is marked as
   lexical in the function definition. Therefore it creates a reference
   to
   it in a HashTable of the COPIED zend_function (that will be stored in
   the resource).

3. The 'lexical $string;' translates into a FETCH opcode that will
   work
   in exactly the same way as 'static' or 'global' - only fetching it
   from
   the additional HashTable in the zend_function structure.

The resource destructor makes sure that the HashTable containing the
references to the lexical veriables is correctly destroyed upon
destruction of the resource. It does NOT destroy other parts of the
function structure because they will be freed when the function is
removed from the global function table.

With these changes, closures work in PHP.

Some caveats / bugs / todo:

• Calling anonymous functions by name directly is problematic if there
  are lexical variables that need to be assigned. I added checks to
  make sure this case does not happen.

• In the opcode handler, error handling needs to be added.

• If somebody removes the function from the global function table,
  (e.g. with runkit), the new opcode will return NULL instead of
  a resource (error handling is missing). Since I do increment
  refcount of the zend_function, it SHOULD not cause segfaults or
  memory leaks, but I haven't tested it.

• $this is kind of a problem, because all the fetch handlers in PHP
  make sure $this is a special kind of variable. For the first version
  of the patch I chose not to care about this because what still works
  is e.g. the following:

  $object = $this;
  $func = function () {
    lexical $object;
    // do something
  };
  

  Also, inside the closures, the class context is not preserved, so
  accessing private / protected members is not possible.

  I'm not sure this actually represents a problem because you can
  always use normal local variables to pass values between closure
  and calling method and make the calling method change the
  properties
  itself.

• I've had some problems with eval(), have a look at the following
  code:

  $func = eval ('return function () { echo "Hello World!\n"; };');
  $func();
  

  With plain PHP, this seems to work, with the VLD extension loaded
  (that shows the Opcodes), it crashes. I don't know if that's a
  problem with eval() or just with VLD and I didn't have time to
  investigate it further.

• Oh, yes, 'lexical' is now a keyword. Although I really don't think
  that TOO many people use that as an identifier, so it probably won't
  hurt THAT much.

Except those above points, it really works, even with complex stuff.
Let
me show you some examples:

1. Customized array_filter:

function filter_larger ($array, $min = 42) {
$filter = function ($value) {
lexical $min;
return ($value >= $min);
};
return array_filter ($array, $filter);
}

$arr = array (41, 43);
var_dump (filter_larger ($arr)); // 43
var_dump (filter_larger ($arr, 40)); // 41, 43
var_dump (filter_larger ($arr, 44)); // empty

2. Jeff's example:

function getAdder($x) {
return function ($y) {
lexical $x;
return $x + $y;
};
}

$plusFive = getAdder(5);
$plusTen = getAdder(10);

echo $plusFive(4)."\n"; // 9
echo $plusTen(7)."\n"; // 17

3. Nested closures

$outer = function ($value) {
return function () {
lexical $value;
return $value * 2;
};
};

$duplicator = $outer (4);
echo $duplicator ()."\n"; // 8
$duplicator = $outer (8);
echo $duplicator ()."\n"; // 16

[Ok, yeah, that example is quite stupid and should NOT be used as an
example for good code. ;-) But it's simple and demonstrates the
possibilities.]

It would be great if somebody could review the patch because I'm shure
some parts can still be cleaned up or improved. And it would be even
better if this feature would make it into PHP. ;-)

Regards,
Christian

PS: I'd like to thank Derick Rethans for his GREAT Vulcan Logic
Disassembler - without it, developement would have been a LOT more
painful.

PPS: Oh, yeah, if it should be legally necessary, I grant the right to
anybody to use this patch under any OSI certified license you may want
to choose.

2007/12/22, Christian Seiler chris_se@gmx.net:

PPS: Oh, yeah, if it should be legally necessary, I grant the right to
anybody to use this patch under any OSI certified license you may want
to choose.

That's very kind of you but, if I was explained right, you don't have
copyright on a patch. If I understood the legalese correctly, you retain
authorship on the code provided, but this kind of derived work, which has no
purpose outside the original, cannot be claimed through copyright. You
specially did it for the php codebase, and has not a meaning outside it.

Consider what happens to the code you make to maintain an in-house
application of a company. You cannot claim copyright on that codebase, it
remains to the company. But, if you were to develop an app from scratch for
the same company, the company has the right to use it in-house, but if they
want to distribute it, they have to require your consent, because you're one
of the copyright holders.

Also consider what happens to reviewers that check and correct works from
other people, such as syntax and grammar checking in literature, or code
reviewing in software. They can claim authorship on their corrections, but
they cannot claim copyright.

Anyway, it's too complicate. The bottom line is that copyright cannot be
claimed on patches. Which makes complete sense, imagine what would happen to
open-source if it wasn't this way... the licensing documentation would
occupy more than the code itself...

2007/12/22, Christian Seiler chris_se@gmx.net:

Hi,

I was following this thread and came upon Jeff's posting on how closures
could be implemented in PHP.

Since I would find the feature to be EXTREMELY useful, I decided to
actually implement it more or less the way Jeff proposed. So, here's the
patch (against PHP_5_3, I can write one against HEAD if you whish):

http://www.christian-seiler.de/temp/closures-php-5-3.patch

Hi,

I finally got some time to check out your patch (the first and the second
version). It looks cool, works as expected in most of the things I've tried.

Nevertheless, I don't need this feature, it doesn't provide any advantage
over the functor objects I've been using since long. Closures and anonymous
functions are harder to maintain and debug, and they are not as elegant as
their counterparts in functional languages (lambda functions). On the other
side, functor objects are more maintainable, scalable, elegant, and
versatile (and php already supports them... it could be improved though).

Bottom line, excellent work. If a considerable part of the php user base
need this, excellent, they have an implementation which seems viable. But,
sorry, I couldn't care less about closures or anonymous functions. Anyway,
my opinion doesn't matter, I'm just one user.

Best Regards,

Martin Alterisio

Hello Christian,

I have put your proposal as a link to a PHP GSoC 2008 idea here:
http://wiki.php.net/gsoc/2008

Feel invited to add to this idea in whatever way you want :-)

marcus

Saturday, December 22, 2007, 4:08:04 PM, you wrote:

Hi,

I was following this thread and came upon Jeff's posting on how closures
could be implemented in PHP.

Since I would find the feature to be EXTREMELY useful, I decided to
actually implement it more or less the way Jeff proposed. So, here's the
patch (against PHP_5_3, I can write one against HEAD if you whish):

http://www.christian-seiler.de/temp/closures-php-5-3.patch

I started with Wez's patch for adding anonymous functions that aren't
closures. I changed it to make sure no shift/reduce or reduce/reduce
error occur in the grammar. Then I started implementing the actual
closure stuff. It was fun because I learned quite a lot about how PHP
actually works.

I had the following main goals while developing the patch:

1. Don't reinvent the wheel.
2. Don't break anything unless absolutely necessary.
3. Keep it simple.

Jeff proposed a new type of zval that holds additional information about
the function that is to be called. Adding a new type of zval would need
changes throughout the ENTIRE PHP source and probably also throughout
quite a few scripts. But fortunately, PHP already posesses a zval that
supports the storage of arbitrary data while being very lightweight:
Resources. So I simply added a new resource type that stores zend
functions. The $var = function () {}; will now make $var a resource (of
the type "anonymous function".

Anonymous functions are ALWAYS defined at compile time, no matter where
they occur. They are simply named __compiled_lamda_1234 and added to the
global function table. But instead of simply returning the string
'__compiled_lambda_1234', I introduced a new opcode that will create
references to the correct local variables that are referenced inside the
function.

For example, if you have:

$func = function () {
echo "Hello World\n";
};

This will result in an anonymous function called '__compiled_lambda_0'
that is added to the function table at compile time. The opcode for the
assignment to $func will be something like:

     1  ZEND_DECLARE_ANON_FUNC     ~0      '__compiled_lambda_0'
     2  ASSIGN                             !0, ~0

The ZEND_DECLARE_ANON_FUNC opcode handler does the following:

It creates a new zend_function, copies the contents of the entire
structure of the function table entry corresponding to
'__compiled_lamda_0' into that new structure, increments the refcount,
registeres it as a resource and returns that resource so it can be
assigned to the variable.

Now, have a look at a real closure:

$string = "Hello World!\n";
$func = function () {
lexical $string;
echo $string;
};

This will result in the same opcode as above. But here, three additional
things happen:

1. The compiler sees the keyword 'lexical' and stores the information,
   that a variable called 'string' should be used inside the closure.

2. The opcode handler sees that a variable named 'string' is marked as
   lexical in the function definition. Therefore it creates a reference to
   it in a HashTable of the COPIED zend_function (that will be stored in
   the resource).

3. The 'lexical $string;' translates into a FETCH opcode that will work
   in exactly the same way as 'static' or 'global' - only fetching it from
   the additional HashTable in the zend_function structure.

The resource destructor makes sure that the HashTable containing the
references to the lexical veriables is correctly destroyed upon
destruction of the resource. It does NOT destroy other parts of the
function structure because they will be freed when the function is
removed from the global function table.

With these changes, closures work in PHP.

Some caveats / bugs / todo:

• Calling anonymous functions by name directly is problematic if there
  are lexical variables that need to be assigned. I added checks to
  make sure this case does not happen.

• In the opcode handler, error handling needs to be added.

• If somebody removes the function from the global function table,
  (e.g. with runkit), the new opcode will return NULL instead of
  a resource (error handling is missing). Since I do increment
  refcount of the zend_function, it SHOULD not cause segfaults or
  memory leaks, but I haven't tested it.

• $this is kind of a problem, because all the fetch handlers in PHP
  make sure $this is a special kind of variable. For the first version
  of the patch I chose not to care about this because what still works
  is e.g. the following:

   $object = $this;
   $func = function () {
     lexical $object;
     // do something
   };

Also, inside the closures, the class context is not preserved, so
accessing private / protected members is not possible.

I'm not sure this actually represents a problem because you can
always use normal local variables to pass values between closure
and calling method and make the calling method change the properties
itself.

• I've had some problems with eval(), have a look at the following
  code:

   $func = eval ('return function () { echo "Hello World!\n"; };');
   $func();

With plain PHP, this seems to work, with the VLD extension loaded
(that shows the Opcodes), it crashes. I don't know if that's a
problem with eval() or just with VLD and I didn't have time to
investigate it further.

• Oh, yes, 'lexical' is now a keyword. Although I really don't think
  that TOO many people use that as an identifier, so it probably won't
  hurt THAT much.

Except those above points, it really works, even with complex stuff. Let
me show you some examples:

1. Customized array_filter:

function filter_larger ($array, $min = 42) {
$filter = function ($value) {
lexical $min;
return ($value >= $min);
};
return array_filter ($array, $filter);
}

$arr = array (41, 43);
var_dump (filter_larger ($arr)); // 43
var_dump (filter_larger ($arr, 40)); // 41, 43
var_dump (filter_larger ($arr, 44)); // empty

2. Jeff's example:

function getAdder($x) {
return function ($y) {
lexical $x;
return $x + $y;
};
}

$plusFive = getAdder(5);
$plusTen = getAdder(10);

echo $plusFive(4)."\n"; // 9
echo $plusTen(7)."\n"; // 17

3. Nested closures

$outer = function ($value) {
return function () {
lexical $value;
return $value * 2;
};
};

$duplicator = $outer (4);
echo $duplicator ()."\n"; // 8
$duplicator = $outer (8);
echo $duplicator ()."\n"; // 16

[Ok, yeah, that example is quite stupid and should NOT be used as an
example for good code. ;-) But it's simple and demonstrates the
possibilities.]

It would be great if somebody could review the patch because I'm shure
some parts can still be cleaned up or improved. And it would be even
better if this feature would make it into PHP. ;-)

Regards,
Christian

PS: I'd like to thank Derick Rethans for his GREAT Vulcan Logic
Disassembler - without it, developement would have been a LOT more painful.

PPS: Oh, yeah, if it should be legally necessary, I grant the right to
anybody to use this patch under any OSI certified license you may want
to choose.

Best regards,
Marcus