# [PATCH] [RFC] Closures and lambda functions in PHP

- Thread: https://externals.io/message/38290
- Original archive: https://news-web.php.net/php.internals/38290
- Started by Christian Seiler on 2008-06-16T17:39:19+00:00
- Messages: 108
- RFC: Short Closures 2.0 (Declined) — https://externals.io/rfc/auto-capture-closure.md
- Page 1 of 5: messages 1 to 25 (25 per page, add `?page=N` to the URL)
- Next page: https://externals.io/message/38290.md?page=2
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · RFCs at https://externals.io/rfc.md · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #38290 — Christian Seiler — 2008-06-16T17:39:19+00:00

URL: https://externals.io/message/38290#38290

Hi,

As a followup to the discussion in January, I'd like post a revised patch to
this list that implements closures and anonymous functions in PHP.

INTRODUCTION
------------

Closures and lambda functions can make programming much easier in 
several ways:

  1. Lambda functions allow the quick definition of throw-away functions
     that are not used elsewhere. Imaging for example a piece of code that
     needs to call `preg_replace_callback()`. Currently, there are three
     possibilities to acchieve this:

      a. Define the callback function elsewhere. This distributes code that
         belongs together throughout the file and decreases readability.

      b. Define the callback function in-place (but with a name). In 
that case
         one has to use `function_exists()` to make sure the function is only
         defined once. Example code:

          <?php
             function replace_spaces ($text) {
               if (!function_exists ('replace_spaces_helper')) {
                 function replace_spaces_helper ($matches) {
                   return str_replace ($matches[1], ' ', '&nbsp;').' ';
                 }
               }
               return preg_replace_callback ('/( +) /', 
'replace_spaces_helper',
                                             $text);
             }
          ?>

         Here, the additional if() around the function definition makes the
         source code difficult to read.

      c. Use the present create_function() in order to create a function at
         runtime. This approach has several disadvantages: First of all, 
syntax
         highlighting does not work because a string is passed to the 
function.
         It also compiles the function at run time and not at compile 
time so
         opcode caches can't cache the function.

  2. Closures provide a very useful tool in order to make lambda 
functions even
     more useful. Just imagine you want to replace 'hello' through 
'goodbye' in
     all elements of an array. PHP provides the `array_map()` function which
     accepts a callback. If you don't wan't to hard-code 'hello' and 
'goodbye'
     into your sourcecode, you have only four choices:

      a. Use create_function(). But then you may only pass literal values
         (strings, integers, floats) into the function, objects at best as
         clones (if `var_export()` allows for it) and resources not at 
all. And
         you have to worry about escaping everything correctly. 
Especially when
         handling user input this can lead to all sorts of security issues.

      b. Write a function that uses global variables. This is ugly,
         non-reentrant and bad style.

      c. Create an entire class, instantiate it and pass the member function
         as a callback. This is perhaps the cleanest solution for this 
problem
         with current PHP but just think about it: Creating an entire 
class for
         this extremely simple purpose and nothing else seems overkill.

      d. Don't use `array_map()` but simply do it manually (foreach). In this
         simple case it may not be that much of an issue (because one simply
         wants to iterate over an array) but there are cases where doing
         something manually that a function with a callback as parameter 
does
         for you is quite tedious.

     [Yes, I know that str_replace also accepts arrays as a third 
parameter so
     this example may be a bit useless. But imagine you want to do a more
     complex operation than simple search and replace.]

PROPOSED PATCH
--------------

I now propose a patch that implements compile-time lambda functions and
closures for PHP while keeping the patch as simple as possible. The patch is
based on a previous patch on mine which was based on ideas discussed here
end of December / start of January.

Userland perspective
--------------------

1. The patch adds the following syntax as a valid expression:

function & (parameters) { body }

(The & is optional and indicates - just as with normal functions - that the
anonymous function returns a reference instead of a value)

Example usage:

$lambda = function () { echo "Hello World!\n"; };

The variable $lambda then contains a callable resource that may be called
through different means:

$lambda ();
call_user_func ($lambda);
call_user_func_array ($lambda, array ());

This allows for simple lambda functions, for example:

function replace_spaces ($text) {
   $replacement = function ($matches) {
     return str_replace ($matches[1], ' ', '&nbsp;').' ';
   };
   return preg_replace_callback ('/( +) /', $replacement, $text);
}

2. The patch implements closures by defining an additional keyword 'lexical'
that allows an lambda function (and *only* an lambda function) to import
a variable from the "parent scope" to the lambda function scope. Example:

function replace_in_array ($search, $replacement, $array) {
   $map = function ($text) {
     lexical $search, $replacement;
     if (strpos ($text, $search) > 50) {
       return str_replace ($search, $replacement, $text);
     } else {
       return $text;
     }
   };
   return array_map ($map, array);
}

The variables $search and $replacement are variables in the scope of the
function replace_in_array() and the lexical keyword imports these variables
into the scope of the closure. The variables are imported as a reference,
so any change in the closure will result in a change in the variable of the
function itself.

3. If a closure is defined inside an object, the closure has full access
to the current object through $this (without the need to use 'lexical' to
import it seperately) and all private and protected methods of that class.
This also applies to nested closures. Essentially, closures inside 
methods are
added as public methods to the class that contains the original method.

4. Closures may live longer as the methods that declared them. It is 
perfectly
possible to have something like this:

function getAdder($x) {
   return function ($y) {
     lexical $x;
     return $x + $y;
   };
}

Zend internal perspective
-------------------------

The patch basically changes the following in the Zend engine:

When the compiler reaches a lambda function, it creates a unique name 
for that
function ("\0__compiled_lambda_FILENAME_N" where FILENAME is the name of the
file currently processed and N is a per-file counter). The use of the 
filename
in the function name ensures compability with opcode caches. The lambda
function is then immediately added to the function table (either the global
function table or that of the current class if declared inside a class 
method).
Instead of a normal ZEND_DECLARE_FUNCTION opcode the new
ZEND_DECLARE_LAMBDA_FUNC is used as an opcode at this point. The op_array
of the new function is initialized with is_lambda = 1 and is_closure = 0.

When parsing a 'lexical' declaration inside an anonymous function the parser
saves the name of the variable that is to be imported in an array stored
as a member of the op_array structure (lexical_names).

The opcode handler for ZEND_DECLARE_LAMBDA_FUNC does the following: First of
all it creates a new op_array and copies the entire memory structure of the
lambda function into it (the opcodes themselves are not copied since they
are only referenced in the op_array structure). Then it sets is_closure = 1
on the new op_array, and for each lexical variable name that the compiler
added to the original op_array it creates a reference to that variable from
the current scope into a HashTable member in the new op_array. It also saves
the current object pointer ($this) as a member of the op_array in order to
allow for the closure to access $this. Finally it registers the new op_array
as a resource and returns that resource.

The opcode handler of the 'lexical' construct simply fetches the variable
from that HashTable and imports it into local scope of the inner function
(just like with 'global' only with a different hash table).

Some hooks were added that allow the 'lambda function' resource to be 
called.
Also, there are several checks in place that make sure the lambda function
is not called directly, i.e. if someone explicitely tries to use the 
internal
function name instead of using the resource return value of the declaration.

The patch
---------

The patch is available here:
<http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-16-1.diff>

Please note that I did NOT include the contents of zend_language_scanner.c
in the patch since that can easily be regenerated and just takes up enormous
amounts of space.

The patch itself applies against the 5.3 branch of PHP.

If I understand the discussion regarding PHP6 on this list correctly, some
people are currently undergoing the task of removing the unicode_semantics
switch and if (UG(unicode)). As soon as this task is finished I will also
provide a patch for CVS HEAD (it doesn't make much sense adopting the patch
now and then having to change it again completely afterwards).

BC BREAKS
---------

  * Introduction of a new keyword 'lexical'. Since it is very improbable 
that
    someone should use it as a function, method, class or property name, I
    think this is an acceptable break.

Other that that, I can find no BC breaks of my patch.

CAVEATS / POSSIBLE WTFS
-----------------------

  * On writing $func = function () { }; there is a semicolon necessary. 
If left
    out it will produce a compile error. Since any attempt to remove that
    necessity would unecessarily bloat the grammar, I suggest we simply keep
    it the way it is. Also, Lukas Kahwe Smith pointed out that a single
    trailing semicolon after a closing brace already exists: do { } 
while ();

  * The fact that 'lexical' creates references may cause certain WTFs:

      for ($i = 0; $i < 10; $i++) {
        $arr[$i] = function () { lexical $i; return $i; };
      }

    This will not work as expected since $i is a reference and thus all
    created closures would reference the same variable. In order to get this
    right one has to do:

      for ($i = 0; $i < 10; $i++) {
        $loopIndex = $i;
        $arr[$i] = function () { lexical $loopIndex; return $loopIndex; };
        unset ($loopIndex);
      }

    This can be a WTF for people that don't expect lexical to create an
    actual reference, especially since other languages such as JavaScript
    don't do it. On the other hand, global and static both DO create
    references so that behaviour is consistent with current PHP.

    But complex constructions such as this will probably not be used by
    beginners so maintaining a good documentation should solve this.

  * The fact that 'lexical' is needed at all may cause WTFs. Other languages
    such as JavaScript implicitely have the entire scope visible to child
    functions. But since PHP does the same thing with global variables, I
    find a keyword like 'lexical' much more consistent than importing the
    entire scope (and always importing the entire scope costs unnecessary
    performance).

FINAL THOUGHTS
--------------

My now proposed patch addresses the two main problems of my previous patch:
Support for closures in objects (with access to $this) and opcode caches. My
patch applies against PHP_5_3 and does not break any tests. It adds a 
valuable
new language feature which I'd like to see in PHP.

Regards,
Christian

---

## #38292 — Christian Seiler — 2008-06-16T18:56:18+00:00

URL: https://externals.io/message/38290#38292 · In reply to: #38290

Hi,

Lukas Kahwe Smith asked me to put my proposal into the PHP wiki, which I
have done:

http://wiki.php.net/rfc/closures

I also have incorporated the last comment by troels knak-nielsen about
JS behaving the same way as my patch and thus not being that much of a
WTF at all (I somehow had a different behaviour for JS in mind, I
probably got confused with yet another language).

Anyway, feel free to comment.

Regards,
Christian

---

## #38306 — Larry Garfield — 2008-06-17T02:23:56+00:00

URL: https://externals.io/message/38290#38306 · In reply to: #38292

> Hi,
>
> Lukas Kahwe Smith asked me to put my proposal into the PHP wiki, which I
> have done:
>
> http://wiki.php.net/rfc/closures
>
> I also have incorporated the last comment by troels knak-nielsen about
> JS behaving the same way as my patch and thus not being that much of a
> WTF at all (I somehow had a different behaviour for JS in mind, I
> probably got confused with yet another language).
>
> Anyway, feel free to comment.

Should comments from user-space folk be posted here or as comments at the 
bottom of the wiki page?

-- 
Larry Garfield			AIM: LOLG42
larry@garfieldtech.com		ICQ: 6817012

"If nature has made any one thing less susceptible than all others of 
exclusive property, it is the action of the thinking power called an idea, 
which an individual may exclusively possess as long as he keeps it to 
himself; but the moment it is divulged, it forces itself into the possession 
of every one, and the receiver cannot dispossess himself of it."  -- Thomas 
Jefferson

---

## #38307 — Philip Olson — 2008-06-17T02:53:20+00:00

URL: https://externals.io/message/38290#38307 · In reply to: #38306

> > […]
>
> Should comments from user-space folk be posted here or as comments  
> at the
> bottom of the wiki page?

We're all users... so here.

Regards,
Philip

---

## #38308 — Larry Garfield — 2008-06-17T03:57:23+00:00

URL: https://externals.io/message/38290#38308 · In reply to: #38292

Thoughts from a user-land denizen:

- I conceptually really like.  lambdas and closures are a feature I've been 
jealous of Javascript for having since I learned how Javascript "really" 
worked. :-)

- I recall earlier discussion pondering if we should be using the 
keyword "function" to describe this new construct.  It is a function, but not 
as we know it.  (Thank you, Mr. Spock.)  Should we consider using the 
keyword "lambda" instead, or do we think re-using "function" is acceptable?  
(I'm cool either way; I just wanted to make sure it was a conscious 
decision.)

- I don't think the lexical keyword is a wtf at all, especially given that it 
works essentially like global.  It simplifies both the internal code and the 
user-space code, because it becomes more self-documenting.  The semi-colon is 
a slight wtf, but since it is the same as Javascript I think most people will 
get used to it.  I have no objection to it.

- I am a little confused about the OOP interaction.  How does a function 
become a public method of the class?

class Example {
  private $a = 2;

  function myMethod($b) {
    $lambda = function() {
      lexical $b;
      return $this->a * $b; // This part I get
    };
    return $lambda;
  }
}

$e = new Example();
$lambda = $e->myMethod();
$e->$lambda(5);

That doesn't seem right at all, but that's how I interpret "Essentially, 
closures inside methods are added as public methods to the class that 
contains the original method."  Can you give an example of what that actually 
means?

- Related to that, would it then be possible to add methods to a class at 
runtime using lambda functions as the added methods?  If so, how?  If not, is 
that something that could reasonably be added here without hosing performance 
(or at least doing so less than stacking __call() and `call_user_func_array()` 
does)?

I guess I just need some code examples to see how this behaves in relation to 
objects, because I'm not really following it as is.

- My minuscule knowledge of the engine internals looks at the description 
there and doesn't see anything to be scared about, but I am far from an 
authority in that subject.  It does seem to side-step the "can't compile at 
compile time" problem that was raised back in December, which is good.

Overall, big +1 pending clarification of how lambdas and classes interact, 
which I still don't grok.


> Hi,
>
> Lukas Kahwe Smith asked me to put my proposal into the PHP wiki, which I
> have done:
>
> http://wiki.php.net/rfc/closures
>
> I also have incorporated the last comment by troels knak-nielsen about
> JS behaving the same way as my patch and thus not being that much of a
> WTF at all (I somehow had a different behaviour for JS in mind, I
> probably got confused with yet another language).
>
> Anyway, feel free to comment.
>
> Regards,
> Christian


-- 
Larry Garfield			AIM: LOLG42
larry@garfieldtech.com		ICQ: 6817012

"If nature has made any one thing less susceptible than all others of 
exclusive property, it is the action of the thinking power called an idea, 
which an individual may exclusively possess as long as he keeps it to 
himself; but the moment it is divulged, it forces itself into the possession 
of every one, and the receiver cannot dispossess himself of it."  -- Thomas 
Jefferson

---

## #38309 — Alexey Zakhlestin — 2008-06-17T04:58:58+00:00

URL: https://externals.io/message/38290#38309 · In reply to: #38308

>  - I am a little confused about the OOP interaction.  How does a function
>  become a public method of the class?
>
>  class Example {
>   private $a = 2;
>
>   function myMethod($b) {
>     $lambda = function() {
>       lexical $b;
>       return $this->a * $b; // This part I get
>     };
>     return $lambda;
>   }
>  }
>
>  $e = new Example();
>  $lambda = $e->myMethod();
>  $e->$lambda(5);
>
>  That doesn't seem right at all, but that's how I interpret "Essentially,
>
> closures inside methods are added as public methods to the class that
>
> contains the original method."  Can you give an example of what that actually
>  means?


As far as I understand, it means following:

class Example
{
    private $a = 1;

	private function b()
	{
		return 2;
	}

	public function getLambda($param)
	{
		$lambda = function($lparam) {
			lexical $param;

			return $this->a + $this->b() + $param + $lparam;
		}

		return $lambda;
	}
}

$obj = new Example();
$lambda = $obj->getLambda(3);

$result = $lambda(4); // 1+2+3+4 => 10

-- 
Alexey Zakhlestin
http://blog.milkfarmsoft.com/

---

## #38310 — Nathan Nobbe — 2008-06-17T05:57:12+00:00

URL: https://externals.io/message/38290#38310 · In reply to: #38308

On Mon, Jun 16, 2008 at 9:57 PM, Larry Garfield <larry@garfieldtech.com>
wrote:

> Thoughts from a user-land denizen:
>
> - Related to that, would it then be possible to add methods to a class at
> runtime using lambda functions as the added methods?  If so, how?  If not,
> is
> that something that could reasonably be added here without hosing
> performance
> (or at least doing so less than stacking __call() and
> `call_user_func_array()`
> does)?


im curious about this as well, and i would welcome such functionality.
adding methods to class instances from within the definition of the class
seems intuitive, the $this keyword should be available, but when adding
methods via a variable which holds a handle to a class instance, well, i
wonder, would the $this keyword be available within those anonymous function
definitions?

class Dynamic {
  private $someVar = 5;
  /// adding a function to instances from within the class
  public function addMethodAtRuntime() {
      $this->dynamicFunc1 = function() {
          return $this->someVar;  // expected to work
      }
  }
}

/// adding a function to an instance externally
$dynamic = new Dynamic();
$anotherVar = 6;
$dynamic->dynamicFunc2 = function() {
  lexical $anotherVar;  // expected to work
  return $this->someVar + $anotherVar;  // would this work ?
}

/// invoking dynamically added methods
/// (anticipated behavior given above definitions)
$dynamic->addMethodAtRuntime();
echo $dynamic->dynamicFunc1(); // 5
echo $dynamic->dynamicFunc2(); // 11

-nathan

---

## #38312 — Christian Seiler — 2008-06-17T07:39:37+00:00

URL: https://externals.io/message/38290#38312 · In reply to: #38308

Hi,

> - I am a little confused about the OOP interaction.  How does a function 
> become a public method of the class?

To clarify: the "public method" ist just the internal representation of
the lambda function and has *nothing* to do with the semantics of
calling the lambda itself. The "method" only means that the lambda
function defined inside another method can access the class members and
"public" only means that the lambda function can still be called from
outside the class.

> class Example {
>   private $a = 2;
> 
>   function myMethod($b) {
>     $lambda = function() {
>       lexical $b;
>       return $this->a * $b; // This part I get
>     };
>     return $lambda;
>   }
> }
> 
> $e = new Example();
> $lambda = $e->myMethod();
> $e->$lambda(5);

No, that's not what my patch does. My patch does:

class Example {
   private $a = 2;

   public function myMethod ($b) {
     return function () {
       lexical $b;
       return $this->a * $b;
     };
   }
}

$e = new Example ();
$lambda = $e->myMethod (4);
var_dump ($lambda ()); // int(8)
$lambda2 = $e->myMethod (6);
var_dump ($lambda2 ()); // int(12)

So esentially, it does not matter whether you define a lambda function
inside a method or a function (or in global scope, for that matter), you
always use it the same way. The in-class-method lambda function only has
the additional advantage of being able to access the private and
protected class members since *internally* it is treated like a public
class method.

> - Related to that, would it then be possible to add methods to a class at 
> runtime using lambda functions as the added methods?

No.

> If not, is  that something that could reasonably be added here without
> hosing performance (or at least doing so less than stacking __call() and
> `call_user_func_array()` does)?

If you want to add methods dynamically to classes, why not use the
runkit extension? I really don't see a point in making lambda functions
and closures something they are not.

Regards,
Christiaan

---

## #38313 — Christian Seiler — 2008-06-17T07:44:58+00:00

URL: https://externals.io/message/38290#38313 · In reply to: #38310

Hi!

> class Dynamic {
>   private $someVar = 5;
>   /// adding a function to instances from within the class
>   public function addMethodAtRuntime() {
>       $this->dynamicFunc1 = function() {
>           return $this->someVar;  // expected to work
>       }
>   }
> }
> 
> /// invoking dynamically added methods
> /// (anticipated behavior given above definitions)
> $dynamic->addMethodAtRuntime();
> echo $dynamic->dynamicFunc1(); // 5

This will not work - for the same reason as this does not work:

class SpecialChars {
   public $process = 'htmlspecialchars';
}

$sc = new SpecialChars ();
var_dump ($sc->process ('<>')); // call to undefined ...

On the other hand, the following will work:

$sc = new SpecialChars ();
$processor = $sc->process;
var_dump ($processor ('<>')); // string(8) "&lt;&gt;"

The same with closures:

echo $dynamic->dynamicFunc1(); // call to undefined ...
$func = $dynamic->dynamicFunc1;
echo $func (); // 5
echo call_user_func ($dynamic->dynamicFunc1); // 5

As I sead in my other mail: I don't see closures as a method for somehow
making it possible to extend classes dynamically - if you want that, use
runkit.

Regards,
Christian

---

## #38318 — Marcus Boerger — 2008-06-17T10:03:20+00:00

URL: https://externals.io/message/38290#38318 · In reply to: #38290

Hello Christian,

very nice work. I think we should really add this to 5.3. The only thing I
don't like is the function naming (“\0__compiled_lambda_FILENAME_N”). Can
we drop the \0? For methods inside classes, do we have to provide real
private methods or do we support visibility fully? Or did you use the \0
prefix to prevent direct invocations? If so, it doesn't help, as the user
can simply create the function call with the \0.

I think the best option would be to force lambda functions being public
always. The last question is about changing visibility and overriding
functions, do we want that, or should we mark lamdas as final? Note that
preceeding the function names with a \0 does not help. In fact it might
confuse reflection. Actually how does it integrate with reflection?

Comments about the implementation:

- ZendEngine2/zend_compile.c:

  Why provide forward declaration for free_filename(), simply putting the
  new function above the first user avoids it and does the same with
  increased maintainability.

  s/static void add_lexical_var (/static void add_lexical_var(/

Other than that it all looks fine.


marcus

Monday, June 16, 2008, 7:39:19 PM, you wrote:

> Hi,

> As a followup to the discussion in January, I'd like post a revised patch to
> this list that implements closures and anonymous functions in PHP.

> INTRODUCTION
> ------------

> Closures and lambda functions can make programming much easier in 
> several ways:

>   1. Lambda functions allow the quick definition of throw-away functions
>      that are not used elsewhere. Imaging for example a piece of code that
>      needs to call `preg_replace_callback()`. Currently, there are three
>      possibilities to acchieve this:

>       a. Define the callback function elsewhere. This distributes code that
>          belongs together throughout the file and decreases readability.

>       b. Define the callback function in-place (but with a name). In 
> that case
>          one has to use `function_exists()` to make sure the function is only
>          defined once. Example code:

>           <?php
>              function replace_spaces ($text) {
>                if (!function_exists ('replace_spaces_helper')) {
>                  function replace_spaces_helper ($matches) {
>                    return str_replace ($matches[1], ' ', '&nbsp;').' ';
>                  }
>                }
>                return preg_replace_callback ('/( +) /', 
> 'replace_spaces_helper',
>                                              $text);
>              }
>           ?>

>          Here, the additional if() around the function definition makes the
>          source code difficult to read.

>       c. Use the present create_function() in order to create a function at
>          runtime. This approach has several disadvantages: First of all, 
> syntax
>          highlighting does not work because a string is passed to the 
> function.
>          It also compiles the function at run time and not at compile 
> time so
>          opcode caches can't cache the function.

>   2. Closures provide a very useful tool in order to make lambda 
> functions even
>      more useful. Just imagine you want to replace 'hello' through 
> 'goodbye' in
>      all elements of an array. PHP provides the `array_map()` function which
>      accepts a callback. If you don't wan't to hard-code 'hello' and 
> 'goodbye'
>      into your sourcecode, you have only four choices:

>       a. Use create_function(). But then you may only pass literal values
>          (strings, integers, floats) into the function, objects at best as
>          clones (if `var_export()` allows for it) and resources not at 
> all. And
>          you have to worry about escaping everything correctly. 
> Especially when
>          handling user input this can lead to all sorts of security issues.

>       b. Write a function that uses global variables. This is ugly,
>          non-reentrant and bad style.

>       c. Create an entire class, instantiate it and pass the member function
>          as a callback. This is perhaps the cleanest solution for this 
> problem
>          with current PHP but just think about it: Creating an entire 
> class for
>          this extremely simple purpose and nothing else seems overkill.

>       d. Don't use `array_map()` but simply do it manually (foreach). In this
>          simple case it may not be that much of an issue (because one simply
>          wants to iterate over an array) but there are cases where doing
>          something manually that a function with a callback as parameter 
> does
>          for you is quite tedious.

>      [Yes, I know that str_replace also accepts arrays as a third 
> parameter so
>      this example may be a bit useless. But imagine you want to do a more
>      complex operation than simple search and replace.]

> PROPOSED PATCH
> --------------

> I now propose a patch that implements compile-time lambda functions and
> closures for PHP while keeping the patch as simple as possible. The patch is
> based on a previous patch on mine which was based on ideas discussed here
> end of December / start of January.

> Userland perspective
> --------------------

> 1. The patch adds the following syntax as a valid expression:

> function & (parameters) { body }

> (The & is optional and indicates - just as with normal functions - that the
> anonymous function returns a reference instead of a value)

> Example usage:

> $lambda = function () { echo "Hello World!\n"; };

> The variable $lambda then contains a callable resource that may be called
> through different means:

> $lambda ();
> call_user_func ($lambda);
> call_user_func_array ($lambda, array ());

> This allows for simple lambda functions, for example:

> function replace_spaces ($text) {
>    $replacement = function ($matches) {
>      return str_replace ($matches[1], ' ', '&nbsp;').' ';
>    };
>    return preg_replace_callback ('/( +) /', $replacement, $text);
> }

> 2. The patch implements closures by defining an additional keyword 'lexical'
> that allows an lambda function (and *only* an lambda function) to import
> a variable from the "parent scope" to the lambda function scope. Example:

> function replace_in_array ($search, $replacement, $array) {
>    $map = function ($text) {
>      lexical $search, $replacement;
>      if (strpos ($text, $search) > 50) {
>        return str_replace ($search, $replacement, $text);
>      } else {
>        return $text;
>      }
>    };
>    return array_map ($map, array);
> }

> The variables $search and $replacement are variables in the scope of the
> function replace_in_array() and the lexical keyword imports these variables
> into the scope of the closure. The variables are imported as a reference,
> so any change in the closure will result in a change in the variable of the
> function itself.

> 3. If a closure is defined inside an object, the closure has full access
> to the current object through $this (without the need to use 'lexical' to
> import it seperately) and all private and protected methods of that class.
> This also applies to nested closures. Essentially, closures inside 
> methods are
> added as public methods to the class that contains the original method.

> 4. Closures may live longer as the methods that declared them. It is 
> perfectly
> possible to have something like this:

> function getAdder($x) {
>    return function ($y) {
>      lexical $x;
>      return $x + $y;
>    };
> }

> Zend internal perspective
> -------------------------

> The patch basically changes the following in the Zend engine:

> When the compiler reaches a lambda function, it creates a unique name 
> for that
> function ("\0__compiled_lambda_FILENAME_N" where FILENAME is the name of the
> file currently processed and N is a per-file counter). The use of the 
> filename
> in the function name ensures compability with opcode caches. The lambda
> function is then immediately added to the function table (either the global
> function table or that of the current class if declared inside a class 
> method).
> Instead of a normal ZEND_DECLARE_FUNCTION opcode the new
> ZEND_DECLARE_LAMBDA_FUNC is used as an opcode at this point. The op_array
> of the new function is initialized with is_lambda = 1 and is_closure = 0.

> When parsing a 'lexical' declaration inside an anonymous function the parser
> saves the name of the variable that is to be imported in an array stored
> as a member of the op_array structure (lexical_names).

> The opcode handler for ZEND_DECLARE_LAMBDA_FUNC does the following: First of
> all it creates a new op_array and copies the entire memory structure of the
> lambda function into it (the opcodes themselves are not copied since they
> are only referenced in the op_array structure). Then it sets is_closure = 1
> on the new op_array, and for each lexical variable name that the compiler
> added to the original op_array it creates a reference to that variable from
> the current scope into a HashTable member in the new op_array. It also saves
> the current object pointer ($this) as a member of the op_array in order to
> allow for the closure to access $this. Finally it registers the new op_array
> as a resource and returns that resource.

> The opcode handler of the 'lexical' construct simply fetches the variable
> from that HashTable and imports it into local scope of the inner function
> (just like with 'global' only with a different hash table).

> Some hooks were added that allow the 'lambda function' resource to be 
> called.
> Also, there are several checks in place that make sure the lambda function
> is not called directly, i.e. if someone explicitely tries to use the 
> internal
> function name instead of using the resource return value of the declaration.

> The patch
> ---------

> The patch is available here:
> <http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-16-1.diff>

> Please note that I did NOT include the contents of zend_language_scanner.c
> in the patch since that can easily be regenerated and just takes up enormous
> amounts of space.

> The patch itself applies against the 5.3 branch of PHP.

> If I understand the discussion regarding PHP6 on this list correctly, some
> people are currently undergoing the task of removing the unicode_semantics
> switch and if (UG(unicode)). As soon as this task is finished I will also
> provide a patch for CVS HEAD (it doesn't make much sense adopting the patch
> now and then having to change it again completely afterwards).

> BC BREAKS
> ---------

>   * Introduction of a new keyword 'lexical'. Since it is very improbable 
> that
>     someone should use it as a function, method, class or property name, I
>     think this is an acceptable break.

> Other that that, I can find no BC breaks of my patch.

> CAVEATS / POSSIBLE WTFS
> -----------------------

>   * On writing $func = function () { }; there is a semicolon necessary. 
> If left
>     out it will produce a compile error. Since any attempt to remove that
>     necessity would unecessarily bloat the grammar, I suggest we simply keep
>     it the way it is. Also, Lukas Kahwe Smith pointed out that a single
>     trailing semicolon after a closing brace already exists: do { } 
> while ();

>   * The fact that 'lexical' creates references may cause certain WTFs:

>       for ($i = 0; $i < 10; $i++) {
>         $arr[$i] = function () { lexical $i; return $i; };
>       }

>     This will not work as expected since $i is a reference and thus all
>     created closures would reference the same variable. In order to get this
>     right one has to do:

>       for ($i = 0; $i < 10; $i++) {
>         $loopIndex = $i;
>         $arr[$i] = function () { lexical $loopIndex; return $loopIndex; };
>         unset ($loopIndex);
>       }

>     This can be a WTF for people that don't expect lexical to create an
>     actual reference, especially since other languages such as JavaScript
>     don't do it. On the other hand, global and static both DO create
>     references so that behaviour is consistent with current PHP.

>     But complex constructions such as this will probably not be used by
>     beginners so maintaining a good documentation should solve this.

>   * The fact that 'lexical' is needed at all may cause WTFs. Other languages
>     such as JavaScript implicitely have the entire scope visible to child
>     functions. But since PHP does the same thing with global variables, I
>     find a keyword like 'lexical' much more consistent than importing the
>     entire scope (and always importing the entire scope costs unnecessary
>     performance).

> FINAL THOUGHTS
> --------------

> My now proposed patch addresses the two main problems of my previous patch:
> Support for closures in objects (with access to $this) and opcode caches. My
> patch applies against PHP_5_3 and does not break any tests. It adds a 
> valuable
> new language feature which I'd like to see in PHP.

> Regards,
> Christian




Best regards,
 Marcus

---

## #38319 — Christian Seiler — 2008-06-17T10:20:02+00:00

URL: https://externals.io/message/38290#38319 · In reply to: #38318

Hi Marcus,

> very nice work.

Thanks!

> The only thing I don't like is the function naming
> (“\0__compiled_lambda_FILENAME_N”). Can we drop the \0?

I used \0 because it is already used in two other places:

  1) create_function (run-time lambda functions) uses \0__lambda_N
  2) build_runtime_defined_function_key uses \0 to start function names.

I can drop it if you like; personally, I don't care for either solution
- it's an internal name that *may* leak to userspace in some
circumstances but is never really useful for userspace anyway..

A minor side-note here: I oriented myself at
build_runtime_defined_function_key at the time of writing but I have
noticed a slight discrepancy between function names generated by
build_runtime_defined_function_key and the normal function names: When
stored in the corresponding function_table hash, for runtime defined
function keys it is opline->op1.u.constant.value.str.len, whereas for
normal function names it is »Z_STRLEN_P(...) + 1« and thus including the
*trailing* (not preceding!) \0 in the hash key for normal function names
but not including it for runtime defined function keys. Any idea why
that is the case? [For the record: I'm refering to the code that is
already used in PHP, not to my patch!]

> Or did you use the \0 prefix to prevent direct invocations?

No, direct invocations are prevented by the is_lambda == 1 &&
is_closure == 0 check.

> I think the best option would be to force lambda functions being public
> always.

They are. If you look at my modified version of
zend_do_begin_function_declaration, you will see that:

if (is_lambda && CG(active_class_entry)) {
	is_method = 1;
	fn_flags = ZEND_ACC_PUBLIC;
	if (CG(active_op_array)->fn_flags & ZEND_ACC_STATIC) {
		fn_flags |= ZEND_ACC_STATIC;
	}
} else if (is_lambda) {
	fn_flags = 0;
}

The only attribute that is "inherited" from the parent function is
whether that function is static or not.

> The last question is about changing visibility and overriding
> functions, do we want that, or should we mark lamdas as final?

Internal representations of lambda fuctions should never be overridden, 
so yes, ZEND_ACC_FINAL would probably be a good idea. Overriding them 
won't work anyway since the new opcode that "instantiates" a closure 
will always use the class in which the closure was defined to look it up.

I'll add that.

> Actually how does it integrate with reflection?

Good question, I will investigate that and come back to you.

> Comments about the implementation:
> 
> - ZendEngine2/zend_compile.c:
> 
>   Why provide forward declaration for free_filename(), simply putting the
>   new function above the first user avoids it and does the same with
>   increased maintainability.
> 
>   s/static void add_lexical_var (/static void add_lexical_var(/

Ok, I will fix that.

Regards,
Christian

---

## #38322 — Edward Z. Yang — 2008-06-17T11:48:27+00:00

URL: https://externals.io/message/38290#38322 · In reply to: #38308

Larry Garfield wrote:
> $e = new Example();
> $lambda = $e->myMethod();
> $e->$lambda(5);
> 
> That doesn't seem right at all, but that's how I interpret "Essentially, 
> closures inside methods are added as public methods to the class that 
> contains the original method."  Can you give an example of what that actually 
> means?

At first blush, this sets off warning bells in my head, I suppose
because my notion of a lambda says that the lambda should not carry any
baggage about the context it was created in.

However, with further thought, I believe that binding the lambda's
lexical scope to the place it was defined is:

* Conducive to good coding (you will always be able to look outside the
lambda to find out where the lexical variables are coming form)
* Adds functionality, since anything you want to pass to the function
via the callee's context can be passed via a parameter

What would be neat, however, is the ability to rebind the lambda to
another context. Also, I don't know how other languages do it (Python?
Lisp?).

-- 
 Edward Z. Yang                        GnuPG: 0x869C48DA
 HTML Purifier <http://htmlpurifier.org> Anti-XSS Filter
 [[ 3FA8 E9A9 7385 B691 A6FC B3CB A933 BE7D 869C 48DA ]]

---

## #38324 — Christian Seiler — 2008-06-17T12:24:01+00:00

URL: https://externals.io/message/38290#38324 · In reply to: #38318

Hi Marcus,

I now have revised my patch to include your suggestions:

http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-17-2.diff

The changes to the previous version:

  - \0 at the start of the compiled lambda function name is dropped.
  - lambdas which are class members are now marked as final
  - the generated name of the lambda is now also stored within the
    op_array (before op_array->function_name was simply "lambda")
  - your suggestions for code cleanups in zend_compile.c

> Actually how does it integrate with reflection?

Consider the following class:

class Example {
   private $x = 0;

   public function getIncer () {
     return function () {
       $this->x++;
     };
   }

   public function show () {
     $this->reallyShow ();
   }

   protected function reallyShow () {
     echo "{$this->x}\n";
   }
}

Running

Reflection::export(new ReflectionClass('Example'));

will yield (among other things):

   - Methods [4] {
     Method [ <user> public method getIncer ] {
       @@ /home/christian/dev/php5.3/c-tests/reflection.php 6 - 10
     }

     Method [ <user> final public method 
__compiled_lambda_/home/christian/dev/php5.3/c-tests/reflection.php_0 ] {
       @@ /home/christian/dev/php5.3/c-tests/reflection.php 7 - 9
     }

     Method [ <user> public method show ] {
       @@ /home/christian/dev/php5.3/c-tests/reflection.php 12 - 14
     }

     Method [ <user> protected method reallyShow ] {
       @@ /home/christian/dev/php5.3/c-tests/reflection.php 16 - 18
     }
   }

So lambda functions appear simply as additional methods in the class,
with their generated name.

Of course, the ReflectionMethod / ReflectionFunction classes could be
extended so that it provides and additional method isLambda() in order
to determine whether a function actually is a lambda function. But I'd
rather do that in a separate step and a separate patch.

Regards,
Christian

---

## #38325 — Chris Stockton — 2008-06-17T16:25:06+00:00

URL: https://externals.io/message/38290#38325 · In reply to: #38324

Hello,

Great patch and a much needed feature. One thing I do not agree with is the
point in the lexical key word, seems it should be natural to inherit the
outer scope. I guess the choice of adding lexical and going slightly against
the grain of typical closure implementations like scheme or ecmascript is
that is not really consistent with php so i can understand disagreement and
your note you made on performance. Seems like the right choice to force
manual inheritance of outer scope. But great work on this, hope it gets
added and none of the core developers say it is not the php way or is only
useful in brainless languages.

-Chris

---

## #38328 — Marcus Boerger — 2008-06-17T19:10:38+00:00

URL: https://externals.io/message/38290#38328 · In reply to: #38324

Hello Christian, Johannes,

Tuesday, June 17, 2008, 2:24:01 PM, you wrote:

> Hi Marcus,

> I now have revised my patch to include your suggestions:

> http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-17-2.diff

> The changes to the previous version:

>   - \0 at the start of the compiled lambda function name is dropped.
>   - lambdas which are class members are now marked as final
>   - the generated name of the lambda is now also stored within the
>     op_array (before op_array->function_name was simply "lambda")
>   - your suggestions for code cleanups in zend_compile.c

> > […]

> Consider the following class:

> class Example {
>    private $x = 0;

>    public function getIncer () {
>      return function () {
>        $this->x++;
>      };
>    }

>    public function show () {
>      $this->reallyShow ();
>    }

>    protected function reallyShow () {
>      echo "{$this->x}\n";
>    }
> }

> Running

> Reflection::export(new ReflectionClass('Example'));

> will yield (among other things):

>    - Methods [4] {
>      Method [ <user> public method getIncer ] {
>        @@ /home/christian/dev/php5.3/c-tests/reflection.php 6 - 10
>      }

>      Method [ <user> final public method 
> __compiled_lambda_/home/christian/dev/php5.3/c-tests/reflection.php_0 ] {
>        @@ /home/christian/dev/php5.3/c-tests/reflection.php 7 - 9
>      }

>      Method [ <user> public method show ] {
>        @@ /home/christian/dev/php5.3/c-tests/reflection.php 12 - 14
>      }

>      Method [ <user> protected method reallyShow ] {
>        @@ /home/christian/dev/php5.3/c-tests/reflection.php 16 - 18
>      }
>    }

> So lambda functions appear simply as additional methods in the class,
> with their generated name.

> Of course, the ReflectionMethod / ReflectionFunction classes could be
> extended so that it provides and additional method isLambda() in order
> to determine whether a function actually is a lambda function. But I'd
> rather do that in a separate step and a separate patch.

Yep, Reflection details can and should be addressed separately.

I think the next step is confirming with the rest of the core team that we
all ewant this. IMO it has often enough been requested and your patch
solves every tiny piece that was missing so far. And if people agree then
unfortunately you need to provide a patch for HEAD first. Actually you
could do so just now.

Johannes, what's your take on this one for 5.3?


Best regards,
 Marcus

---

## #38329 — Stanislav Malyshev — 2008-06-17T19:19:56+00:00

URL: https://externals.io/message/38290#38329 · In reply to: #38328

Hi!

> Johannes, what's your take on this one for 5.3?

I'm not Johannes and I didn't review the proposal in detail yet, but I 
think we have enough for 5.3 right now. I'd think we better concentrate 
on tying the loose ends and rolling beta out and then moving towards the 
release than adding more and more features and never releasing it. 5.3 
is not the final release until the end of times, there will be 5.4 etc. 
and 6, so there will be ample opportunity to add stuff. And 5.3 has 
enough stuff to be released, there's no rush to add more new things, 
especially radically new ones. My opinion is that we better take some 
time with it and not tie it to 5.3.
-- 
Stanislav Malyshev, Zend Software Architect
stas@zend.com   http://www.zend.com/
(408)253-8829   MSN: stas@zend.com

---

## #38330 — Steph Fox — 2008-06-17T19:24:20+00:00

URL: https://externals.io/message/38290#38330 · In reply to: #38329

> I'm not Johannes and I didn't review the proposal in detail yet, but I 
> think we have enough for 5.3 right now. I'd think we better concentrate 
> on tying the loose ends and rolling beta out and then moving towards the 
> release than adding more and more features and never releasing it. 5.3 
> is not the final release until the end of times, there will be 5.4 etc. 
> and 6, so there will be ample opportunity to add stuff. And 5.3 has 
> enough stuff to be released, there's no rush to add more new things, 
> especially radically new ones. My opinion is that we better take some 
> time with it and not tie it to 5.3.

Amen to that.

- Steph

> -- 
> Stanislav Malyshev, Zend Software Architect
> stas@zend.com   http://www.zend.com/
> (408)253-8829   MSN: stas@zend.com

---

## #38331 — Alexey Zakhlestin — 2008-06-17T19:35:44+00:00

URL: https://externals.io/message/38290#38331 · In reply to: #38329

>  I'm not Johannes and I didn't review the proposal in detail yet, but I
> think we have enough for 5.3 right now. I'd think we better concentrate on
> tying the loose ends and rolling beta out and then moving towards the
> release than adding more and more features and never releasing it. 5.3 is
> not the final release until the end of times, there will be 5.4 etc. and 6,
> so there will be ample opportunity to add stuff. And 5.3 has enough stuff to
> be released, there's no rush to add more new things, especially radically
> new ones. My opinion is that we better take some time with it and not tie it
> to 5.3.

although I really want to have this functionality right now, I agree with Stas.
5.3 has a lot of new things already. this should go to HEAD, and,
hopefully, it will attract more people to actually try php-6

-- 
Alexey Zakhlestin
http://blog.milkfarmsoft.com/

---

## #38332 — Marcus Boerger — 2008-06-17T19:51:15+00:00

URL: https://externals.io/message/38290#38332 · In reply to: #38329

Hello Stanislav,

  nicely put but not in agreement with the PHP world. First we cannot add
a new feature like this in a mini release as it comes with an API change.
And second PHP is not anywhere close so we'd have to do it in a PHP 5.4
and personally I would like to avoid it.

marcus

Tuesday, June 17, 2008, 9:19:56 PM, you wrote:

> Hi!

> > […]

> I'm not Johannes and I didn't review the proposal in detail yet, but I 
> think we have enough for 5.3 right now. I'd think we better concentrate 
> on tying the loose ends and rolling beta out and then moving towards the 
> release than adding more and more features and never releasing it. 5.3 
> is not the final release until the end of times, there will be 5.4 etc. 
> and 6, so there will be ample opportunity to add stuff. And 5.3 has 
> enough stuff to be released, there's no rush to add more new things, 
> especially radically new ones. My opinion is that we better take some 
> time with it and not tie it to 5.3.
> -- 
> Stanislav Malyshev, Zend Software Architect
> stas@zend.com   http://www.zend.com/
> (408)253-8829   MSN: stas@zend.com




Best regards,
 Marcus

---

## #38333 — Lars Strojny — 2008-06-17T19:52:05+00:00

URL: https://externals.io/message/38290#38333 · In reply to: #38329

Hi Markus, hi Stas,

Am Dienstag, den 17.06.2008, 12:19 -0700 schrieb Stanislav Malyshev:
[...]
> I'm not Johannes and I didn't review the proposal in detail yet, but I 
> think we have enough for 5.3 right now. I'd think we better concentrate 
> on tying the loose ends and rolling beta out and then moving towards the 
> release than adding more and more features and never releasing it. 5.3 
> is not the final release until the end of times, there will be 5.4 etc. 
> and 6, so there will be ample opportunity to add stuff. And 5.3 has 
> enough stuff to be released, there's no rush to add more new things, 
> especially radically new ones. My opinion is that we better take some 
> time with it and not tie it to 5.3.

I would like to see 5.3 released first before we add really cool
features like this. I'm really +1 for closures but I please for 5.4 and
6.

cu, Lars

---

## #38334 — Stanislav Malyshev — 2008-06-17T20:12:49+00:00

URL: https://externals.io/message/38290#38334 · In reply to: #38332

Hi!

>   nicely put but not in agreement with the PHP world. First we cannot add
> a new feature like this in a mini release as it comes with an API change.
> And second PHP is not anywhere close so we'd have to do it in a PHP 5.4
> and personally I would like to avoid it.

You meant "PHP 6 not anywhere close"? well, if we keep adding completely 
new stuff to 5.3 then 5.3 would not be anywhere close so what we earn by 
that? I understand that we all want new and cool stuff in PHP, but 
endlessly delaying 5.3 is not the answer.
And this feature is a big thing, it's not adding yet another module or 
function - it should be thoroughly reviewed and seen how it affect all 
other things. It's probably very cool addition, but yet more reason to 
consider all the side effects and surprises on the way.
Pushing it in now would mean we'd either have to delay 5.3 yet another 
half-year or we'd release buggy 5.3 and then we'd be bound by API 
compatibility and couldn't fix things that we might want to fix. I think 
as much as we want to add yet another cool feature - we have to release 
versions for the users to actually be able to enjoy features we already 
added, from time to time. And time for 5.3 is very much due, and we 
still have a bunch of work to do on it.
-- 
Stanislav Malyshev, Zend Software Architect
stas@zend.com   http://www.zend.com/
(408)253-8829   MSN: stas@zend.com

---

## #38335 — Christian Seiler — 2008-06-17T20:43:02+00:00

URL: https://externals.io/message/38290#38335 · In reply to: #38333

Hi!

> > […]
> 
> I would like to see 5.3 released first before we add really cool
> features like this. I'm really +1 for closures but I please for 5.4 and
> 6.

If I may add my own personal (and biased ;-)) opinion (which may not
count much but I'd like to present the arguments): I'd like to see it in
PHP 5.3. Mainly because of two reasons:

First: My patch is quite non-intrusive, it only adds things in a few
places (new opcode, a few checks). If you only look at the non-generated
files (i.e. excluding files generated by re2c or zend_vm_gen.php), the
patch is actually not even that long:
http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-17-3-redux.diff
Except for the introduction of a 'lexical' keyword I carefully designed
the patch not to have *any* impact *at all* on PHPs other behaviour. I'd
be genuinely surprised if any code breaks with my patch. I also don't
see how this would delay 5.3 - of course things have to be tested but at
least as far as I can tell the major showstoppers currently are class
inheritance rules and namespaces which still cause quite a few headaches
of their own.

Second: If closures are not supported in PHP 5.3, even with the release
of PHP 6 backwards compability will be a hindrance in using them. Since
PHP 6 will have Unicode support and thus quite a few semantic changes,
this will of course not matter much for the actual PHP applications
since these will have to change anyway. But think of class libraries:
There are many things that can be implemented in class libraries where
unicode support doesn't matter at all - such as for example advanced
date and time calculations (beyond timelib), financial calculations etc.
Such libraries will probably want to maintain compability with PHP 5.3
as long as possible. But these libraries may profit from closures.

If you still decide to include closures only from post PHP 5.3, I
suggest to at least declare 'lexical' a reserved keyword in PHP 5.3.

Just my 2 cents...

As to the patch for HEAD: I thought it best to wait for
unicode.semantics to go away along with the if (UG(unicode)) checks
before implementing it (everything else would be a waste of time - since
if I'm not mistaken someone is actually currently removing those). If I
really am mistaken in my interpretation of the discussions here on this
topic and they are not going away (at least not in the short term), I
can of course provide one now (meaning the next few days).

Regards,
Christian

---

## #38337 — Christopher Jones — 2008-06-17T23:23:20+00:00

URL: https://externals.io/message/38290#38337 · In reply to: #38290

Christian Seiler wrote:
 > As a followup to the discussion in January, I'd like post a revised
 > patch to this list that implements closures and anonymous functions
 > in PHP.

Did I miss seeing its phpt tests?
A really thorough test suite might the case for inclusion in PHP 5.3.

Chris

-- 
Christopher Jones, Oracle
Email: christopher.jones@oracle.com    Tel:  +1 650 506 8630
Blog:  http://blogs.oracle.com/opal/   Free PHP Book: http://tinyurl.com/f8jad

---

## #38338 — Larry Garfield — 2008-06-18T01:59:25+00:00

URL: https://externals.io/message/38290#38338 · In reply to: #38312

> Hi,
>
> > […]
>
> To clarify: the "public method" ist just the internal representation of
> the lambda function and has *nothing* to do with the semantics of
> calling the lambda itself. The "method" only means that the lambda
> function defined inside another method can access the class members and
> "public" only means that the lambda function can still be called from
> outside the class.

If one knew how to access it, which it seems is not possible/feasible for 
user-space code.  

> > […]
>
> No, that's not what my patch does. My patch does:
>
> class Example {
>    private $a = 2;
>
>    public function myMethod ($b) {
>      return function () {
>        lexical $b;
>        return $this->a * $b;
>      };
>    }
> }
>
> $e = new Example ();
> $lambda = $e->myMethod (4);
> var_dump ($lambda ()); // int(8)
> $lambda2 = $e->myMethod (6);
> var_dump ($lambda2 ()); // int(12)
>
> So esentially, it does not matter whether you define a lambda function
> inside a method or a function (or in global scope, for that matter), you
> always use it the same way. The in-class-method lambda function only has
> the additional advantage of being able to access the private and
> protected class members since *internally* it is treated like a public
> class method.

I see.  It would be great if you could update the RFC with this information so 
that it's clearer.  

> If you want to add methods dynamically to classes, why not use the
> runkit extension? I really don't see a point in making lambda functions
> and closures something they are not.

I was asking if they could be used for that, not to make them into a different 
animal.  As for using runkit, do I really need to answer that? :-)

Two other questions that just occurred to me:

1) What is the interaction with namespaces, if any?  Are lambdas as 
implemented here ignorant of namespace, or do they take the namespace where 
they are lexically defined?

2) What happens with the following code?

class Foo {
  private $a;
}

$f = new Foo();

$b = 5;

$f->myfunc = function($c) {
  lexical $b;
  print $a; // This generates an error, no?
  print $b; // This prints 5, right?
  print $c; // Should print whatever $c is.
}

$f->myfunc(3);

Or is the above a parse error entirely?

-- 
Larry Garfield			AIM: LOLG42
larry@garfieldtech.com		ICQ: 6817012

"If nature has made any one thing less susceptible than all others of 
exclusive property, it is the action of the thinking power called an idea, 
which an individual may exclusively possess as long as he keeps it to 
himself; but the moment it is divulged, it forces itself into the possession 
of every one, and the receiver cannot dispossess himself of it."  -- Thomas 
Jefferson

---

## #38339 — Andi Gutmans — 2008-06-18T06:01:34+00:00

URL: https://externals.io/message/38290#38339 · In reply to: #38290

Hi Christian,

This is a very nice piece of work. Definitely addresses a lot of the issues we have raised in the past.
I would like to see such a solution make its way into PHP (see below re: timing).	

There are some things I'd like to consider:
1) I am not sure that the current semantics of the "lexical" keyword is great in all cases. Is the reason why you don't allow by-value binding so that we don't have to manage more than one lambda instance per declaration?
2) [minor curiosity - do we want to consider reusing "parent" instead of "lexical"? I guess that could be confusing but it's not the first time we reuse a keyword when it's clear that the usage is in two different places (this is minor and I don't mind much either way although lexical doesn't mean too much to me).]
3) I am concerned about binding to classes. First of all we need to look into more detail what the implications are for bytecode caches when changing class entries at run-time. We may want to also consider an option where the lambda binds to the object and only has public access although I realize that may be considered by some as too limiting. We'll review these two things in the coming days.

Re: timing, I think the biggest issue we have right now with PHP 5.3 is that we are not making a clear cut on features. There's always pressure on release managers to include more (I went through the same with 5.0) but at some point you just have to stop at some place or things will never go out as there are always good ideas flowing in. Unfortunately with 5.3 that cut isn't happening and it seems to drag out longer than needed. I prefer having this discussion in the context of a hard date for a beta release after which we'll be especially strict with accepting new features. Each new feature will drag out the beta/RC cycle as they need enough time for testing/feedback/tweaks.

Andi	

> -----Original Message-----
> From: Christian Seiler [mailto:chris_se@gmx.net]
> Sent: Monday, June 16, 2008 10:39 AM
> To: php-dev List
> Subject: [PHP-DEV] [PATCH] [RFC] Closures and lambda functions in PHP
> 
> Hi,
> 
> As a followup to the discussion in January, I'd like post a revised patch to
> this list that implements closures and anonymous functions in PHP.
> 
> INTRODUCTION
> ------------
> 
> Closures and lambda functions can make programming much easier in
> several ways:
> 
>   1. Lambda functions allow the quick definition of throw-away functions
>      that are not used elsewhere. Imaging for example a piece of code that
>      needs to call `preg_replace_callback()`. Currently, there are three
>      possibilities to acchieve this:
> 
>       a. Define the callback function elsewhere. This distributes code that
>          belongs together throughout the file and decreases readability.
> 
>       b. Define the callback function in-place (but with a name). In
> that case
>          one has to use `function_exists()` to make sure the function is only
>          defined once. Example code:
> 
>           <?php
>              function replace_spaces ($text) {
>                if (!function_exists ('replace_spaces_helper')) {
>                  function replace_spaces_helper ($matches) {
>                    return str_replace ($matches[1], ' ', '&nbsp;').' ';
>                  }
>                }
>                return preg_replace_callback ('/( +) /',
> 'replace_spaces_helper',
>                                              $text);
>              }
>           ?>
> 
>          Here, the additional if() around the function definition makes the
>          source code difficult to read.
> 
>       c. Use the present create_function() in order to create a function at
>          runtime. This approach has several disadvantages: First of all,
> syntax
>          highlighting does not work because a string is passed to the
> function.
>          It also compiles the function at run time and not at compile
> time so
>          opcode caches can't cache the function.
> 
>   2. Closures provide a very useful tool in order to make lambda
> functions even
>      more useful. Just imagine you want to replace 'hello' through
> 'goodbye' in
>      all elements of an array. PHP provides the `array_map()` function which
>      accepts a callback. If you don't wan't to hard-code 'hello' and
> 'goodbye'
>      into your sourcecode, you have only four choices:
> 
>       a. Use create_function(). But then you may only pass literal values
>          (strings, integers, floats) into the function, objects at best as
>          clones (if `var_export()` allows for it) and resources not at
> all. And
>          you have to worry about escaping everything correctly.
> Especially when
>          handling user input this can lead to all sorts of security issues.
> 
>       b. Write a function that uses global variables. This is ugly,
>          non-reentrant and bad style.
> 
>       c. Create an entire class, instantiate it and pass the member function
>          as a callback. This is perhaps the cleanest solution for this
> problem
>          with current PHP but just think about it: Creating an entire
> class for
>          this extremely simple purpose and nothing else seems overkill.
> 
>       d. Don't use `array_map()` but simply do it manually (foreach). In this
>          simple case it may not be that much of an issue (because one simply
>          wants to iterate over an array) but there are cases where doing
>          something manually that a function with a callback as parameter
> does
>          for you is quite tedious.
> 
>      [Yes, I know that str_replace also accepts arrays as a third
> parameter so
>      this example may be a bit useless. But imagine you want to do a more
>      complex operation than simple search and replace.]
> 
> PROPOSED PATCH
> --------------
> 
> I now propose a patch that implements compile-time lambda functions and
> closures for PHP while keeping the patch as simple as possible. The patch is
> based on a previous patch on mine which was based on ideas discussed here
> end of December / start of January.
> 
> Userland perspective
> --------------------
> 
> 1. The patch adds the following syntax as a valid expression:
> 
> function & (parameters) { body }
> 
> (The & is optional and indicates - just as with normal functions - that the
> anonymous function returns a reference instead of a value)
> 
> Example usage:
> 
> $lambda = function () { echo "Hello World!\n"; };
> 
> The variable $lambda then contains a callable resource that may be called
> through different means:
> 
> $lambda ();
> call_user_func ($lambda);
> call_user_func_array ($lambda, array ());
> 
> This allows for simple lambda functions, for example:
> 
> function replace_spaces ($text) {
>    $replacement = function ($matches) {
>      return str_replace ($matches[1], ' ', '&nbsp;').' ';
>    };
>    return preg_replace_callback ('/( +) /', $replacement, $text);
> }
> 
> 2. The patch implements closures by defining an additional keyword 'lexical'
> that allows an lambda function (and *only* an lambda function) to import
> a variable from the "parent scope" to the lambda function scope. Example:
> 
> function replace_in_array ($search, $replacement, $array) {
>    $map = function ($text) {
>      lexical $search, $replacement;
>      if (strpos ($text, $search) > 50) {
>        return str_replace ($search, $replacement, $text);
>      } else {
>        return $text;
>      }
>    };
>    return array_map ($map, array);
> }
> 
> The variables $search and $replacement are variables in the scope of the
> function replace_in_array() and the lexical keyword imports these variables
> into the scope of the closure. The variables are imported as a reference,
> so any change in the closure will result in a change in the variable of the
> function itself.
> 
> 3. If a closure is defined inside an object, the closure has full access
> to the current object through $this (without the need to use 'lexical' to
> import it seperately) and all private and protected methods of that class.
> This also applies to nested closures. Essentially, closures inside
> methods are
> added as public methods to the class that contains the original method.
> 
> 4. Closures may live longer as the methods that declared them. It is
> perfectly
> possible to have something like this:
> 
> function getAdder($x) {
>    return function ($y) {
>      lexical $x;
>      return $x + $y;
>    };
> }
> 
> Zend internal perspective
> -------------------------
> 
> The patch basically changes the following in the Zend engine:
> 
> When the compiler reaches a lambda function, it creates a unique name
> for that
> function ("\0__compiled_lambda_FILENAME_N" where FILENAME is the name of the
> file currently processed and N is a per-file counter). The use of the
> filename
> in the function name ensures compability with opcode caches. The lambda
> function is then immediately added to the function table (either the global
> function table or that of the current class if declared inside a class
> method).
> Instead of a normal ZEND_DECLARE_FUNCTION opcode the new
> ZEND_DECLARE_LAMBDA_FUNC is used as an opcode at this point. The op_array
> of the new function is initialized with is_lambda = 1 and is_closure = 0.
> 
> When parsing a 'lexical' declaration inside an anonymous function the parser
> saves the name of the variable that is to be imported in an array stored
> as a member of the op_array structure (lexical_names).
> 
> The opcode handler for ZEND_DECLARE_LAMBDA_FUNC does the following: First of
> all it creates a new op_array and copies the entire memory structure of the
> lambda function into it (the opcodes themselves are not copied since they
> are only referenced in the op_array structure). Then it sets is_closure = 1
> on the new op_array, and for each lexical variable name that the compiler
> added to the original op_array it creates a reference to that variable from
> the current scope into a HashTable member in the new op_array. It also saves
> the current object pointer ($this) as a member of the op_array in order to
> allow for the closure to access $this. Finally it registers the new op_array
> as a resource and returns that resource.
> 
> The opcode handler of the 'lexical' construct simply fetches the variable
> from that HashTable and imports it into local scope of the inner function
> (just like with 'global' only with a different hash table).
> 
> Some hooks were added that allow the 'lambda function' resource to be
> called.
> Also, there are several checks in place that make sure the lambda function
> is not called directly, i.e. if someone explicitely tries to use the
> internal
> function name instead of using the resource return value of the declaration.
> 
> The patch
> ---------
> 
> The patch is available here:
> <http://www.christian-seiler.de/temp/closures-php-5.3-2008-06-16-1.diff>
> 
> Please note that I did NOT include the contents of zend_language_scanner.c
> in the patch since that can easily be regenerated and just takes up enormous
> amounts of space.
> 
> The patch itself applies against the 5.3 branch of PHP.
> 
> If I understand the discussion regarding PHP6 on this list correctly, some
> people are currently undergoing the task of removing the unicode_semantics
> switch and if (UG(unicode)). As soon as this task is finished I will also
> provide a patch for CVS HEAD (it doesn't make much sense adopting the patch
> now and then having to change it again completely afterwards).
> 
> BC BREAKS
> ---------
> 
>   * Introduction of a new keyword 'lexical'. Since it is very improbable
> that
>     someone should use it as a function, method, class or property name, I
>     think this is an acceptable break.
> 
> Other that that, I can find no BC breaks of my patch.
> 
> CAVEATS / POSSIBLE WTFS
> -----------------------
> 
>   * On writing $func = function () { }; there is a semicolon necessary.
> If left
>     out it will produce a compile error. Since any attempt to remove that
>     necessity would unecessarily bloat the grammar, I suggest we simply keep
>     it the way it is. Also, Lukas Kahwe Smith pointed out that a single
>     trailing semicolon after a closing brace already exists: do { }
> while ();
> 
>   * The fact that 'lexical' creates references may cause certain WTFs:
> 
>       for ($i = 0; $i < 10; $i++) {
>         $arr[$i] = function () { lexical $i; return $i; };
>       }
> 
>     This will not work as expected since $i is a reference and thus all
>     created closures would reference the same variable. In order to get this
>     right one has to do:
> 
>       for ($i = 0; $i < 10; $i++) {
>         $loopIndex = $i;
>         $arr[$i] = function () { lexical $loopIndex; return $loopIndex; };
>         unset ($loopIndex);
>       }
> 
>     This can be a WTF for people that don't expect lexical to create an
>     actual reference, especially since other languages such as JavaScript
>     don't do it. On the other hand, global and static both DO create
>     references so that behaviour is consistent with current PHP.
> 
>     But complex constructions such as this will probably not be used by
>     beginners so maintaining a good documentation should solve this.
> 
>   * The fact that 'lexical' is needed at all may cause WTFs. Other languages
>     such as JavaScript implicitely have the entire scope visible to child
>     functions. But since PHP does the same thing with global variables, I
>     find a keyword like 'lexical' much more consistent than importing the
>     entire scope (and always importing the entire scope costs unnecessary
>     performance).
> 
> FINAL THOUGHTS
> --------------
> 
> My now proposed patch addresses the two main problems of my previous patch:
> Support for closures in objects (with access to $this) and opcode caches. My
> patch applies against PHP_5_3 and does not break any tests. It adds a
> valuable
> new language feature which I'd like to see in PHP.
> 
> Regards,
> Christian

---

Next page: https://externals.io/message/38290.md?page=2