[RFC] Pipe Operator (again)

Hi, Larry, That's super! I hope it will pass!

Oladoyinbo, IMO the way it is described right now (e.g. explicit closures)
is much more elegant than a new way of doing things that's not so obvious
and will be necessary to keep in mind and support anyway.

If it'd be necessary to simplify the stuff, like passing particular
parameter from the input pipe into the function at the particular position,

• I think it would be possible to do it with partial function application I
  hope to see in the future. (e.g. bind callback for array_map function,
  making a new function for the pipe that will accept the only parameter -
  input array)

Thank you

On Fri, Feb 7, 2025, 9:16 AM Oladoyinbo Vincent oladoyinbov@gmail.com
wrote:

PHP codebase in general is quite unreadable due to robust way of doing
things. Pipe operator might make things more complicated even more...

But after reading the RFC, something came to my mind, a way to simplify
this stuff

What if we implement it this way:

$pipe = " hello world "
           |> strtoupper(self)
           |> trim(self, ' ')
           |> htmlentities(self)
           |> fn (self): string => ....

Maybe 'self' or '$this' can be used as the keyword param.

It's a suggestion anyways.

On Fri, 7 Feb 2025, 5:58 am Larry Garfield, larry@garfieldtech.com
wrote:

Hi folks. A few years ago I posted an RFC for a pipe operator, as seen
in many other languages. At the time it didn't pass, in no small part
because the implementation was a bit shaky and it was right before freeze.
Nonetheless, there are now even more (bad) user-space implementations in
the wild, as it gets brought up frequently in "what do you want in PHP?"
threads (though nowhere near generics or better async, of course), so it
seems clear there is demand in the market for it.

It is now back with a better implementation (many thanks to Ilija for his
help and guidance in that), and it's nowhere close to freeze, so here we go
again:

https://wiki.php.net/rfc/pipe-operator-v3

Of particular note, since the last RFC I have concluded that a compose
operator is a necessary complement to a pipe operator. However, it's also
going to be notably more work, and the two operators don't actually
interact at all at the code level, so since people keep saying "Small
RFCs!", here's a small RFC. :-)

--
Larry Garfield
larry@garfieldtech.com

Merging a few replies together here, since they overlap. Also reordering a few of Tim's comments...

Hi

Am 2025-02-07 05:57, schrieb Larry Garfield:

It is now back with a better implementation (many thanks to Ilija for
his help and guidance in that), and it's nowhere close to freeze, so
here we go again:

https://wiki.php.net/rfc/pipe-operator-v3

There's some editorial issues:

1. Status: Draft needs to be updated.
2. The RFC needs to be added to the overview page.
3. List formatting issues in “Future Scope” and “Patches and Tests”.

Would also help having a closed voting widget in the “Proposed Voting
Choices” section to be crystal clear on what is being voted on (see
below the next quote).

I split pipes off from the Composition RFC late last night right before posting; I guess I missed a few things while doing so. :-/ Most notably, the Compose section is now removed from pipes, as it is not in scope for this RFC. (As noted, it's going to be more work so has its own RFC.) Sorry for the confusion. I think it should all be handled now.

5. The “References” (as in reference variables) section would do well
   with an example of what doesn't work.

Example block added.

9. In the “Why in the engine?” section: The RFC makes a claim about
   performance.

Do you have any numbers?

Not currently. The statements here are based on simply counting the number of function calls necessary, and PHP function calls are sadly non-cheap. In previous benchmarks of my own libraries using my Crell/fp library, I did find that the number of function calls involved in some tight pipe operations was both a performance and debugging concern, but I don't have any hard numbers laying about at present to share.

If you think that's critical, please advise on how to best get meaningful numbers here.

Regarding the equivalency of pipes:

Tim Düsterhus wrote:

4. “That is, the following two code fragments are also exactly
   equivalent:”.

I do not believe this is true (specifically referring to the “exactly”
word in there), since the second code fragment does not have the short
closures, which likely results in an observable behavioral difference
when throwing Exceptions (in the stack trace) and also for debuggers. Or
is the implementation able to elide the the extra closure? (Of course
there's also the difference between the temporary variable existing,
with would be observable for get_defined_vars() and possibly
destructors / object lifetimes).

Thomas Hruska wrote:

The repeated assignment to $temp in your second example is not
actually equal to the earlier example as you claim. The second example
with all of the $temp variables should, IMO, just be:

$temp = "Hello World";
$result = array_filter(array_map('strtoupper',
str_split(htmlentities($temp))), fn($v) { return $v != 'O'; });

Juris Evertovskis wrote:

3. Does the implementation actually turn 1 |> f(...) |> g(...) into
   $π = f(1); g($π)? Is g(f(1)) not performanter? Or is the engine
   clever enough with the var reuse anyways?

There's some subtlety here on these points. The v2 RFC used the lexer to mutate $a |> $b |> $c into the same AST as $c($b($a)), which would then compile as though that had been written in the first place. However, that made addressing references much harder, and there's an important caveat around order of operations. (See below.) The v3 RFC instead uses a compile function to take the AST of $a |> $b |> $c and produce opcodes that are effectively equivalent to $t = $b($a); $t = $c($t); I have not compared to see if they are the precise same opcodes, but they net effect is the same. So "effectively equivalent" may be a more accurate statement.

In particular, Tim is correct that, technically, the short lambdas would be used as-is, so you'd end up with the equivalent of:

$temp = (fn($x) => array_map(strtoupper(...), $x))($temp);

I'm not sure if there's a good way to automatically unwrap the closure there. (If someone knows of one, please share; I'm fine with including it.) However, the intent is that it would be largely unnecessary in the future with a revised PFA implementation, which would obviate the need for the explicit wrapping closure. You would instead write

$a |> array_map(strtoupper(...), ?);

Alternatively, one can use higher order user-space functions already. In trivial cases:

function amap(Closure $fn): Closure {
return fn(array $x) => array_map($fn, $x);
}

$a |> amap(strtoupper(...));

Which I am already using in Crell/fp and several libraries that leverage it, and it's quite ergonomic.

There's a whole bunch of such simple higher order functions here:
https://github.com/Crell/fp/blob/master/src/array.php
https://github.com/Crell/fp/blob/master/src/string.php

Which leads to the subtle difference between that and the v2 implementation, and why Thomas' statement is incorrect. If the expression on the right side that produces a Closure has side effects (output, DB interaction, etc.), then the order in which those side effects happen may change with the different restructuring. With all pure functions, that won't make a practical difference, and normally one should be using pure functions, but that's not something PHP can enforce.

I don't think there would be an appreciable performance difference between the two compiled versions, either way, but using the temp-var approach makes dealing with references easier, so it's what we're doing.

Juris Evertovskis wrote:

1. Do you think it would be hard to add some shorthand for |> $condition ? $callable : fn($😐) => $😐?

I'm not sure I follow here. Assuming you're talking about "branch in the next step", the standard way of doing that is with a higher order user-space function. Something like:

function cond(bool $cond, Closure $t, Closure $f): Closure {
return $cond ? $t : $f;
}

$a |> cond($config > 10, bigval(...), smallval(...)) |> otherstuff(...);

I think it's premature to try and bake that logic into the language, especially when I don't know of any other function-composition-having language that does so at the language level rather than the standard library level. (There are a number of fun operations people build into pipelines, but they are all generally done in user space.)

--Larry Garfield

Of particular note, since the last RFC I have concluded that a compose operator is a necessary complement to a pipe operator. However, it's also going to be notably more work, and the two operators don't actually interact at all at the code level, so since people keep saying "Small RFCs!", here's a small RFC. :-)

Fair enough. And with the pipe operator, one might live without a
compose operator, e.g.

$f1 = fn($x) => 2 * $x;
$f2 = fn($x) => $x + 3;
// $f3 = $f2 ∘ $f1
$f3 = fn($x) => $x |> $f1 |> $f2;

Christoph

The v2 RFC took that position, that compose was easy enough to emulate via pipe. Indeed, pipe and compose can both be implemented in terms of each other. However, since the previous RFC I've concluded[1] that both are sufficiently useful that we really out to include both of them. PIpes are just way easier to implement in practice. :-)

--Larry Garfield

[1] https://peakd.com/hive-168588/@crell/aoc2021-review

The repeated assignment to $temp in your second example is not
actually equal to the earlier example as you claim. The second example
with all of the $temp variables should, IMO, just be:

$temp = "Hello World";
$result = array_filter(array_map('strtoupper',
str_split(htmlentities($temp))), fn($v) { return $v != 'O'; });

Tbh, this is unreadable. Larry's example with an intermediate variable is a
magnitude times more readable. This is exactly why we need pipe operator.

I also consider the above compact code to be plenty readable and not

particularly necessary to span multiple lines, but that's obviously
subjective.

It is not, the functions are being applied from in to out (or right to
left), which become hard to read with addition of each new function. Pipe
operator makes it natural as they are applied from left to right which is
how you read code, literally 0 cognitive load.

Just because someone can do something doesn't mean that they should.

More than likely, users trying to do pipe-like operations in PHP
shouldn't be doing them in the first place.

Why not? It clearly makes code more readable and in future with PFA (🤞)
will allow composing non-unary functions.
PHP is and always has been a multi paradigm language, there is no reason to
not add stuff which makes using functional paradigm easier.

Kind regards,
Faizan

Hello,

I’m also wondering when I see code examples in the RFC like:

$profit = [1, 4, 5]
|> loadMany(...)
|> fn(array $records) => array_map(makeWidget(...), $records)
|> fn(array $ws) => array_filter(isOnSale(...), $ws)
|> fn(array $ws) => array_map(sellWidget(...), $ws)
|> array_sum(...);

This would be way better on performances as a single foreach, no?
I feel like this pipe operator encourages coders to use array_* functions with closures, which is often terrible performances compared to a loop.

How would the performance of the above compare with:

$profit = 0;
foreach (loadMany($input) as $item) {
$widget = makeWidget($item);
if (!isOnSale($widget)) {
continue;
}
$profit += sellWidget($widget);
}

Côme

Hi

Am 2025-02-07 05:57, schrieb Larry Garfield:

https://wiki.php.net/rfc/pipe-operator-v3

After also having taken a look at the implementation and then the
updated “Precedence” section, I'd like to argue in favor of moving |>
to have a higher precedence than the comparison operators (i.e. between
string concatenation and <). This would mean that |> has higher
precedence than ??, but looking at the following examples, that
appears to be the more useful default anyways.

I'm rather interested in handling a null pipe result:

 $user = $request->get('id')
     |> $database->fetchUser(...)
     ?? new GuestUser();

Than handling a null callback (using the RFC example, because I can't
even think of a real-world use-case):

 $res1 = 5
     |> $null_func ?? defaultFunc(...);

To give some more examples of what would be possible without parentheses
then:

 $containsNotOnlyZero = $someString
     |> fn ($str) => str_replace('0', '', $str)
     |> strlen(...)
     > 0;

Which is not particularly pretty, but appears to be more useful than
either passing a boolean into a single-argument function or piping into
a boolean (which would error).

Best regards
Tim Düsterhus

I have updated the patch and RFC accordingly. I think you're right, it does make a bit more sense this way.

--Larry Garfield

Hi Ilija and Larry,

thank you so much for your great work bringing PHP forward. I have been passively reading this list for a while and would like to chime in with two thoughts.

Pipes would no longer have the form of expr |> expr, where the right-hand-side is expected to return a callable. Instead, it would have the form of expr |> function_call, where the left-hand-side is implicitly inserted as the first parameter of the call.

namespace Iter {
function map(iterable $iterable, \Closure $callback): \Iterator;
function filter(iterable $iterable, \Closure $callback): \Iterator;
}

namespace {
use function Iter{map, filter};

$result = "Hello World"
|> str_split()
|> map(strtoupper(...))
|> filter(fn($v) => $v != 'O');
}

With named parameters, you could even make this approach work without the suggested (but still useful) new Iterator API:

$result = "Hello World"
|> str_split()
|> array_map(callback: strtoupper(...))
|> array_filter(callback: fn($v) => $v != 'O');

or

$result = "Hello World"
|> str_split()
|> array_map(callback: strtoupper(...))
|> array_filter(fn($v) => $v != 'O');

I am also wondering whether |> and -> should have the same operator precedence.

Best regards,

Olaf Schmidt-Wischhöfer

Hi Larry

Sorry for the late response.

https://wiki.php.net/rfc/pipe-operator-v3

We have already discussed this topic extensively off-list, so let me
bring the list up-to-date.

The current pipes proposal is elegantly simple. This has many upsides,
but it comes with an obvious limitation:
It only works well when the called function takes only a single argument.

$sourceCode |> lexer(...) |> parser(...) |> compiler(...) |> vm(...)

Such code is nice, but is also quite niche. I have argued off-list
that the predominant use-case for pipes are arrays and iterators
(including strings immediately split into chunks), and it seems most
agree. However, most array/iterator functions (e.g. filter, map,
reduce, first, all, etc.) don't fall into the one-parameter category.

A slightly simplified example from the RFC:

$result = "Hello World"
|> str_split(...)
|> fn($x) => array_map(strtoupper(...), $x)
|> fn($x) => array_filter($x, fn($v) => $v != 'O');

IMO, this is harder to understand than the alternative of using
multiple statements with a temporary variable.

$tmp = "Hello World";
$tmp = str_split($tmp);
$tmp = array_map(strtoupper(...), $tmp);
$result = array_filter($tmp, fn($v) => $v != 'O');

The RFC has a solution for this: Partial function application [1].

$result = "Hello World"
|> str_split(...)
|> array_map(strtoupper(...), ?)
|> array_filter(?, fn($v) => $v != 'O');

This still causes more cognitive overhead than it should, at least to me.

• The placement of ? is hard to detect, especially when it's not the
  first argument.
• The user now has to think about immediately-invoked closures that
  exist solely for argument-reordering. The closure can be elided
  through the optimizer, but we cannot elide the additional cognitive
  overhead in the user.
• The implementation of ? is significantly more complex than that of
  pipes, making the supposed simplicity of pipes somewhat misleading.

If my assumption is correct that the primary use-case for pipes are
arrays, it might be worth investigating the possibility of introducing
a new iterator API, which has been proposed before [2], optimized for
pipes. Specifically, this API would ensure consistent placement of the
subject, i.e. the iterable in this case, as the first argument. Pipes
would no longer have the form of expr |> expr, where the
right-hand-side is expected to return a callable. Instead, it would
have the form of expr |> function_call, where the left-hand-side is
implicitly inserted as the first parameter of the call.

namespace Iter {
function map(iterable $iterable, \Closure $callback): \Iterator;
function filter(iterable $iterable, \Closure $callback): \Iterator;
}

namespace {
use function Iter{map, filter};

$result = "Hello World"
    |> `str_split()`
    |> map(strtoupper(...))
    |> filter(fn($v) => $v != 'O');

}

This is the same approach taken by Elixir [3]. It has a few benefits:

• We don't need to think about closures that are immediately invoked,
  because there are none. The code is exactly the same as if you had
  written it through nested function calls. This simplifies things
  significantly for both the engine and the user.
• It closely resembles code that would be written in an
  object-oriented manner, making it more familiar.
• It is the shortest and most readable of all the proposed options.

As with everything, there are downsides.

• It only works well for subject-first APIs. There are not an
  insignificant number of existing functions that do not follow this
  convention (e.g. explode(), preg_match(), etc.). That said, explode('
  ', $s) |> filter($c1) |> map($c2) still composes well, given explode()
  is usually first first in the chain, while preg_match() is rarely
  chained at all.
• People have voiced concerns for potential confusion regarding the
  right-hand-side. It may not be any arbitrary expression, but is
  restricted to a function call. Hence, $param |> $myClosure is not
  valid code, requiring additional braces: $param |> $myClosure().
  This approach resembles the -> operator, where at least conceptually,
  the left-hand-side is implicitly passed as a $this parameter. However,
  the spaces between |> do not signal this fact as well, making it look
  like the right-hand-side is evaluated separately. Potentially, a
  different symbol might work better.

Internal reactions to this idea were mixed, so I'm interested to hear
what the community thinks about it.

Ilija

[1] https://wiki.php.net/rfc/partial_function_application
[2] https://externals.io/message/118896
[3] https://elixirschool.com/en/lessons/basics/pipe_operator

To clarify my stance on the above: I am open to this, and I agree with Ilija that in the typical case it would be more convenient. The argument that it would be confusing to have a "hidden" first param is valid, but as with any new feature I think it's obvious once you know it, so that's a small issue. I didn't propose it originally as I suspected folks would balk at the added complexity, but I do like the concept.

Part of Ilija's proposal does include offering $val |> ($expr) (or similar) to allow arbitrary expressions on the left, which would need to return a unary function. Basically the () would make it the same as what the RFC is doing now.

However, it also received significant pushback off-list from folks who felt it was too much magic. I don't want to torpedo pipes on over-reaching. But without feedback from other voters, I don't know if this is over-reaching. Is it? Please, someone tell me which approach you'd be more willing to vote for. :-)

One concern of this approach is that it gets even closer to "real" extension functions. But real extension functions (which let you write code that looks like you're adding arbitrary methods to arbitrary objects, even though under the hood it's just a plain function that takes an object as a parameter) also run into a lot of additional complexity. Chief among them, they don't handle name collisions, so you can have only one "map" function rather than one-per-class. Unless you have an alternate syntax for the extension functions to specify the type they work on (which is what Kotlin does), but then you run into questions around inheritance and polymorphism that are hard to resolve in a runtime-centric environment. I haven't fully thought through all of these details.

It's also been proposed to use +> as an operator for extension functions and/or first-param pipes like Elixir. I'm not sure how I feel about that; my main concern is which one it would apply to, since as noted above full extension functions introduce a lot of extra considerations.

But I really don't want to hold up pipes on speculation on multiple future maybe-features. As the RFC notes, there are a number of follow ups that I want to try and get at least some of into the same release.

So, consider this me begging for voters to actually speak up on this issue and give feedback on a way forward, because right now I have no idea what to do with it.

--Larry Garfield