[RFC] Pipe Operator

Hi Sara,

I have to take exception to the section of the RFC titled "File
collection Example", where you take a block of code and re-write it to
be more readable, using the RFC's syntax.

However, it is not the proposed new syntax that makes the code more
readable - it is solely due to writing it in the correct order that
makes the code more readable. This can be done using current PHP
syntax, so that the code is:

$dirEntries = scandir($arg);
$files = array_filter($filesAndDirectories, function($x) { return
$x !== '.' && $x != '..'; });
$fullPaths = array_map(function ($x) use ($arg) { return $arg . '/' .
$x; }, $files)
$fileInfo = getFileArg($fullPaths);
$ret = array_merge($ret, $fileListInfo);

Using the proposed new syntax to rewrite it without the intermediate
variables like this.

$ret = scandir($arg)
|> array_filter($$, function($x) { return $x !== '.' && $x != '..'; })
|> array_map(function ($x) use ($arg) { return $arg . '/' . $x; }, $$)
|> getFileArg($$)
|> array_merge($ret, $$);

Does not now show a marked improvement in readability.

Showing the benefits of an RFC by comparing 'apples to oranges', is at
least to some extent 'pulling a fast one'. Do you have any examples
where the original code is not terrible, that show an increase in
readability? (By implication, I am not persuaded by the PSR 7 ones.)

Larry Garfield wrote:

The absence of an intermediate variable name does not inherently hinder readability/debuggability (nor does its presence inherently improve it).

I disagree strongly, the presence of intermedite variables does help
the code be more understandable. It gives semantic meaning to the
steps in involved in the process, as well as giving variables that can
be watched in a debugger.

If people want the excitement of not being able to figure out what the
intermediate variables mean, then they can already do so by using
terrible variable names.

From the first "PSR7 Example", we could write the code like:

$v = getGlobals();
$v = parseRequest($v);
$request = buildPsr7Request($v);

If you truly believe that not having semantically meaningful
intermediate variable names, you would have no problem with code being
written like this, right?

Personally, I always use semantically meaningful variable names, as
they make life so much easier for any person who needs to read the
code.

cheers
Dan

I like the proposal, but readability should be on the forefront of things
that need to be done right. Yes, there are people that will make it an
unreadable mess, but we have code reformat, so if the syntax is verbose
enought to be read at a glance, I'm all for it.

+1 as long as syntax gets a hard look at and examples in RFC are amended to
show it better :)

I would say that this makes the entire functional approach:

1. more readable
2. easier to debug (woah, we can haz line numbers for failures!)

Here's the pseudo-code for a typical request/response dispatch cycle:

$request = getGlobals()
|> parseRequest($$)
|> buildPsr7Request($$);

$response = loadConfig()
|> buildDic($$)
|> getApp($$)
|> getRouter($app)
|> getDispatcher($$, $request)
|> dispatchBusinessLogic($$, $request, new Response())
|> renderResponse($$)
|> buildPsr7Response($$)
|> emit($$);

Here's what it would look like with a separate approach (more
business-oriented):

buildRequest() // (basically the first part of the previous example here)
|> validate($$)
|> convertToCommand($$)
|> execute($$)
|> convertToViewModel($$)
|> render($$)
|> convertToHttpResponse($$)
|> emit($$)

I think this is much more readable/clear than any event-driven or
procedural approach.
We know exactly what is going on, we will get clear stack traces (this
could need some fine-tuning of the current PR - I didn't try it out yet)
and we will be able to introduce intermediate steps with ease. In addition
to all that, this will ease reactive programming by a lot, as piping is a
natural fit for streams and all abstractions around streams.

The syntax is also already effective in F# (you can write very expressive
and easy to understand code with it) and now also landed in Hack.

I'd say +10 to this proposal :D

Marco Pivetta

http://twitter.com/Ocramius

http://ocramius.github.com/

Hi!

This is one of my favorites out of HackLang. It's pure syntactic
sugar, but it goes a long way towards improving readability.
https://wiki.php.net/rfc/pipe-operator

I think this takes the syntax too much towards Perl 6 direction. There
is a lot of fun in inventing cool signs like |> and making them do
nontrivial cool stuff. There's much less fun when you try to figure out
what such code is actually doing - and especially why it is not doing
what you thought it is doing. Brian Kernigan once said:

"Everyone knows that debugging is twice as hard as writing a program in
the first place. So if you're as clever as you can be when you write it,
how will you ever debug it?"

I think this applies not only to debugging, but also to reading programs
in general. If we are "as clever as we can be" with syntax, the program
becomes unreadable.

For this feature specifically, I see nothing wrong with assigning
intermediate results to variables - moreover, I see an advantage to it
both in reading (you tell people what you expect to have happened in
this stage) and in debugging (you can actually see what happened and
check whether what you thought should be happening actually happened).

If we could achieve the elegance of Unix pipes, I'd agree that it is
simple enough that we can see what is going on - but having it rely on
two magic things which are in different part of expression and
context-dependent ($$ means different things in different places) is too
much for me.

--
Stas Malyshev
smalyshev@gmail.com

I think this is much more readable/clear than any event-driven or
procedural approach.
We know exactly what is going on, we will get clear stack traces (this
could need some fine-tuning of the current PR - I didn't try it out yet)
and we will be able to introduce intermediate steps with ease. In addition
to all that, this will ease reactive programming by a lot, as piping is a
natural fit for streams and all abstractions around streams.

The syntax is also already effective in F# (you can write very expressive
and easy to understand code with it) and now also landed in Hack.

I'd say +10 to this proposal :D

I am with Stanislav on this one or rather meh. It also goes against what
you usually advocate: there should be only one way to achieve things (I
remember your /fluent interface/ example very well). Unless you changed
your mind regarding that. Hence, it is nice syntactic sugar but it does
not allow us to solve problems that we cannot solve already.

--
Richard "Fleshgrinder" Fussenegger

Here's the pseudo-code for a typical request/response dispatch cycle:

Hi Marco,

Could you clarify something for me?

If I submitted a pull-request to you, with variables named like this:

$v = buildRequest();
$v = validate($v);
$v = convertToCommand($v);
$v = execute($v);
$v = convertToViewModel($v);
$v = render($v);
$v = convertToHttpResponse($v);
emit($v);

There is no chance that you would accept it. You would (correctly)
observe that the variables for the intermediate steps have meaningless
names and so the code is hard to reason about. I don't think removing
the presence of the intermediate variables and replacing them with $$
improves this situation.

What is it that having a special pipe operator that make this
acceptable, other than it covers up the lack of names for the
intermediate variables, by using a different syntax?

Marco wrote:

Relevant: https://youtu.be/UvD1VjRvGIk

I could imagine how having inline branches could be a useful thing for
functional programming, for various scenarios, including being able to
'inline' error handling, to be nearer the source of errors. However
this RFC does not propose that.

cheers
Dan

However,
passing an array to array_filter() then passing the result to array_map() is always type safe (because array_filter still returns an
array even if it filters down to 0 items), and array_map on an empty
array is essentially a no-op, so I'm comfortable doing so, and wish I
could do so more often with less fugly syntax. :-)

Hi!

See, I take that quote in the exact opposite direction. I find chaining
methods to be far, far less "clever" than deeply nesting them inside

If you tell me that syntax like "foo() |> bar($$)" is more natural or
intuitive or readily understandable to anyone with any programming and
PHP background than "$fooResult = foo(); bar($fooResult); " then we
indeed have exactly opposite understanding of what is natural and intuitive.

I think that overly clever is inventing cryptic syntax to save a couple
of keystrokes and rearrange code in unusual pattern that looks unlike
the code used to look before and resembles some other language (this
time it's F#? how many languages PHP should be at once - can we get a
dozen?)

each other. We shouldn't force pointless intermediate variables on

Why you think they are pointless? And you do produce them, you just hide
them behind $$ so you can not actually neither check them nor understand
which value goes where without tracing through the code with your finger.

people for the sake of readability. The fair comparison is the deeply
nested version vs. the chained version, and in that comparison the
readability of the chained version is far, far better.

No, it's not a fair comparison. You shouldn't do deep nesting, and you
shouldn't do cryptic syntax either. Variables are not evil. They are
there to make things easier for you. Use them.

intermediate variable to analyze and nothing has changed. However,
passing an array to array_filter() then passing the result to
array_map() is always type safe (because array_filter still returns an
array even if it filters down to 0 items), and array_map on an empty
array is essentially a no-op, so I'm comfortable doing so, and wish I
could do so more often with less fugly syntax. :-)

I appreciate the careful choice of the example on RFC - indeed, in some
very carefully chosen cases (mostly with arrays) you could get away with
chaining without getting into trouble. In most cases where it would be
used, though, that would mean not checking for errors, forgetting corner
cases and so on. And that's how people would use this, because none of
the existing functions were developed to work with this pattern, so you
can only get away with it due to luck.

In strictly typed languages like Haskell, they use types like Option and
Maybe to enable this pattern - essentially, to enable functions to
operate on more than one simple type of value. However, it doesn't look
in place in PHP, and it would require much more deep restructuring to
actually make it work than just having |> as an operator.

of promises", but more "is there something we can do here to be forward
thinking, since lots of people want to see async in core PHP?")

I'm not sure what promises have to do with inventing strange syntax to
pass result of one function as a parameter to another.

Stas Malyshev
smalyshev@gmail.com

Hi!

See, I take that quote in the exact opposite direction. I find chaining
methods to be far, far less "clever" than deeply nesting them inside

If you tell me that syntax like "foo() |> bar($$)" is more natural or
intuitive or readily understandable to anyone with any programming and
PHP background than "$fooResult = foo(); bar($fooResult); " then we
indeed have exactly opposite understanding of what is natural and intuitive.

I think that overly clever is inventing cryptic syntax to save a couple
of keystrokes and rearrange code in unusual pattern that looks unlike
the code used to look before and resembles some other language (this
time it's F#? how many languages PHP should be at once - can we get a
dozen?)

each other. We shouldn't force pointless intermediate variables on

Why you think they are pointless? And you do produce them, you just hide
them behind $$ so you can not actually neither check them nor understand
which value goes where without tracing through the code with your finger.

people for the sake of readability. The fair comparison is the deeply
nested version vs. the chained version, and in that comparison the
readability of the chained version is far, far better.

No, it's not a fair comparison. You shouldn't do deep nesting, and you
shouldn't do cryptic syntax either. Variables are not evil. They are
there to make things easier for you. Use them.

intermediate variable to analyze and nothing has changed. However,
passing an array to array_filter() then passing the result to
array_map() is always type safe (because array_filter still returns an
array even if it filters down to 0 items), and array_map on an empty
array is essentially a no-op, so I'm comfortable doing so, and wish I
could do so more often with less fugly syntax. :-)

I appreciate the careful choice of the example on RFC - indeed, in some
very carefully chosen cases (mostly with arrays) you could get away with
chaining without getting into trouble. In most cases where it would be
used, though, that would mean not checking for errors, forgetting corner
cases and so on. And that's how people would use this, because none of
the existing functions were developed to work with this pattern, so you
can only get away with it due to luck.

In strictly typed languages like Haskell, they use types like Option and
Maybe to enable this pattern - essentially, to enable functions to
operate on more than one simple type of value. However, it doesn't look
in place in PHP, and it would require much more deep restructuring to
actually make it work than just having |> as an operator.

of promises", but more "is there something we can do here to be forward
thinking, since lots of people want to see async in core PHP?")

I'm not sure what promises have to do with inventing strange syntax to
pass result of one function as a parameter to another.

Stas Malyshev
smalyshev@gmail.com

In a way... Sara, this syntax feels like it's only one step removed from
promises; if any of the chained steps involve IO, it becomes basically
what promises are for. Am I barking down the wrong tree, or is there
something we could connect there? (I don't mean "reject this in favor
of promises", but more "is there something we can do here to be forward
thinking, since lots of people want to see async in core PHP?")

--Larry Garfield

I see the connection to promises too; it's not just you :)

I might be thinking about it wrong, but I think that "one step" you're
looking for is handling the failure case (Sara kinda mentioned it in a
later post). Promises are essentially matching two cases:
Success/Ok/Some, and Failure/Error/None. This RFC seems to only handle
the first case. Sure, you could use exception handling:

try {
$ret = scandir($arg)
|> array_filter($$, function($x) { return $x !== '.' && $x !=
'..'; })
|> array_map(function ($x) use ($arg) { return $arg . '/' . $x;
}, $$)
|> getFileArg($$)
|> array_merge($ret, $$);
catch (Throwable $e) {
}

But not all failures or "errors" are actually exceptions. I'm not sure
how to approach this cleanly without monads, really.

The other issue is with closures and callbacks. Unless, at some point in
the future when the heroes of PHP gift us with async in the core pipe
operator is redefined, async operations need to be resumable later.

Consider a small hypothetical I/O example:

$data = new Socket()
|> (function ($socket) {
// Um, how do I yield/await here? The next pipe function
// expects an immediate return...
return $socket->connect(/* ... */);
})($$) // <-- also, this is awkward...
|> readUntil($$, '\n')

The benefit of promises is that everything is eventually defined in
terms of callbacks. That way things can be executed whenever. From my
understanding (correct me if I'm wrong) the pipe operator does not
enable this.

Also, +1 for the idea, and +1 for considering the promises case! Using
the pipe operator for async operations would be super cool!

--
Stephen

On Sun, May 1, 2016 at 4:05 AM, Larry Garfield larry@garfieldtech.com
wrote:

In a way... Sara, this syntax feels like it's only one step removed from
promises; if any of the chained steps involve IO, it becomes basically what
promises are for. Am I barking down the wrong tree, or is there something
we could connect there? (I don't mean "reject this in favor of promises",
but more "is there something we can do here to be forward thinking, since
lots of people want to see async in core PHP?")

If you had async/await, you could of course use |> to chain async
operations in an async function by using "await $$" in place of "$$" (since
$$ would be the Promise/Awaitable):

$ret = scandir_async($arg)
|> array_filter(await $$, ...)
|> ...

That's quite neat, IMO, and the features are still orthogonal.

You could introduce a new operator to combine the "|>" and "await", say
"a|>", but to add an operator just to combine two others would seem a bit
arbitrary.

You could also generalise |> as a monadic bind (with Promise as the monad
for asynchronicity) but I don't know how that would fit in PHP land without
a meaningful way to turn an expression into a closure. It could be added
later as something like "|>>" if need be.

Relevant: https://youtu.be/UvD1VjRvGIk

Trimming the now useless error code ....

As I said in the message ... no problem if you simply have a SINGLE
pathway through the code. That video simply assumes that anything that
is not success is an error. MUCH of my code base has turntables or three
way switches which in addition to possible errors, you have a lot more
than one 'success' output.

The pipe operator does not provide a work flow as demonstrated in that
video ...

--
Lester Caine - G8HFL

Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

Let's say I want to add a condition just before getFileArg(); with the
current version I've got to:

• go to the beginning of the chain, and assign to something other than
  $ret
• go to where I want to break the chain, and reintroduce the $ret
  assignment
• add my check in the gap, using the variable I just added at the
  beginning of the chain

$fileList = scandir($arg)
|> array_filter($$, function($x) { return $x !== '.' && $x != '..';
})
|> array_map(function ($x) use ($arg) { return $arg . '/' . $x; },
$$);
if ( someCheck($fileList) {
something();
}
$ret = getFileArg($$)
|> array_merge($ret, $$);

The syntax is fighting me here, making me jump around the code. But if
assignment was always on the end of the chain, I would only need to make
changes at the point where I was breaking the chain. The basic pattern
would be:

|=> $tempVar; // terminate the chain and capture the value
// do stuff with $tempVar
$tempVar // restart the chain

So:

scandir($arg)
|> array_filter($$, function($x) { return $x !== '.' && $x != '..';
})
|> array_map(function ($x) use ($arg) { return $arg . '/' . $x; },
$$)
|=> $fileList;
if ( someCheck($fileList) {
something();
}
$fileList
|> getFileArg($$)
|> array_merge($ret, $$)
|=> $ret;

If I don't need the condition any more, I can delete lines 4 to 8, and
I've got back my original chain.

Could you use a closure instead to accomplish this? (Again yes, Sara could
you clarify if this is permitted?)

$ret = scandir($arg)
|> array_filter($$, function($x) { return $x !== '.' && $x !=
'..'; })
|> array_map(function ($x) use ($arg) { return $arg . '/' . $x; },
$$)
|> (function($fileList) {
if (someCheck($fileList)) {
something();
}
return $fileList;
})($$)
|> getFileArg($$)
|> array_merge($ret, $$);

Not completely the best, but perhaps there's some sort of an idea here?

--
Stephen

--

​Doesn't Nikic's scalar objects (https://github.com/nikic/scalar_objects)
more or less achieve the same thing while also cleaning up the std lib?

$ret = scandir($arg)
->filter(function(){})
->map(function(){})
->merge($someOtherArray);

Terry Cullen

Could you use a closure instead to accomplish this? (Again yes, Sara could
you clarify if this is permitted?)

$ret = scandir($arg)
|> array_filter($$, function($x) { return $x !== '.' && $x != '..';
})
|> array_map(function ($x) use ($arg) { return $arg . '/' . $x; },
$$)
|> (function($fileList) {
if (someCheck($fileList)) {
something();
}
return $fileList;
})($$)
|> getFileArg($$)
|> array_merge($ret, $$);

You could, but IMO you lose a lot of the elegance if you go that
route. This syntax works best when you have a series of functions
which /don't/ have failure cases (and many many functions don't), or
who fail via exceptions.

Using current syntax, the above would look like:

$ret = array_merge($ret, getFileArg(function($fileList) {
if (someCheck($fileList)) { something(); }
return $fileList;
})(array_map(function($x) use ($arg) { return $arg . '/' . $x; },
array_filter(scandir($arg), function($x) { return $x !== '.' &&
$x !== '..'; }))
)));

It's way more readable in the pipe syntax version, but it's
overloading a single statement in both places.

-Sara

Stephen Coakley wrote on 03/05/2016 01:57:

The basic pattern would be:

|=> $tempVar; // terminate the chain and capture the value
// do stuff with $tempVar
$tempVar // restart the chain

So:

scandir($arg)
|> array_filter($$, function($x) { return $x !== '.' && $x !=
'..'; })
|> array_map(function ($x) use ($arg) { return $arg . '/' . $x;
}, $$)
|=> $fileList;
if ( someCheck($fileList) {
something();
}
$fileList
|> getFileArg($$)
|> array_merge($ret, $$)
|=> $ret;

If I don't need the condition any more, I can delete lines 4 to 8, and
I've got back my original chain.

Could you use a closure instead to accomplish this? (Again yes, Sara
could you clarify if this is permitted?)

$ret = scandir($arg)
|> array_filter($$, function($x) { return $x !== '.' && $x !=
'..'; })
|> array_map(function ($x) use ($arg) { return $arg . '/' .
$x; }, $$)
|> (function($fileList) {
if (someCheck($fileList)) {
something();
}
return $fileList;
})($$)
|> getFileArg($$)
|> array_merge($ret, $$);

Not completely the best, but perhaps there's some sort of an idea here?

The disadvantage here is that you're still having to mutate your code to
fit the piped style, even if it's just wrapping into the closure. I was
aiming for a style that would let me freely mix piped and unpiped code.

Meanwhile, I still see having the assignment at the end of the chain as
an end in itself - the ability to use the pipe operator as an expression
seems like a nightmare waiting to happen to me. As currently proposed,
you could write this, which is just horrible:

if (
scandir($arg)
|> array_filter($$, function($x) { return $x !== '.' && $x !=
'..'; })
|> count($$)
>
scandir($arg)
|> count($$)
) { ... }

Obviously, it's always possible to show horrible abuses of a feature,
but I can't think of any sensible place to use a pipe as an expression
except in order to either assign it to a variable, or return it from a
function.

Regards,

Rowan Collins
[IMSoP]

On Tue, May 17, 2016 at 9:14 PM, Rowan Collins rowan.collins@gmail.com
wrote:

To reiterate my comments from earlier in the thread, I think the "$id =
$id" looks really weird here, and spoils the step-by-step layout - and, in
this case, the symmetry.

We have the same thing already when chaining methods on value/immutable
objects, like with PSR-7:

$message = $message
->withHeader('foo', 'bar')
->withAddedHeader('foo', 'baz');

So depending on often that pattern is used, many are probably already used
to seeing "$var = $var ....", myself included.

$id
|> StringList::decode($$)
|> array_replace($$, $replace)
|> StringList::encode($$)
|> $id;

Sara, what do you think of this tweak, and my subsequent suggestion that the
pipe should only be usable as a statement, not an expression?

I'm not strictly against it, but I'm not a huge fan. Primarily, it's
the statement-only restriction that bothers me since (apart from
control flow structures), most constructs in PHP are expressions, not
statements, and this feature naturally fits in the composable
expressions category, not the top-level statement category. From an
implementation standpoint, I'm fairly confident that the patch would
be much more complex as a result of enforcing that statement-only
requirement.

Meanwhile, if we remove the statement-only requirement, then using
"simple-variable" as a signal to switch to expression termination,
then we introduce a whole class of bug where a user fat-fingers the $$
token in what SHOULD be another link in a chain expression, but it's
not detectable as an error because a simple-var is valid.

And one more counter-example:

$x
|> foo($$)
|> $y[$$];

This is no longer a simple var, should that terminate our statement by
writing to $y[$$]? Or should that be seen as the next link in a chain
where we read the value from $y[$$] (which could well have getter side
effects)? I would probably expect the latter, but maybe the former
was intended?

So yeah, I'm really not a fan. That said, if enough other people
chime in that they think these issues are surmountable in "a PHP way",
then I'll take a crack at a PR and we can include it in the
discussion, but as-is.... I can't quite get behind it.

-Sara

If we restrict usage to function calls, we can quite easily get rid of the $$ by
associating to each function the position where the lhs must be inserted
when used as a 'piped function call'.

So we'd have to audit all 4k+ functions in the PHP runtime (and
provide a mechanism for defining it on userspace functions)?

That feels like a LOT of work for an alternate approach to something
which takes much less work as proposed. I'm hoping I misunderstand
what you're suggesting...

Getting rid of the $$ would resolve most issues about parameter order in the
existing library, which is probably the biggest PHP sadness.

I will admit that anything which alleviates the sadness from argument
ordering in the standard lib will be nice. I'm just skeptical of the
work it'll take to get there.

My question, before I write a detailed proposal for this, is : do you want
to keep the possiblility to use any expression between pipe operators, or
would it be acceptable to restrict this to function calls?

I'd like to keep the ability to use arbitrary expressions as it keeps
the language overall more consistent. That said, nothing is set in
stone and the point of a discussion period is to discuss. Maybe with
your proposal written out it'll make more sense to me.

-Sara