[RFC] Explicit call-site pass-by-reference (again)

Am 21.02.2020 um 00:04 schrieb Larry Garfield larry@garfieldtech.com:

Just chiming in to voice strong support for this RFC. This is a key
piece toward making PHP code statically analyzable. If it becomes
required at the call site, such as in an edition of the language, it
will significantly enhance the ability to reason about code and
probably make it more correct as well. As a small example, consider
this method on an Optional type class:

function map(callable $f): Optional {
if ($this->enabled) {
return new Optional($f($this->data));
} else {
return $this;
}
}

The intent is to return a new optional or an empty one, but if you
pass a closure that accepts something by reference you can change the
original, which is not intended at all. For people who defend against
it, it requires saving $this->data to a local variable, then passing
in the local. Then if the user does a call-by-reference it will affect
the local, not the object's data.

If $this->data is itself an object, then you have a concern for data manipulation (spooky action at a distance) even if it's passed by value. Given how much data these days is objects, and thus the problem exists regardless of whether it's by value or by reference passing, adding steps to make pass-by-reference harder doesn't seem to help much.

+1

The whole discussion about being worried about 'malicious' libraries altering your precious scalar values misses the fact that PHP is not a pure language, there are many ways a function can have side-effects, Larry pointing out one obvious one.
Speaking of language editions: Trying to solve one obscure case (and one which is easily enough detectable by statical analysis) by introducing such a big BC break could render a whole edition ineligible for a software project. So beware, features bundled in one (hypothetical) edition better not break too many different things at the same time.

If you don't trust your library code then you're in deep trouble anyway.

• Chris

On Fri, Feb 21, 2020 at 12:05 AM Larry Garfield larry@garfieldtech.com
wrote:

Just chiming in to voice strong support for this RFC. This is a key
piece toward making PHP code statically analyzable. If it becomes
required at the call site, such as in an edition of the language, it
will significantly enhance the ability to reason about code and
probably make it more correct as well. As a small example, consider
this method on an Optional type class:

function map(callable $f): Optional {
if ($this->enabled) {
return new Optional($f($this->data));
} else {
return $this;
}
}

The intent is to return a new optional or an empty one, but if you
pass a closure that accepts something by reference you can change the
original, which is not intended at all. For people who defend against
it, it requires saving $this->data to a local variable, then passing
in the local. Then if the user does a call-by-reference it will affect
the local, not the object's data.

If $this->data is itself an object, then you have a concern for data
manipulation (spooky action at a distance) even if it's passed by value.
Given how much data these days is objects, and thus the problem exists
regardless of whether it's by value or by reference passing, adding steps
to make pass-by-reference harder doesn't seem to help much.

If you will allow me some exaggeration, what you're basically saying here
is that all the const / readonly / immutability features in (nearly) all
programming languages are useless, because they (nearly) always allow for
interior mutability in one way or another. "const" in JavaScript doesn't
allow you to rebind the object, but you can still modify the object. Same
with "final" in Java. Similar things hold in C/C++/Rust when it comes to
const pointers/references to structs that contain non-const
pointers/references. And of course, the "readonly" RFC for PHP that is
currently under discussion has the same characteristics.

What I'm trying to say here: All of these features do not guarantee
recursive immutability, but that doesn't render them useless in the least.
In fact, the outer-most layer is where immutability is the most important,
because there's a lot of difference between

$i = 0;
var_dump($i); // int(0)
foo($i);
var_dump($i); // array(7) { ... }
// WTF just happened???

and

$o = new Foo();
var_dump($o); // object(Foo) #42 { xxx }
foo($o);
var_dump($o); // object(Foo) #42 { yyy }
// Did something change in there? Doesn't really matter for this code!

One of the big differences is that by-reference passing can change the
type of the variable, while by-object passing cannot. It cannot even
change object identity.

On a closing note: I don't think this RFC makes passing by reference
"harder" in any meaningful sense. Yes, you do need to write one extra
character. In exchange, every time you read code you will immediately see
that by-reference passing is used, here be dragons.

Regards,
Nikita

On Feb 21, 2020, at 5:20 PM, Rowan Tommins rowan.collins@gmail.com
wrote:
My instinctive reaction is still one of frustration that the pain of
removing call-site ampersands was in vain, and I will now be asked to
put most of them back in.

That is a great example of what is known as a "sunken cost."

Perhaps, yes. I freely admit it's an emotional reaction rather than a rational one.

One of the reasons it is confusing is because developers are currently
required to use the ampersand in one place and not the other. Making
it always used removes said confusion as they would no longer be a
reason to have to remember when and when not to use the ampersand
anymore.

Maybe. I think a larger part of it is that references themselves are a slightly confusing concept, and the fact that & looks like an operator of its own (and is often documented that way) but is really an annotation on other operators/commands. That is, the & in $foo = &$bar and return &$bar doesn't modify $bar, it modifies = and return, respectively.

Making the rules more logical and symmetrical would perhaps be more helpful to new users than it is to established users, particularly those who've known multiple versions of the language already.

There is a potential "PR" cost of this change that should be weighed
against the advantages.

To say "We fixed something that in hindsight we've since determined was
a problem." How is this a concern?

The concern is that the costs will be much more visible to users than the benefits, and they will resent the core developers pushing that requirement onto them, rather than thanking then for their hard work.

As I said, that's not an absolute reason not to do it, it's a cost to be weighed.

I'm also not very keen on internal functions being able to do things
that can't be replicated on userland, and this RFC adds two: additional
behaviour for existing "prefer-ref" arguments, and new "prefer-value"
arguments

So what specific problems would having these enhancement cause for the
language?

There are two problems I have with internal-only features in general: the inability to polyfill and extend, and the requirement for a separate mental model.

As an example of the first, the RFC mentions using call_user_func with a call-site annotation to forward the parameter by reference. The reason for allowing that also applies to a user-defined wrapper like call_with_current_user or call_with_swapped_parameters, but there's no syntax for those to be marked "prefer-val".

As an example of the second, even under strict settings, calls to certain internal functions will have an optional & at the call site, which changes their behaviour. To those without knowledge of the core, those functions simply have to be remembered as "magic", because their behaviour can't be modelled as part of the normal language.

My current opinion is that I'd rather wait for the details of out and
inout parameters to be worked out, and reap higher gains for the same
cost. For instance, if preg_match could mark $matches as "out", I'd be
more happy to run in a mode where I needed to add a call-site keyword.

This sounds like preferring perfect in the (potentially distant) future
vs. much better today.

No, it's preferring to hold out for a little bit more value to weigh against my evaluation of the cost.

This is, when followed through to its conclusion of mandatory marking, a disruptive change to every piece of code, so we need to decide if the disruption is worth it.

It's also the second change in the same place, and we should be sure that we've got it right this time, and won't require a third change in the near future. For instance, if out parameters were added, would the same line of code end up going from optional &, to forbidden &, to mandatory &, to mandatory "out"?

I'm not strongly against the idea, but the advantages just don't feel quite strong enough, so if I had a vote, I'd currently be inclined to vote no.

Regards,

--
Rowan Tommins
[IMSoP]

The prefer-ref/prefer-val thing is indeed a bit peculiar. It's an
artifact of the current way of implicit by-reference passing, where
the decision of whether to pass by-value or by-reference has to be
made based on an "educated guess" at the call-site. That leaves us
with always-val, always-ref, prefer-val and prefer-ref as the possible
passing modes. In the explicit by-ref passing regime, the latter two
consolidate, and we have by-val, by-ref and "either" as the options,
which is a lot more obvious.

Thanks, that's a good summary of how this all relates to the RFC. If
there is a use case for such a mode, perhaps we need a way to annotate
userland functions as "either", so that they too can take advantage of
the call-site annotation. In a sense, they'd be opting in to the pre-5.4
behaviour, for that particular parameter.

Instead of having __call(), what we really should have is __get_method().

That is a really interesting idea. The lack of function signature is
currently a big turn-off for using __call, because it means manually
recreating a lot of the unpacking and type checking that the language
would normally do for you.

I believe we talked about this in some detail in the previous
discussion on this topic. My basic stance on in/out is that it's
probably not worth the complexity, unless it is part of an effort to
eliminate references from PHP entirely (which would be hugely
beneficial). Unfortunately I don't really see a clear pathway towards
that. "out" parameters can remove one use-case of references, and I
can see how that would work both in terms of semantics and
implementation. The case of "inout" parameters is much more
problematic. While these can nominally work without references, I
don't see how they could do so efficiently (we would have to "move
out" the value from the original location to avoid COW). Similarly, I
don't have any answer to how &__get() and &offsetGet() would work
without references.

That's fair enough. I guess the reason I've fixated on them is that I'd
really like "out" parameters, independent of what else happens with
references, in order to get the clear signal of "variable is initialised
here" on a call like "preg_match($foo, $bar, out $matches)". That would
make out parameters more attractive to build APIs around, e.g. when you
want multiple strongly typed outputs from one call.

Even if we can't eliminate references entirely, perhaps there's value in
reducing the use cases where they're necessary? A bit like how property
accessor syntax wouldn't allow us to remove __get and __set, but would
mean fewer cases where people needed to deal with them.

Regards,

--
Rowan Tommins (né Collins)
[IMSoP]