[RFC] Context Managers

Hello.

Thank you for the RFC.
An excellent tool for a language that supports interfaces.

--
Ed

Arnaud and I would like to present another RFC for consideration: Context
Managers.

https://wiki.php.net/rfc/context-managers

You'll probably note that is very similar to the recent proposal from Tim
and Seifeddine. Both proposals grew out of casual discussion several
months ago; I don't believe either team was aware that the other was also
actively working on such a proposal, so we now have two. C'est la vie. :-)

Naturally, Arnaud and I feel that our approach is the better one. In
particular, as Arnaud noted in an earlier reply, __destruct() is unreliable
if timing matters. It also does not allow differentiating between a
success or failure exit condition, which for many use cases is absolutely
mandatory (as shown in the examples in the context manager RFC).

The Context Manager proposal is a near direct port of Python's approach,
which is generally very well thought-out. However, there are a few open
questions as listed in the RFC that we are seeking feedback on.

Discuss. :-)

Great idea, I'm definitely behind this.

I've also read through all the PR code.

I have a few questions

1. Apart from wrapping zend_resource into ResourceContext, has there been
   discussions or ideas to wrap other things ?

2. Are there any scenarios where using with() is a bad idea or has "side
   effects"?

3. In the implementation code there is a lot of mention of "list" and
   zend_list .. why? Maybe the answer is obvious but I can't see, at first
   glance, why we are implementing list under the hood.

Thanks, and great work to both of you!

--
Larry Garfield
larry@garfieldtech.com

Arnaud and I would like to present another RFC for consideration: Context
Managers.

https://wiki.php.net/rfc/context-managers

You'll probably note that is very similar to the recent proposal from Tim
and Seifeddine. Both proposals grew out of casual discussion several
months ago; I don't believe either team was aware that the other was also
actively working on such a proposal, so we now have two. C'est la vie. :-)

Naturally, Arnaud and I feel that our approach is the better one. In
particular, as Arnaud noted in an earlier reply, __destruct() is unreliable
if timing matters. It also does not allow differentiating between a
success or failure exit condition, which for many use cases is absolutely
mandatory (as shown in the examples in the context manager RFC).

The Context Manager proposal is a near direct port of Python's approach,
which is generally very well thought-out. However, there are a few open
questions as listed in the RFC that we are seeking feedback on.

Discuss. :-)

--
Larry Garfield
larry@garfieldtech.com

Great RFC and I really like how much more readable the code can become with
this approach.

Out of curiosity, what happens if GOTO is used inside a context block to
jump away from it?

Could the RFC clarify the relation between Context and switch/case? I
thought it was really odd that something that triggers a warning on
switch/case is being introduced into a brand new language construct
basically creating the possibility for new code to fall into the same trap
as opposed to avoiding it in the first place. Specially a construct like
switch/case that has been in decline for over a decade and ever since match
came out on 8.0, switch case is practically deprecated without actually
being deprecated yet. What’s the importance/relevance of being consistent
with it?

While we’re at it, do we really need break; statements inside context
blocks? If you want out you can:

• return
• throw

In the case of a nested block (break 2;) where I don’t want to wrap the
entire thing in try/catch, it seems like a GOTO out of it would be more
meaningful with text-based identifiers rather than number-based, which
leads to my first question (although I was more curious than actually
making an argument for it because I would rather avoid nested with as much
as possible).

Marco Deleu

<larry@garfieldtech.com

Arnaud and I would like to present another RFC for consideration: Context Managers.

https://wiki.php.net/rfc/context-managers

You'll probably note that is very similar to the recent proposal from Tim and Seifeddine. Both proposals grew out of casual discussion several months ago; I don't believe either team was aware that the other was also actively working on such a proposal, so we now have two. C'est la vie. :-)

Naturally, Arnaud and I feel that our approach is the better one. In particular, as Arnaud noted in an earlier reply, __destruct() is unreliable if timing matters. It also does not allow differentiating between a success or failure exit condition, which for many use cases is absolutely mandatory (as shown in the examples in the context manager RFC).

The Context Manager proposal is a near direct port of Python's approach, which is generally very well thought-out. However, there are a few open questions as listed in the RFC that we are seeking feedback on.

Discuss. :-)

Larry, Arnaud,

I really like this RFC but have a couple of things to discuss:

• automatically re-throwing exceptions: I think that this behavior, especially with a boolean return value deciding if it happens or not is not intuitive. I think a better approach is to do nothing with the exception and let the user re-throw it if desired. I can't think of anywhere else we re-throw exceptions unless the user indicates otherwise. I'd rather leave the return value for return values; we could expand this allow access to the return value like: with (foo() as $foo return $bar) { }, and $bar would be set to null on void returns.

• context variable and scope: I know that you explicitly are not creating a new scope, this means that the context variable will clash with the enclosing scope namespace, and then the variable will be unset after the context ends, this doesn't sit so well with me. I think I'd rather see the same behavior as arrow function arguments, where it does not override variables of the same name in the enclosing scope and whatever value it has is lost at the end of the context, leaving the outer scope version intact.

At worst though, I'm sure IDE and static analyzers will be able to detect the "use after unset" behavior with clashing variable names, causing the developer to resolve it, and it'll be fine either way.

Thanks for the great RFC!

• Davey

Hi

Is the keyword "with" reserved, semi-reserved, ... ?
Asking because I've seen global "with" functions before, like in Laravel for example.

What really is a footgun of the resource type is that they can represent an illegal state.
This proposal leans into having close functions (or equivalent) for resources/objects that make it possible to represent such illegal state; something I'm fundamentally against.

As an aside, under "Rejected Features", there's a false statement:

Destructors are not always called immediately when the refcount hits zero.

Kind regards
Niels

Arnaud and I would like to present another RFC for consideration:
Context Managers.

https://wiki.php.net/rfc/context-managers

You'll probably note that is very similar to the recent proposal from
Tim and Seifeddine. Both proposals grew out of casual discussion
several months ago; I don't believe either team was aware that the
other was also actively working on such a proposal, so we now have two.
C'est la vie. :-)

Naturally, Arnaud and I feel that our approach is the better one. In
particular, as Arnaud noted in an earlier reply, __destruct() is
unreliable if timing matters. It also does not allow differentiating
between a success or failure exit condition, which for many use cases
is absolutely mandatory (as shown in the examples in the context
manager RFC).

The Context Manager proposal is a near direct port of Python's
approach, which is generally very well thought-out. However, there are
a few open questions as listed in the RFC that we are seeking feedback
on.

Discuss. :-)

Hi all. I'm going to reply to several people at once in a single message, for simplicity:

1. Apart from wrapping zend_resource into ResourceContext, has there
   been discussions or ideas to wrap other things ?

Not really. The intent is that users can create their own "setup and teardown" logic packages and do with them as they please. Resources are just an oddball case in PHP, because reasons. I'm not sure what other auto-wrapping cases would make sense.

That said, PHP could absolutely ship context managers for people to use explicitly. My ideal way to address async would be exactly that: The Scope example from the RFC, where the only way to get to a scope (and therefore start coroutines) is via a context manager, and PHP iself provides the scope types we want to support. Nothing else.

There may be other managers that PHP would want to ship in the future for whatever reason, but that's out of scope for now. (No pun intended.)

2. Are there any scenarios where using with() is a bad idea or has
   "side effects"?

If you have a setup/teardown routine that is only used once or twice, then making a context manager for just that one use case is likely overkill. Just write try/catch/finally as normal.

I don't believe a context manager could handle this, although I've only rarely seen it in the wild:

$success = true;
try {
// ...
}
catch (\Exception $e) {
$success = false;
}

if ($success) { ... }

(That is, leaking a variable from the setup/teardown code into the surrounding scope. Though, I suppose this would work in a pinch:

$success = false;
with (new Foo() as $f) {
// ...
$success = true;
}

if ($success) { ... }

Not ideal, but would work.

We're still exploring the possibility of making the context manager keyword an expression rather than a statement, which might offer other alternatives. Still brainstorming.

3. In the implementation code there is a lot of mention of "list" and
   zend_list .. why? Maybe the answer is obvious but I can't see, at first
   glance, why we are implementing list under the hood.

I will defer to Arnaud here.

Have you considered returning enum instead of ?bool? It would have a
clear self explanatory meaning.

We have discussed that a bit, actually. The main concern is usability. The typical case will be to allow exceptions to propagate. If, say, a TypeError gets thrown 4 function calls down, you probably do want that to propagate to your top level handler, but still want to rollback your transaction or close your file or whatever. So the typical case should be easy, hence why we said null means the default behavior. And since null is falsy`, that fits neatly into a ?bool return; that is also what Python uses.

With an enum, you'd have a much longer thing to type, plus it's less self-evident what no-return means. Ie, you'd have:

function exitContext(?Exception $e): ContextResult
{
if ($e) {
$this->conn->rollback();
// This line becomes required.
return ContextResult::Propagate;
}

$this->conn->commit();
return ContextResult::Done; // Or, eh, what?
}

And not returning becomes a type error.

Alternatively, we could assume null implies one of the other cases; but as shown above, there's still the issue that the return type is only meaningful in case of an exception, so it's unclear how that interacts.

We're still open to discussion here. It also would play into the outstanding question of with as an expression.

Out of curiosity, what happens if GOTO is used inside a context block
to jump away from it?

That would be a success case, just like break or return. Basically anything other than an exception is a success case. (That said, please don't use Goto. :-) )

Could the RFC clarify the relation between Context and switch/case? I
thought it was really odd that something that triggers a warning on
switch/case is being introduced into a brand new language construct
basically creating the possibility for new code to fall into the same
trap as opposed to avoiding it in the first place. Specially a
construct like switch/case that has been in decline for over a decade
and ever since match came out on 8.0, switch case is practically
deprecated without actually being deprecated yet. What’s the
importance/relevance of being consistent with it?

break and continue are interesting keywords. (In the "may you live in interesting times" sense.) Sometimes they have the same effect, if a control structure is non-looping. Or they may have different effects in case it is. The main non-looping case is switch, where for reasons that were before my time the decision was made to deprecate continue in favor of just supporting break. However, blocking it entirely is a problem, because that would change where continue 2 would go (as switch would be removed as a "level" that it could go to). It is kind of a mess.

with is a non-looping control structure, and thus it seems logical to be consistent with other non-looping control structures. But, as noted, the other non-looping control structure is a mess. :-) Therefore, we get to choose between "a consistent mess" and "an inconsistent non-mess in one place and a mess in another."

Neither is a fantastic option. We're open to both, depending on what the consensus is.

While we’re at it, do we really need break; statements inside context
blocks? If you want out you can:

• return
• throw

Both of those exit the function the with statement is in, which is not always desireable.

In the case of a nested block (break 2;) where I don’t want to wrap the
entire thing in try/catch, it seems like a GOTO out of it would be more
meaningful with text-based identifiers rather than number-based, which
leads to my first question (although I was more curious than actually
making an argument for it because I would rather avoid nested with as
much as possible).

Because Goto was added to PHP as a troll, and not a feature you should actually use in production code 99.999% of the time. :-)

I really like this RFC but have a couple of things to discuss:

• automatically re-throwing exceptions: I think that this behavior,
  especially with a boolean return value deciding if it happens or not is
  not intuitive. I think a better approach is to do nothing with the
  exception and let the user re-throw it if desired. I can't think of
  anywhere else we re-throw exceptions unless the user indicates
  otherwise. I'd rather leave the return value for return values; we
  could expand this allow access to the return value like: with (foo() as
  $foo return $bar) { }, and $bar would be set to null on void returns.

Using the return value of exitContext() as the result of a "with expression" is something we are considering.

However, we're modeling on Python (the most robust such functionality we are aware of), and they rethrow by default. Essentially, the concept is that exitContext() (and it's Python equivalent magic method), is mostly a finally block, not a catch block. finally blocks do propagate exceptions. In practice, many exitContext() methods will not need to differentiate; they'll just close a file or whatever and move on with life, which is why you would want an exception to propagate. The inclusion of the exception parameter makes it a sort of combined catch/finally, so it has some behavior of each.

Another option we kicked around was splitting it into two methods; catchContext(Throwable $e) and exitContext(). However, that creates two other problems:

1. Because it's an interface, you would need to implement catchContext() all the time, even if you don't need it. That's very inconvenient. (Shamless plug for revisiting Levi's Interface Default Methods RFC, which would solve this issue: https://wiki.php.net/rfc/interface-default-methods) Using magic methods instead would avoid that problem, but then we're dealing with magic methods rather than a clearly-detectable interface.

2. If you need to run logic in both methods, do you duplicate it? Or worse, if you have logic that runs only on a success case, then what? Most likely you'd need to have your own $wasItAnError property inside the context manager object, which is ugly and annoying.

That said, we're open to other ways to structure this logic. But I think in practice it's true that most use cases will want to propagate the exception, after doing appropriate local cleanup.

• context variable and scope: I know that you explicitly are not
  creating a new scope, this means that the context variable will clash
  with the enclosing scope namespace, and then the variable will be unset
  after the context ends, this doesn't sit so well with me. I think I'd
  rather see the same behavior as arrow function arguments, where it does
  not override variables of the same name in the enclosing scope and
  whatever value it has is lost at the end of the context, leaving the
  outer scope version intact.

Arnaud says that masking the context variable itself is probably fairly straightforward, so we can go ahead and do that. However, masking every variable that gets created doesn't make sense. This construct is not creating a new "block scope" in the language. It's just desugaring into a reusable try-catch-finally construct.

If we wanted to have an actual local scope specific to the with block, then instead of the statement list we should have a callable, which in most cases would be an anon function. However, PHP's anon functions suck to use because of the need to explicitly use variables. That would effectively eliminate any benefit this feature offers, because you can already do $someWrapper->do($aCallable). But $aCallable needs a long list of use statements, which makes it fugly.

If anon functions were fixed, that would make that approach easier to do. However, that's been tried at least twice and it's been shot down both times, so I'm assuming we're stuck with a clunky anon function syntax indefinitely.

Also, off-list discussion has shown an interest in multiple context managers in one with block, which was one of the outstanding open questions. It looks like we'll probably include that, as it should be easy enough to do.

And now the big one... also in off-list discussion, Seifeddine noted that Laravel already defines a global function named with: https://github.com/laravel/framework/blob/12.x/src/Illuminate/Support/helpers.php#L510

And since this RFC would require with to be a semi-reserved keyword at the parser/token level, that creates a conflict. (This would be true even if it was namespaced, although Laravel is definitely Doing It Wrong(tm) by using an unnamespaced function.) Rendering all Laravel deployments incompatible with PHP 8.6 until it makes a breaking API change would be... not good for the ecosystem.

So that means using Python's with keyword here is not going to work. Damn.

A couple of other options have presented themselves, but we're open to other suggestions, too:

1. Java uses a parenthetical block on try for similar functionality (though without a separate context manager). That would look like:

try (new Foo() as $foo) {
// ...
}
// catch and finally become optional if there is a context.

Pros here is that it introduces no new keywords, and context managers are effectively "packaged try-catch-finally" logic, so it fits. Downsides are that it gets more confusing now that try only sometimes requires a catch or finally. The ordering between the context manager and explicit catch/finally blocks is also non-obvious. It would also entirely preclude context blocks being an expression, as try is already non-expressional.

2. Either use or using. The semantics here would be identical to the current with proposal.

Pros here are that use is already a reserved word, and using is, I hope, still available in practice. They could also be implemented as expressions if we figure out a way to do so. Downsides are that use is already used in a bunch of places to mean different things, so adding yet another contextual meaning just increases the complexity/confusion. using wouldn't have that issue, but we would still need to verify if it's available.

--Larry Garfield

Hey Larry, Tim, Seifeddine and Arnauld,

Arnaud and I would like to present another RFC for consideration: Context Managers.

https://wiki.php.net/rfc/context-managers

You'll probably note that is very similar to the recent proposal from Tim and Seifeddine. Both proposals grew out of casual discussion several months ago; I don't believe either team was aware that the other was also actively working on such a proposal, so we now have two. C'est la vie. :-)

Naturally, Arnaud and I feel that our approach is the better one. In particular, as Arnaud noted in an earlier reply, __destruct() is unreliable if timing matters. It also does not allow differentiating between a success or failure exit condition, which for many use cases is absolutely mandatory (as shown in the examples in the context manager RFC).

The Context Manager proposal is a near direct port of Python's approach, which is generally very well thought-out. However, there are a few open questions as listed in the RFC that we are seeking feedback on.

Discuss. :-)

I've been looking at both RFCs and I don't think either RFC is good
enough yet.

As for this RFC:

It makes it very easy to not call the exitContext() method when calling
enterContext() manually. The language (obviously) doesn't prevent
calling enterContext() - and that's a good thing. But also, it will not
enforce that exitContext() gets ever called (and it also cannot,
realistically).

Thus, we have a big pitfall, wherein APIs may expect enterContext() and
exitContext() to be called in conjunction, but users don't - with
possibly non-trivial side-effects (locks not cleared, transactions not
committed etc.). Thus, to be safe, implementers of the interface will
also likely need the destructor to forward calls to exitContext() as
well. But it's an easy thing to forget - after all, the intended usage
of the API just works. Why would I think of that, as an implementer of
the interface, before someone complains?

Ultimately you definitely will need the capability of calling
enterContext() and exitContext() manually too (i.e. restricting that is
not realistic either), as lifetimes do not necessarily cleanly nest - as
a trivial example, you might want to obtain access to a handle which is
behind a lock. You'll have to enter the context of the lock, enter the
context of the handle, and close the lock (because more things are
locked behind that lock, including the handle). ... But you don't
necessarily want the hold on the lock to outlive the inner handle. In
short: The proposed approach only allows nesting contexts, but not
interleaving them.

Further, calling with() twice on an object is quite bad in general. But
it might easily happen - you have a function which wants a transaction.
e.g. function writeSomeData(DatabaseTransaction $t) { with ($t) {
$t->query("..."); } }. A naive caller might think, DatabaseTransaction
implements ContextManager ... so let's wrap it: with($db->transaction()
as $t) { writeSomeData($t); }. But now you are nesting a transaction,
which may have unexpected side effects - and the code probably not
prepared to handle it. So, you have to add yet another safeguard into
your implementation: check whether enterContext() is only active once.
... Or, maybe a caller assumes that $t = $db->transaction(); with ($t) {
$t->query("..."); } with ($t) { $t->query("..."); } is fine - but the
implementation is not equipped to handle multiple calls to enterContext().

Additionally, I would expect implementers to want to provide methods,
which can be called while the context is active. However, it's not
impossible to call these methods without wrapping it into with() or
calling the enterContext() method explicitly. One more failure mode,
which needs handling.
Like for example, calling $t->query() on a transaction without starting it.

I don't like that design, which effectively forces you to put safety
checks for all but the simplest cases onto the ContextManager
implementation.
And it forces the user to recognize "this returned object
DatabaseTranscation actually implements ContextManager, thus I should
put it into with() and not immediately call methods on it". (A problem
which the use() proposal from Tim does not have by design.)

The choice of adding the exception to the exitContext() is interesting,
but also very opinionated:

• It means, that the only way to abort, in non-exceptional cases, is to
  throw yourself an exception. And put a try/catch around the with() {}
  block. Or manually use enterContext() & exitContext() - with a fake "new
  Exception" essentially.
• Maybe you want to hold a transaction, but just ensure that everything
  gets executed together (i.e. atomicity), but not care about whether
  everything actually went through (i.e. not force a rollback on
  exception). You'll now have to catch the exception, store it to a
  variable, use break and check for the exception after the with block.
  Or, yes, manually using enterContext() and exitContext().

It feels like with() is designed to be covering 70% of the use cases,
with a load of hidden pitfalls and advanced usage requiring manual
enterContext() and exitContext() calls. It's not a very good solution.

As to the destructors (and also in reply to that other email from
Arnauld talking about PDO::disconnect() etc.):

It's already possible today to have live objects which are already
destructed. It's extremely common to have in shutdown code. It's
sometimes a pain, I agree. But it's an already known pain, and an
already handled pain in a lot of code.
If your object only knows "create" and "destruct", there's no way for a
double enterContext() (nested or consecutive) situation to ever happen.
(Well, yes, you could theoretically manually call __destruct(), but
why would you ever do that?)

Last thing - proper API usage forces you to use that construct.

To the use() proposal from Tim:

This proposal makes it very simple to inadvertently leak the use()'d
value. I don't think the current proposed form goes far enough.

However we could decide to force-destruct an object (just like we do in
shutdown too). It's just one single flag for child methods to check as
well - the object is either destructed or not. We could also trivially
prohibit nested use() calls by throwing an AlreadyDestructedError when
an use()'d and inside destructed object crosses the use() boundary.

The only disadvantage is that there's no information about thrown
exceptions. I.e. you cannot add a default behaviour of "on exception,
please do this", like rolling transactions back. But:

• Is it actually a big problem? Where is the specific disadvantage over
  simply $db->transaction(function($t) { /* do stuff */ }); - where the
  call of the passed Closure can be trivially wrapped in try/catch.
• If yes, can we e.g. add an interface ExceptionDestructed { public bool
  $destructedDuringException; }? Which will set that property if the
  property is still undefined - to true if the destructor gets triggered
  inside ZEND_HANDLE_EXCEPTION. To false otherwise. And, if an user
  desires to manually force success/failure handling, he may set
  $object->destructedDuringException = true; himself as a very simple -
  one-liner - escape hatch.

The use() proposal is not a bad one, but I feel like requiring the RC to
drop to zero first, misses a bit of potential to save users from mistakes.
The other nice thing about use() is that it's optional. You don't have
to use it. You use it if you want some scoping, otherwise the scope is
simply the function scope.

To both proposals:

It remains possible by default to call methods on the object, after
leaving the with or use block. So some checking on methods for a safe
API is probably still required.

I don't think it's possible to solve that problem at all with the
ContextManager RFC, except manual checking by the implementer in every
single method. But it's possibly possible to solve it with the use() RFC
in orthogonal ways - like a #[ProhibitDestructed] attribute, which can
be added onto a class (making it apply to all methods) or a specific
method and causes method calls to throw an exception when the object is
destructed.
Which is possible to provide by the language, as the language knows
about whether objects are already destructed, unlike e.g. the
ContextManager, where it would be object state, which has to be
maintained by the user.

TL;DR: ContextManagers are a buttload of pitfalls. use() is probably
better, with much less inherent problems. And with the remaining
problems of the proposal being actually solvable.

Thanks,

Bob

Arnaud and I would like to present another RFC for consideration: Context Managers.

https://wiki.php.net/rfc/context-managers

I haven't had a chance to read the RFC in detail yet, but am really
pleased to see it. Ever since I read the description (and design
rationale) for Python's implementation, I have been thinking this would
be a useful addition to PHP.

One small thing I noticed: you list "Generator decorator managers" in
Future Scope, and while I agree that a magic attribute would need a bit
of thought, including the "standard boilerplate class" under some
suitable name seems worth considering.

--
Rowan Tommins
[IMSoP]