# [RFC] Configurable callback to dump results of expressions in `php -a`

- Thread: https://externals.io/message/112568
- Original archive: https://news-web.php.net/php.internals/112568
- Started by tyson andre on 2020-12-20T22:36:38+00:00
- Messages: 10
- RFC: Dump results of expressions in `php -a` (Declined) — https://externals.io/rfc/readline_interactive_shell_result_function.md
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · RFCs at https://externals.io/rfc.md · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #112568 — tyson andre — 2020-12-20T22:36:38+00:00

URL: https://externals.io/message/112568#112568

Hi internals,

I've created a new RFC https://wiki.php.net/rfc/readline_interactive_shell_result_function
adding a way to configure a callback to be called to dump the value of single-expression statements from `php -a`
(and an ini setting that can be disabled to prevent that callback from being called).

Many REPLs (Read-Eval-Print Loops) for other programming languages that I'm familiar with
print a (possibly truncated) representation of the result of expressions, but PHP doesn't.
It would be useful to allow users to extend the functionality of the default interactive php shell (php -a),
possibly with auto_prepend_file or through use of command wrappers/aliases, or after loading the shell.
Prior to this RFC, there was no way to extend the interactive php shell in this way.
(I've seen https://github.com/bobthecow/psysh mentioned as an alternative for php -a while investigating this,
but that's a shell written from scratch, and doesn't have some functionality from php -a such as tolerance of fatal errors)

Because PHP's interactive shell is written in C, adding new features or bug fixes is inconvenient - it would require a lot
of time getting familiar with C programming, PHP's internals and memory management,
and with PHP's internal C ast representation.
It would be easier and more accessible to extend PHP's interactive shell through code written in PHP rather than code written in C.

This was among the proposed enhancements I brought up in https://externals.io/message/111073 .
An earlier version of the implementation was also created months ago -  https://github.com/php/php-src/pull/5962

Thanks,
- Tyson

---

## #112710 — tyson andre — 2021-01-02T15:16:27+00:00

URL: https://externals.io/message/112568#112710 · In reply to: #112568

Hi internals,

> I've created a new RFC https://wiki.php.net/rfc/readline_interactive_shell_result_function
> adding a way to configure a callback to be called to dump the value of single-expression statements from `php -a`
> (and an ini setting that can be disabled to prevent that callback from being called).
> 
> Many REPLs (Read-Eval-Print Loops) for other programming languages that I'm familiar with
> print a (possibly truncated) representation of the result of expressions, but PHP doesn't.
> It would be useful to allow users to extend the functionality of the default interactive php shell (php -a),
> possibly with auto_prepend_file or through use of command wrappers/aliases, or after loading the shell.
> Prior to this RFC, there was no way to extend the interactive php shell in this way.
> (I've seen https://github.com/bobthecow/psysh mentioned as an alternative for php -a while investigating this,
> but that's a shell written from scratch, and doesn't have some functionality from php -a such as tolerance of fatal errors)

A reminder that I plan to start the vote on https://wiki.php.net/rfc/readline_interactive_shell_result_function on 2021-01-03

Thanks,
- Tyson

---

## #112719 — Nikita Popov — 2021-01-02T23:17:36+00:00

URL: https://externals.io/message/112568#112719 · In reply to: #112568

On Sun, Dec 20, 2020 at 11:36 PM tyson andre <tysonandre775@hotmail.com>
wrote:

> Hi internals,
>
> I've created a new RFC
> https://wiki.php.net/rfc/readline_interactive_shell_result_function
> adding a way to configure a callback to be called to dump the value of
> single-expression statements from `php -a`
> (and an ini setting that can be disabled to prevent that callback from
> being called).
>
> Many REPLs (Read-Eval-Print Loops) for other programming languages that
> I'm familiar with
> print a (possibly truncated) representation of the result of expressions,
> but PHP doesn't.
> It would be useful to allow users to extend the functionality of the
> default interactive php shell (php -a),
> possibly with auto_prepend_file or through use of command
> wrappers/aliases, or after loading the shell.
> Prior to this RFC, there was no way to extend the interactive php shell in
> this way.
> (I've seen https://github.com/bobthecow/psysh mentioned as an alternative
> for php -a while investigating this,
> but that's a shell written from scratch, and doesn't have some
> functionality from php -a such as tolerance of fatal errors)
>
> Because PHP's interactive shell is written in C, adding new features or
> bug fixes is inconvenient - it would require a lot
> of time getting familiar with C programming, PHP's internals and memory
> management,
> and with PHP's internal C ast representation.
> It would be easier and more accessible to extend PHP's interactive shell
> through code written in PHP rather than code written in C.
>
> This was among the proposed enhancements I brought up in
> https://externals.io/message/111073 .
> An earlier version of the implementation was also created months ago -
> https://github.com/php/php-src/pull/5962
>

Reading through the linked earlier mail, you have quite a few additional
hooks in mind, which might need a significant amount of additional userland
code (such as a parser implementation) to usefully implement. At that point
I'm wondering what the benefit of this hybrid approach is, relatively to a
userland implementation like psysh.

That is: a) Assuming all the hooks have been implemented, what additional
functionality is the interactive shell implementation itself actually
providing? b) Is it possible to go the other way around, and expose that
additional functionality to userland instead? You do mention fatal error
tolerance as one distinguishing feature -- is there anything beyond that?

Regards,
Nikita

---

## #112729 — tyson andre — 2021-01-03T16:11:35+00:00

URL: https://externals.io/message/112568#112729 · In reply to: #112719

> Reading through the linked earlier mail, you have quite a few additional hooks in mind, which might need a significant amount of additional userland code (such as a parser implementation) to usefully implement. At that point I'm wondering what the benefit of this hybrid approach is, relatively to a userland implementation like psysh.
>
> That is: 
> a) Assuming all the hooks have been implemented, what additional functionality is the interactive shell implementation itself actually providing?
> b) Is it possible to go the other way around, and expose that additional functionality to userland instead? You do mention fatal error tolerance as one distinguishing feature -- is there anything beyond that?

For https://wiki.php.net/rfc/readline_interactive_shell_result_function in particular,
compared to `php -a` in php 8.0, a subsequent RFC would provide additional functionality using these hooks: it would add default hooks to print a short representation of results of expressions to the interactive shell implementation (that could be turned off or replaced to dump objects)

- The benefit is that dumping the result of expressions improves the default experience.
  psysh wouldn't be installed by default when a new developer is learning php through the php manual, or when sshed into a remote server.
- The reason to allow hooking it is because some objects wouldn't have a user-friendly representation for var_dump/var_export (e.g. recursive data structures)
- Even if the code size is large, it may be doable by making the userland implementation part of ext/phpi/ (--enable-phpi), which is disabled by default and can be installed in separate packages created by operating system package maintainers (and shrunk by using minifiers)

Compared to psysh, the main distinguishing feature is definitely the ability to detect/tolerate fatal errors when compiling snippets or inheriting classes, and fewer dependencies to include to integrate an interactive shell with utilities for a project.
I don't think it should be exposed to regular processes or web servers, though, due to possible memory corruption or leaks after zend_error_noreturn (e.g. class inheritance errors after autoloading), etc.).

- It would possibly be an improvement to throw an error instead of causing a fatal error for common mistakes in interactive shell sessions such as duplicate functions/parameters but I'm not sure how likely that is, especially since classes and functions currently get added as the file is being compiled.

Integrating userland shells like `psysh` deeply into `php -a` may wish to avoid readline entirely and call a callback instead of printing `php>` and directly processing input like those projects already do.
Two hooks may help with enabling that approach, which can be added in `auto_prepend_file`

1. A hook to call a callback instead of printing "php >" and C readline reading stdin.
    e.g. `readline_replace_interactive_shell_initializer(function () { ... read and process stdin in a loop })`
2. Adding a hook to call a function every time an uncatchable fatal error was encountered, e.g. to resume the userland shell.
    e.g. `readline_replace_interactive_fatal_error_handler(function ($errcode, $errmsg, $file, $line, $errcount): bool { /* process or exit */ })`

Thanks,
- Tyson

---

## #112737 — Rowan Tommins — 2021-01-03T18:44:32+00:00

URL: https://externals.io/message/112568#112737 · In reply to: #112729

> - The benefit is that dumping the result of expressions improves the default experience.
>    psysh wouldn't be installed by default when a new developer is learning php through the php manual, or when sshed into a remote server.


It doesn't feel to me that you've really answered Nikita's question: if 
all the code using these hooks is going to be distributed as userland 
code anyway, then they're not going to improve the default experience.

It feels like we need to go in one of two directions:

a) Build a full-featured official REPL with all of these improvements 
completely implemented out of the box. Limited extension hooks might 
still be desirable to build custom versions for frameworks etc, but they 
could be more targeted - for custom input, it could be "register 
meta-command"; for custom output, we already have __debugInfo() at the 
class level.

b) Expose the magic behaviour needed for something like PsySh to do 
everything `php -a` already can, and leave the rest to userland. So far, 
the only mentioned requirement is a special form of eval() that swallows 
fatal errors.

My feeling is that the current mood of the community favours (b) rather 
than (a); the most obvious example is that PHP used to bundle a PEAR 
executable, but Composer remains an entirely external project. Is there 
a reason not to aim for the same "de facto standard" for a REPL?

Regards,

-- 
Rowan Tommins
[IMSoP]

---

## #112739 — tyson andre — 2021-01-03T20:45:42+00:00

URL: https://externals.io/message/112568#112739 · In reply to: #112737

Hi Rowan Tommins,

> > […]
> 
> It doesn't feel to me that you've really answered Nikita's question: if 
> all the code using these hooks is going to be distributed as userland 
> code anyway, then they're not going to improve the default experience.

I was saying that I'd planned to propose defaults if this passed.
__debugInfo() is a slight improvement, but the human-readable representation of an object isn't always the same thing as the debug representation of the object.
A human-readable representation might be `Point(x: 1, y: 2)`, where var_dump or var_export(`__set_state`)
is much longer, and I don't believe var_dump is a one size fits all solution for both simple and recursive data structures,
especially since `__debugInfo` predates the repl.

```
php > class Point { public function __construct(public int $x, public int $y) {} }
php > var_export(new Point(1, 2));
Point::__set_state(array(
   'x' => 1,
   'y' => 2,
))
```

> It feels like we need to go in one of two directions:
> a) Build a full-featured official REPL with all of these improvements 
> completely implemented out of the box. Limited extension hooks might 
> still be desirable to build custom versions for frameworks etc, but they 
> could be more targeted - for custom input, it could be "register 
> meta-command"; for custom output, we already have __debugInfo() at the 
> class level.

I'd be happy as long as we made progress on improving the interactive shell.

Psysh is 2.4MB as a compiled phar release and larger if distributed with library/application releases (e.g. on remote servers).
default extension hooks would likely be much smaller.

> b) Expose the magic behaviour needed for something like PsySh to do 
> everything `php -a` already can, and leave the rest to userland. So far, 
> the only mentioned requirement is a special form of eval() that swallows 
> fatal errors.

That may or may not be possible to do through creating a new `unsafe_eval` PECL (or only exposing it for interactive sessions) - I'd have to check.
Something like `unsafe_eval(string $code, array &$variables): mixed result` (or throw UnsafeFatalError)
The usual caveats about not using it in production would apply - the php compiler treats eval differently in that it has access to the caller's scope.

https://github.com/bobthecow/psysh/blob/master/src/ExecutionLoopClosure.php uses eval(),
but there's actually a lot of heuristics to avoid calling eval() on code with common known fatal errors.
(it uses the readline/libedit PHP module if it's available, but doesn't load readline_cli.c)

The ext/readline/readline_cli.c overrides `EG(bailout)` with zend_try macros to recover from fatal errors

```c
				zend_try {
					zend_eval_stringl(code, prompt_end - prompt_spec - 1, NULL, "php prompt code");
				} zend_end_try();
```

> My feeling is that the current mood of the community favours (b) rather 
> than (a); the most obvious example is that PHP used to bundle a PEAR 
> executable, but Composer remains an entirely external project. Is there 
> a reason not to aim for the same "de facto standard" for a REPL?

Even if we do start distributing a better alternative shell with php or endorsing an alternative in docs,
it's still possible to incrementally improve `php -a`

It seems like a significant missing feature to omit printing expression results from `php -a`;
people who do use `php -a` may wish to continue using
that feature set (e.g. `#setting_name=value` to set/dump ini variables) but still benefit from minor improvements such as this RFC.
https://github.com/bobthecow/psysh/issues/462 mentions psysh isn't a drop-in replacement.

E.g. on shared hosting or when debugging on a remote host, it may be inconvenient to download psysh and faster to use `php -a`

https://www.php.net/manual/en/features.commandline.interactive.php only mentions `php -a`.
Perhaps it should mention external userland shells with a note that they're developed independently
if the REPL rarely receives updates in favor of other functionality.

- Someone learning from the php.net manual or a tutorial with minimal dependencies wouldn't install psysh right now.

Aside: the php manual doesn't mention composer except in a few PECL extension docs,
but people seem to figure out how to use composer out of necessity and library installation docs.
Interactive shells wouldn't be as advertised

Cheers,
-Tyson

---

## #112744 — Rowan Tommins — 2021-01-04T10:28:37+00:00

URL: https://externals.io/message/112568#112744 · In reply to: #112739

> I was saying that I'd planned to propose defaults if this passed.


Then I guess my feedback can be summarised as: propose these default 
implementations first, rather than last. Start by improving the 
out-of-the-box experience, and then look at what hooks userland tools 
might want to customise that (which might involve something "smarter" 
than replacing the entire output routine).


> Psysh is 2.4MB as a compiled phar release and larger if distributed with library/application releases (e.g. on remote servers).
> default extension hooks would likely be much smaller.


That was a lot of data when I used to carry floppy disks around in my 
school backpack. It sounds pretty reasonable for a feature-rich 
command-line tool in 2021.


> Someone learning from the php.net manual or a tutorial with minimal dependencies wouldn't install psysh right now.


To be honest, I doubt most users ever come across php -a either, or know 
that they need to install ext/readline to unlock its features (I'm 
pretty sure that's not installed by default on Ubuntu, for instance).

I'm supportive of the idea of changing that, but I don't think the RFC 
as currently presented particularly helps that cause.


Regards,

-- 
Rowan Tommins
[IMSoP]

---

## #112763 — Rowan Tommins — 2021-01-05T13:39:36+00:00

URL: https://externals.io/message/112568#112763 · In reply to: #112739

> __debugInfo() is a slight improvement, but the human-readable representation of an object isn't always the same thing as the debug representation of the object.
> A human-readable representation might be `Point(x: 1, y: 2)`, where var_dump or var_export(`__set_state`)
> is much longer, and I don't believe var_dump is a one size fits all solution for both simple and recursive data structures,
> especially since `__debugInfo` predates the repl.


A couple of notes on this:

- The readline-based shell for "php -a" was added in PHP 5.1 
[https://www.php.net/manual/en/features.commandline.interactive.php] and 
__debugInfo not until 5.6 [https://wiki.php.net/rfc/debug-info]

- I agree that the existing debug outputs are quite verbose, but I don't 
think that's a problem unique to the REPL. `var_export()` is constrained 
to render valid PHP code, but `print_r()` and `var_dump()` could and perhaps 
should represent objects more compactly.

- That compact representation of a Point would be useful pretty much 
everywhere anyone wanted debug output. It would also be possible to 
build it on top of the existing functionality (key-value pairs from 
__debugInfo if defined, else all properties).

- Allowing objects to overload the output seems much preferable to the 
formatting function having to know all the special cases, so I'm not 
convinced of the need to hook the entire output for the shell.

- I can't find any references off-hand, but I'm pretty sure popular 
REPLs in other languages take that approach: use existing 
pretty-printing mechanisms from the language, which in turn can be 
overloaded by individual types / classes.


Regards,

-- 
Rowan Tommins
[IMSoP]

---

## #112772 — tyson andre — 2021-01-06T03:42:33+00:00

URL: https://externals.io/message/112568#112772 · In reply to: #112568

Hi internals,

> I've created a new RFC https://wiki.php.net/rfc/readline_interactive_shell_result_function
> adding a way to configure a callback to be called to dump the value of single-expression statements from `php -a`
> (and an ini setting that can be disabled to prevent that callback from being called).
> 
> Many REPLs (Read-Eval-Print Loops) for other programming languages that I'm familiar with
> print a (possibly truncated) representation of the result of expressions, but PHP doesn't.
> It would be useful to allow users to extend the functionality of the default interactive php shell (php -a),
> possibly with auto_prepend_file or through use of command wrappers/aliases, or after loading the shell.
> Prior to this RFC, there was no way to extend the interactive php shell in this way.
> (I've seen https://github.com/bobthecow/psysh mentioned as an alternative for php -a while investigating this,
> but that's a shell written from scratch, and doesn't have some functionality from php -a such as tolerance of fatal errors)
> 
> Because PHP's interactive shell is written in C, adding new features or bug fixes is inconvenient - it would require a lot
> of time getting familiar with C programming, PHP's internals and memory management,
> and with PHP's internal C ast representation.
> It would be easier and more accessible to extend PHP's interactive shell through code written in PHP rather than code written in C.
> 
> This was among the proposed enhancements I brought up in https://externals.io/message/111073 .
> An earlier version of the implementation was also created months ago -  https://github.com/php/php-src/pull/5962

Some of the feedback had mentioned that adding the ability to dump the results of expressions in the default shell
only made sense if there was a default implementation included with this RFC.

I plan to start a straw poll shortly, to measure interest with/without a proposed result expression dumper loaded by default.
https://wiki.php.net/rfc/readline_interactive_shell_result_function_straw_poll

Thanks,
- Tyson

---

## #112911 — tyson andre — 2021-01-16T21:20:39+00:00

URL: https://externals.io/message/112568#112911 · In reply to: #112568

Hi internals,

> I've created a new RFC https://wiki.php.net/rfc/readline_interactive_shell_result_function
> adding a way to configure a callback to be called to dump the value of single-expression statements from `php -a`
> (and an ini setting that can be disabled to prevent that callback from being called).
> 
> Many REPLs (Read-Eval-Print Loops) for other programming languages that I'm familiar with
> print a (possibly truncated) representation of the result of expressions, but PHP doesn't.
> It would be useful to allow users to extend the functionality of the default interactive php shell (php -a),
> possibly with auto_prepend_file or through use of command wrappers/aliases, or after loading the shell.
> Prior to this RFC, there was no way to extend the interactive php shell in this way.
> (I've seen https://github.com/bobthecow/psysh mentioned as an alternative for php -a while investigating this,
> but that's a shell written from scratch, and doesn't have some functionality from php -a such as tolerance of fatal errors)

I've updated https://wiki.php.net/rfc/readline_interactive_shell_result_function after feedback 
and plan to start voting in a few days.
This RFC will make the interactive shell dump non-null expression results with `var_dump()`/var_export() by default
(https://externals.io/message/112568#112719 , https://wiki.php.net/rfc/readline_interactive_shell_result_function_straw_poll)

- The dumping of expression results can be disabled through a system ini setting or at runtime

```
$ php -a
Interactive shell

php > 0.5*2;
=> 1.0
```

Thanks,
- Tyson