Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:127277
Precedence: bulk
MIME-Version: 1.0
Date: Sun, 4 May 2025 03:34:05 -0400
Message-ID: <CAEUnE0c88jp9LeoHnngqvU+XjL_fpwPcfKi1g7hg6k6PBN6KCQ@mail.gmail.com>
Subject: [PHP-DEV] Modules, again.
To: PHP internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary="00000000000048764a06344a696f"
From: tendoaki@gmail.com (Michael Morris)

--00000000000048764a06344a696f
Content-Type: text/plain; charset="UTF-8"

It's been 9 months. Been researching, working on other projects, mulling
over
points raised the last time I brought this up. And at the moment I don't
think
PHP 8.5 is in its final weeks so this isn't a distraction for that.  The
previous discussion got seriously, seriously derailed and I got lost even
though
I started it. I'm not going to ask anyone to dig into the archives, let's
just
start afresh.

--------------------------------------------------------------------------------
THE PROBLEM
--------------------------------------------------------------------------------
PHP has no way of dealing with userland code trying to write to the same
entry
on the symbol tables.  Namespaces are a workaround of this and combined
with
autoloaders they've yielded the package environments we currently have
(usually
composer), but at the end of the day if two code blocks want different
versions
of the same package invoked at the same time there's going to be a crash.
This
is seen most visibly in WordPress plugins that use composer, as they resort
to
monkey-typing the composer packages they consume to avoid such collisions.

--------------------------------------------------------------------------------
PROPOSAL
--------------------------------------------------------------------------------
Modules - blocks of PHP code with independent symbol tables and autoload
queues.

Instead of using any new keywords along with the backwards compatibility
problems that creates three existing keywords will be used in a new way:
"use",
"require", and "yield"

The first file that PHP loads will always be on the "main thread". To bring
in
code as a module the use require structure is used. The simplest possible
version of this is as follows:

PHP Code
-----------------------------------------------------------------------

use require 'mymodule.php';
--------------------------------------------------------------------------------

The contents of 'mymodule.php' have two requirements. First, a namespace is
*required* of a module. Second, yield statements are used to mark the
functions
classes and constants the module exports, and at least one such yield must
be
present. Hence the file may look something like this:

PHP Code
-----------------------------------------------------------------------

namespace MyModule;

yield function sum(a, b) { return a + b; }

--------------------------------------------------------------------------------

Returning to our caller, it could make use of this function as follows.

PHP Code
-----------------------------------------------------------------------

use require 'mymodule.php';

echo MyModule::sum(3, 4);
--------------------------------------------------------------------------------

So far there is nothing here that couldn't have been done with a static
class.
The important difference though is in behavior.

1. The module does not affect or see variables on the main thread -
including
the superglobals. A module can only get to them if it receives them as an
argument in some sort of setter yield function.

2. The module does not affect or see constants or functions established on
the
main thread or in other modules. It can see and autoload classes from the
main
thread if the module author opts into this (discussed below).

The use case above is not typical - usually inclusions from modules are more
targeted.

PHP Code
-----------------------------------------------------------------------

use sum require 'mymodule.php';

echo sum(3, 4); // 7
--------------------------------------------------------------------------------

Here the class is not created in the main thread, only the yielded function
is.

Aliases and multiple objects can be declared in the use just as is the case
now.

PHP Code
-----------------------------------------------------------------------

use sum, difference as subtract require 'mymodule.php';
--------------------------------------------------------------------------------

And if desired the namespace of the module can be aliased on the fly using
the
wildcard operator

PHP Code
-----------------------------------------------------------------------

use * as AliasModule require 'mymodule.php';

AliasModule::sum(3,4);
--------------------------------------------------------------------------------

Require continues to invoke autoloaders, even when used in the context of
use
require.  The callback defined in spl_autoload_register will receive a
second
argument from this context, boolean true if the main thread is loading a
module,
and if a module is loading a module its namespace string will be sent. Hence

PHP Code
-----------------------------------------------------------------------

require('./vendor/composer/autoload.php')

use require 'mymodule';
// Callback will receive args ('mymodule', true)
class A;
// Callback will receive args ('A', false) - the false case should preserve
BC.
--------------------------------------------------------------------------------

And in a module

PHP Code
-----------------------------------------------------------------------
namespace MyModule;

/*
 * Mod Author can elect to use the global autoloader
 * by passing the string "global" instead of a callback
 */
spl_autoload_register('global');

use require 'othermodule';
// Callback will receive args ('othermodule', 'MyModule')
class A;
// Callback will receive args ('A', 'mymodule')
--------------------------------------------------------------------------------

Autoload callbacks are currently required to directly load the file. For
modules this *will not work* because the loading of a module file involves
the
setup of a new symbol table, autoload queue, and slightly different parsing
rules (again - namespace is required, not optional, and at least one top
level
yield statement must be present). So when an autoloader is asked about a
module
it must return the absolute path to the module, or false if it can't
resolve it
(handing off to the next loader in the queue). PHP will then load the
module as
if the filepath had been placed in the use require statement in the first
place.


--------------------------------------------------------------------------------
CLOSING REMARKS
--------------------------------------------------------------------------------

I want to take a moment to ruminate on what doors are opened by the above,
but
all that follows is NOT part of my proposal.  The above gets what I feel is
needed - a way to cleanly run disparate packages in an application whose
authors
refuse to update it to embrace composer (cough - WordPress - cough). But
even
for projects that do embrace composer and its package management the above
makes
the prospect of a large API change in a major version far less frightening.
In
the current setup all the extensions have to be kept current and more or
less
on the same page. The larger the extension authoring set becomes the less
feasible this is - projects get abandoned, stagnate and if you have a site
that
needs such then finding a replacement or upgrading it personally can be a
pain.

The string that gets passed to the autoloader could be anything btw - rules
for
which versions the module might accept like 'mymodule@7.x' can work if the
package manager is written to parse out such, but the rules for such, not to
mention what the composer.json file or equivalent would need to look like is
outside the scope of this proposal.

It should be noted that Modules offer a much more black box behavior than
the
current namespaces and autoloaders can provide.  The only part of a module
the
outside world can see is what it yields. If a class isn't yielded the
outside
world can't make an instance of it.  This shielding of internal API's
should be
useful because no matter how big a note you make in the comments about how
a
class shouldn't be used by outside code sooner or later someone will do it
and
their code will break when you change the internal API. While it is their
fault
for doing such, it can be a concern especially if the use of such "internal"
API's becomes commonplace (Drupal has several such instances of this)

Finally, my previous writings on this have mused about possibly having
module files
possess vastly different parsing rules. I bring this up as a possibility
but I
won't dwell on it as it could easily become a thread derailing distraction.
That said, it should be possible to use the autoload callback to signal to
PHP
that a block of code should be loaded using the module parsing rules if
having
them be different in any way is desired. That or a new require_module
statement
could be used to pull in code under the module parsing rules.

As to why this might be desirable - I'm no expert on the engine but I'm
going
to guess that giving each module an independent symbol table will incur
overhead, possibly significant overhead. One way to claw back performance
is to
fix bugs that have been unfixable for backwards compatibility reasons.
There is
no existing module code, so in one fell swoop this code can step away from
those
problems.  If this is done though it has to be done right as the window
closes
once projects with modules start appearing.

--00000000000048764a06344a696f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>It&#39;s been 9 months. Been researching, working on =
other projects, mulling over <br>points raised the last time I brought this=
 up. And at the moment I don&#39;t think <br>PHP 8.5 is in its final weeks =
so this isn&#39;t a distraction for that.=C2=A0 The <br>previous discussion=
 got seriously, seriously derailed and I got lost even though <br>I started=
 it. I&#39;m not going to ask anyone to dig into the archives, let&#39;s ju=
st <br>start afresh.<br><br>-----------------------------------------------=
---------------------------------<br>THE PROBLEM<br>-----------------------=
---------------------------------------------------------<br>PHP has no way=
 of dealing with userland code trying to write to the same entry <br>on the=
 symbol tables.=C2=A0 Namespaces are a workaround of this and combined with=
 <br>autoloaders they&#39;ve yielded the package environments we currently =
have (usually <br>composer), but at the end of the day if two code blocks w=
ant different versions <br>of the same package invoked at the same time the=
re&#39;s going to be a crash. This <br>is seen most visibly in WordPress pl=
ugins that use composer, as they resort to <br>monkey-typing the composer p=
ackages they consume to avoid such collisions.<br><br>---------------------=
-----------------------------------------------------------<br>PROPOSAL<br>=
---------------------------------------------------------------------------=
-----<br>Modules - blocks of PHP code with independent symbol tables and au=
toload queues. <br><br>Instead of using any new keywords along with the bac=
kwards compatibility <br>problems that creates three existing keywords will=
 be used in a new way: &quot;use&quot;,<br>&quot;require&quot;, and &quot;y=
ield&quot;<br><br>The first file that PHP loads will always be on the &quot=
;main thread&quot;. To bring in<br>code as a module the use require structu=
re is used. The simplest possible<br>version of this is as follows:<br><br>=
PHP Code ------------------------------------------------------------------=
-----<br><br>use require &#39;mymodule.php&#39;;<br>-----------------------=
---------------------------------------------------------<br><br>The conten=
ts of &#39;mymodule.php&#39; have two requirements. First, a namespace is<b=
r>*required* of a module. Second, yield statements are used to mark the fun=
ctions<br>classes and constants the module exports, and at least one such y=
ield must be<br>present. Hence the file may look something like this:<br><b=
r>PHP Code ----------------------------------------------------------------=
-------<br><br>namespace MyModule;<br><br>yield function sum(a, b) { return=
 a + b; }<br><br>----------------------------------------------------------=
----------------------<br><br>Returning to our caller, it could make use of=
 this function as follows.<br><br>PHP Code --------------------------------=
---------------------------------------<br><br>use require &#39;mymodule.ph=
p&#39;;<br><br>echo MyModule::sum(3, 4);<br>-------------------------------=
-------------------------------------------------<br><br>So far there is no=
thing here that couldn&#39;t have been done with a static class.<br>The imp=
ortant difference though is in behavior. <br><br>1. The module does not aff=
ect or see variables on the main thread - including <br>the superglobals. A=
 module can only get to them if it receives them as an <br>argument in some=
 sort of setter yield function.<br><br>2. The module does not affect or see=
 constants or functions established on the <br>main thread or in other modu=
les. It can see and autoload classes from the main<br>thread if the module =
author opts into this (discussed below).<br><br>The use case above is not t=
ypical - usually inclusions from modules are more<br>targeted.<br><br>PHP C=
ode -----------------------------------------------------------------------=
<br><br>use sum require &#39;mymodule.php&#39;;<br><br>echo sum(3, 4); // 7=
<br>-----------------------------------------------------------------------=
---------<br><br>Here the class is not created in the main thread, only the=
 yielded function is.<br><br>Aliases and multiple objects can be declared i=
n the use just as is the case now.<br><br>PHP Code ------------------------=
-----------------------------------------------<br><br>use sum, difference =
as subtract require &#39;mymodule.php&#39;;<br>----------------------------=
----------------------------------------------------<br><br>And if desired =
the namespace of the module can be aliased on the fly using the<br>wildcard=
 operator<br><br>PHP Code -------------------------------------------------=
----------------------<br><br>use * as AliasModule require &#39;mymodule.ph=
p&#39;;<br><br>AliasModule::sum(3,4);<br>----------------------------------=
----------------------------------------------<br><br>Require continues to =
invoke autoloaders, even when used in the context of use <br>require.=C2=A0=
 The callback defined in spl_autoload_register will receive a second<br>arg=
ument from this context, boolean true if the main thread is loading a modul=
e,<br>and if a module is loading a module its namespace string will be sent=
. Hence<br><br>PHP Code ---------------------------------------------------=
--------------------<br><br>require(&#39;./vendor/composer/autoload.php&#39=
;)<br><br>use require &#39;mymodule&#39;;<br>// Callback will receive args =
(&#39;mymodule&#39;, true)<br>class A;<br>// Callback will receive args (&#=
39;A&#39;, false) - the false case should preserve BC.<br>-----------------=
---------------------------------------------------------------<br><br>And =
in a module<br><br>PHP Code -----------------------------------------------=
------------------------<br>namespace MyModule;</div><div><br></div><div>/*=
</div><div>=C2=A0* Mod Author can elect to use the global autoloader</div><=
div>=C2=A0* by passing the string &quot;global&quot; instead of a callback<=
/div><div>=C2=A0*/</div><div>spl_autoload_register(&#39;global&#39;);<br><b=
r>use require &#39;othermodule&#39;;<br>// Callback will receive args (&#39=
;othermodule&#39;, &#39;MyModule&#39;)<br>class A;<br>// Callback will rece=
ive args (&#39;A&#39;, &#39;mymodule&#39;)<br>-----------------------------=
---------------------------------------------------<br><br>Autoload callbac=
ks are currently required to directly load the file. For<br>modules this *w=
ill not work* because the loading of a module file involves the<br>setup of=
 a new symbol table, autoload queue, and slightly different parsing<br>rule=
s (again - namespace is required, not optional, and at least one top level<=
br>yield statement must be present). So when an autoloader is asked about a=
 module<br>it must return the absolute path to the module, or false if it c=
an&#39;t resolve it<br>(handing off to the next loader in the queue). PHP w=
ill then load the module as<br>if the filepath had been placed in the use r=
equire statement in the first place.<br><br><br>---------------------------=
-----------------------------------------------------<br>CLOSING REMARKS<br=
>--------------------------------------------------------------------------=
------<br><br>I want to take a moment to ruminate on what doors are opened =
by the above, but<br>all that follows is NOT part of my proposal.=C2=A0 The=
 above gets what I feel is<br>needed - a way to cleanly run disparate packa=
ges in an application whose authors<br>refuse to update it to embrace compo=
ser (cough - WordPress - cough). But even<br>for projects that do embrace c=
omposer and its package management the above makes<br>the prospect of a lar=
ge API change in a major version far less frightening. In<br>the current se=
tup all the extensions have to be kept current and more or less<br>on the s=
ame page. The larger the extension authoring set becomes the less<br>feasib=
le this is - projects get abandoned, stagnate and if you have a site that<b=
r>needs such then finding a replacement or upgrading it personally can be a=
 pain.<br><br>The string that gets passed to the autoloader could be anythi=
ng btw - rules for<br>which versions the module might accept like &#39;mymo=
dule@7.x&#39; can work if the<br>package manager is written to parse out su=
ch, but the rules for such, not to<br>mention what the composer.json file o=
r equivalent would need to look like is<br>outside the scope of this propos=
al.<br><br>It should be noted that Modules offer a much more black box beha=
vior than the<br>current namespaces and autoloaders can provide.=C2=A0 The =
only part of a module the<br>outside world can see is what it yields. If a =
class isn&#39;t yielded the outside<br>world can&#39;t make an instance of =
it.=C2=A0 This shielding of internal API&#39;s should be<br>useful because =
no matter how big a note you make in the comments about how a <br>class sho=
uldn&#39;t be used by outside code sooner or later someone will do it and<b=
r>their code will break when you change the internal API. While it is their=
 fault<br>for doing such, it can be a concern especially if the use of such=
 &quot;internal&quot;<br>API&#39;s becomes commonplace (Drupal has several =
such instances of this)<br><br>Finally, my previous writings on this have m=
used about possibly having module files <br>possess vastly different parsin=
g rules. I bring this up as a possibility but I <br>won&#39;t dwell on it a=
s it could easily become a thread derailing distraction.<br>That said, it s=
hould be possible to use the autoload callback to signal to PHP<br>that a b=
lock of code should be loaded using the module parsing rules if having<br>t=
hem be different in any way is desired. That or a new require_module statem=
ent <br>could be used to pull in code under the module parsing rules.<br><b=
r>As to why this might be desirable - I&#39;m no expert on the engine but I=
&#39;m going<br>to guess that giving each module an independent symbol tabl=
e will incur<br>overhead, possibly significant overhead. One way to claw ba=
ck performance is to<br>fix bugs that have been unfixable for backwards com=
patibility reasons. There is<br>no existing module code, so in one fell swo=
op this code can step away from those<br>problems.=C2=A0 If this is done th=
ough it has to be done right as the window closes<br>once projects with mod=
ules start appearing.</div></div>

--00000000000048764a06344a696f--