Module or Class Visibility, Season 2

Ok, the conversation is getting sidetracked, but I think some progress is being made.

I started this latest iteration last year with a thread about introducing something similar to the ES module system of JavaScript to PHP. What attracts me to this particular model is that it should already be familiar to the vast majority of PHP users. Prior to ES modules browsers had no natural module import mechanic. Prior to ES modules all symbols were attached to the window. You can see this if you serve open this index.html from a server (Note that opening the file locally will result in the js being blocked by modern browser security. )

<!DOCTYPE html>
<html>
  <head>
    <script>
      var a = 1234
    </script>
  </head>
  <body>
    <script>
      console.log(a)
      console.log(window.a)
    </script>
  </body>
</html>

The above spits 1234 into the console twice. Second example - let's put a module in.

<!DOCTYPE html>
<html>
  <head>
    <script>
      var a = 1234
    </script>
    <script type="module">
      const a = 5678
      var b = 9123
    </script>
  </head>
  <body>
    <script>
      console.log(a)
      console.log(window.a)
      console.log(b)
    </script>
  </body>
</html>

This outputs 1234 twice and an error is raised about b being undefined.

I bring the above up to demonstrate that is the desired behavior of what I originally called a PHP module and have been bullied over and taken to task about not understanding the meaning of "module". Rowain seems to be more comfortable characterizing this as containers. If everyone is happy with that term I really don't care - I just want a way to isolate a code block so that whatever happens in there stays in there unless I explicitly export it out, and the only way I see things in that scope is if I bring them in.

The other thing that was done with ES is that the syntax for the modules was tightened. JavaScripters cannot dictate what browser a user chooses, so the bad decisions of the early days of JS never really went away until ES came along which enforced their strict mode by default. PHP has no such strict mode - it has a strict types mode but that isn't the same thing. There are multiple behaviors in PHP that can't go away because of backwards compatibility problems, and one of those might indeed be how namespaces are handled. In PHP a namespace is just a compile shortcut for resolving symbol names. The namespace is prefixed to the start of every symbol within it. Unlike Java or C#, PHP has no concept of namespace visibility. At the end of the day it's a shortcut and its implementation happens entirely at compile time.

Previously in the discussion Alwin Garside made a long but insightful post on namespaces and their workings that I've been thinking on and trying to digest for the last several days. What I've arrived at is the discussions about composer and autoloaders are indeed a red herring to the discussion. At the end of the day, PHP's include statements are a means to separate the php process into multiple files. In his email he explored some of the rewriting that could be done, and myself and Rowain have also explored this in the form of namespace pathing and aliasing.

We've gotten away from the original focus of containing this code and how that would work. So once again this moron is going to take a stab at it.

Container modules are created with require_module('file/path'). All code that executes as a result of this call is isolated to its container. That includes the results of any require or include calls made by the module file itself or any file it requires.

Since the module file is cordoned off to its own container from the rest of the application whatever namespaces it uses are irrelevant to outside code. Any symbols created in the module will not be established in the script that made the require_module() call. Since it is coming into being with a new require mechanism it could be subjected to more efficient parsing rules if that is desired, but that's a massive can of worms for later discussion. One of those will be necessary - it will need to return something to the php code that called it. The simplest way to go about this is to just require that it have a return. So...

$myModule = require_module('file/path');

or perhaps

const myModule = require_module('file/path');

The module probably should return a static class or class instance, but it could return a closure. In JavaScript the dynamic import() statement returns a module object that is most similar to PHP's static classes, with each export being a member or method of the module object.

Circling back to a question I know will be asked - what about autoloaders? To which I answer, what about them? If the module wants to use an autoloader it has to require one just as the initial php file that required it had to have done at some point. The container module is for all intents and purposes its own php process that returns some interface to allow it to talk to the process that spawned it.

Will this work? I think yes. Will it be efficient? Hell no. Can it be optimized somehow? I don't know.

This could work! I have a couple of critiques, but they aren’t negative:

I think I like it. It might be worth pointing out that JavaScript "hoists" the imports to file-level during compilation — even if you have the import statement buried deep in a function call. Or, at least it used to. I haven’t kept track of the language that well in the last 10 years, so I wouldn’t be surprised if it changed; or didn’t. I don’t think this is something we need to worry about too much here.

It’s also worth pointing out that when PHP compiles a file, every file has either an explicit or implicit return. https://www.php.net/manual/en/function.include.php#:~:text=Handling%20Returns%3A,from%20included%20files.

So, in other words, what is it about require_module that is different from require or include? Personally, I would then change PHP from "compile file" mode when parsing the file to "compile module" mode. From a totally naive point-of-view, this would cause PHP to:

1. if we already have a module from that file; return the module instead of compiling it again.
2. swap out symbol tables to the module’s symbol table.
3. start compiling the given file.
4. concatenate all files as included/required.
5. compile the resulting huge file.
6. switch back to the calling symbol table (which may be another module).
7. return the module.
   For a v1, I wouldn’t allow autoloading from inside a module — or any autoloaded code automatically isn’t considered to be part of the module (it would be the responsibility of the main program to handle autoloading). This is probably something that needs to be solved, but I think it would need a whole new approach to autoloading which should be out of scope for the module RFC (IMHO).

In other words, you can simply include/require a module to load the entire module into your current symbol table; or use require_module to "contain" it.

As for what should a module return? I like your idea of just returning an object or closure.

— Rob

I just had another thought; sorry about the back-to-back emails. This wouldn’t preclude something like composer (or something else) from being used to handle dependencies, it would just mean that the package manager might export a "Modules" class + constants — we could also write a composer plugin that does just this:

require_once 'vendor/autoload.php';

$module = require_module Vendor\Module::MyModule;

where Vendor\Module is a generated and autoloaded class containing consts to the path of the exported module.

— Rob

This could work! I have a couple of critiques, but they aren’t negative:

I think I like it. It might be worth pointing out that JavaScript "hoists"
the imports to file-level during compilation — even if you have the import
statement buried deep in a function call. Or, at least it used to. I
haven’t kept track of the language that well in the last 10 years, so I
wouldn’t be surprised if it changed; or didn’t. I don’t think this is
something we need to worry about too much here.

As I pointed out in detail to Rob off list JavaScript has 2 import
mechanisms that are subtly different from each other. Those interested can
read here:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import

It’s also worth pointing out that when PHP compiles a file, every file has
either an explicit or implicit return.
https://www.php.net/manual/en/function.include.php#:~:text=Handling%20Returns%3A,from%20included%20files
.

True, but it's a rarely used mechanism and a return statement isn't
required - hence an implicit return is possible.

So, in other words, what is it about require_module that is different from
require or include? Personally, I would then change PHP from "compile
file" mode when parsing the file to "compile module" mode. From a totally
naive point-of-view, this would cause PHP to:

1. if we already have a module from that file; return the module
   instead of compiling it again.
2. swap out symbol tables to the module’s symbol table.
3. start compiling the given file.
4. concatenate all files as included/required.
5. compile the resulting huge file.
6. switch back to the calling symbol table (which may be another
   module).
7. return the module.

For a v1, I wouldn’t allow autoloading from inside a module — or any
autoloaded code automatically isn’t considered to be part of the module (it
would be the responsibility of the main program to handle autoloading).
This is probably something that needs to be solved, but I think it would
need a whole new approach to autoloading which should be out of scope for
the module RFC (IMHO).

In other words, you can simply include/require a module to load the entire
module into your current symbol table; or use require_module to "contain"
it.

Yes, that is certainly possible but comes at an opportunity cost from an
engine design standpoint. This is an exceedingly rare opportunity to go
back and fix mistakes that have dogged the language for sometime that
simply can't be fixed without creating large backwards compatibility
problems. For instance, say for the sake of example that PHP files could be
compiled 10 times faster if the object parser could assume the whole file
was code and there's not going to be any <?php ?> tags or Heredoc or Nowdoc
blocks. It might be worth it then to have modules not allow such, and if
an author had a block of code that they really wanted this templating
behavior to apply to they still could issue a require. Maybe it's time to
dig up some downvoted RFC's that got killed for these reasons.

As for what should a module return? I like your idea of just returning an
object or closure.

I'm leaning towards something akin to a static class. If modules have
export keywords (note that if they have their own parser they can also have
keywords without disrupting the existing PHP ecosystem) then compiling one
would be a static class with members and methods matching what was exported.

I just had another thought; sorry about the back-to-back emails. This
wouldn’t preclude something like composer (or something else) from being
used to handle dependencies, it would just mean that the package manager
might export a "Modules" class + constants — we could also write a composer
plugin that does just this:
require_once 'vendor/autoload.php';
$module = require_module Vendor\Module::MyModule;
where Vendor\Module is a generated and autoloaded class containing consts
to the path of the exported module.

That leads into some thoughts I have on loading modules in general.
require_module is the simplest expression. That said, 'use module' might
also be appropriate.

require_once 'vendor/autoload.php';

use module Vendor/Module as MyModule;

Something like that?

JavaScript has a distinction between global scope import, which is almost
analogous to our namespaces and use statement, and dynamic scope import,
which is almost analogous to our require statements. Don't know if it's
useful to us to draw such a distinction.

I think there's a key assumption here still that is at the root of much of the disagreement in this thread.

Given that code from multiple files is clustered together into a "thing"
and Given we can use that "thing" to define a boundary for:

• name resolution (what Michael is after);
• visibility (what I am after);
• more efficient optimizations (what Arnaud showed is possible);
• various other things
  Then the key question is: Who defines that boundary?

Is it the codeauthor that defines that boundary? Or is it the codeconsumer?

Similarly, is it the code author or consumer that has to Do Work(tm) in order to leverage the desired capability? Or both?

My take on this is that both need to exist as separate features.

The author should define the boundary for things like visibility and
optimisation. It should be possible to take an existing Composer
package, add some metadata to it in some way, and have the engine make
some useful assumptions. The consumer of that package should not need to
care about the difference, except in edge cases like trying to define
additional classes with the same prefix as a third-party package.

On the other hand, the consumer should define the boundary for isolating
name resolution. It should be possible to take any folder of PHP files,
with no metadata about package management, and load it in a context
where duplicate class names are allowed. The author of the package
shouldn't need to make any changes, except in edge cases like highly
dynamic code which needs additional hints to work correctly when sandboxed.

The first feature is an add-on to the extremely successful ecosystem
based on the assumption that packages will inter-operate based on agreed
names. The second feature is a bridge between that ecosystem and the
very different world of plugin-based web applications, which want to
manage multiple pieces of code which are not designed to co-operate, and
run them side by side.

JS has come up a few times as a comparison, because there the two
features overlap heavily. That's because the language has always started
from the opposite assumption to PHP: declarations in JS are local by
default, and can only be referenced outside of the current function
scope if explicitly passed outwards in some way. In PHP - and Java, C#,
and many others - the opposite is true, and declarations have a global
name by default; keywords like "private" and "internal" are then used to
indicate that although code can name something, it's not allowed to use it.

Both have their strengths and weaknesses, but at this point every JS
module going back 15+ years (CommonJS was founded in 2009, to
standardise existing practices) is based on the "interact by export"
model; and every PHP package going back 25+ years (PEAR founded in 1999;
Composer in 2011) is based on the "interact by name" model.

--
Rowan Tommins
[IMSoP]