Module or Class Visibility, Season 2

Hi!

It's been a few days since I wanted to send this email to internals, but
real life has been a bit chaotic so I apologize if it comes off as if I
didn't research the archives enough. I glossed over the Module conversation
from 10 months ago and the one that recently surfaced and after deeply
thinking about Rowan's and Larry's comments I wanted to throw this idea
into the pit.

Lets preface the conversation with the fact that 1) a module system for
PHP has been discussed for several years and 2) if there was an easy and
perfect solution it would have been long implemented by now.

If we consider how GitHub, Composer and Docker Hub works, we can pin a
very important aspect of "namespaces": {entity}/{project}.

Hi, thank you for starting this thread.

I have a similar thing, an idea that was sitting on my mind that I want to
put in writing, so I will share it here.

First of all, I want to acknowledge the problems with big projects, big
monoliths that can only have one composer.json and many libraries needed,
that would eventually clash and make upgrades harder than they need to be.

Can we develop the modules concept without linking ourselves to namespaces
or any other new entity?

I think it might be possible, if every symbol that is defined is attached
to a module when it is loaded.
And we can use something like:

module(string $name, callable $closure);

The $closure is defined in the current module, but is executed in the new
module.
Anything else follows the logic: the functions and methods are executed in
the module where they are defined and associated with. Any new symbols
defined inherit the current module.

When a module is defined by another parent module, a relation will be
created between them and the parent module would get access to the child
module symbols, but not also the other way around.

There is no need to change anything on the libraries code but only on
composer and autoloading. That is the key point for easy adoption.
Composer-related, each installed package will be loaded in their own module.

What is yet to figure out:

• can we define internal modules, where the child module symbols are
  visible only in the parent defining module and not any level up?
• once a module defined, can the parent - child relation be
  redefined/changed/removed?

I apologize if this changes the solution discussion, but I didn't wanted to
start a new thread on the same topic.

--
Alex

Hi!

This would allow public, private and protected classes in a way that I
believe to be useful for the large ecosystem that surrounds Composer. From
my extremely limited understanding of the engine, I think the easy/natural
step would be to allow private/protected classes to be received outside
its namespace because a type declaration does not trigger autoload.

This has been discussed before - making Namespaces something beyond what
they are - a convenience string replace. That is given

namespace foo;
use otherns\boo;

function bar () {}
bar();
boo();

The engine quietly rewrites as

function foo\bar () {}

foo\bar();
otherns\boo();

The engine doesn't store any notion of namespace directly, it's just a
string replace on the symbol table. And if I recall correctly this is also
the reason that \ is the namespace operator - the original proposal was for
namespaces to have the :: operator, same as static classes, but late in the
implementation it was found that there were disambiguation problems and the
choice was made to use the \ operator rather than solve those problems. I
assume the problems were judged to be intractable, but I can't say for sure.

Also, a namespace block could appear anywhere, so if a stubborn programmer
really wanted to get access to a private or protected block they could put
a namespace block into their own code with the same namespace.

This also doesn't isolate a module with shared dependencies from changes to
those dependencies. This makes composer libraries impossible to use
effectively in the absence of a mechanism for coordinating those conflicts
from the core. And for worse, the WP core team doesn't want to provide
such. And yes, it really isn't the PHP dev team's direct responsibility to
step in and directly fix that problem. Indirectly perhaps? Only if it is
a benefit to everyone who uses the language.

Neither PHP nor JavaScript deal with directories or other file collections.
We have to look to Java and its jar files or golang modules for examples of
this. And no, the PSR standard of mapping namespaces to directories has
nothing to do with how the engine itself sees packages, namespaces, or the
world. In truth it sees none of these because they don't exist in the
runtime in any meaningful way that I'm aware of. Phar perhaps? I know
little of that system beyond the fact is the manner in which composer and a
few other libs are distributed.

I sense, and I could be wrong here, that there is no appetite for moving
beyond the one file at a time model. So the system I proposed last go round
was still a one file solution.

Other ideas I've seen kicked around - file based privacy. In this schema a
file can declare itself private (public is the assumed and Backward
compatible default) and then mark exceptions to this as public. But for
this to truly be useful I fear some file structure awareness would be
needed and again, at the moment, PHP doesn't do that.

A very long time ago I proposed a require into namespace mechanism. The
engine by default attaches all incoming namespaces to the root /. I
suggested (because I naively thought require was a function construct, not
a statement, and was ignorant of the difference at the time) that "target
namespace" could be the second argument of require and if provided all the
symbols established by that file become attached to that namespace. This
could only work if the autoloader correctly dispatched symbols to the
correct namespaces, and I don't have a clue how it could do that.

The need of the plugin community is code that just plugs into the app and
doesn't need to care about what the other applications are doing. This is
currently possible only if the plugin provides all of its own code and if
it does use composer libraries, it resorts to monkey-typing to change the
names of all the symbols to something prefixed with the plugin name to
avoid collisions. This approach is NOT optimal but it is the only one at
present.

However, an important question is whether this is enough groundwork that

could lead to optimizations that have been discussed when the topic of
module is brought up. For instance, if type-hint outside the module is
disallowed, could that make it easier to pack and optimize an entire module
if we could instruct PHP how to load all symbols of a namespace all at
once? I don't know.

Doing that would likely involve giving the engine some notion of directory,
again as Java does with JAR files, and PHP might do in PHAR files but I
know embarrassingly little about them. Could the existing PHAR structure be
used in some way for a starting point on this? I just don't know, not
without research.

If we consider how GitHub, Composer and Docker Hub works, we can pin a very important aspect of "namespaces": {entity}/{project}. Entity may either be an individual or an organization, but the concept is mostly the same. Although it can be argued that PHP has nothing to do with that, I think that could be a "good-enough" foundation considering the complexity of the subject.

While a two-level namespace root for a project is common, it's far from universal. Picking two examples from the first page of "popular packages" on packagist.org, Guzzle's root namespace is one level ("GuzzleHttp") and the Symfony Console component's root namespace is three levels ("Symfony\Component\Console").

So I think any module or visibility tied to namespaces would need a way to declare that prefix explicitly, not have the language assume it's a particular length.

If we just want namespace visibility, we could use Scala's approach, where the private modifier itself can be qualified:

private[\Symfony\Component\Console] class Foo { ... }
private[\Symfony] class Bar { ... }

If we want modules to have more existence - module-wide declares, optimisation, etc - then we need some way of declaring "this namespace prefix is a module" - a "module" keyword, or "declare_module" function, or something. Those are the lines that Larry and Arnaud were exploring along a while ago - see https://github.com/Crell/php-rfcs/blob/master/modules/spec-brainstorm.md and https://github.com/arnaud-lb/php-src/pull/10

What Michael Morris is talking about is really a completely different concept - it's more like "containers", in the sense of Docker, Kubernetes, etc, where different sections of code can be isolated, and declare classes with conflicting fully-qualified names. I don't think it's what most applications and libraries would want "modules" to be; it's probably best thought of as a completely separate feature.

--
Rowan Tommins
[IMSoP]

Hi!

It's been a few days since I wanted to send this email to internals, but
real life has been a bit chaotic so I apologize if it comes off as if I
didn't research the archives enough. I glossed over the Module conversation
from 10 months ago and the one that recently surfaced and after deeply
thinking about Rowan's and Larry's comments I wanted to throw this idea
into the pit.

Lets preface the conversation with the fact that 1) a module system for
PHP has been discussed for several years and 2) if there was an easy and
perfect solution it would have been long implemented by now. With that in
mind, I think there are mainly two major "camps": the ones that would
support something new similar to Node ESM vs CommonJS and those who won't.
Having dealt with this mess on the NodeJS side, I'm still on the side that
would support it because even though it's been 10 years worth of "mess", it
has greatly empowered progress. But I think PHP is too conservative to
indulge this camp, so I'm going to focus on Rowan's and Larry's position of
"we need something that builds on top of namespace, not replace it".

If we consider how GitHub, Composer and Docker Hub works, we can pin a
very important aspect of "namespaces": {entity}/{project}. Entity may
either be an individual or an organization, but the concept is mostly the
same. Although it can be argued that PHP has nothing to do with that, I
think that could be a "good-enough" foundation considering the complexity
of the subject. Here is what we could do:

<?php declare(strict_types=1);

namespace Acme\ProjectOne
{
    public class Foo {} // same as class Foo {}

    private class Bar {} // only visible inside Acme\ProjectOne

    protected class Baz {} // visible inside Acme
}

namespace Acme\ProjectTwo
{
    new \Acme\ProjectOne\Foo; // Work as always

    new \Acme\ProjectOne\Bar; // Fatal error: Uncaught Error: Cannot instantiate private class \Acme\ProjectOne\Bar from \Acme\ProjectTwo

    new \Acme\ProjectOne\Baz; // Works
}

namespace Corp\Corp
{
    new \Acme\ProjectOne\Foo; // Work as always

    new \Acme\ProjectOne\Bar; // Fatal error: Uncaught Error: Cannot instantiate private class \Acme\ProjectOne\Bar from \Corp\Corp

    new \Acme\ProjectOne\Baz; // Fatal error: Uncaught Error: Cannot instantiate protected class \Acme\ProjectOne\Baz from \Corp\Corp
}


function (\Acme\ProjectOne\Foo $foo) {} // Works as always

function (\Acme\ProjectOne\Bar $bar) {} // Open question: allow or disallow it?

function (\Acme\ProjectOne\Baz $baz) {} // Open question: allow or disallow it?

This would allow public, private and protected classes in a way that I
believe to be useful for the large ecosystem that surrounds Composer. From
my extremely limited understanding of the engine, I think the easy/natural
step would be to allow private/protected classes to be received outside
its namespace because a type declaration does not trigger autoload.
However, an important question is whether this is enough groundwork that
could lead to optimizations that have been discussed when the topic of
module is brought up. For instance, if type-hint outside the module is
disallowed, could that make it easier to pack and optimize an entire module
if we could instruct PHP how to load all symbols of a namespace all at
once? I don't know.

As I'm writing this down I don't know if it could be related or if its
something only making sense inside my head, but I see the above proposal
paired with a potential amendment to PSR-4 (and Composer), to stimulate the
community to pack small related symbols in a single file with an opt-in
approach:

composer.json:

// ...

"autoload": {
    "psr-4-with-module": {
        "App\\": "app/",
    }
},

// ...

<?php declare(strict_types=1);

// app/Foo/Bar.php
namespace App\Foo;
class Bar {}


// app/Foo.module.php
namespace App\Foo;
enum Baz {}
enum Qux {}


new \App\Foo\Bar; // loads app/Foo/Bar.php
\App\Foo\Baz::option; // file app/Foo/Baz.php does not exist, tries app/Foo.module.php before giving up
\App\Foo\Qux::option; // app/Foo.module.php has been loaded and Qux has been registered already

Thoughts?

--
Marco Deleu

Hey all,

Anyone familiar with C++'s friend keyword? It’s not a direct replacement
for modules, but it solves similar problems — allowing trusted classes or
functions to access private/protected members without making them public.

The idea: allow one class to explicitly grant access to another class or
function. Useful for tightly coupled code that still wants to maintain
encapsulation. Since friend would be a new keyword, it’s safe to add
(currently a parse error).

Examples:
`
class Engine {
private string $status = 'off';
friend class Car;
friend function debugEngine;
}

class Car {
public function start(Engine $e) {
$e->status = 'on'; // allowed
}
}

function debugEngine(Engine $e) {
echo $e->status; // also allowed
}
`
This avoids reflection, awkward internal APIs, or overly permissive
visibility. Could be useful in frameworks, testing tools, or any place
where selective trust is helpful.

Thoughts?

Hammed