[Initial Feedback] PHP User Modules - An Adaptation of ES6 from JavaScript

Not replying to anyone in particular and instead doing a mild reset taking
into account the discussion that has gone before.

So, I want to import a package. I'll create an index.php file at the root
of my website and populate it with this.

<?php
import "./src/mymodule";

Now I'll create that directory and run a command php mod init in that
directory. Stealing this from Go, it's fairly straightforward though. Now
if we look in the directory we will see two files.

php.mod

php.sum

The second file I'll not be touching on but exists to track checksums of
downloaded packages - Composer does the same with its composer-lock.json
file which in turn was inspired by node's package-lock.json.

The php.mod file stands in for composer.json, but it isn't a json file. It
would start something like this:

namespace mymodule

php 10.0

registry packagist.org/packages

We start with three directives - the root namespace is presumed to be the
directory name. If that isn't true this is a text file, change it. PHP min
version should be straightforward. Registry details where we are going to
go get code from. Suppose we want to use our own registry but fallback to
packagist. That would be this:

namespace mymodule

php 10.0

registry (

github.com/myaccount

packagist.org/packages

)

Multiple registry entries will be checked for the code in order. Handling
auth tokens for restricted registries is outside of scope at the moment.

So let's build the module. We'll make a file called hello.phm. The reason
for phm and not php is so that web SAPIs will not try to parse this code.
Further they can be configured to not even allow direct https access to
these files at all.

import "twig/twig";

use \Twig\Loader\ArrayLoader;

use \Twig\Environment;

$loader = new ArrayLoader([

'index' => 'Hello {{ name }}'

]);

$twig = new Environment($loader);

export $twig;

As mentioned in previous discussions, modules have their own variable
scope. Back in our index we need to receive the variable

<?php

import $twig from "./src/mymodule"

$twig->render('index', ['name' => 'World']);

If we load index.php in the web browser we should see "Hello World". If
we look back in the mymodules folder we'll see the php.mod file has been
updated

namespace mymodule

php 10.0

registry packagist.org/packages

imports (

twig/twig v3.10.3

symfony/deprecation-contracts v2.5 //indirect

symfony/polyfill-mbstring v1.3 //indirect

symfony/polyfill-php80 v1.22 //indirect

)

Note the automatically entered comment that marks the imported dependencies
of twig. Meanwhile the php.sum file will also be updated with the checksums
of these packages.

So why this instead of composer? Well, a native implementation should be
faster, but also it might be able to deal with php extensions.

import "@php_mysqli"

The @ marks that the extension is either a .so or .dll library, as I'll
hazard a guess that the resolution mechanic will be radically different
from the php language modules themselves - if it is possible at all. If it
can be done it will make working with packages that require extensions a
hell of a lot easier since it will no longer be necessary to monkey the
php.ini file to include them. At a minimum the parser needs to know that
the import will not be in the registry and instead it should look to the
extensions directory, hence the lead @. Speaking of, having the extension
directory location be a directive of php.mod makes sense here. Each module
can have its own extension directory, but if this is kept within the
project instead of globally then web SAPIs definitely need to stay out of
those directories.

Final thing to touch on is how the module namespaces behave. The export
statement is used to call out what is leaving the module - everything else
is private to that module.

class A {} // private

export class B {} // public

All the files of the package effectively have the same starting namespace -
whatever was declared in php.mod. So it isn't necessary to repeat the
namespace on each file of the package. If a namespace is given, it will be
a sub-namespace

namespace tests;

export function foo() {}

Then in the importing file

import "./src/mymodule"

use \mymodule\tests\foo

Notice here that if there is no from clause everything in the module grafts
onto the symbol table. Subsequent file loads need only use the use
statement. Exported variables however must be explicitly pulled because the
variable symbol table isn't affected by namespaces (if I recall correctly,
call me an idiot if I'm wrong).

The from clause is useful for permanently aliasing - if something is
imported under an alias it will remain under that alias. Continuing the
prior example

import tests\foo as boo from "./src/mymodule";

boo()

That's enough to chew on I think.

Namespaces don't require autoloading, and autoloading doesn't require one file per class.

No they do not, but the design of each was heavily intertwined with each other resulting in a less than optimal design, IMO.

So, are you arguing to keep one and eject the other for modules, and if so which are you arguing we eject? Autoloading?

Or are you arguing to keep both for modules, in which case your argument above is moot?

To compile a program with multiple source files, in any language, you need one of two things:

a) A list of files you want to compile. Maybe auto-generated, maybe done with a recursive iteration over a directory, but ultimately the compiler needs a file path to process.

Recursion is only needed if modules are hierarchical in nature.

b) A way for the compiler to tell, based on some symbol it wants to resolve, which file should be compiled.

That presumes the compiler did not simply generate an AST from the list of files.

PHP originally provided only option (a), via the include and require keywords. Autoloading adds option (b), where you provide a function which takes a class name and does whatever you want to find the definition.

And that "whatever you want" takes execution time (and tracing through when you are debugging.) But when you look at many other languages loading is an implementation detail that PHP chose to hoist onto userland developers when PHP could have established the rules to handle it more performantly without userland involvement.

Or is there some aspect of autoloading that could not be handled by PHP itself? Note I am asking only within the propose scope of modules, which we could constrain to optimize their runtime use.

I think it might be time to re-visit the tooling around option (a), as OpCache makes the cost of eagerly loading a list of files much lower than it was when autoloading was added.

Now you are getting somewhere.

Imagine that each module — which could equal a single directory — could have a pre-compiled op-cache which is essentially what I proposed in other recent emails.

That could be as simple as include_all($directory), or as fancy as include_from_manifest_file($some_crazy_binary_file_format); either could be implemented right now in userland, because it all eventually comes down to calling include or require.

meh.

That sounds like a way to avoid discussing the ways in which smartly designed modules could really improve PHP.

My opinions match Larry's almost exactly: I want package-level optimisation, and package-private declarations. But I don't want to rewrite my entire codebase to start using a completely different naming system.
I can't see how package-privates would be of any value to you unless you rewrite your codebase.
Simple: I have private Composer packages, right now, that group all their classes under a particular namespace prefix. I want to be able to mark some classes in that namespace as "internal".

Not simple, although I admit I am being pedantic about words used here, but for a reason.

I asked about "package-privates," you responded with "namespace-privates."

Adding private to namespaces is orthogonal to the discussion of packages.

To require that packages be constrained to have all the same warts as namespaces and existing PHP code simply so you can have namespace-privates is short-sighted (and IMO a bit selfish.)

Alternately, namespaces could get private scope in parallel to having modules be considered.

That would allow modules to gain improvements that we could not get by having to maintain BC with namespaces.

Which causes me to ask: If you have really wanted namespace private why has it been six years since it was even last mentioned on the list, and four years since last discussed?

https://externals.io/message/101323 https://externals.io/message/101323

Why has there not been an RFC since this one https://wiki.php.net/rfc/namespace-visibility https://wiki.php.net/rfc/namespace-visibility six years ago, that was not even voted on?

Why is it that when the topic of addressing modules/packages comes up — which has been talked about numerous times in the past six years — do you now bring up namespace privates in a manner that would effectively torpedo goals of the modules discussion, at least from the perspective of the OP and myself?

If namespace private were really something important to you, why haven't you championed it before, rather than hijack a discussion about the benefits we could get from modules if not constrained by namespaces?

I do not want to change every place that uses the existing classes to reference "ModuleName@ClassName" instead of "NamespacePrefix\ClassName", or change every "use" to "import from".

Then don't. Champion this RFC https://wiki.php.net/rfc/namespace-visibility https://wiki.php.net/rfc/namespace-visibility and get what you want.

But please don't argue against a discussion on modules because you want a feature that can be gotten orthogonally. (If you must argue against it, make arguments for which accommodations for your preferences cannot be found.)

Code doesn't existing in isolation; if Symfony Mailer is re-published as a "module", every single application that uses it needs to change their code from referencing namespaced classes, to having "import" statements.

And that is bad, how?

But before you answer, it just means that instead of a use statement in your existing code you change to a import statement.

You'd then of course need to changes — if applicable — to call the new Symphony Mailer, but you'd have to do that with or without modules.

Or is there something else I am missing?

As for package-level optimisation, you'll need to give examples of what you mean there as I don't want to wrongly assume.

Currently, OpCache only optimises per file, because it can't guarantee how files will be used together.

A simple example is function fallback: if you could declare a package as "completely loaded", OpCache could replace all references to "strlen" with "\strlen", knowing that no namespaced function with that name could be added later.

Thank you for elaborating on that.

So, champion an RFC to improve OpCache for namespaces. That need not impose on the discussion about modules.

Further, and this is what is nice about being able to discuss modules not having to be compatible with namespaces, if there are aspect of namespaces that make optimization hard or impossible then we could potentially set up rules of modules that make similar optimizations easy and/or possible.

EVEN further, consider the fact that in PHP all class members are public by default. One thing we could have in modules is to go back to short var and eliminate both private and protected modifiers and only have public with the default behavior being what is private outside of modules. protected would no longer be needed as we would have module scope which is defacto-protected.

Classes could be final by default in modules and then we could modify them with an open keyword (thanks to Lynn for that one.)

And so on. In other words, if we could treat modules as their own sandbox, we could get fix many of the regrettable former design choices of the PHP language — some of which are to make PHP be beginner friendly — and potentially re-energize people who once looked at PHP and dismissed it to give it another look.

What I meant was: we can't just treat namespaces and modules as completely separate things, and assume that every code file will be using one style or the other. We have to imagine the user experience when there is a mix of the two.

Why can we not just treat namespaces and modules as completely separate things?

I can't imagine it being pleasant to have a mix of "import" and "use" statements at the top of a file, with different and even conflicting semantics.

That feels like a frivolous concern when compared to the benefits we could see with modules, especially when there would be ways to mitigate your stated concerns here.

If you don't like to see imports, but your imports in a namespace and then "use" that namespace.

Or we could allow "use module" instead of (or in addition to) "import" and then it could look more pleasant for you.

As for conflicting semantics:

1.) I'm not seeing how those could be significant in the using/importing file, and
2.) Isn't dealing with conflicting semantics just a part of programming?
3.) Don't "use" and "use function" have conflicting semantics?

God knows that "use" by itself has many confusing semantics, which "import" could avoid.

Perhaps I didn't word the question well. What I'm really asking, as someone who's never used JS or Go modules, is why I'd want to write "import", rather than referencing a global name in a hierarchy.

"use module" would work just as well as "import"; the "import" is not special, the module scoping and features are what is valuable here.

For specifics see my other recent emails on the subject. If they do not explain, please ask again with specifics.

That's really all I mean by "making it compatible with namespace": I want "new \Foo\Bar\Baz;" to be able to refer to a "packaged" class, probably by having a way to mark all classes under "\Foo\Bar" as belonging to a particular package.

And that is what I am trying to get away from.

First the backslash — because when using in reflection or other dynamic programming they have to be escaped which can lead to escaping errors. I know you don't care, but I and others do.

Second, the hierarchy. Because there is no constraint on hierarchy PHP subtly encourages developers — as if sirens of the Odyssey — to create large hierarchies. I even find myself doing it as I fighting myself against it.

The reasons hierarchy is bad is:

1.) larger hierarchies grow conceptual complexity,
2.) they place no limit on package growth as you can always create subdirectories,
3.) they make it harder to "see" all the code files in one place (a single directory),
4.) they constrain where code is located when there are benefits to a different layout

That's really all I mean by "making it compatible with namespace": I want "new \Foo\Bar\Baz;" to be able to refer to a "packaged" class, probably by having a way to mark all classes under "\Foo\Bar" as belonging to a particular package.

Revisiting this, why is it important to you that "new \Foo\Bar\Baz" refer to a "packaged" class vs a namespaced class, assuming you had namespace-private and OpCache improvements?

Why can't you still just use the namespaces you prefer and let "packages" (modules) improve in other ways?

I am trying my best not to make this ad-hominem so forgive me but I do have to ask if this is just not a case of "I am comfortable doing it the way I have been doing it and do not want to consider changing," maybe? Note I am asking that question limited to the one statement I quoted above, not on the broader discussion.

6. Modules and packages are typically small-scoped to a single directory and it is a code smell to have many different packages tightly coupled, as is the case with namespaces. Forcing modules to munge with namespaces would mean most modules would be written with those code smells for years because that will be how everyone doing a quick shift from namespace to module will write their modules.

Again, this is entirely about code style, and not something the language can control.

A language cannot control it, but a language can encourage or discourse it.

And the PHP language encourages a large amount of file and directory bloat.

One only need to compare the number of files in most PHP libraries to the number of files in JS or Go package to see that the nature of a language clearly does not influence.

To bring stats vs. opinion I asked ChatGPT what the two equivalent packages are to Symphony for JS and Go respectively and it suggested ExpressJS and Gin. So I cloned them to see the number of files and directories each has. From the root of each repo:

Project			Files 	Dirs

Symfony: 12,504 2,162
ExpressJS: 259 87
Gin(GoLang): 145 30

The comparison might not be completely fair given how much longer Symfony has been around, but they all target the same use-case so even if there is less functionality in ExpressJS or Gin.

Given that I think that well over an order of magnitude more files is a really odiferous code smell, and is thanks to the language which admittedly cannot "control" layout, but definitely influences it.

Am I wrong? Present any other relatively equivalent project comparisons you please. Here are the bash commands to count files and dirs:

find /path/to/subdirectory -type f | wc -l
find /path/to/subdirectory -type d | wc -l

Also, the JS insistence on having a separate package for every tiny function is a common source of criticism, so personally I am very happy that PHP packages are generally larger than that.

I can't speak for the OP, but nothing I am proposing is advocating for separate packages for every tiny functions. Nothing.

Instead I am advocating for packages that are mostly in a few directories instead of almost two magnitudes more!

That said, maybe the best solution is to NOT put the stake in the ground right now and say "They must be namespace compatible" or "They must not be namespace compatible" but move forward with an open mind so that we can tease out exactly how namespaces would constrain modules and and then make the decision later for what would be in the best interest of moving PHP into the future.

If and when an actual problem arises, let's discuss it.

Not "problems" but instead "opportunities."

I have already pointed out numerous opportunities in this email and one of my recent emails.

What specifically stops us doing all the things you've been discussing around loading, and visibility, etc, in a way that's compatible with the
400_000 packages available on Packagist, and billions of lines of existing code?

You speak as if I am proposing getting rid of namespaces and making those 400_000 packages available on Packagist, and billions of lines of existing code not work in PHP. Of course not.

No, I'm saying that every one of those packages could benefit if we make incremental changes.

Maybe.

What benefits can you envision you would get if PHP made namespaces==modules compared with the benefits I have mentioned for making modules not be constrained to compatibility with namespaces (besides private and OpCache as we already discussed you pursue for namespaces?)

Can we get precompiling for modules in a directory and written to a .php.module file? We can't do that with namespaces because scanning recursively could take too long at runtime.

Can we get default private for all symbols and class members in namespaces? No, that would be a huge BC break.

Can we get namespaces to be first-class AST participants? If yes, why have we not done it before?

I could go on, but this email is getting loooong.

I don't want to couple it so that you can't have "package private" without also switching to some new "advanced" dialect of the language, and I don't see any reason why we need to do so.

And I am not advocating that. I am advocating you should get "namespace private." Hey RFC is already written! https://wiki.php.net/rfc/namespace-visibility https://wiki.php.net/rfc/namespace-visibility

And most of the other benefits of modules as I am proposing would be BC breaks so you could not get them in namespaces anyway.

Unless you can come up with something besides private and opCache I had not considered.

Maybe package scoped declares could allow opting in to certain checks, but I don't think "is in a package" and "has been audited for a load of extra breaking changes" should be set by the same flag.

I am not aware of any discussion of opting in, flags, nor auditing with respect to modules.

-Mike

This is a very long reply to several emails.

__
The angle I am coming at this from is improving the developer experience around "packages" or "modules" or whatever you want to call them, and so much of this proposal doesn't seem to be about that.

Ok, first problem - not a proposal really, but a ramble trying to get to a proposal. Before I made the first post the idea was knocking around in my head and wouldn't go away, so I just stream of consciousness listed what's going through my head. That leads to the second point you made.

I could have made that point in other ways, and I'm sorry that my first attempt came off as insulting. It really concerned me when I already saw discussion about taking this off-list and going into the weeds on technical details when the problem that is being addressed by this proposal is extremely unclear to me.

It is unclear even to me. Perhaps I shouldn't have posted out something this half baked. That said, pruning off large sections of language functionality is a distraction. For now let's just note that it is a possibility to improve the language this way afforded by the fact that import would be new way of bringing scripts in. Could isn't should. Also, at the moment again it's a distraction. Let's focus down on how code is imported.

First though, a history review, partially to get this straight in my own head but hopefully of use for those following along. Why? Knowing how we got we are is important to some degree to chart a way forward.

PHP started as a template engine. By modern standards, and compared to the likes of twig, it's a very bad template engine, but that doesn't really matter because it's evolved into a programming language in it's own right over the last nearly 20 years.

How do you think twig works, exactly? You should probably check it out, because templates compile down to regular PHP templates -- at least it did the last time I looked at it a few years ago. How do you think emails are templated by php code? How do you think anything is output? By either "echo" or ?> content <?php, or fwrite/file_put_contents

Without that, there is literally no purpose to php code (or any code).

Include, include_once, require, and require_once have been around since the beginning as the way to splice code files together. The behavior of these statements calls back to PHP's origin as a template engine as they do things similar mechanisms like JavaScript's import do not do (and for that matter, their equivalents in C# and Java). Their scope behavior is very different from import mechanisms in other languages, as they see the variables in the scope of the function they were invoked from or the global scope when called from there. Their parsing can be aborted early with a return. They can return a value, which is quite unusual to be honest. None of this is bad per se, but it is different and the question arises is it necessary.

How do you think javascript import works, exactly? They load a file which returns a value via export.

There is nothing inherently wrong with requires/includes, it's literally required by every language via some mechanism or another (C's include statements, go's go.mod file, javascripts import/package.json, C#'s project config, etc). There's no magical thing here, just abstractions and different levels of it.

One artifact of their behavior that is bad in my opinion is that they start from the standpoint of being text or html files.

Every single language starts with a text file... There's nothing inherently special about the bytes in any source code file and they only have meaning due to creating a parser that can make sense of the stream of bytes. The fact that they also have meaning to humans is what makes it source code and not object code/byte code.

If the included file has no PHP tags then the contents get echoed out. If there are no output buffers running this can cause headers to be set and fun errors to be had. So they can't be used to create files that can only echo explicitly (that is, a call to the echo statement or the like).

For headers, this is largely up to the SAPI (the program that executes the PHP code, eg, frankenphp, php-fpm, mod-cgi, roadrunner, etc) and the fact that most SAPIs want to be able to run existing code where developers have certain expectations of behavior. FrankenPHP did a little something different with the support of the 103 status code which isn't supported in any other SAPI (AFAIK). The CLI sapi doesn't output any headers whatsoever.

As far as PHP opening tags go... I don't even notice it. They're there, just like the "package" declaration on the top of every one of my Go files.

Fast forward a bit - PHP 5.3, and the introduction of namespaces were introduced to deal with the overloaded symbol tables. They are a bit a hotwire as (if I'm not mistaken, it's been a couple years since I read the discussion on it) they just quietly prepend the namespace string in front of the name of all new symbols declared in the namespace for use elsewhere. As a result, PHP namespaces don't do some of the things we see in the namespaces of other languages (looking at Java and C# here). For example, privacy modifiers within a namespace aren't a thing.

This would be nice to have ... maybe. But namespace have been around, what, 10-15 years? I think if someone wanted to "fix" it, it would have been fixed by now.

Very quickly after PHP 5.3 released autoloaders showed up. At some point support for multiple autoloaders was added. Several schema were added, PSR-4 won out, and composer showed up to leverage this. Composer is based on NPM, even to the point where json is used to configure it, and the composer.json file is fairly close to npm's package.json file even now. It's a userland solution, but to my knowledge WordPress is the only widely used PHP application out there that doesn't use it directly (there is a Composer Wordpress project).

WordPress predates composer et. al., by more than 10 years (if you count the B2 code it was forked from). Why would it use it? From working at Automattic (I left a couple of years ago, and this is merely what I saw as an observer, I wasn't closely involved with any of the open-source side), there was a bit of a push to make it happen as it looked like Composer would be around for awhile, but then there was talk about an "official" loader/thing from php itself and I think they'd rather use that instead. When you have a project that has literally been around decades, you don't change out your whole system on the whims of what is fashionable at the time; you either innovate or wait and see what becomes standard. Composer has only been around 50% of the time WordPress has been around now; and it didn't start out as immediately popular.

Before composer, and before namespaces there was PECL. Composer has eclipsed it because PECL has the limitation of being server-wide. It never really caught on in the age of virtual hosting with multiple PHP sites running on one box. Today we have Docker, but that didn't help PECL make a comeback because by the time docker deployment of PHP sites became the norm composer had won out. Also, composer library publishing is more permissive than PECL. I'll stop here lest this digress into a Composer v PECL discussion - suffice to say stabs a bringing code packages into PHP isn't a new idea, and a survey of what's been done before, what was right about those attempts and what was wrong needs to be considered before adding yet another php package system into the mix.

I don't think PECL and Composer have much in common... at all. Like they are not even comparable, and it is still the best way to install extensions (in fact, it is the only way in a docker container AFAIK) on a self-compiled php.

The main influence of composer and autoloaders for preparing packages is that PHP has become far more Object Oriented than it was before. Prior to PHP 5.3 object oriented programming was a great option, but since autoloaders cannot bring in functions (at least not directly, they can be cheated in by bundling them in static classes which are all but namespaces) the whole ecosystem has become heavily object oriented.

require/include still works fine, or using the "file" key on the autoloader in composer.json.

I don't find most of these "problems" actually valid. There is some merit to the conclusions, but the premise feels shaky.

— Rob

node_modules IMO is one of the worse things about the JavaScript ecosystem. Who has not seen the meme about node_modules being worse than a black hole?

Fair enough. Or maybe import maps would be a better way forward.

Import maps are really a small part of what PHP actually needs. For example, is it a class, an interface, or a function? For a module, it is a property?

I envision basically that this file, whatever it would be called would be a pre-compilation of everything that PHP can pre-compile about the files that are contained within the module/directory.

See below where I talk about a pre-compiled .php.module

But ensuring that it is possible to disallow loading needs to be contemplated in the design. PHP has to be able to know what is a module and what isn't without expensive processes.

One possible solution is that if modules do not have <?php ?> tags, ever, and someone directly tries to load a module through http(s) the file won't execute. Only files with <?php ?> tags are executable by the web sapi.

Except that would require parsing all of the entire files in the directory to know (unless everything were pre-compiled as I am advocating.).

Still, I think it would be better to be explicit, and for that I would propose the first line in the file needs to start with "module" and have the name of the module.

I've only touched the surface on how GoLang does things. Some of it was confusing to me at first. It's also been awhile so I'd need to refresh my memory to speak to it.

In Go modules or, in this context more correctly named "packages" are:

• A collection of files grouped into a directory and thus all files in that directory are in the same package.

• Public or private scope are determined by case of symbols; lowercase are private and uppercase are public. People coming from other languages tend to hate this, but I have come to love it because it makes code less dense while employing the same information as a "public" and "private" keywords. It also makes code across different developers more consistent.

• Packages can be nested in package directories, but...

• There is no concept of a "sub" package, meaning there are no hierarchies when packages are used in code (there is a file path hierarchy but that is only relevant for importing the package.) When I started working with Go I thought that was unfortunate. Now after 5+ years working with Go I see it as a really good decision.

• Package files must have a "package" statement at the top, and all files in the directory must have the same "package" statement, with one caveat.

• That caveat is that package files can have package <packagename>_test as a package name and that file is assumed to contains a test but it cannot see private members in package <packagename>.

• Test files are typically named to pair with a <filename>.go and would be named <filename>_test.go. That file's package name can either be just <packagename> or <packagename>_test, depending on if you want to reach into private members or not.

• You can also find test packages that contain all <filename>_test.go files.

• Testing is build into Go with go test ./... to run all tests in current and all subdirectories. (Idiomatic testing in Go is so much easier that idiomatic testing in PHP resulting in a culture of testing among almost all Go developers.)

• Package files can have types, vars, consts, and funcs as well as imports and directives, of course.

• Types in Go can be struct (which is the closest Go has to a class), slice of type e.g. []<type>, array of type e.g. [<n>]<type>, map[<key>]<value>, and a few more that I won't go into as I think they are out of scope for this explanation.

• Packages can have one or more init() functions that are all called before the program's main() func is called. There can also be multiple init() functions even in the same file.

• vars can be initialized and those initializations are run before the program's main() func is called.

• consts are initialized before the program's main() func is called but can only be initialized by literal scalar types. Unfortunately.

• imports take the form of import "<package>" for standard library types and where a <package> can contain parent paths.

• For local types imports take the form of import "<module>/<package>" where a <module> is defining by having a go.mod file in the directory or a parent directory, and a <package> can contain parent paths. A go.mod file has a module directive, a go version, and one or more require statements (I'm ignoring a bit of minutia here.)

• Modules allow grouping of packages together and were added in recent years to provide versioning for the collective dependencies of a module. The version information is stored in go.sum and is managed automatically with Go CLI commands.

• For external third party modules imports take the form of import "<domain>/<package>" where <package> can contain parent paths and almost always does. An example is "github.com/stretchr/testify/assert http://github.com/stretchr/testify/assert"

• External modules are by definition HTTP(S) GETable, and Go developers use go get <module> on the command line to download the module. Go does not have or need a 3rd party package manager as that can become a single point of failure and is definitely a single point of control. To download testify for use in their Go module a Go dev would run go get github.com/stretchr/testify <http://github.com/stretchr/testify/assert>;

• Most external third party modules for Go are hosted on Github but can be hosted on a custom domain, Bitbucket, GitLab, etc.

• The Go team manages a standard proxy for go get but organizations can run their own if desired.

• imports are referenced by name internally where the package name is the last segment of the import after a /, or just the name if no slash. So "github.com/stretchr/testify/assert http://github.com/stretchr/testify/assert" is referenced in code as assert. For example, assert.Equal(t,1,value) would assert that value!=1 then it would use the testing variable t to mark this assertion as an error and generate appropriate output.

• imports can be aliases so you could import check "github.com/stretchr/testify/assert <http://github.com/stretchr/testify/assert>"; and then call check.Equal(t,1,value) instead of assert.Equal(t,1,value)` but needing to alias a package frequently is a code smell for a badly named package.

• You can use . as an alias and then not need to use the alias, so we could import "github.com/stretchr/testify/assert http://github.com/stretchr/testify/assert"and then just callEqual(t,1,value) instead of assert.Equal(t,1,value) but this is frowned on in the Go community except for in very specific use-cares.

• You can use _ to bring in a package even if you are not referencing it in case it has an init() function that you need to run. If that applied to testify it would look like this: import _ "github.com/stretchr/testify/assert http://github.com/stretchr/testify/assert".`

• All of import, var, and const support a multiline for using parenthesis like so:

  var (
  x = 1
  y = 2
  )

• Module names are idiomatically one word w/o underscores and lowercase.

• There is no need to import specific symbols from a Go package like there is in JavaScript. I have programmed in both Go and JS, and I have not found a real benefit to having to reference everything explicitly in the import — since you have to mention the package name everywhere you use any package symbol — but I have noticed a benefit to not having nearly as much boilerplate at the top of the file for import when working with Go vs. working with Javascript. And my GoLang IDE just manages imports for me whereas WebStorm just calls out when I haven't imported function names in Javascript.

I am sure there is more I missed, but that should cover the highlights.

The takeaways that I think would be useful are PHP modules are:

1. Imports
2. Import aliases
3. Module-level consts
4. Module-level init() functions
5. Module-level vars with initialization
6. Module-level functions
7. One directory == one module
8. No hierarchy for modules
9. Single word module names in lowercase.
10. Module sytax being <module><operator><symbol>, e.g. mymodule->MySymbol

Takeaways I wish the PHP community would consider but doubt there is any chance:

1. Having modules be HTTP(S) GETtable with php get <module>
2. Uppercase being public, lowercase being private, and no need for protected
3. Test packages with testing build into PHP e.g. php test ./...

I'm not fond of this either.

There will need to be a way to define the entrypoint php. I think index.php is reasonable, and if another entry point is desired it can be called out -> "mypackage/myentry.php"

Why is an entry point needed? If there is a module metadata file as I am proposing PHP can get all the information it needs from that file. Maybe that is the .phm file?

Maybe. Again, I need to look over this meta data format. Also, how does it get created?

As I am envisioning, PHP at the command line would have the ability to pre-compile a module — aka all files in the module directory — and then write a module-specific file, maybe .php.module? That could ideally be optimized for loading by PHP and have everything it needs to know to run the code in that module.

That file could be completely self-contained include all source code similar to a .phar file, or it could just have a complete symbol table and still require the PHP source code to exist as well. I have not pondered all the pros and cons of these alternatives yet.

Clearly though even if it compiled to a self-contained file the .PHP files would still be needed during development. Thus I envision that the PHP CLI would need a --watch option to watch directories and recompile the .php.module file upon PHP file change. IDEs like PhpStorm could run php --watch for users and non-IDE users could run it themselves.

When PHP would come across an import statement pointing to a module directory it would first look for the compiled .php.module file and if found use it but if not found it would recreate it. Maybe it could write to disk, or generate an error if it cannot write to disk. OTOH writing to disk might be a security issue in which case it could issue a warning that the .php.module file does not exists and then compile the module to memory and continue on.

It would be nice if there was a mode where PHP would check the timestamps of all PHP files in the module directory and if the compiled .php.module was earlier than any of the .php file then recompile but you'd want that off for production. That could be a new function set_dev_mode(boolean) or a CLI option to create a .phpdev.module instead of a .php.module.

Clearly anyone using deployments could have their build generate all the required .php.module files for deployment, and hosting companies that host apps that don't use deployments like WordPress could have processes that build the .php.module files for their users.

I think I have thought through this enough to identify there are no technical blockers, but I could certainly have missed something so please call it out if anyone can identify something that would keep this from working and/or significantly change the nature of PHP development.

BTW, this pre-compiling would ONLY apply to modules, so people not using modules would not have to be concerned about any of this at all.

-Mike

P.S.

I remember when the choice to use \ was made. I've rarely been so angry about a language design choice before or since. I've gotten used to it, but seeing \ all over the place in strings is still.. yuck.

Ditto.

2. Autoloading effectively necessitates that every symbol be in its own separate file.

How so? While Composer "recommends" PSR4 autoloading with one class per file, other configurations are entirely possible, either with Composer's autoload.classmap (which uses a precomputed class -> filename table), or with a custom autoloader.

On Jul 1, 2024, at 7:57 AM, Arvids Godjuks arvids.godjuks@gmail.com
wrote:

TL;DR: As a userland developer, in my opinion, this is just a downgrade
from what we have now. Enhance namespaces to have the ability to have
internal/private classes, interfaces, enums and constants. That's about it.

Please note my comments that follow do not mean I am in support of this
package proposal as presented.

Autoloading is one of the best killer features of PHP - love it or hate
it - it's your personal preference.

Two really solid reasons to hate autoloading as implemented in PHP:

1. Autoloading runs userland code. This means it has the potential
   conflict between different packages with different autoloaders, it means
   there can be buggy autoloaders, and it means that when using XDEBUG every
   time a new symbol is found when the developer is single-step debugging the
   developer will be dropped into the autoloader and then best case they then
   immediately trace out. All of these aspects a major PITA and time waster
   and make debugging more exhausting than it needs to be.

But that is why interoperability in PHP world is so high. When it was
introduced, it allowed enabling autoloading for most code bases out there
regardless of what their structure was and still is. Sure, you have to be
careful how you do it, but that is also not really a userland concern -
most of those have been implemented once and then almost never touched :)
The community has settled on a general approach since naturally and that's
how vast majority of people write their code. Buggy code is buggy code, it
really has not much to do with the autoloading and with how people write
buggy code :) You are blaming the hammer for user trying to nail the screw
into wood :)

You can already sidestep autoloader by adding a require statement to any
file and loading everything without triggering autoload. You can add your
own autoloader that has a map of high-level namespaces where you want to
load it as a package and recursively include everything that way. The tools
are there, you just need to use them. If anything, since PHP now has
attributes, you can just make yourself an attribute and handler for it and
have a #[Package('name')] that can find all files with that attribute and
load it all as a package.
It's far more powerful in userland because it allows people to do whatever
they want with it if they do not like the standard autoloader(s). If
anything, combined with composer folks, PHP-FIG could come up with a
community based #[Package] tag and make packages a thing.

2. Autoloading effectively necessitates that every symbol be in its own
   separate file. This needlessly bloats number of files and directories by
   more than an order of magnitude — see my numbers from recent discussion —
   and that also mean related code is located farther away from other related
   code. This can be worked around but the workarounds I've seen are all
   fragile and unable to be generic, and few 3rd party packages do this.

That's just how PSR-4 standard was written and works, and it's a good
default for a reason. On small projects - sure, I can see it being
overkill. But I haven't worked on a small project in a decade and I rarely
have that little code in one file that I would want to stuff all together
in one file. I do use PHPStorm, so it is very adapted to how most PHP
projects are and provides excellent navigation abilities that fit the PSR-4
and that way of structuring projects. Vast part of the community uses it.
It's a standard in a lot of companies for a reason.

I've seen a sizeable chunk of developers that come from other languages
discover PHP's autoloading and their minds just get blown.

It is unclear to me if by saying their "minds just get blown" if that
means you think they see it as a positive or negative?

In a positive manner - a lot of people love that they do not have to fiddle
with import statements and just can leave that part to the ecosystem and
IDE to figure out.

As a developer who spent a decade in PHP and then branched out and added
Go to my repertoire I can tell you one of the nicest differences I
experienced was not having to deal with an autoloader during debugging, and
not being so constrained was to have to create a new file for every symbol.
Go projects need an order of magnitude fewer files. It is just so much
easier to grok the source code of a Go project compared to a PHP because of
this one simple fact. Now when I program in PHP I find myself constantly
cursing the fact that I have to deal with the autoloader.

BTW, I know Go is a pre-compiled language unlike PHP, but that does not
necessarily preclude PHP from having a better solution for code loading and
organization.

See my reply to autoloading things above - you can eliminate the autoloader
triggering easily in probably 15-30 minutes flat.

Performance has not been an issue for a long time due to opcache and all
the optimizations that have been done to it and ability to preload
bytecode. Then there are things like FrankenPHP, Swoole, ReactPHP and
others that entirely sidestep that issue. And then there's the active
development of JIT engine - just let the people working on the
implementation time to cook.

This reads to me like Stockholm syndrome, e.g. "My captors still hold me
captive, but they no longer beat me every day."

It's not that, it's literally true. PHP is one of the fastest-interpreted
languages, butting heads with nodejs only losing to it in default simple
application implementations because PHP is not event loop first (but
Franken PHP and others have a few things to say about that and then there
have been recently people playing with fibers and stuff and got performance
results that showed PHP being faster than nodejs at higher loads. Sorry I
do not have a link handly :\ )

It works, worked for a long time and there are not so many things wrong
with it to entirely upend the whole ecosystem and split the language.
Here's your HARD REMINDER about Python 2 => Python 3 and how that went and
is still somewhat ongoing.

Totally agree on that.

-Mike

There are aspects to this that go beyond just technical aspects. Wherever
when implementing autoloading people were secret geniuses, stumbled into it
accidentally or just had practical needs themselves that they just
implemented in this way - it has been a major turning point in PHP's life
and has transformed the ecosystem into what it is today together with the
rise of the composer package manager. I have seen people talk about
visibility in namespaces, package concept and all that, but every time it
was building upon existing autoloading mechanics - some adding capabilities
to them, modifying them, having new type of autoloader. But I have never
encountered anyone in this community in 20 years i have been an active part
of it to propose a radical change like this and I'm fairly certain after
keeping up with the thread that it is almost universally not what people
want. Most people just want the toolbox be "finished" so to speak, not get
a completely new one in addition that has no compatibility with the old one.
To be frank, the PHP ecosystem just does not have the resources to eat a
change on that level and support more than one implementation. Unlike many
other languages, PHP does not really get support and investment from the
likes of google, Microsoft, meta and so on. PHP Foundation is great, but
it's not google that can singlehandedly throw 1000 devs at supporting a
language and not even really feel it.

Arvīds Godjuks
+371 26 851 664
arvids.godjuks@gmail.com
Telegram: @psihius https://t.me/psihius

TL;DR: As a userland developer, in my opinion, this is just a downgrade from what we have now. Enhance namespaces to have the ability to have internal/private classes, interfaces, enums and constants. That's about it.

Please note my comments that follow do not mean I am in support of this package proposal as presented.

Autoloading is one of the best killer features of PHP - love it or hate it - it's your personal preference.

Two really solid reasons to hate autoloading as implemented in PHP:

1. Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders, it means there can be buggy autoloaders, and it means that when using XDEBUG every time a new symbol is found when the developer is single-step debugging the developer will be dropped into the autoloader and then best case they then immediately trace out. All of these aspects a major PITA and time waster and make debugging more exhausting than it needs to be.

FWIW, (in Intellij at least), you can set it to skip those files.

— Rob

Sent from my iPhone

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders

Can run userland code. It doesn't have to; FYI spl_autoload (https://www.php.net/manual/en/function.spl-autoload.php) has existed since php5.1 and works amazingly well.

That "standards" like psr-whatever can't (read: choose not to) use it says more about people and maintaining their little fiefdoms than anything else.