[Initial Feedback] PHP User Modules - An Adaptation of ES6 from JavaScript

4 months ago by Michael Morris — view source

unread

Hello all. This is a ramble of an idea that's managed to run around my head
for a few days now. It isn't fully formed, but I've ran the thought
experiment as far as I can on my own and want to share it with all of you.

I've mostly been a lurker and I've seen a lot of RFC's come and go. Of
those not accepted many have been passed over because of background
compatibility. And then there is the issue that PHP has multiple design
flaws that seem impossible to get rid of. Finally, I sense from
conversations I've read that there are a lot of engine parser optimizations
that haven't been tried because of the background compatibility problems
present.

JavaScript was in this position as well 10 years ago when JavaScript
modules were introduced with the ES6 syntax. Only recently have these
modules finally begun to become first class members of node.js. The
existing CommonJS require mechanism remains and will remain in Node for the
foreseeable future, but the ES6 syntax allows an opportunity to sidestep
the issue. The most significant of these is JavaScript modules run in
strict mode, which actually removes features that are problematic for the
engine or make it difficult to create optimized code.

Something similar could be done in PHP, and that's what the remainder of
this letter is about, but before I continue I want to make clear my vantage
point: I am but a humble user of the code, I'm no expert on the
underpinnings of the Zend engine. In the text to follow I'm going to make
wrong calls on some things - maybe multiple things. I'm not married to
anything here. Further, even if I were a master of the engine and knew
where to start, the scope of this is too large for any one person to
undertake.

So all that said, I'll begin.

PHP User Modules are php files that are brought into the runtime through a
new parser that is able to generate faster and more concise runtime code by
removing support for problematic features and imposing a strict mode by
default. They focus on PHP as a language and not as a template engine.

The only background compatibility break is the introduction of three
keywords: "import", "export" and "from"

The PHP interpreter does not load PHP files as modules unless it is
directed to do so in an ini file or an .htaccess file using the
default_filetype directive. If this directive is missing its value will be
"default" - the value "module" will be used to trigger loading the initial
PHP file as a module, and further types could in theory be introduced at a
far later date.

Again, this setting only affects the INITIAL PHP script file loaded by the
interpreter, such as the index.php of Drupal. Files that are included with
include, include_once, require, or require_once will be imported as they
always have. Files that are included with import are PHP User Modules.

User Module Files
PHP User Modules have the following properties (Proposed, and very much
subject to change):

They are code files. They have no <?php or ?> tags, and the inclusion of
those tags is a parse exception. I know this will be problematic for PHP
storm and other IDE's, but it's not an insurmountable problem.
If the removal of HEREDOC and NOWDOC syntax would simplify the parser,
then these too would be removed from User Modules.
They have no starting symbol table. Each class, function or constant to
be used must be imported with the import statement. Symbols declared in a
user module do not affect the symbol tables of the rest of the runtime.
They have their own variable scope. They do not by default see globals or
superglobals. Variables declared in a module remain in that module.
Superglobals can be imported (Ideally this is an opportunity to provide new
more secure ways of accessing this data as several userland libraries have
done).
They have no support for braceless syntax (which is only really useful
when PHP is used as a template engine).
User Modules run in strict mode.
Exceptions only. trigger_error will cause a parse exception.
The @ error suppression operator is not supported.
Top level return to stop parsing of the file early (as in include and
require) is not supported.
If at all possible, . as a connotation operator will not be supported and
instead that operator will be used for scope resolution instead of the
three scope resolution operators currently in use for legacy reasons (::,
-> and \ )
Other language features whose use is considered bad practice are also up
for chopping.

Import Statement
PHP User modules are loaded by importing them, not by using include or
require. It's syntax is similar to JavaScript's, but not exact - for one
unlike JavaScript there need not be a from clause.

User Modules can't use code that hasn't been imported to their symbol
table. So if you want to use str_pos you need to import it

import str_pos

An import of a symbol will search for that symbol using the existing
resolution rules, and if the symbol is not found the autoloaders are
invoked. Once all have ran the import is retried and if the symbol now
exists globally it can be imported. This somewhat weird approach ensures
that user modules aren't cut off from the existing ecosystem.

Why not just require, or bother with importing existing symbols? My idea
here is clarity. It should be easier on the IDE's and probably on the
parser if these are called out. Also, explicitly importing functions makes
it easier to get to fixed versions, which is a repeated stumbling block of
many an RFC. I hereby invoke the ghost of PHP 6 and unicode. That failed
because it was too much to do in one pass. Import allows language
improvements to arrive piecemeal, and allows some of them to be userland.
More on that in a bit.

As with Javascript, aliasing is allowed.

import str_pos as strPos

The fun really starts when the from clause shows up.

import foo from "foo.php"

The search order of import is as follows:

Is the file in the same directory as the importing file? Yes, load.
Is there a php_modules directory? If so, is the file in there?
If the importing file is within the tree of the cwd (established by the
first file loaded), then recursively look for a php_modules directory until
at the cwd until the file is found (this is identical to the seek process
of node with it's analogous node_modules directory
As a final try, consider the PHP include_paths.

This will of course require a package manager similar to composer to become
part of core. However, composer will not be eclipsed as the import package
manager (phppm?) is only concerned with user modules. These modules must
explicitly export any symbols being fetched from them, whereas composer
will continue to load files using require.

Imports can also be done against directories

import foo from "mypackage"

In this case the parser will look for "mypackage/index.php"

All exports of a file can be brought in with a wildcard

import * from "file.php"

Should a wildcard be allowed without the from clause? That is import *.
To me this would mean "bring in the master symbol table" I'm not sure if
that's a good idea as it feels like a bad practice.

Also note, if foo.php doesn't export foo, the import will fail with an
exception. Which brings us to...

Export statement

PHP User Modules export code out using an export statement. If this didn't
happen there wouldn't be much point to them. Constants, classes and
functions can be exported. Unlike JavaScript, there is no default export
as there isn't an export object in the same sense as JavaScript.

export class Animal {}
export const pi = 3.141527
export function foo {}

As in Javascript exports can be sourced with a from clause. This is most
frequently seen in packages.

export MyClass from "./MyClass.php"
export * from "./methods.php"

The wildcard allowing for all exports of another file to be exported at a
common point, simplifying package interfaces.

Aliases are also possible. For example, say you want to use multibyte
string functions by default. You can do this in one file now

export mb_str_pad as strPad;
export mb_str_split as strSplit;

And so on, then import

import * from "myMbAliases.php"

If you got this far, thank you. This overall idea to take one of the better
things to happen to JavaScript in the last decade and incorporate it into
PHP has been bothering me for awhile so I figured I'd share. I don't know
how much merit there is to this though.

Note there's a lot more to JavaScript's implementation of import and export
that I only touched on here, but this letter has gone on long enough for a
surface level idea pitch.

Mod Note: It's been so long since I've sent any mail to the list that I'm
getting mail from an address I no longer have access to -
dmgx.michael@gmail.com. Is it possible to unsubscribe that email?

4 months ago by Rob Landers — view source

unread

Hello all. This is a ramble of an idea that's managed to run around my head for a few days now. It isn't fully formed, but I've ran the thought experiment as far as I can on my own and want to share it with all of you.

I've mostly been a lurker and I've seen a lot of RFC's come and go. Of those not accepted many have been passed over because of background compatibility. And then there is the issue that PHP has multiple design flaws that seem impossible to get rid of. Finally, I sense from conversations I've read that there are a lot of engine parser optimizations that haven't been tried because of the background compatibility problems present.

JavaScript was in this position as well 10 years ago when JavaScript modules were introduced with the ES6 syntax. Only recently have these modules finally begun to become first class members of node.js. The existing CommonJS require mechanism remains and will remain in Node for the foreseeable future, but the ES6 syntax allows an opportunity to sidestep the issue. The most significant of these is JavaScript modules run in strict mode, which actually removes features that are problematic for the engine or make it difficult to create optimized code.

Something similar could be done in PHP, and that's what the remainder of this letter is about, but before I continue I want to make clear my vantage point: I am but a humble user of the code, I'm no expert on the underpinnings of the Zend engine. In the text to follow I'm going to make wrong calls on some things - maybe multiple things. I'm not married to anything here. Further, even if I were a master of the engine and knew where to start, the scope of this is too large for any one person to undertake.

So all that said, I'll begin.

PHP User Modules are php files that are brought into the runtime through a new parser that is able to generate faster and more concise runtime code by removing support for problematic features and imposing a strict mode by default. They focus on PHP as a language and not as a template engine.

FYI, in non-strict mode, this produces a deprecation warning that can be caught and thrown from:

(fn(int $x) => print($x))(123.456); // deprecation warning

but this will work

(fn(int $x) => print($x))(123.000); // this is fine

Both of those are errors in strict types, so you might be tempted to do

fn(int $x) => print($x))((int)$some_var);

but $some_var might not actually be an integer-like (such as null or a string that becomes zero).

Some of us prefer the more strict, non-strict mode as the built-in strict mode is actually ... uhh, problematic, to say the least, in some business cases. So forcing strict mode is probably a non-starter.

The only background compatibility break is the introduction of three keywords: "import", "export" and "from"

The PHP interpreter does not load PHP files as modules unless it is directed to do so in an ini file or an .htaccess file using the default_filetype directive. If this directive is missing its value will be "default" - the value "module" will be used to trigger loading the initial PHP file as a module, and further types could in theory be introduced at a far later date.

Again, this setting only affects the INITIAL PHP script file loaded by the interpreter, such as the index.php of Drupal. Files that are included with include, include_once, require, or require_once will be imported as they always have. Files that are included with import are PHP User Modules.

One of the advantages of the current autoloading file-loading system is that files are not included until the are used. This allows you to run MASSIVE projects that realistically only need to load dozens of files. For example, I know of some code-bases that have literally millions of PHP files collected over the last 15 years and hundreds of developers working on them every day.

User Module Files
PHP User Modules have the following properties (Proposed, and very much subject to change):

They are code files. They have no <?php or ?> tags, and the inclusion of those tags is a parse exception. I know this will be problematic for PHP storm and other IDE's, but it's not an insurmountable problem.

So, will ?> work? I have turned on output buffering and then just wrote out what I needed --- ie, a template, then get the output.

If the removal of HEREDOC and NOWDOC syntax would simplify the parser, then these too would be removed from User Modules.

Are we sure we want multiple PHP parsers to maintain?

— Rob

4 months ago by Rob Landers — view source

unread

Hello all. This is a ramble of an idea that's managed to run around my head for a few days now. It isn't fully formed, but I've ran the thought experiment as far as I can on my own and want to share it with all of you.

I've mostly been a lurker and I've seen a lot of RFC's come and go. Of those not accepted many have been passed over because of background compatibility. And then there is the issue that PHP has multiple design flaws that seem impossible to get rid of. Finally, I sense from conversations I've read that there are a lot of engine parser optimizations that haven't been tried because of the background compatibility problems present.

JavaScript was in this position as well 10 years ago when JavaScript modules were introduced with the ES6 syntax. Only recently have these modules finally begun to become first class members of node.js. The existing CommonJS require mechanism remains and will remain in Node for the foreseeable future, but the ES6 syntax allows an opportunity to sidestep the issue. The most significant of these is JavaScript modules run in strict mode, which actually removes features that are problematic for the engine or make it difficult to create optimized code.

Something similar could be done in PHP, and that's what the remainder of this letter is about, but before I continue I want to make clear my vantage point: I am but a humble user of the code, I'm no expert on the underpinnings of the Zend engine. In the text to follow I'm going to make wrong calls on some things - maybe multiple things. I'm not married to anything here. Further, even if I were a master of the engine and knew where to start, the scope of this is too large for any one person to undertake.

So all that said, I'll begin.

PHP User Modules are php files that are brought into the runtime through a new parser that is able to generate faster and more concise runtime code by removing support for problematic features and imposing a strict mode by default. They focus on PHP as a language and not as a template engine.

FYI, in non-strict mode, this produces a deprecation warning that can be caught and thrown from:

(fn(int $x) => print($x))(123.456); // deprecation warning

but this will work

(fn(int $x) => print($x))(123.000); // this is fine

Both of those are errors in strict types, so you might be tempted to do

fn(int $x) => print($x))((int)$some_var);

but $some_var might not actually be an integer-like (such as null or a string that becomes zero).

Some of us prefer the more strict, non-strict mode as the built-in strict mode is actually ... uhh, problematic, to say the least, in some business cases. So forcing strict mode is probably a non-starter.

If you want to see what I mean:

non-strict: https://3v4l.org/kZ09l
strict: https://3v4l.org/5kVSG

— Rob

4 months ago by Claude Pache — view source

unread

Le 27 juin 2024 à 09:41, Rob Landers rob@bottled.codes a écrit :

PHP User Modules are php files that are brought into the runtime through a new parser that is able to generate faster and more concise runtime code by removing support for problematic features and imposing a strict mode by default. They focus on PHP as a language and not as a template engine.

FYI, in non-strict mode, this produces a deprecation warning that can be caught and thrown from:

(fn(int $x) => print($x))(123.456); // deprecation warning

but this will work

(fn(int $x) => print($x))(123.000); // this is fine

Both of those are errors in strict types, so you might be tempted to do

fn(int $x) => print($x))((int)$some_var);

but $some_var might not actually be an integer-like (such as null or a string that becomes zero).

Some of us prefer the more strict, non-strict mode as the built-in strict mode is actually ... uhh, problematic, to say the least, in some business cases. So forcing strict mode is probably a non-starter.

Hi,

There is no equivalent of “strict mode” in PHP. Do not confuse it with “strict_types”, which has nothing in common except the word “strict” in its name..

Strict mode in JS was introduced to disable design mistakes that couldn’t be removed due to very strong (almost inflexible) BC contraints. In PHP, we have deprecations followed by removals in next major version.

(And some people, including me, will argue: while JS strict mode disables design mistakes, PHP strict_types mode is a design mistake.)

—Claude

4 months ago by Deleu — view source

unread

Hi Michael,

Hello all. This is a ramble of an idea that's managed to run around my
head for a few days now. It isn't fully formed, but I've ran the thought
experiment as far as I can on my own and want to share it with all of you.

I've mostly been a lurker and I've seen a lot of RFC's come and go. Of
those not accepted many have been passed over because of background
compatibility. And then there is the issue that PHP has multiple design
flaws that seem impossible to get rid of. Finally, I sense from
conversations I've read that there are a lot of engine parser optimizations
that haven't been tried because of the background compatibility problems
present.

JavaScript was in this position as well 10 years ago when JavaScript
modules were introduced with the ES6 syntax. Only recently have these
modules finally begun to become first class members of node.js. The
existing CommonJS require mechanism remains and will remain in Node for the
foreseeable future, but the ES6 syntax allows an opportunity to sidestep
the issue. The most significant of these is JavaScript modules run in
strict mode, which actually removes features that are problematic for the
engine or make it difficult to create optimized code.

Working with Typescript a little bit does give a vibe that PHP could borrow
some of the concepts there and be improved greatly and I share this
sentiment.

Something similar could be done in PHP, and that's what the remainder of
this letter is about, but before I continue I want to make clear my vantage
point: I am but a humble user of the code, I'm no expert on the
underpinnings of the Zend engine. In the text to follow I'm going to make
wrong calls on some things - maybe multiple things. I'm not married to
anything here. Further, even if I were a master of the engine and knew
where to start, the scope of this is too large for any one person to
undertake.

Who would build it is an extremely key aspect of making changes to PHP.
Ideas are hard enough to survive the RFC process when there's already an
implementation. Finding a sponsor to work on this would be the first step.

So all that said, I'll begin.

PHP User Modules are php files that are brought into the runtime through a
new parser that is able to generate faster and more concise runtime code by
removing support for problematic features and imposing a strict mode by
default. They focus on PHP as a language and not as a template engine.

The only background compatibility break is the introduction of three
keywords: "import", "export" and "from"

The PHP interpreter does not load PHP files as modules unless it is
directed to do so in an ini file or an .htaccess file using the
default_filetype directive. If this directive is missing its value will be
"default" - the value "module" will be used to trigger loading the initial
PHP file as a module, and further types could in theory be introduced at a
far later date.

Again, this setting only affects the INITIAL PHP script file loaded by the
interpreter, such as the index.php of Drupal. Files that are included with
include, include_once, require, or require_once will be imported as they
always have. Files that are included with import are PHP User Modules.

Given that ini settings are frowned upon nowadays, I think having a <?php declare(modules=1); for the initial file might make the idea more likely
to pass a vote? Or maybe I'd even try to go one step further and say that
whatever file is being executed by SAPI (the first PHP file) could be
interpreted with a dumb lookahead. If the file has import / export syntax,
treat it like PHP Module, otherwise fallback.

The fun really starts when the from clause shows up.

import foo from "foo.php"
The search order of import is as follows:

Is the file in the same directory as the importing file? Yes, load.

Is there a php_modules directory? If so, is the file in there?

If the importing file is within the tree of the cwd (established by the
first file loaded), then recursively look for a php_modules directory until
at the cwd until the file is found (this is identical to the seek process
of node with it's analogous node_modules directory

As a final try, consider the PHP include_paths.

I'm not familiar enough with Javascript / Typescript ecosystem, but I've
only ever seen / used the ability to import using direct filepath. The fact
there's weird behaviors as result of trying to import a file and suddenly a
file all the way from include_paths or php_modules seems like a no-go
to me. I'd favor using only simple file path navigation and if the file
doesn't exist, error.

Perhaps if the idea gains merit, Composer could offer something similar to
Vite where we can create an alias to a specific folder and then import
things like from '@package/path/to/file.

This will of course require a package manager similar to composer to
become part of core. However, composer will not be eclipsed as the import
package manager (phppm?) is only concerned with user modules. These modules
must explicitly export any symbols being fetched from them, whereas
composer will continue to load files using require.

Imports can also be done against directories
import foo from "mypackage"
In this case the parser will look for "mypackage/index.php"

I'm not fond of this either.

Overall, I think PHP has already reached the limit of surviving with only
PSR-4 and Composer. Single class files were a great solution to get us out
of the nightmare of require and import on top of PHP files. But more
than once I have had the desire to declare a couple of interfaces in a
single file, or a handful of Enums, etc. It seems like PHP Modules could
also address the issue with function autoloading and package-level
visibility. I like the idea but I'm a bit skeptical until we have some
buy-in from someone that could actually get this implemented.

--
Marco Deleu

4 months ago by Jordan LeDoux — view source

unread

Who would build it is an extremely key aspect of making changes to PHP.
Ideas are hard enough to survive the RFC process when there's already an
implementation. Finding a sponsor to work on this would be the first step.

...

I like the idea but I'm a bit skeptical until we have some buy-in from

someone that could actually get this implemented.

--
Marco Deleu

Perhaps, though a conversation like this is helpful. Some rather
complicated RFCs do get approved/voted on before an implementation is done
when contributors who are familiar with the Zend engine get on board early.
Conversely, there are some extremely thoroughly implemented complicated
RFCs that get rejected because most voters don't participate in discussion
until voting is actually started. Something as broad as this probably
requires an off-list discussion with key active contributors, because
participation on list is so hit-and-miss.

Jordan

4 months ago by Mike Schinkel — view source

unread

This is a long reply rather than send a bunch of shorter emails.

Overall, I think PHP has already reached the limit of surviving with only PSR-4 and Composer. Single class files were a great solution to get us out of the nightmare of require and import on top of PHP files. But more than once I have had the desire to declare a couple of interfaces in a single file, or a handful of Enums, etc.

This.

I cannot overemphasize how nice it is to work in Go where I can put almost any code I want in any file I want without having to think about autoloading.

It is great when writing proofs of concept and having all the code in one place makes it easier to reason about. Once fleshed out you can then organize into multiple files, but still get to keep highly related code in the same files.

Thanks. The sticking point is what degree of change should be occurring. PHP isn't as behind an 8-ball as JavaScript is since the dev can choose their PHP version and hence deprecation works most of the time for getting rid of old stuff. But not always. Changes that are incompatible with what came before need a way to do things the old way during transition.

As I understand the proposal, this would have no BC issues for code not in modules. PHP could then set rules for code in modules that would not to be directly compatible with code outside modules.

That's how it works in JavaScript, at least as I have experienced, and I'd say it works pretty well.

Again, see PHP 6 and unicode, which snowballed until it was clear that even if PHP 6 had been completed it wouldn't be able to run most PHP 5 code.

At least to me this does not feel as big as trying to implement unicode.

No need for autoloaders with modules; I assume this would be obvious, right?

Depends largely on whether modules can include and require to get access to old code. I also didn't discuss how they behave - do they share their variables with includes and requires?

I was presuming that all old code would use autoloaders but modules would be free to do it a better way.

If you need to call code from a namespace from inside a module, sure, the autoloader would be needed.

Modules should be directories, not .php files. Having each file be a module makes code org really hard.

Yes, but that is how JavaScript currently handles things. It is currently necessary when making large packages to have an index.js that exports out the public members of the module. This entry point is configurable through the package.json of the module.

I am envisioning that there could be a module metadata file that would have everything that PHP needs to handle the module. It could even be binary, using protobufs:

https://github.com/protobuf-c/protobuf-c

The php CLI could have an option to generate this file making it easy for IDEs to generate the file, or generic file watchers to generate. This would mean that within a module there would be no need for an autoloader.

If the module metadata file does n0t exist, PHP could generate it on the fly. If the file is obviously out-of-date given a new file, PHP could re-generate. If PHP can't write the file, such as on a production server, it throws a warning and regenerates for in-memory use each page load.

It iss also possible that instead of protobuf the module file could actually be a phar file, or the equivalent of a phar file optimized to allow PHP to load, access and execute that code as fast as possible.

Modules would have a symbol table metadata file generated by IDEs and during deployment.

Node.js uses package.json and the attendant npm to do this sort of prep work. And it's a critical part of this since modules can be versioned, and different modules may need to run different specific versions of other modules.

node_modules IMO is one of the worse things about the JavaScript ecosystem. Who has not seen the meme about node_modules being worse than a black hole?

I would argue that PHP itself not be involved in trying to manage versions. Let Composer do that, or whatever other tool developers currently use to manage versions, or new tools developed later.

.php files in modules as identified by metadata file should not be loadable via HTTP(S).

Those are implementation details a little further down the road than we're ready for, I think.

But ensuring that it is possible to disallow loading needs to be contemplated in the design. PHP has to be able to know what is a module and what isn't without expensive processes.

Having exports separate from functions and classes seems like it would be problematic.

Again, this is how they work in JavaScript. Not saying that's the best approach, but even if problematic it's a solved problem.

I have evidently not written enough JavaScript to realize that.

I'm also interested in learning on how other module systems out there do work.

I am very familiar with modules (packages) in GoLang and think PHP could benefit from considering how they work, too.

Composer would need a massive rewrite to be a part of this since it currently requires the file once it determines it should do so. If we do a system where import causes the parser to act differently then that alone means imports can't be dealt with in the same manner as other autoloads.

That is why I am strongly recommending a modern symbol resolution system within modules vs. autoloading.

I'm not fond of this either.

There will need to be a way to define the entrypoint php. I think index.php is reasonable, and if another entry point is desired it can be called out -> "mypackage/myentry.php"

Why is an entry point needed? If there is a module metadata file as I am proposing PHP can get all the information it needs from that file. Maybe that is the .phm file?

Thanks. The sticking point is what degree of change should be occurring. PHP isn't as behind an 8-ball as JavaScript is since the dev can choose their PHP version and hence deprecation works most of the time for getting rid of old stuff. But not always. Changes that are incompatible with what came before need a way to do things the old way during transition. Again, see PHP 6 and unicode, which snowballed until it was clear that even if PHP 6 had been completed it wouldn't be able to run most PHP 5 code.

It’s not just up to the dev, but the libraries we use and whether or not we can easily upgrade (or remove) them to upgrade the php version.

By "upgrade" then, do you mean convert them into modules, or just be able to use them as-is.

As I read it and am envisioning it, there would be no changes needed to be able to use them as-is.

I think it would be a mistake to exclude old code and/or prevent templating. Not only are there now decades old code in some orgs, but how would you write an email sender that sent templated emails, provide html, generate code, etc? There has to be an output from the code to be useful.

Excluding old code or templates from modules would not exclude them from working as they currently do outside modules. As I see it, modules would be more about exporting classes and functions, not generating output per se.

So all that decades of old code could continue to exist outside modules, as it currently does today.

I think it’s fine to use js as an inspiration, but it isn’t the only one out there. There is some precedent to consider directories as modules (go calls them “packages”) and especially in PHP where namespaces (due to PSR-4 autoloading) typically match directory structures.

Totally agree about inspiration for modules outside JS, but not sure that PHP namespaces are the best place to look for inspiration.

Namespaces by their very nature were designed to enable autoloading with a one-to-one file to class or interface, and by nature add conceptual scope and complexity to a project that would not be required if a modern module/package system were added to PHP.

Modules could and IMO should be a rethink that learns the lessons other languages have learned over the past decade+.

Node.js uses package.json and the attendant npm to do this sort of prep work. And it's a critical part of this since modules can be versioned, and different modules may need to run different specific versions of other modules.

Please, please, please do not make a json file a configuration language. You can’t comment in them, you can’t handle “if php version <9, load this, or if this extension is installed, use this.”

Maybe that is desirable, but doing things slightly different based on extensions loaded is def a thing.

I don't think commenting is important in this file, or even desired.

As I proposed above, these could be protobuf or phar. These should be build artifacts that can be generated on the fly during development or for newbies even during deployment, not hand-managed.

I could see the generation of two files; one in binary form and one that is readonly so a developer can double-check what is in the current protobuf or phar file.

Those are implementation details a little further down the road than we're ready for, I think.

Personally, if these are going to have any special syntax, we probably shouldn’t call them .php files. Maybe .phm?

I was going to suggest that, and then remembered earlier PHP when there were multiple file extensions and that was a nightmare.

This does remind me to mention that I think there should be a required "module" declaration at the top of each file just like Go requires a "package" declaration at the top of each file. That would make it trivial for tooling to differentiate, even with grep.

the only thing I don’t like about this import/export thing is that it reminds me of the days when we had to carefully order our require_once directives to make sure files were loaded before they were used. So, I think it is worth thinking about how loading will work and whether loading can be dynamic, hoisted out of function calls (like js), how order matters, whether packages can enrich other packages (like doctrine packages) and if so, how much they can gain access to internal state, etc. This is very much not “a solved problem.”

That is why I proposed having a "compiled" module symbol table to eliminate most (all?) of those issues.

I do think PHP badly needs a native concept of "module" or "package" - in fact, I'm increasingly convinced it's the inevitable path we'll end up on at some point. BUT I think any such concept needs to be built on top of what we have right now. That means:

It should build on or work in harmony with namespaces, not ignore or replace them

It may be an unpopular opinion, but I would argue that namespaces were optimized for autoloading and the one class/interface per file paradigm, not to mention to regrettable choice of using the escape operator to seperate namespaces and that fact that PHP throws away a lot of information about namespaces at runtime.

IMO allowing modules to eventually deprecate namespaces — at least in a defacto form of deprecation — would allow modules to be much better than if the try to cling to a less desirable past.

It should be easy to take existing code, and convert it to a module/package

Maybe, but not if that means modules retain baggage that should really be jettisoned.

and namespaces have proved an extremely successful way of sharing code without those names colliding.

At the expense of a lot more complexity than necessary, yes.

Managing symbols in a module need not be a hard problem if PHP recognizes modules internally rather than trying to munge everything into a global namespace like with namespaces.

Other parts of your e-mail are essentially an unrelated idea, to have some new "PHP++" dialect, where a bunch of "bad" things are removed. You're not the first person to be tempted by this, but I think the history HHVM and Hack is educational here: initially, PHP and Hack were designed to interoperate on one run-time, but the more they tried to optimise for Hack, the harder it became to support PHP, and now Hack is a completely independent language.

While I agree that some things are unnecessary — such as unifying scope resolution operators for existing concepts — past failure does not guarantee future failure.

Hack tried to create an entirely new language yet still be PHP compatible. Learn from the Hack experience and rather than create an entirely new language, PHP modules could simply add constraints for code in modules, and then any "new" language features that are not module-specific by-nature should be considered to work everywhere in PHP, or not at all.

What problem would packages/modules/whatever be solving that isn't already adequately solved?

Not speaking for Michael, obviously, but speaking for what I envision:

Adding a module/package system to PHP with modern module features
- including module private, module function, and module properties
Providing an alternative to auto-loader-optimized namespaces.
- better code management and better page load performance

Do we want:

Packages and namespaces are synonymous? (This is roughly how JVM languages work, I believe.)

Packages and files are synonymous? (This is how Python and Javascript work.)

All packages correspond to a namespace, but not all namespaces are a package?

I would argue packages (modules) should be orthogonal to namespaces to allow modules to be optimized for what other languages have learned about packages/modules over the past decade+.

The fact that namespaces use the escape character as a separator, that PHP does not keep track of namespace after parsing is enough reason to move on from them, and that they were optimize for one-to-one symbol to file autoload are enough reasons IMO to envision a way to move on from them.

And given the near-universality of PSR-4 file structure, what impact would each of those have in practice?

Orthogonal. Old way vs new way. But still completely usable, just not as modules without conversion.

The fact PSR-4 exists is an artifact of autoloading single-symbol files and thus a sunken cost does not mean that PHP should not cling to for modules just because they currently exist.

-Mike

4 months ago by Rowan Tommins [IMSoP] — view source

unread

It may be an unpopular opinion, but I would argue that namespaces were optimized for autoloading and the one class/interface per file paradigm

I don't see any particular relationship between namespaces and autoloading, or any reason we need to throw them away to introduce different conventions for loading files.

My opinions match Larry's almost exactly: I want package-level optimisation, and package-private declarations. But I don't want to rewrite my entire codebase to start using a completely different naming system.

Not to mention that working with a combination of existing namespaced packages and "new shiny module" packages is going to be inevitable, so we can't just hand-wave that away.

Adding a module/package system to PHP with modern module features

I find that "modern" often just means "fashionable". Please, let's be specific. What is different between imports and namespaces, and why is it a good thing?

What specifically stops us doing all the things you've been discussing around loading, and visibility, etc, in a way that's compatible with the 400_000 packages available on Packagist, and billions of lines of existing code?

Rowan Tommins
[IMSoP]

4 months ago by Rowan Tommins [IMSoP] — view source

unread

I don't see any particular relationship between namespaces and autoloading, or any reason we need to throw them away to introduce different conventions for loading files.

Sure, you can make the argument they are not related, but then you have to ask if namespaces would look the way they do if it were not for the need to map them to be able to autoload symbols. I do not think they would.

Autoloading is by-nature one symbol per file. Namespaces were designed for mapping with "<namespace>/<className>.php" to allow autoloading.

Having worked in languages that do not require having to think about or run userland code to handle autoloading nor have to be concerned about loading in the proper order has been such a joy when compared to the pain of working PHP.

Namespaces don't require autoloading, and autoloading doesn't require one file per class.

To compile a program with multiple source files, in any language, you need one of two things:

a) A list of files you want to compile. Maybe auto-generated, maybe done with a recursive iteration over a directory, but ultimately the compiler needs a file path to process.
b) A way for the compiler to tell, based on some symbol it wants to resolve, which file should be compiled.

PHP originally provided only option (a), via the include and require keywords. Autoloading adds option (b), where you provide a function which takes a class name and does whatever you want to find the definition.

I think it might be time to re-visit the tooling around option (a), as OpCache makes the cost of eagerly loading a list of files much lower than it was when autoloading was added. That could be as simple as include_all($directory), or as fancy as include_from_manifest_file($some_crazy_binary_file_format); either could be implemented right now in userland, because it all eventually comes down to calling include or require.

My opinions match Larry's almost exactly: I want package-level optimisation, and package-private declarations. But I don't want to rewrite my entire codebase to start using a completely different naming system.

I can't see how package-privates would be of any value to you unless you rewrite your codebase.

Simple: I have private Composer packages, right now, that group all their classes under a particular namespace prefix. I want to be able to mark some classes in that namespace as "internal".

I do not want to change every place that uses the existing classes to reference "ModuleName@ClassName" instead of "NamespacePrefix\ClassName", or change every "use" to "import from".

And rewrite your entire codebase? Why? No need to rewrite if you don't need the specific features. Just because there is a new feature doesn't mean you have to use it if there is no benefit to you using it.

Code doesn't existing in isolation; if Symfony Mailer is re-published as a "module", every single application that uses it needs to change their code from referencing namespaced classes, to having "import" statements.

As for package-level optimisation, you'll need to give examples of what you mean there as I don't want to wrongly assume.

Currently, OpCache only optimises per file, because it can't guarantee how files will be used together.

A simple example is function fallback: if you could declare a package as "completely loaded", OpCache could replace all references to "strlen" with "\strlen", knowing that no namespaced function with that name could be added later.

Not to mention that working with a combination of existing namespaced packages and "new shiny module" packages is going to be inevitable, so we can't just hand-wave that away.

I do not follow your train of thought here.

What specifically are you accusing me of "hand-waving away?"

I didn't intend it as a personal accusation, apologies if it came across that way.

What I meant was: we can't just treat namespaces and modules as completely separate things, and assume that every code file will be using one style or the other. We have to imagine the user experience when there is a mix of the two.

I can't imagine it being pleasant to have a mix of "import" and "use" statements at the top of a file, with different and even conflicting semantics.

Adding a module/package system to PHP with modern module features

I find that "modern" often just means "fashionable". Please, let's be specific.
What is different between imports and namespaces, and why is it a good thing?

Namespaces are a parsing construct but not an AST construct beyond scoping. This has many ramifications which have often been mentioned as limitations on this mailing list.

Namespaces cannot provide code isolation and encapsulation, unlike more "fashionable" modules/packages. ;-)

Namespaces have no runtime behavior, but more "fashionable" modules/packages often do.

Perhaps I didn't word the question well. What I'm really asking, as someone who's never used JS or Go modules, is why I'd want to write "import", rather than referencing a global name in a hierarchy.

That's really all I mean by "making it compatible with namespace": I want "new \Foo\Bar\Baz;" to be able to refer to a "packaged" class, probably by having a way to mark all classes under "\Foo\Bar" as belonging to a particular package.

The usage of the escape character for namespace separator makes dynamic programming tedious and error prone.

Sorry, not interested.

Because of one-to-one symbol-to-file for autoloading, Namespaces by nature result in a large number of files in a large number of directories and do not allow code organization optimized for cohesiveness.

See above - this is not related to namespaces.

Modules and packages are typically small-scoped to a single directory and it is a code smell to have many different packages tightly coupled, as is the case with namespaces. Forcing modules to munge with namespaces would mean most modules would be written with those code smells for years because that will be how everyone doing a quick shift from namespace to module will write their modules.

Again, this is entirely about code style, and not something the language can control.

Also, the JS insistence on having a separate package for every tiny function is a common source of criticism, so personally I am very happy that PHP packages are generally larger than that.

In designing modules, if modules and namespaces were munged together then every single design decision made for modules will have to be compatible with namespaces. I cannot currently know what all constraints will emerge but I can almost guarantee that modules would be less well-designed if they have to be shoehorned to be fully compatible with namespaces.

That said, maybe the best solution is to NOT put the stake in the ground right now and say "They must be namespace compatible" or "They must not be namespace compatible" but move forward with an open mind so that we can tease out exactly how namespaces would constrain modules and and then make the decision later for what would be in the best interest of moving PHP into the future.

If and when an actual problem arises, let's discuss it.

What specifically stops us doing all the things you've been discussing around loading, and visibility, etc, in a way that's compatible with the
400_000 packages available on Packagist, and billions of lines of existing code?

You speak as if I am proposing getting rid of namespaces and making those 400_000 packages available on Packagist, and billions of lines of existing code not work in PHP. Of course not.

No, I'm saying that every one of those packages could benefit if we make incremental changes.

I don't want to couple it so that you can't have "package private" without also switching to some new "advanced" dialect of the language, and I don't see any reason why we need to do so.

Maybe package scoped declares could allow opting in to certain checks, but I don't think "is in a package" and "has been audited for a load of extra breaking changes" should be set by the same flag.

Leaving the rest of your reply here, since you accidentally sent it privately:

What I am saying is that we should design modules from a cleaner slate than namespaces, and allow solving problems that concerns for BC have always stopped PHP from solving.

Besides, when a language evolves and adds new features, it rarely works to shoehorn existing code AS-IS into the new constructs because doing so does not take advantage of the new capabilities. Just because you have a ton of code written for namespaces doesn't mean modules should be constrained to make it easy for you to move your namespaces to modules without a redesign.

But as I am proposing your namespaced code would continue to work exactly as before.

By their nature, beginners would not be as likely to use modules and would be likely to stick to existing PHP style. Intermediate to advanced programmers could instead be the target market for modules.

There has always been a divide in PHP between those who want a really advanced language and are happy to break compatibility to get there, and those who want PHP just the way it has always been. Modules could easily require typing for all things that can be typed, for example. Modules could be the thing that finally addresses the needs of intermediate to advanced developers while keeping everyone else who wants to keep PHP as more beginner friendly language happy.

--
Rowan Tommins
[IMSoP]

4 months ago by Michael Morris — view source

unread

Not replying to anyone in particular and instead doing a mild reset taking
into account the discussion that has gone before.

So, I want to import a package. I'll create an index.php file at the root
of my website and populate it with this.

<?php
import "./src/mymodule";

Now I'll create that directory and run a command php mod init in that
directory. Stealing this from Go, it's fairly straightforward though. Now
if we look in the directory we will see two files.

php.mod

php.sum

The second file I'll not be touching on but exists to track checksums of
downloaded packages - Composer does the same with its composer-lock.json
file which in turn was inspired by node's package-lock.json.

The php.mod file stands in for composer.json, but it isn't a json file. It
would start something like this:

namespace mymodule

php 10.0

registry packagist.org/packages

We start with three directives - the root namespace is presumed to be the
directory name. If that isn't true this is a text file, change it. PHP min
version should be straightforward. Registry details where we are going to
go get code from. Suppose we want to use our own registry but fallback to
packagist. That would be this:

namespace mymodule

php 10.0

registry (

github.com/myaccount

packagist.org/packages

)

Multiple registry entries will be checked for the code in order. Handling
auth tokens for restricted registries is outside of scope at the moment.

So let's build the module. We'll make a file called hello.phm. The reason
for phm and not php is so that web SAPIs will not try to parse this code.
Further they can be configured to not even allow direct https access to
these files at all.

import "twig/twig";

use \Twig\Loader\ArrayLoader;

use \Twig\Environment;

$loader = new ArrayLoader([

'index' => 'Hello {{ name }}'

]);

$twig = new Environment($loader);

export $twig;

As mentioned in previous discussions, modules have their own variable
scope. Back in our index we need to receive the variable

<?php

import $twig from "./src/mymodule"

$twig->render('index', ['name' => 'World']);

If we load index.php in the web browser we should see "Hello World". If
we look back in the mymodules folder we'll see the php.mod file has been
updated

namespace mymodule

php 10.0

registry packagist.org/packages

imports (

twig/twig v3.10.3

symfony/deprecation-contracts v2.5 //indirect

symfony/polyfill-mbstring v1.3 //indirect

symfony/polyfill-php80 v1.22 //indirect

)

Note the automatically entered comment that marks the imported dependencies
of twig. Meanwhile the php.sum file will also be updated with the checksums
of these packages.

So why this instead of composer? Well, a native implementation should be
faster, but also it might be able to deal with php extensions.

import "@php_mysqli"

The @ marks that the extension is either a .so or .dll library, as I'll
hazard a guess that the resolution mechanic will be radically different
from the php language modules themselves - if it is possible at all. If it
can be done it will make working with packages that require extensions a
hell of a lot easier since it will no longer be necessary to monkey the
php.ini file to include them. At a minimum the parser needs to know that
the import will not be in the registry and instead it should look to the
extensions directory, hence the lead @. Speaking of, having the extension
directory location be a directive of php.mod makes sense here. Each module
can have its own extension directory, but if this is kept within the
project instead of globally then web SAPIs definitely need to stay out of
those directories.

Final thing to touch on is how the module namespaces behave. The export
statement is used to call out what is leaving the module - everything else
is private to that module.

class A {} // private

export class B {} // public

All the files of the package effectively have the same starting namespace -
whatever was declared in php.mod. So it isn't necessary to repeat the
namespace on each file of the package. If a namespace is given, it will be
a sub-namespace

namespace tests;

export function foo() {}

Then in the importing file

import "./src/mymodule"

use \mymodule\tests\foo

Notice here that if there is no from clause everything in the module grafts
onto the symbol table. Subsequent file loads need only use the use
statement. Exported variables however must be explicitly pulled because the
variable symbol table isn't affected by namespaces (if I recall correctly,
call me an idiot if I'm wrong).

The from clause is useful for permanently aliasing - if something is
imported under an alias it will remain under that alias. Continuing the
prior example

import tests\foo as boo from "./src/mymodule";

boo()

That's enough to chew on I think.

4 months ago by Rob Landers — view source

unread

Not replying to anyone in particular and instead doing a mild reset taking into account the discussion that has gone before.

So, I want to import a package. I'll create an index.php file at the root of my website and populate it with this.

<?php
import "./src/mymodule";

Now I'll create that directory and run a command php mod init in that directory. Stealing this from Go, it's fairly straightforward though. Now if we look in the directory we will see two files.

php.mod
php.sum

The second file I'll not be touching on but exists to track checksums of downloaded packages - Composer does the same with its composer-lock.json file which in turn was inspired by node's package-lock.json.

I don't think that is correct... package-lock.json didn't come about until what, 2016-7ish? with pressure from yarn which did a yarn.lock file. Pretty sure composer was doing that since the beginning. I remember this being a BIG reason we switched from npm to yarn when it came out, because dev A would have different versions of libraries than dev B. Bug hunting was FUN when it was in a library.

The php.mod file stands in for composer.json, but it isn't a json file. It would start something like this:

namespace mymodule
php 10.0
registry packagist.org/packages

We start with three directives - the root namespace is presumed to be the directory name. If that isn't true this is a text file, change it. PHP min version should be straightforward. Registry details where we are going to go get code from. Suppose we want to use our own registry but fallback to packagist. That would be this:

namespace mymodule
php 10.0
registry (
github.com/myaccount
packagist.org/packages
)

Multiple registry entries will be checked for the code in order. Handling auth tokens for restricted registries is outside of scope at the moment.

While this looks good on paper, you're going to have to standardize how packages are accessed (API calls, etc) so they can be used in this file, or literally anyone who wants to add a competing registry will have to create an RFC to allow accessing their own registry, which is a ton of politics for something that is strictly technical -- not to mention a bunch of if-this-registry-do-that type statements scattered throughout the code, which makes it harder to maintain.

So let's build the module. We'll make a file called hello.phm. The reason for phm and not php is so that web SAPIs will not try to parse this code. Further they can be configured to not even allow direct https access to these files at all.

import "twig/twig";
use \Twig\Loader\ArrayLoader;
use \Twig\Environment;

$loader = new ArrayLoader([
'index' => 'Hello {{ name }}'
]);

$twig = new Environment($loader);

export $twig;

SAPIs are the programs that parse ALL php code and return it to the server (ie, nginx, apache, caddy, etc) to be displayed. The SAPI absolutely needs to parse these files in order to execute them. Servers are designed to display files, so any server configured today will just output the contents of these files because it won't be configured to send the request to the SAPI instead. It's better to suggest moving these files out of the web-root so it's a non-issue.

In other news, I'm not a fan of how many times I have to write "twig" just to get Twig in the current file. The module already registers a namespace, why can't the use-statement implicitly import the module?

As mentioned in previous discussions, modules have their own variable scope. Back in our index we need to receive the variable

<?php
import $twig from "./src/mymodule"

$twig->render('index', ['name' => 'World']);

If we load index.php in the web browser we should see "Hello World". If we look back in the mymodules folder we'll see the php.mod file has been updated

In real life, my code is going to be in a module/framework and I'm going to need to render it there. This example of exporting a dependency also kinda breaks encapsulation principles, and even though it is an example, things like this end up in documentation of a feature and cause all kinds of bad practices (like Symfony and anemic objects).

namespace mymodule
php 10.0
registry packagist.org/packages

imports (
twig/twig v3.10.3
symfony/deprecation-contracts v2.5 //indirect
symfony/polyfill-mbstring v1.3 //indirect
symfony/polyfill-php80 v1.22 //indirect
)

Note the automatically entered comment that marks the imported dependencies of twig. Meanwhile the php.sum file will also be updated with the checksums of these packages.

One of the first things I do in a composer.json file is remove polyfills through the replace key. It's unnecessary, annoys me in my IDE with having multiple classes of the same name, and hides the fact that I should probably install an extension for better performance. How do we do that with this new setup?

In fact, it is worth pointing out that how would this system work with polyfills in-general? Polyfills have their uses -- especially for library/framework code where you don't control the runtime environment. Like how would someone polyfill mb_string since people will be adding import @mbstring and not import symfony/polyfill-mbstring?

So why this instead of composer? Well, a native implementation should be faster, but also it might be able to deal with php extensions.

import "@php_mysqli"

The @ marks that the extension is either a .so or .dll library, as I'll hazard a guess that the resolution mechanic will be radically different from the php language modules themselves - if it is possible at all. If it can be done it will make working with packages that require extensions a hell of a lot easier since it will no longer be necessary to monkey the php.ini file to include them. At a minimum the parser needs to know that the import will not be in the registry and instead it should look to the extensions directory, hence the lead @. Speaking of, having the extension directory location be a directive of php.mod makes sense here. Each module can have its own extension directory, but if this is kept within the project instead of globally then web SAPIs definitely need to stay out of those directories.

So ... if we want to round, we have to use import @math and then we can call the global round() function? Or if we want to use DateTimeImmutable we have to add import @date? That seems like a step in the wrong direction since most people don't even know that most (if not all) global library functions come from extensions -- and virtually nobody knows the name of each extension and what functions they have. Also, installing extensions is not 100% straightforward as some environments need to use pecl, some need to use OS package managers.

Final thing to touch on is how the module namespaces behave. The export statement is used to call out what is leaving the module - everything else is private to that module.

class A {} // private
export class B {} // public

All the files of the package effectively have the same starting namespace - whatever was declared in php.mod. So it isn't necessary to repeat the namespace on each file of the package. If a namespace is given, it will be a sub-namespace

namespace tests;

export function foo() {}

Then in the importing file

import "./src/mymodule"
use \mymodule\tests\foo

Notice here that if there is no from clause everything in the module grafts onto the symbol table. Subsequent file loads need only use the use statement. Exported variables however must be explicitly pulled because the variable symbol table isn't affected by namespaces (if I recall correctly, call me an idiot if I'm wrong).

The from clause is useful for permanently aliasing - if something is imported under an alias it will remain under that alias. Continuing the prior example

import tests\foo as boo from "./src/mymodule";

boo()

That's enough to chew on I think.

— Rob

4 months ago by Mike Schinkel — view source

unread

Not replying to anyone in particular and instead doing a mild reset taking into account the discussion that has gone before.

So, I want to import a package. I'll create an index.php file at the root of my website and populate it with this.

<?php
import "./src/mymodule";

Now I'll create that directory and run a command php mod init in that directory. Stealing this from Go, it's fairly straightforward though. Now if we look in the directory we will see two files.

php.mod
php.sum

The second file I'll not be touching on but exists to track checksums of downloaded packages - Composer does the same with its composer-lock.json file which in turn was inspired by node's package-lock.json.

The php.mod file stands in for composer.json, but it isn't a json file. It would start something like this:

namespace mymodule
php 10.0
registry packagist.org/packages http://packagist.org/packages

We start with three directives - the root namespace is presumed to be the directory name. If that isn't true this is a text file, change it. PHP min version should be straightforward. Registry details where we are going to go get code from. Suppose we want to use our own registry but fallback to packagist. That would be this:

namespace mymodule
php 10.0
registry (
github.com/myaccount http://github.com/myaccount
packagist.org/packages http://packagist.org/packages
)

Multiple registry entries will be checked for the code in order. Handling auth tokens for restricted registries is outside of scope at the moment.

That is very Go-like, as you stated.

However, be aware that in a Go project repo you are likely to have only one go.mod — or multiple if you have numerous CLI apps being generated — whereas every directory with Go code is a package (which I think is equivalent to what you are calling "module."

So I think your use of them here is conflating the two concepts. One is a project-wide concept and the other is a "package" concept.

Maybe you would be better to adopt module to mean project and package to mean packaged code as Go has them?

From here on I will refer to directory rather than module or package to avoid confusion. By directory I will mean what Go calls a "package" and what I think your original proposal called a "module."

A big difference between Go and PHP is that Go have a compiler that compiles into an executable before it runs. That is clearly not compatible with PHP, and why I was proposing that each directory could have a pre-compiled .php.module that could be pre-compiled, or compiled on the fly at first import.

Also, it is problematic to have php.mod and php.sum because web servers would serve them if not carefully configured hence why I went with a leading dot, e.g. .php.module

So let's build the module. We'll make a file called hello.phm. The reason for phm and not php is so that web SAPIs will not try to parse this code. Further they can be configured to not even allow direct https access to these files at all.

import "twig/twig";
use \Twig\Loader\ArrayLoader;
use \Twig\Environment;

$loader = new ArrayLoader([
'index' => 'Hello {{ name }}'
]);

$twig = new Environment($loader);

export $twig;

As mentioned in previous discussions, modules have their own variable scope. Back in our index we need to receive the variable

<?php
import $twig from "./src/mymodule"

$twig->render('index', ['name' => 'World']);

Aside from being familiar per Javascript, what is the argument to requiring the import of specific symbols vs just a package import, e.g.:

<?php
import "./src/mymodule"

mymodule->twig->render('index', ['name' => 'World']);

To me is seems to just add to boilerplate required. Note that having mymodule everywhere you reference twig makes code a lot more self-documenting, especially on line 999 of a PHP file. 🙂

If we load index.php in the web browser we should see "Hello World". If we look back in the mymodules folder we'll see the php.mod file has been updated

namespace mymodule
php 10.0
registry packagist.org/packages http://packagist.org/packages

imports (
twig/twig v3.10.3
symfony/deprecation-contracts v2.5 //indirect
symfony/polyfill-mbstring v1.3 //indirect
symfony/polyfill-php80 v1.22 //indirect
)

Having a php.sum file is interesting but again, it should start with a period if so.

That said, I wonder if incorporating versioning does not make the scope of modules too big to complete?

Note the automatically entered comment that marks the imported dependencies of twig. Meanwhile the php.sum file will also be updated with the checksums of these packages.

So why this instead of composer? Well, a native implementation should be faster, but also it might be able to deal with php extensions.

import "@php_mysqli"

I would like this, but I think hosting vendors would block it since extensions can have C bugs and create vulnerabilities for servers.

I have long thought PHP should kick off a new type of extension using WASM, which can be sandboxed.

But I digress.

The @ marks that the extension is either a .so or .dll library, as I'll hazard a guess that the resolution mechanic will be radically different from the php language modules themselves - if it is possible at all. If it can be done it will make working with packages that require extensions a hell of a lot easier since it will no longer be necessary to monkey the php.ini file to include them. At a minimum the parser needs to know that the import will not be in the registry and instead it should look to the extensions directory, hence the lead @. Speaking of, having the extension directory location be a directive of php.mod makes sense here. Each module can have its own extension directory, but if this is kept within the project instead of globally then web SAPIs definitely need to stay out of those directories.

Final thing to touch on is how the module namespaces behave. The export statement is used to call out what is leaving the module - everything else is private to that module.

class A {} // private
export class B {} // public

All the files of the package effectively have the same starting namespace - whatever was declared in php.mod. So it isn't necessary to repeat the namespace on each file of the package. If a namespace is given, it will be a sub-namespace

namespace tests;

export function foo() {}

Then in the importing file

import "./src/mymodule"
use \mymodule\tests\foo

Notice here that if there is no from clause everything in the module grafts onto the symbol table. Subsequent file loads need only use the use statement. Exported variables however must be explicitly pulled because the variable symbol table isn't affected by namespaces (if I recall correctly, call me an idiot if I'm wrong).

The from clause is useful for permanently aliasing - if something is imported under an alias it will remain under that alias. Continuing the prior example

import tests\foo as boo from "./src/mymodule";

boo()

That's enough to chew on I think.

I don't think it is wise to intertwine this concept of modules with namespaces like that, but I am replied out for the night. :-)

-Mike

4 months ago by Rowan Tommins [IMSoP] — view source

unread

So why this instead of composer? Well, a native implementation should be faster, but also it might be able to deal with php extensions.

Building a package manager is hard, and getting a package manager adopted requires the network effect of a community using it. Jordi, Nils, et al have done a fantastic job with Composer, and it has a near 100% buy-in from the community, with hundreds of thousands of published packages.

It already supports requiring extensions; being able to install extensions is a much harder job, but that's being worked on now.

I would need an extremely persuasive argument to pay any attention at all to an incompatible alternative.

Rowan Tommins
[IMSoP]

4 months ago by Mike Schinkel — view source

unread

Namespaces don't require autoloading, and autoloading doesn't require one file per class.

No they do not, but the design of each was heavily intertwined with each other resulting in a less than optimal design, IMO.

So, are you arguing to keep one and eject the other for modules, and if so which are you arguing we eject? Autoloading?

Or are you arguing to keep both for modules, in which case your argument above is moot?

To compile a program with multiple source files, in any language, you need one of two things:

a) A list of files you want to compile. Maybe auto-generated, maybe done with a recursive iteration over a directory, but ultimately the compiler needs a file path to process.

Recursion is only needed if modules are hierarchical in nature.

b) A way for the compiler to tell, based on some symbol it wants to resolve, which file should be compiled.

That presumes the compiler did not simply generate an AST from the list of files.

PHP originally provided only option (a), via the include and require keywords. Autoloading adds option (b), where you provide a function which takes a class name and does whatever you want to find the definition.

And that "whatever you want" takes execution time (and tracing through when you are debugging.) But when you look at many other languages loading is an implementation detail that PHP chose to hoist onto userland developers when PHP could have established the rules to handle it more performantly without userland involvement.

Or is there some aspect of autoloading that could not be handled by PHP itself? Note I am asking only within the propose scope of modules, which we could constrain to optimize their runtime use.

I think it might be time to re-visit the tooling around option (a), as OpCache makes the cost of eagerly loading a list of files much lower than it was when autoloading was added.

Now you are getting somewhere.

Imagine that each module — which could equal a single directory — could have a pre-compiled op-cache which is essentially what I proposed in other recent emails.

That could be as simple as include_all($directory), or as fancy as include_from_manifest_file($some_crazy_binary_file_format); either could be implemented right now in userland, because it all eventually comes down to calling include or require.

meh.

That sounds like a way to avoid discussing the ways in which smartly designed modules could really improve PHP.

My opinions match Larry's almost exactly: I want package-level optimisation, and package-private declarations. But I don't want to rewrite my entire codebase to start using a completely different naming system.
I can't see how package-privates would be of any value to you unless you rewrite your codebase.
Simple: I have private Composer packages, right now, that group all their classes under a particular namespace prefix. I want to be able to mark some classes in that namespace as "internal".

Not simple, although I admit I am being pedantic about words used here, but for a reason.

I asked about "package-privates," you responded with "namespace-privates."

Adding private to namespaces is orthogonal to the discussion of packages.

To require that packages be constrained to have all the same warts as namespaces and existing PHP code simply so you can have namespace-privates is short-sighted (and IMO a bit selfish.)

Alternately, namespaces could get private scope in parallel to having modules be considered.

That would allow modules to gain improvements that we could not get by having to maintain BC with namespaces.

Which causes me to ask: If you have really wanted namespace private why has it been six years since it was even last mentioned on the list, and four years since last discussed?

https://externals.io/message/101323 https://externals.io/message/101323

Why has there not been an RFC since this one https://wiki.php.net/rfc/namespace-visibility https://wiki.php.net/rfc/namespace-visibility six years ago, that was not even voted on?

Why is it that when the topic of addressing modules/packages comes up — which has been talked about numerous times in the past six years — do you now bring up namespace privates in a manner that would effectively torpedo goals of the modules discussion, at least from the perspective of the OP and myself?

If namespace private were really something important to you, why haven't you championed it before, rather than hijack a discussion about the benefits we could get from modules if not constrained by namespaces?

I do not want to change every place that uses the existing classes to reference "ModuleName@ClassName" instead of "NamespacePrefix\ClassName", or change every "use" to "import from".

Then don't. Champion this RFC https://wiki.php.net/rfc/namespace-visibility https://wiki.php.net/rfc/namespace-visibility and get what you want.

But please don't argue against a discussion on modules because you want a feature that can be gotten orthogonally. (If you must argue against it, make arguments for which accommodations for your preferences cannot be found.)

Code doesn't existing in isolation; if Symfony Mailer is re-published as a "module", every single application that uses it needs to change their code from referencing namespaced classes, to having "import" statements.

And that is bad, how?

But before you answer, it just means that instead of a use statement in your existing code you change to a import statement.

You'd then of course need to changes — if applicable — to call the new Symphony Mailer, but you'd have to do that with or without modules.

Or is there something else I am missing?

As for package-level optimisation, you'll need to give examples of what you mean there as I don't want to wrongly assume.

Currently, OpCache only optimises per file, because it can't guarantee how files will be used together.

A simple example is function fallback: if you could declare a package as "completely loaded", OpCache could replace all references to "strlen" with "\strlen", knowing that no namespaced function with that name could be added later.

Thank you for elaborating on that.

So, champion an RFC to improve OpCache for namespaces. That need not impose on the discussion about modules.

Further, and this is what is nice about being able to discuss modules not having to be compatible with namespaces, if there are aspect of namespaces that make optimization hard or impossible then we could potentially set up rules of modules that make similar optimizations easy and/or possible.

EVEN further, consider the fact that in PHP all class members are public by default. One thing we could have in modules is to go back to short var and eliminate both private and protected modifiers and only have public with the default behavior being what is private outside of modules. protected would no longer be needed as we would have module scope which is defacto-protected.

Classes could be final by default in modules and then we could modify them with an open keyword (thanks to Lynn for that one.)

And so on. In other words, if we could treat modules as their own sandbox, we could get fix many of the regrettable former design choices of the PHP language — some of which are to make PHP be beginner friendly — and potentially re-energize people who once looked at PHP and dismissed it to give it another look.

What I meant was: we can't just treat namespaces and modules as completely separate things, and assume that every code file will be using one style or the other. We have to imagine the user experience when there is a mix of the two.

Why can we not just treat namespaces and modules as completely separate things?

I can't imagine it being pleasant to have a mix of "import" and "use" statements at the top of a file, with different and even conflicting semantics.

That feels like a frivolous concern when compared to the benefits we could see with modules, especially when there would be ways to mitigate your stated concerns here.

If you don't like to see imports, but your imports in a namespace and then "use" that namespace.

Or we could allow "use module" instead of (or in addition to) "import" and then it could look more pleasant for you.

As for conflicting semantics:

1.) I'm not seeing how those could be significant in the using/importing file, and
2.) Isn't dealing with conflicting semantics just a part of programming?
3.) Don't "use" and "use function" have conflicting semantics?

God knows that "use" by itself has many confusing semantics, which "import" could avoid.

Perhaps I didn't word the question well. What I'm really asking, as someone who's never used JS or Go modules, is why I'd want to write "import", rather than referencing a global name in a hierarchy.

"use module" would work just as well as "import"; the "import" is not special, the module scoping and features are what is valuable here.

For specifics see my other recent emails on the subject. If they do not explain, please ask again with specifics.

That's really all I mean by "making it compatible with namespace": I want "new \Foo\Bar\Baz;" to be able to refer to a "packaged" class, probably by having a way to mark all classes under "\Foo\Bar" as belonging to a particular package.

And that is what I am trying to get away from.

First the backslash — because when using in reflection or other dynamic programming they have to be escaped which can lead to escaping errors. I know you don't care, but I and others do.

Second, the hierarchy. Because there is no constraint on hierarchy PHP subtly encourages developers — as if sirens of the Odyssey — to create large hierarchies. I even find myself doing it as I fighting myself against it.

The reasons hierarchy is bad is:

1.) larger hierarchies grow conceptual complexity,
2.) they place no limit on package growth as you can always create subdirectories,
3.) they make it harder to "see" all the code files in one place (a single directory),
4.) they constrain where code is located when there are benefits to a different layout

That's really all I mean by "making it compatible with namespace": I want "new \Foo\Bar\Baz;" to be able to refer to a "packaged" class, probably by having a way to mark all classes under "\Foo\Bar" as belonging to a particular package.

Revisiting this, why is it important to you that "new \Foo\Bar\Baz" refer to a "packaged" class vs a namespaced class, assuming you had namespace-private and OpCache improvements?

Why can't you still just use the namespaces you prefer and let "packages" (modules) improve in other ways?

I am trying my best not to make this ad-hominem so forgive me but I do have to ask if this is just not a case of "I am comfortable doing it the way I have been doing it and do not want to consider changing," maybe? Note I am asking that question limited to the one statement I quoted above, not on the broader discussion.

Modules and packages are typically small-scoped to a single directory and it is a code smell to have many different packages tightly coupled, as is the case with namespaces. Forcing modules to munge with namespaces would mean most modules would be written with those code smells for years because that will be how everyone doing a quick shift from namespace to module will write their modules.

Again, this is entirely about code style, and not something the language can control.

A language cannot control it, but a language can encourage or discourse it.

And the PHP language encourages a large amount of file and directory bloat.

One only need to compare the number of files in most PHP libraries to the number of files in JS or Go package to see that the nature of a language clearly does not influence.

To bring stats vs. opinion I asked ChatGPT what the two equivalent packages are to Symphony for JS and Go respectively and it suggested ExpressJS and Gin. So I cloned them to see the number of files and directories each has. From the root of each repo:

Project			Files 	Dirs

Symfony: 12,504 2,162
ExpressJS: 259 87
Gin(GoLang): 145 30

The comparison might not be completely fair given how much longer Symfony has been around, but they all target the same use-case so even if there is less functionality in ExpressJS or Gin.

Given that I think that well over an order of magnitude more files is a really odiferous code smell, and is thanks to the language which admittedly cannot "control" layout, but definitely influences it.

Am I wrong? Present any other relatively equivalent project comparisons you please. Here are the bash commands to count files and dirs:

find /path/to/subdirectory -type f | wc -l
find /path/to/subdirectory -type d | wc -l

Also, the JS insistence on having a separate package for every tiny function is a common source of criticism, so personally I am very happy that PHP packages are generally larger than that.

I can't speak for the OP, but nothing I am proposing is advocating for separate packages for every tiny functions. Nothing.

Instead I am advocating for packages that are mostly in a few directories instead of almost two magnitudes more!

That said, maybe the best solution is to NOT put the stake in the ground right now and say "They must be namespace compatible" or "They must not be namespace compatible" but move forward with an open mind so that we can tease out exactly how namespaces would constrain modules and and then make the decision later for what would be in the best interest of moving PHP into the future.

If and when an actual problem arises, let's discuss it.

Not "problems" but instead "opportunities."

I have already pointed out numerous opportunities in this email and one of my recent emails.

What specifically stops us doing all the things you've been discussing around loading, and visibility, etc, in a way that's compatible with the
400_000 packages available on Packagist, and billions of lines of existing code?

You speak as if I am proposing getting rid of namespaces and making those 400_000 packages available on Packagist, and billions of lines of existing code not work in PHP. Of course not.

No, I'm saying that every one of those packages could benefit if we make incremental changes.

Maybe.

What benefits can you envision you would get if PHP made namespaces==modules compared with the benefits I have mentioned for making modules not be constrained to compatibility with namespaces (besides private and OpCache as we already discussed you pursue for namespaces?)

Can we get precompiling for modules in a directory and written to a .php.module file? We can't do that with namespaces because scanning recursively could take too long at runtime.

Can we get default private for all symbols and class members in namespaces? No, that would be a huge BC break.

Can we get namespaces to be first-class AST participants? If yes, why have we not done it before?

I could go on, but this email is getting loooong.

I don't want to couple it so that you can't have "package private" without also switching to some new "advanced" dialect of the language, and I don't see any reason why we need to do so.

And I am not advocating that. I am advocating you should get "namespace private." Hey RFC is already written! https://wiki.php.net/rfc/namespace-visibility https://wiki.php.net/rfc/namespace-visibility

And most of the other benefits of modules as I am proposing would be BC breaks so you could not get them in namespaces anyway.

Unless you can come up with something besides private and opCache I had not considered.

Maybe package scoped declares could allow opting in to certain checks, but I don't think "is in a package" and "has been audited for a load of extra breaking changes" should be set by the same flag.

I am not aware of any discussion of opting in, flags, nor auditing with respect to modules.

-Mike

4 months ago by Michael Morris — view source

unread

On Jun 28, 2024, at 10:12 AM, Rowan Tommins [IMSoP] imsop.php@rwec.co.uk
wrote:

Also, the JS insistence on having a separate package for every tiny
function is a common source of criticism, so personally I am very happy
that PHP packages are generally larger than that.

I can't speak for the OP, but nothing I am proposing is advocating for
separate packages for every tiny functions. Nothing.

Nothing in JavaScript encourages micro-packages, but nothing prevents it
either. It's not something anyone is advocating.

4 months ago by Jim Winstead — view source

unread

PHP User Modules are php files that are brought into the runtime through a new parser that is able to generate faster and more concise runtime code by removing support for problematic features and imposing a strict mode by default. They focus on PHP as a language and not as a template engine.

I think the problem I have with this proposal is calling these "PHP User Modules". Here's an admittedly uncharitable rephrase of this:

"NewLanguage User Modules are NewLanguage files that are brought into the PHP runtime through a new parser that may theoretically be able to generate faster and more concise runtime code by implementing a different language based on much of the syntax from PHP. This new language does not prioritize its use as a template language for HTML."

The only background compatibility break is the introduction of three keywords: "import", "export" and "from"

"We will add three new keywords to PHP to support accessing variables, classes, and functions implemented in NewLanguage."

If you got this far, thank you. This overall idea to take one of the better things to happen to JavaScript in the last decade and incorporate it into PHP has been bothering me for awhile so I figured I'd share. I don't know how much merit there is to this though.

I think there is a lot of ground to be covered in improving PHP's concept of packages or modules that has largely been punted to user-space with autoloading and Composer and it's always good to address the seams that has left, but I feel like this proposal, as sketched out so far, reminds me more of what became Raku (from the Perl world) or perhaps Hack than an actual way forward for PHP itself.

Thanks.

Jim

4 months ago by Michael Morris — view source

unread

PHP User Modules are php files that are brought into the runtime through a
new parser that is able to generate faster and more concise runtime code by
removing support for problematic features and imposing a strict mode by
default. They focus on PHP as a language and not as a template engine.

I think the problem I have with this proposal is calling these "PHP User
Modules". Here's an admittedly uncharitable rephrase of this:

"NewLanguage User Modules are NewLanguage..

If you know you're being insulting why do it? It's completely unhelpful.

4 months ago by Jordan LeDoux — view source

unread

On Thu, Jun 27, 2024 at 12:53 PM Jim Winstead jimw@trainedmonkey.com
wrote:

PHP User Modules are php files that are brought into the runtime through a
new parser that is able to generate faster and more concise runtime code by
removing support for problematic features and imposing a strict mode by
default. They focus on PHP as a language and not as a template engine.

I think the problem I have with this proposal is calling these "PHP User
Modules". Here's an admittedly uncharitable rephrase of this:

"NewLanguage User Modules are NewLanguage files that are brought into the
PHP runtime through a new parser that may theoretically be able to generate
faster and more concise runtime code by implementing a different language
based on much of the syntax from PHP. This new language does not prioritize
its use as a template language for HTML."

Do you feel that Phar is a separate language? Is PHP no longer PHP if the @
error suppression is removed? I'm really unclear about the point you are
making here, even if I ignore the "uncharitable" rephrase.

Jordan

4 months ago by Jim Winstead — view source

unread

__

PHP User Modules are php files that are brought into the runtime through a new parser that is able to generate faster and more concise runtime code by removing support for problematic features and imposing a strict mode by default. They focus on PHP as a language and not as a template engine.

I think the problem I have with this proposal is calling these "PHP User Modules". Here's an admittedly uncharitable rephrase of this:

"NewLanguage User Modules are NewLanguage files that are brought into the PHP runtime through a new parser that may theoretically be able to generate faster and more concise runtime code by implementing a different language based on much of the syntax from PHP. This new language does not prioritize its use as a template language for HTML."

Do you feel that Phar is a separate language? Is PHP no longer PHP if the @ error suppression is removed? I'm really unclear about the point you are making here, even if I ignore the "uncharitable" rephrase.

If I read through the 11 bullet points under "User Module Files" in the original proposal, I see two that are actually related to modules and most of them are just lopping off features from the PHP language in ways both small (no need for <?php) and huge (changing the scoping operator to '.' instead of '::', '->', and '').

The angle I am coming at this from is improving the developer experience around "packages" or "modules" or whatever you want to call them, and so much of this proposal doesn't seem to be about that.

I could have made that point in other ways, and I'm sorry that my first attempt came off as insulting. It really concerned me when I already saw discussion about taking this off-list and going into the weeds on technical details when the problem that is being addressed by this proposal is extremely unclear to me.

Jim

4 months ago by Jordan LeDoux — view source

unread

On Thu, Jun 27, 2024 at 12:53 PM Jim Winstead jimw@trainedmonkey.com
wrote:

PHP User Modules are php files that are brought into the runtime through a
new parser that is able to generate faster and more concise runtime code by
removing support for problematic features and imposing a strict mode by
default. They focus on PHP as a language and not as a template engine.

I think the problem I have with this proposal is calling these "PHP User
Modules". Here's an admittedly uncharitable rephrase of this:

"NewLanguage User Modules are NewLanguage files that are brought into the
PHP runtime through a new parser that may theoretically be able to generate
faster and more concise runtime code by implementing a different language
based on much of the syntax from PHP. This new language does not prioritize
its use as a template language for HTML."

Do you feel that Phar is a separate language? Is PHP no longer PHP if the
@ error suppression is removed? I'm really unclear about the point you are
making here, even if I ignore the "uncharitable" rephrase.

If I read through the 11 bullet points under "User Module Files" in the
original proposal, I see two that are actually related to modules and most
of them are just lopping off features from the PHP language in ways both
small (no need for <?php) and huge (changing the scoping operator to '.'
instead of '::', '->', and '').

The angle I am coming at this from is improving the developer experience
around "packages" or "modules" or whatever you want to call them, and so
much of this proposal doesn't seem to be about that.

I could have made that point in other ways, and I'm sorry that my first
attempt came off as insulting. It really concerned me when I already saw
discussion about taking this off-list and going into the weeds on technical
details when the problem that is being addressed by this proposal is
extremely unclear to me.

Jim

Ah, yes, THAT'S a fair point. While the idea of optimizing the
engine/parser for modules has merit as part of a user modules proposal, I
agree that many of the specifics proposed here feel pretty scatter-shot and
unclear.

The scoping operator change I simply ignored, as that feels to me like just
asking "I would like to program in Node" and there's no clear benefit to
changing the scoping operator outlined, while there is a clear detriment to
eliminating the concatenation operator entirely.

Mostly I ignored that aspect of it, because I assumed that all the people
capable of implementing this proposal would just refuse stuff like that
outright, and that the inclusion of it would guarantee the RFC fails, so no
point in worrying.

But the broader question you are presenting about the focus and goals of
the proposal, and how the specifics relate to that, is actually a question
that I share.

Jordan

4 months ago by Michael Morris — view source

unread

This is a very long reply to several emails.

The angle I am coming at this from is improving the developer experience
around "packages" or "modules" or whatever you want to call them, and so
much of this proposal doesn't seem to be about that.

Ok, first problem - not a proposal really, but a ramble trying to get to a
proposal. Before I made the first post the idea was knocking around in my
head and wouldn't go away, so I just stream of consciousness listed what's
going through my head. That leads to the second point you made.

I could have made that point in other ways, and I'm sorry that my first
attempt came off as insulting. It really concerned me when I already saw
discussion about taking this off-list and going into the weeds on technical
details when the problem that is being addressed by this proposal is
extremely unclear to me.

It is unclear even to me. Perhaps I shouldn't have posted out something
this half baked. That said, pruning off large sections of language
functionality is a distraction. For now let's just note that it is a
possibility to improve the language this way afforded by the fact that
import would be new way of bringing scripts in. Could isn't should. Also,
at the moment again it's a distraction. Let's focus down on how code is
imported.

First though, a history review, partially to get this straight in my own
head but hopefully of use for those following along. Why? Knowing how we
got we are is important to some degree to chart a way forward.

PHP started as a template engine. By modern standards, and compared to the
likes of twig, it's a very bad template engine, but that doesn't really
matter because it's evolved into a programming language in it's own right
over the last nearly 20 years.

Include, include_once, require, and require_once have been around since the
beginning as the way to splice code files together. The behavior of these
statements calls back to PHP's origin as a template engine as they do
things similar mechanisms like JavaScript's import do not do (and for that
matter, their equivalents in C# and Java). Their scope behavior is very
different from import mechanisms in other languages, as they see the
variables in the scope of the function they were invoked from or the global
scope when called from there. Their parsing can be aborted early with a
return. They can return a value, which is quite unusual to be honest. None
of this is bad per se, but it is different and the question arises is it
necessary.

One artifact of their behavior that is bad in my opinion is that they start
from the standpoint of being text or html files. If the included file has
no PHP tags then the contents get echoed out. If there are no output
buffers running this can cause headers to be set and fun errors to be had.
So they can't be used to create files that can only echo explicitly (that
is, a call to the echo statement or the like).

Fast forward a bit - PHP 5.3, and the introduction of namespaces were
introduced to deal with the overloaded symbol tables. They are a bit a
hotwire as (if I'm not mistaken, it's been a couple years since I read the
discussion on it) they just quietly prepend the namespace string in front
of the name of all new symbols declared in the namespace for use elsewhere.
As a result, PHP namespaces don't do some of the things we see in the
namespaces of other languages (looking at Java and C# here). For example,
privacy modifiers within a namespace aren't a thing.

Very quickly after PHP 5.3 released autoloaders showed up. At some point
support for multiple autoloaders was added. Several schema were added,
PSR-4 won out, and composer showed up to leverage this. Composer is based
on NPM, even to the point where json is used to configure it, and the
composer.json file is fairly close to npm's package.json file even now.
It's a userland solution, but to my knowledge WordPress is the only widely
used PHP application out there that doesn't use it directly (there is a
Composer Wordpress project).

Before composer, and before namespaces there was PECL. Composer has
eclipsed it because PECL has the limitation of being server-wide. It never
really caught on in the age of virtual hosting with multiple PHP sites
running on one box. Today we have Docker, but that didn't help PECL make a
comeback because by the time docker deployment of PHP sites became the norm
composer had won out. Also, composer library publishing is more permissive
than PECL. I'll stop here lest this digress into a Composer v PECL
discussion - suffice to say stabs a bringing code packages into PHP isn't a
new idea, and a survey of what's been done before, what was right about
those attempts and what was wrong needs to be considered before adding yet
another php package system into the mix.

The main influence of composer and autoloaders for preparing packages is
that PHP has become far more Object Oriented than it was before. Prior to
PHP 5.3 object oriented programming was a great option, but since
autoloaders cannot bring in functions (at least not directly, they can be
cheated in by bundling them in static classes which are all but namespaces)
the whole ecosystem has become heavily object oriented.

That isn't a bad thing. But it does need to be acknowledged. Before I go
further I'll now respond to some other points made by others in this
thread.

On Thu, Jun 27, 2024 at 6:01 PM Jordan LeDoux jordan.ledoux@gmail.com
wrote:

Ah, yes, THAT'S a fair point. While the idea of optimizing the
engine/parser for modules has merit as part of a user modules proposal, I
agree that many of the specifics proposed here feel pretty scatter-shot and
unclear.

The scoping operator change I simply ignored, as that feels to me like
just asking "I would like to program in Node" and there's no clear benefit
to changing the scoping operator outlined, while there is a clear detriment
to eliminating the concatenation operator entirely.

Mostly I ignored that aspect of it, because I assumed that all the people
capable of implementing this proposal would just refuse stuff like that
outright, and that the inclusion of it would guarantee the RFC fails, so no
point in worrying.

But the broader question you are presenting about the focus and goals of
the proposal, and how the specifics relate to that, is actually a question
that I share.

I hope the above begins to address that. Package management I think should
be the main topic, and from here forward I'll leave aside any unnecessary
parser changes that might occur when code is imported as there are
distractions. Those I continue to bring up I'll state why, and those who
are more familiar with how the engine works can speak to whether such
changes truly are useful or unecessary. If I'm wrong, then dropping such
suggestions entirely is the way to go.

Internals has made it pretty clear: no more declare or ini entries (unless
it is absolutely needed).

Noted.

I personally don’t like it because it uses arrays, which are opaque, easy
to typo, and hard to document/check.

Instead, maybe consider a new Reflection API?

(new ReflectionModule)->import('MyModule')->run()

That doesn't solve the problem of how the parser figures out where the code
is. That's got to happen somewhere. I'll come back to this in a moment.

Keep in mind that extensions typically expose functions automatically, and
under the original proposal those functions have to be imported to be used:
import mysql_query

they also do now, unless you either prefix them with \ or rely on the
fallback resolution system. I’m honestly not sure we need a new syntax for
this, but maybe just disable the global fallback system in modules?

I'm not sure that's a good idea, neither was this.

Perhaps PHP imports, unlike their JavaScript or even Java C# counterparts,
could be placed in try/catch blocks, with the catch resolving what to do if
the import misses.

Which is something I wrote, yet a day later - yuck. I do not like. But I'm
in brainstorm mode, playing with ideas with everyone.

I really don't like the extension games seen in node with js, cjs and mjs,
but there's a precedent for doing it that way. In their setup if you've
set modules as the default parse method then cjs can be used to identify
files that still need to use CommonJS. And mjs can force the ES6 even in
default mode. But it is a bit of a pain and feels like it should be
avoided.

I would argue that it be something seriously considered. Scanning a
directory in the terminal, in production systems, while diagnosing ongoing
production issues, it can be very handy to distinguish between the “old
way” and “new way”, at a glance.

Fair point.

the only thing I don’t like about this import/export thing is that it
reminds me of the days when we had to carefully order our require_once
directives to make sure files were loaded before they were used. So, I
think it is worth thinking about how loading will work and whether loading
can be dynamic, hoisted out of function calls (like js), how order matters,
whether packages can enrich other packages (like doctrine packages) and if
so, how much they can gain access to internal state, etc. This is very much
not “a solved problem.”

In JavaScript import must be top of the file - you'll get an error if you
try an import following any other statement unless it's a dynamic import(),
which is a whole other Promise/Async/Kettle of fish that thankfully PHP
does not have to take into account as, until you get used to it (and even
after), async code is a headache.

Are you sure? I don’t remember them removing import hoisting, but it’s
probably more of a typical linting rule because it is hard to reason about.

Likely correct - I do use linters heavily. Hoisting is evil (necessary,
but still evil).

On Thu, Jun 27, 2024 at 6:13 PM Rowan Tommins [IMSoP] imsop.php@rwec.co.uk
wrote:

Thank you for sharing. I think it's valuable to explore radical ideas
sometimes.

I do think PHP badly needs a native concept of "module" or "package" -
in fact, I'm increasingly convinced it's the inevitable path we'll end
up on at some point. BUT I think any such concept needs to be built on
top of what we have right now. That means:

It should build on or work in harmony with namespaces, not ignore or
replace them

It should be compatible with Composer, but not dependent on it

It should be easy to take existing code, and convert it to a
module/package

It should be easy to carry on using that module/package after it's
been converted

On all these points, agreed.

If we can learn from other languages while we do that, I'm all for it;
but we have to remember that those languages had a completely different
set of constraints to work with.

For instance, JS has no concept of "namespaces", but does treat function
names as dynamically scoped alongside variables. So the module system
needed to give a way of managing how you imported names from one scope
to another. That's not something PHP needs, because it treats all names
as global, and namespaces have proved an extremely successful way of
sharing code without those names colliding.

Very good point.

Other parts of your e-mail are essentially an unrelated idea, to have
some new "PHP++" dialect, where a bunch of "bad" things are removed.

Let's set that aside then. Better package management is a big enough
dragon to slay.

This is a long reply rather than send a bunch of shorter emails.

Overall, I think PHP has already reached the limit of surviving with
only PSR-4 and Composer. Single class files were a great solution to get us
out of the nightmare of require and import on top of PHP files. But
more than once I have had the desire to declare a couple of interfaces in a
single file, or a handful of Enums, etc.

This.

I cannot overemphasize how nice it is to work in Go where I can put almost
any code I want in any file I want without having to think about
autoloading.

Go is cool. I need to use it more. These days JavaScript gets most of my
time, but PHP will always be the language that got me into
programming professionally and for that I'll be eternally grateful.

As I understand the proposal, this would have no BC issues for code not in
modules. PHP could then set rules for code in modules that would not to be
directly compatible with code outside modules.

That is the goal. Module code should be allowed to be different if the
optimization makes for faster running and easier to understand code (for
the programmer, the IDE, and the parser itself). Changing things for the
sake of changing them, no.

At least to me this does not feel as big as trying to implement unicode.

I would hope not, because that turned out to be well night impossible.

No need for autoloaders with modules; I assume this would be
obvious, right?

Depends largely on whether modules can include and require to get access
to old code. I also didn't discuss how they behave - do they share their
variables with includes and requires?

I was presuming that all old code would use autoloaders but modules would
be free to do it a better way.

If you need to call code from a namespace from inside a module, sure, the
autoloader would be needed.

This is correct and what I had in mind.

Modules should be directories, not .php files. Having each file be a
module makes code org really hard.

Yes, but that is how JavaScript currently handles things. It is
currently necessary when making large packages to have an index.js that
exports out the public members of the module. This entry point is
configurable through the package.json of the module.

I am envisioning that there could be a module metadata file that would
have everything that PHP needs to handle the module. It could even be
binary, using protobufs:

An interesting idea. I need to research this some.

node_modules IMO is one of the worse things about the JavaScript
ecosystem. Who has not seen the meme about node_modules being worse than a
black hole?

Fair enough. Or maybe import maps would be a better way forward.

But ensuring that it is possible to disallow loading needs to be
contemplated in the design. PHP has to be able to know what is a module and
what isn't without expensive processes.

One possible solution is that if modules do not have <?php ?> tags, ever,
and someone directly tries to load a module through http(s) the file won't
execute. Only files with <?php ?> tags are executable by the web sapi.

Having exports separate from functions and classes seems like it
would be problematic.

Again, this is how they work in JavaScript. Not saying that's the best
approach, but even if problematic it's a solved problem.

I have evidently not written enough JavaScript to realize that.

JavaScript is an odd prototypical duck. Everything ultimately is an
object. Tha

I'm also interested in learning on how other module systems out there do
work.

I am very familiar with modules (packages) in GoLang and think PHP could
benefit from considering how they work, too.

I've only touched the surface on how GoLang does things. Some of it was
confusing to me at first. It's also been awhile so I'd need to refresh my
memory to speak to it.

Composer would need a massive rewrite to be a part of this since it
currently requires the file once it determines it should do so. If we do a
system where import causes the parser to act differently then that alone
means imports can't be dealt with in the same manner as other autoloads.

That is why I am strongly recommending a modern symbol resolution system
within modules vs. autoloading.

Ok.

I'm not fond of this either.

There will need to be a way to define the entrypoint php. I think
index.php is reasonable, and if another entry point is desired it can be
called out -> "mypackage/myentry.php"

Why is an entry point needed? If there is a module metadata file as I am
proposing PHP can get all the information it needs from that file. Maybe
that is the .phm file?

Maybe. Again, I need to look over this meta data format. Also, how does it
get created?

Thanks. The sticking point is what degree of change should be
occurring. PHP isn't as behind an 8-ball as JavaScript is since the dev can
choose their PHP version and hence deprecation works most of the time for
getting rid of old stuff. But not always. Changes that are incompatible
with what came before need a way to do things the old way during
transition. Again, see PHP 6 and unicode, which snowballed until it was
clear that even if PHP 6 had been completed it wouldn't be able to run most
PHP 5 code.

It’s not just up to the dev, but the libraries we use and whether or not
we can easily upgrade (or remove) them to upgrade the php version.

By "upgrade" then, do you mean convert them into modules, or just be able
to use them as-is.

As I read it and am envisioning it, there would be no changes needed to be
able to use them as-is.

Any system that blocks existing code from being used would be a non-starter
for inclusion.

I think it would be a mistake to exclude old code and/or prevent
templating. Not only are there now decades old code in some orgs, but how
would you write an email sender that sent templated emails, provide html,
generate code, etc? There has to be an output from the code to be useful.

Excluding old code or templates from modules would not exclude them from
working as they currently do outside modules. As I see it, modules would
be more about exporting classes and functions, not generating output per se.

So all that decades of old code could continue to exist outside modules,
as it currently does today.

Exactly this.

I think it’s fine to use js as an inspiration, but it isn’t the only one
out there. There is some precedent to consider directories as modules (go
calls them “packages”) and especially in PHP where namespaces (due to PSR-4
autoloading) typically match directory structures.

Totally agree about inspiration for modules outside JS, but not sure that
PHP namespaces are the best place to look for inspiration.

Namespaces by their very nature were designed to enable autoloading with a
one-to-one file to class or interface, and by nature add conceptual scope
and complexity to a project that would not be required if a modern
module/package system were added to PHP.

Modules could and IMO should be a rethink that learns the lessons other
languages have learned over the past decade+.

Agreed.

Node.js uses package.json and the attendant npm to do this sort of prep
work. And it's a critical part of this since modules can be versioned, and
different modules may need to run different specific versions of other
modules.

Please, please, please do not make a json file a configuration language.
You can’t comment in them, you can’t handle “if php version <9, load this,
or if this extension is installed, use this.”

Maybe that is desirable, but doing things slightly different based on
extensions loaded is def a thing.

I don't think commenting is important in this file, or even desired.

As I proposed above, these could be protobuf or phar. These should be
build artifacts that can be generated on the fly during development or for
newbies even during deployment, not hand-managed.

Hand management has value in learning the underlying concepts though.

I could see the generation of two files; one in binary form and one that
is readonly so a developer can double-check what is in the current protobuf
or phar file.

Those are implementation details a little further down the road than
we're ready for, I think.

Personally, if these are going to have any special syntax, we probably
shouldn’t call them .php files. Maybe .phm?

I was going to suggest that, and then remembered earlier PHP when there
were multiple file extensions and that was a nightmare.

This does remind me to mention that I think there should be a required
"module" declaration at the top of each file just like Go requires a
"package" declaration at the top of each file. That would make it trivial
for tooling to differentiate, even with grep

Fun idea, if the @ operator is ditched as an error suppression operator it
could be used as the package operator. (If I manage to talk everyone into
getting rid of one thing, it's @).

.

the only thing I don’t like about this import/export thing is that it
reminds me of the days when we had to carefully order our require_once
directives to make sure files were loaded before they were used. So, I
think it is worth thinking about how loading will work and whether loading
can be dynamic, hoisted out of function calls (like js), how order matters,
whether packages can enrich other packages (like doctrine packages) and if
so, how much they can gain access to internal state, etc. This is very much
not “a solved problem.”

That is why I proposed having a "compiled" module symbol table to
eliminate most (all?) of those issues.

The more you bring it up, the more I am reminded of the import-map
directive added to client-side JavaScript.

On Jun 27, 2024, at 6:00 PM, Rowan Tommins [IMSoP] imsop.php@rwec.co.uk
wrote:
I do think PHP badly needs a native concept of "module" or "package" -
in fact, I'm increasingly convinced it's the inevitable path we'll end up
on at some point. BUT I think any such concept needs to be built on top of
what we have right now. That means:

It should build on or work in harmony with namespaces, not ignore or
replace them

It may be an unpopular opinion, but I would argue that namespaces were
optimized for autoloading and the one class/interface per file paradigm,
not to mention to regrettable choice of using the escape operator to
seperate namespaces and that fact that PHP throws away a lot of information
about namespaces at runtime.

I remember when the choice to use \ was made. I've rarely been so angry
about a language design choice before or since. I've gotten used to it,
but seeing \ all over the place in strings is still.. yuck.

IMO allowing modules to eventually deprecate namespaces — at least in a
defacto form of deprecation — would allow modules to be much better than if
the try to cling to a less desirable past.

It should be easy to take existing code, and convert it to a
module/package

Maybe, but not if that means modules retain baggage that should really be
jettisoned.

and namespaces have proved an extremely successful way of sharing code
without those names colliding.

At the expense of a lot more complexity than necessary, yes.

Managing symbols in a module need not be a hard problem if PHP recognizes
modules internally rather than trying to munge everything into a global
namespace like with namespaces.

I'm inclined to agree on these points, but I also don't know the engine
internals that wall. Intuitively it would seem keeping the symbol table
small would make the code go faster.

On Jun 27, 2024, at 6:41 PM, Larry Garfield larry@garfieldtech.com
wrote:
What problem would packages/modules/whatever be solving that isn't
already adequately solved?

Not speaking for Michael, obviously, but speaking for what I envision:

Adding a module/package system to PHP with modern module features
- including module private, module function, and module properties

Providing an alternative to auto-loader-optimized namespaces.
- better code management and better page load performance

Couldn't have said it better myself.

Do we want:

Packages and namespaces are synonymous? (This is roughly how JVM
languages work, I believe.)

Packages and files are synonymous? (This is how Python and
Javascript work.)

All packages correspond to a namespace, but not all namespaces are a
package?

I would argue packages (modules) should be orthogonal to namespaces to
allow modules to be optimized for what other languages have learned about
packages/modules over the past decade+.

The fact that namespaces use the escape character as a separator, that PHP
does not keep track of namespace after parsing is enough reason to move on
from them, and that they were optimize for one-to-one symbol to file
autoload are enough reasons IMO to envision a way to move on from them.

And given the near-universality of PSR-4 file structure, what impact
would each of those have in practice?

Orthogonal. Old way vs new way. But still completely usable, just not as
modules without conversion.

The fact PSR-4 exists is an artifact of autoloading single-symbol files
and thus a sunken cost does not mean that PHP should not cling to for
modules just because they currently exist.

I have nothing to add to the above.

4 months ago by Rob Landers — view source

unread

This is a very long reply to several emails.

__
The angle I am coming at this from is improving the developer experience around "packages" or "modules" or whatever you want to call them, and so much of this proposal doesn't seem to be about that.

Ok, first problem - not a proposal really, but a ramble trying to get to a proposal. Before I made the first post the idea was knocking around in my head and wouldn't go away, so I just stream of consciousness listed what's going through my head. That leads to the second point you made.

I could have made that point in other ways, and I'm sorry that my first attempt came off as insulting. It really concerned me when I already saw discussion about taking this off-list and going into the weeds on technical details when the problem that is being addressed by this proposal is extremely unclear to me.

It is unclear even to me. Perhaps I shouldn't have posted out something this half baked. That said, pruning off large sections of language functionality is a distraction. For now let's just note that it is a possibility to improve the language this way afforded by the fact that import would be new way of bringing scripts in. Could isn't should. Also, at the moment again it's a distraction. Let's focus down on how code is imported.

First though, a history review, partially to get this straight in my own head but hopefully of use for those following along. Why? Knowing how we got we are is important to some degree to chart a way forward.

PHP started as a template engine. By modern standards, and compared to the likes of twig, it's a very bad template engine, but that doesn't really matter because it's evolved into a programming language in it's own right over the last nearly 20 years.

How do you think twig works, exactly? You should probably check it out, because templates compile down to regular PHP templates -- at least it did the last time I looked at it a few years ago. How do you think emails are templated by php code? How do you think anything is output? By either "echo" or ?> content <?php, or fwrite/file_put_contents

Without that, there is literally no purpose to php code (or any code).

Include, include_once, require, and require_once have been around since the beginning as the way to splice code files together. The behavior of these statements calls back to PHP's origin as a template engine as they do things similar mechanisms like JavaScript's import do not do (and for that matter, their equivalents in C# and Java). Their scope behavior is very different from import mechanisms in other languages, as they see the variables in the scope of the function they were invoked from or the global scope when called from there. Their parsing can be aborted early with a return. They can return a value, which is quite unusual to be honest. None of this is bad per se, but it is different and the question arises is it necessary.

How do you think javascript import works, exactly? They load a file which returns a value via export.

There is nothing inherently wrong with requires/includes, it's literally required by every language via some mechanism or another (C's include statements, go's go.mod file, javascripts import/package.json, C#'s project config, etc). There's no magical thing here, just abstractions and different levels of it.

One artifact of their behavior that is bad in my opinion is that they start from the standpoint of being text or html files.

Every single language starts with a text file... There's nothing inherently special about the bytes in any source code file and they only have meaning due to creating a parser that can make sense of the stream of bytes. The fact that they also have meaning to humans is what makes it source code and not object code/byte code.

If the included file has no PHP tags then the contents get echoed out. If there are no output buffers running this can cause headers to be set and fun errors to be had. So they can't be used to create files that can only echo explicitly (that is, a call to the echo statement or the like).

For headers, this is largely up to the SAPI (the program that executes the PHP code, eg, frankenphp, php-fpm, mod-cgi, roadrunner, etc) and the fact that most SAPIs want to be able to run existing code where developers have certain expectations of behavior. FrankenPHP did a little something different with the support of the 103 status code which isn't supported in any other SAPI (AFAIK). The CLI sapi doesn't output any headers whatsoever.

As far as PHP opening tags go... I don't even notice it. They're there, just like the "package" declaration on the top of every one of my Go files.

Fast forward a bit - PHP 5.3, and the introduction of namespaces were introduced to deal with the overloaded symbol tables. They are a bit a hotwire as (if I'm not mistaken, it's been a couple years since I read the discussion on it) they just quietly prepend the namespace string in front of the name of all new symbols declared in the namespace for use elsewhere. As a result, PHP namespaces don't do some of the things we see in the namespaces of other languages (looking at Java and C# here). For example, privacy modifiers within a namespace aren't a thing.

This would be nice to have ... maybe. But namespace have been around, what, 10-15 years? I think if someone wanted to "fix" it, it would have been fixed by now.

Very quickly after PHP 5.3 released autoloaders showed up. At some point support for multiple autoloaders was added. Several schema were added, PSR-4 won out, and composer showed up to leverage this. Composer is based on NPM, even to the point where json is used to configure it, and the composer.json file is fairly close to npm's package.json file even now. It's a userland solution, but to my knowledge WordPress is the only widely used PHP application out there that doesn't use it directly (there is a Composer Wordpress project).

WordPress predates composer et. al., by more than 10 years (if you count the B2 code it was forked from). Why would it use it? From working at Automattic (I left a couple of years ago, and this is merely what I saw as an observer, I wasn't closely involved with any of the open-source side), there was a bit of a push to make it happen as it looked like Composer would be around for awhile, but then there was talk about an "official" loader/thing from php itself and I think they'd rather use that instead. When you have a project that has literally been around decades, you don't change out your whole system on the whims of what is fashionable at the time; you either innovate or wait and see what becomes standard. Composer has only been around 50% of the time WordPress has been around now; and it didn't start out as immediately popular.

Before composer, and before namespaces there was PECL. Composer has eclipsed it because PECL has the limitation of being server-wide. It never really caught on in the age of virtual hosting with multiple PHP sites running on one box. Today we have Docker, but that didn't help PECL make a comeback because by the time docker deployment of PHP sites became the norm composer had won out. Also, composer library publishing is more permissive than PECL. I'll stop here lest this digress into a Composer v PECL discussion - suffice to say stabs a bringing code packages into PHP isn't a new idea, and a survey of what's been done before, what was right about those attempts and what was wrong needs to be considered before adding yet another php package system into the mix.

I don't think PECL and Composer have much in common... at all. Like they are not even comparable, and it is still the best way to install extensions (in fact, it is the only way in a docker container AFAIK) on a self-compiled php.

The main influence of composer and autoloaders for preparing packages is that PHP has become far more Object Oriented than it was before. Prior to PHP 5.3 object oriented programming was a great option, but since autoloaders cannot bring in functions (at least not directly, they can be cheated in by bundling them in static classes which are all but namespaces) the whole ecosystem has become heavily object oriented.

require/include still works fine, or using the "file" key on the autoloader in composer.json.

I don't find most of these "problems" actually valid. There is some merit to the conclusions, but the premise feels shaky.

— Rob

4 months ago by Mike Schinkel — view source

unread

Fast forward a bit - PHP 5.3, and the introduction of namespaces were introduced to deal with the overloaded symbol tables. They are a bit a hotwire as (if I'm not mistaken, it's been a couple years since I read the discussion on it) they just quietly prepend the namespace string in front of the name of all new symbols declared in the namespace for use elsewhere. As a result, PHP namespaces don't do some of the things we see in the namespaces of other languages (looking at Java and C# here). For example, privacy modifiers within a namespace aren't a thing.

This would be nice to have ... maybe. But namespace have been around, what, 10-15 years? I think if someone wanted to "fix" it, it would have been fixed by now.

Or not.

Never underestimate the power of inertia for maintaining a less than ideal status-quo, especially when the decision to change has to be approved by a 2/3rd vote of committee. 🤷‍♂️

-Mike

4 months ago by Rob Landers — view source

unread

Fast forward a bit - PHP 5.3, and the introduction of namespaces were introduced to deal with the overloaded symbol tables. They are a bit a hotwire as (if I'm not mistaken, it's been a couple years since I read the discussion on it) they just quietly prepend the namespace string in front of the name of all new symbols declared in the namespace for use elsewhere. As a result, PHP namespaces don't do some of the things we see in the namespaces of other languages (looking at Java and C# here). For example, privacy modifiers within a namespace aren't a thing.

This would be nice to have ... maybe. But namespace have been around, what, 10-15 years? I think if someone wanted to "fix" it, it would have been fixed by now.
Or not.

Never underestimate the power of inertia for maintaining a less than ideal status-quo, especially when the decision to change has to be approved by a 2/3rd vote of committee. 🤷‍♂️

-Mike

I disagree that it is inertia, as there is quite a bit of flexibility and robustness in the current way things are. If you've ever had to get access to "internal" methods/fields in other languages, there are quite a few hoops.

Java: as of 11.0 or maybe 8.0 -- it's been awhile since I've had to do this, but you have to create a class in the target "namespace" and expose whatever you need.
C#: you have to use reflection to gain access to it.

Right now, in PHP, you just "use it" and deal with the consequences of doing so. It's great when you need to work around a bug in a library and very little friction. As a library author, I want some friction, but I also don't want to force people to come to me and open a PR just for their use-case (hence why I almost never make classes final as well).

— Rob

4 months ago by Mike Schinkel — view source

unread

node_modules IMO is one of the worse things about the JavaScript ecosystem. Who has not seen the meme about node_modules being worse than a black hole?

Fair enough. Or maybe import maps would be a better way forward.

Import maps are really a small part of what PHP actually needs. For example, is it a class, an interface, or a function? For a module, it is a property?

I envision basically that this file, whatever it would be called would be a pre-compilation of everything that PHP can pre-compile about the files that are contained within the module/directory.

See below where I talk about a pre-compiled .php.module

But ensuring that it is possible to disallow loading needs to be contemplated in the design. PHP has to be able to know what is a module and what isn't without expensive processes.

One possible solution is that if modules do not have <?php ?> tags, ever, and someone directly tries to load a module through http(s) the file won't execute. Only files with <?php ?> tags are executable by the web sapi.

Except that would require parsing all of the entire files in the directory to know (unless everything were pre-compiled as I am advocating.).

Still, I think it would be better to be explicit, and for that I would propose the first line in the file needs to start with "module" and have the name of the module.

I've only touched the surface on how GoLang does things. Some of it was confusing to me at first. It's also been awhile so I'd need to refresh my memory to speak to it.

In Go modules or, in this context more correctly named "packages" are:

A collection of files grouped into a directory and thus all files in that directory are in the same package.
Public or private scope are determined by case of symbols; lowercase are private and uppercase are public. People coming from other languages tend to hate this, but I have come to love it because it makes code less dense while employing the same information as a "public" and "private" keywords. It also makes code across different developers more consistent.
Packages can be nested in package directories, but...
There is no concept of a "sub" package, meaning there are no hierarchies when packages are used in code (there is a file path hierarchy but that is only relevant for importing the package.) When I started working with Go I thought that was unfortunate. Now after 5+ years working with Go I see it as a really good decision.
Package files must have a "package" statement at the top, and all files in the directory must have the same "package" statement, with one caveat.
That caveat is that package files can have package <packagename>_test as a package name and that file is assumed to contains a test but it cannot see private members in package <packagename>.
Test files are typically named to pair with a <filename>.go and would be named <filename>_test.go. That file's package name can either be just <packagename> or <packagename>_test, depending on if you want to reach into private members or not.
You can also find test packages that contain all <filename>_test.go files.
Testing is build into Go with go test ./... to run all tests in current and all subdirectories. (Idiomatic testing in Go is so much easier that idiomatic testing in PHP resulting in a culture of testing among almost all Go developers.)
Package files can have types, vars, consts, and funcs as well as imports and directives, of course.
Types in Go can be struct (which is the closest Go has to a class), slice of type e.g. []<type>, array of type e.g. [<n>]<type>, map[<key>]<value>, and a few more that I won't go into as I think they are out of scope for this explanation.
Packages can have one or more init() functions that are all called before the program's main() func is called. There can also be multiple init() functions even in the same file.
vars can be initialized and those initializations are run before the program's main() func is called.
consts are initialized before the program's main() func is called but can only be initialized by literal scalar types. Unfortunately.
imports take the form of import "<package>" for standard library types and where a <package> can contain parent paths.
For local types imports take the form of import "<module>/<package>" where a <module> is defining by having a go.mod file in the directory or a parent directory, and a <package> can contain parent paths. A go.mod file has a module directive, a go version, and one or more require statements (I'm ignoring a bit of minutia here.)
Modules allow grouping of packages together and were added in recent years to provide versioning for the collective dependencies of a module. The version information is stored in go.sum and is managed automatically with Go CLI commands.
For external third party modules imports take the form of import "<domain>/<package>" where <package> can contain parent paths and almost always does. An example is "github.com/stretchr/testify/assert http://github.com/stretchr/testify/assert"
External modules are by definition HTTP(S) GETable, and Go developers use go get <module> on the command line to download the module. Go does not have or need a 3rd party package manager as that can become a single point of failure and is definitely a single point of control. To download testify for use in their Go module a Go dev would run go get github.com/stretchr/testify <http://github.com/stretchr/testify/assert>;
Most external third party modules for Go are hosted on Github but can be hosted on a custom domain, Bitbucket, GitLab, etc.
The Go team manages a standard proxy for go get but organizations can run their own if desired.
imports are referenced by name internally where the package name is the last segment of the import after a /, or just the name if no slash. So "github.com/stretchr/testify/assert http://github.com/stretchr/testify/assert" is referenced in code as assert. For example, assert.Equal(t,1,value) would assert that value!=1 then it would use the testing variable t to mark this assertion as an error and generate appropriate output.
imports can be aliases so you could import check "github.com/stretchr/testify/assert <http://github.com/stretchr/testify/assert>"; and then call check.Equal(t,1,value) instead of assert.Equal(t,1,value)` but needing to alias a package frequently is a code smell for a badly named package.
You can use . as an alias and then not need to use the alias, so we could import "github.com/stretchr/testify/assert http://github.com/stretchr/testify/assert"and then just callEqual(t,1,value) instead of assert.Equal(t,1,value) but this is frowned on in the Go community except for in very specific use-cares.
You can use _ to bring in a package even if you are not referencing it in case it has an init() function that you need to run. If that applied to testify it would look like this: import _ "github.com/stretchr/testify/assert http://github.com/stretchr/testify/assert".`
All of import, var, and const support a multiline for using parenthesis like so:

var (
x = 1
y = 2
)
Module names are idiomatically one word w/o underscores and lowercase.
There is no need to import specific symbols from a Go package like there is in JavaScript. I have programmed in both Go and JS, and I have not found a real benefit to having to reference everything explicitly in the import — since you have to mention the package name everywhere you use any package symbol — but I have noticed a benefit to not having nearly as much boilerplate at the top of the file for import when working with Go vs. working with Javascript. And my GoLang IDE just manages imports for me whereas WebStorm just calls out when I haven't imported function names in Javascript.

I am sure there is more I missed, but that should cover the highlights.

The takeaways that I think would be useful are PHP modules are:

Imports
Import aliases
Module-level consts
Module-level init() functions
Module-level vars with initialization
Module-level functions
One directory == one module
No hierarchy for modules
Single word module names in lowercase.
Module sytax being <module><operator><symbol>, e.g. mymodule->MySymbol

Takeaways I wish the PHP community would consider but doubt there is any chance:

Having modules be HTTP(S) GETtable with php get <module>
Uppercase being public, lowercase being private, and no need for protected
Test packages with testing build into PHP e.g. php test ./...

I'm not fond of this either.

There will need to be a way to define the entrypoint php. I think index.php is reasonable, and if another entry point is desired it can be called out -> "mypackage/myentry.php"

Why is an entry point needed? If there is a module metadata file as I am proposing PHP can get all the information it needs from that file. Maybe that is the .phm file?

Maybe. Again, I need to look over this meta data format. Also, how does it get created?

As I am envisioning, PHP at the command line would have the ability to pre-compile a module — aka all files in the module directory — and then write a module-specific file, maybe .php.module? That could ideally be optimized for loading by PHP and have everything it needs to know to run the code in that module.

That file could be completely self-contained include all source code similar to a .phar file, or it could just have a complete symbol table and still require the PHP source code to exist as well. I have not pondered all the pros and cons of these alternatives yet.

Clearly though even if it compiled to a self-contained file the .PHP files would still be needed during development. Thus I envision that the PHP CLI would need a --watch option to watch directories and recompile the .php.module file upon PHP file change. IDEs like PhpStorm could run php --watch for users and non-IDE users could run it themselves.

When PHP would come across an import statement pointing to a module directory it would first look for the compiled .php.module file and if found use it but if not found it would recreate it. Maybe it could write to disk, or generate an error if it cannot write to disk. OTOH writing to disk might be a security issue in which case it could issue a warning that the .php.module file does not exists and then compile the module to memory and continue on.

It would be nice if there was a mode where PHP would check the timestamps of all PHP files in the module directory and if the compiled .php.module was earlier than any of the .php file then recompile but you'd want that off for production. That could be a new function set_dev_mode(boolean) or a CLI option to create a .phpdev.module instead of a .php.module.

Clearly anyone using deployments could have their build generate all the required .php.module files for deployment, and hosting companies that host apps that don't use deployments like WordPress could have processes that build the .php.module files for their users.

I think I have thought through this enough to identify there are no technical blockers, but I could certainly have missed something so please call it out if anyone can identify something that would keep this from working and/or significantly change the nature of PHP development.

BTW, this pre-compiling would ONLY apply to modules, so people not using modules would not have to be concerned about any of this at all.

-Mike

P.S.

I remember when the choice to use \ was made. I've rarely been so angry about a language design choice before or since. I've gotten used to it, but seeing \ all over the place in strings is still.. yuck.

Ditto.

4 months ago by Rowan Tommins [IMSoP] — view source

unread

The takeaways that I think would be useful are PHP modules are:

Imports

Import aliases

Module-level consts

Module-level init() functions

Module-level vars with initialization

Module-level functions

One directory == one module

No hierarchy for modules

Single word module names in lowercase.

Module sytax being <module><operator><symbol>, e.g. mymodule->MySymbol

This all sounds like an interesting set of ideas for building a new language. Most of it sounds completely impractical to apply in retrospect to an existing one with millions of users - apart from the bits we actually already have, like points 3 and 6.

Rather than looking at languages which have done things completely differently, I think it would be more useful to look for inspiration for ones which are similar to PHP's approach, but have extra features.

For instance, .net has both "assemblies" (multiple files compiled as one redistributable unit) and namespaces (which are hierarchical, like PHP's). "Package private" modifiers work at the assembly level, not the namespace one.

.Net assemblies don't have to be limited to one namespace root, but in practice generally are. For PHP, I think there would be some benefits to making that a fixed rule, with some tricks to "re-open" a namespace, or explicitly add "friends" to it.

I don't know much about modern Java, but it too has hierarchical namespaces, so there might be good and bad experiences we can learn from there, as well. And I'm sure there are others that are much less alien than JS or Go.

Rowan Tommins
[IMSoP]

4 months ago by Mike Schinkel — view source

unread

On 29 June 2024 08:06:57 BST, Mike Schinkel mike@newclarity.net wrote: >The takeaways that I think would be useful are PHP modules are: > >1. Imports >2. Import aliases >3. Module-level consts >4. Module-level init() functions >5. Module-level vars with initialization >6. Module-level functions >7. One directory == one module >8. No hierarchy for modules >9. Single word module names in lowercase. >10. Module sytax being <module><operator><symbol>, e.g. mymodule->MySymbol This all sounds like an interesting set of ideas for building a new language.

Maybe, if a new language only had a tiny set of the features needed to actually have a useful language.

That list is just package-specific, nothing about syntax, data types, control structures, package management, etc. etc.

Most of it sounds completely impractical to apply in retrospect to an existing one with millions of users - apart from the bits we actually already have, like points 3 and 6.

You say it is impractical, you claim millions of users, but you don't address why the specific features are impractical.

They are no more impractical than any other new language features PHP has added in recent years (and I am not being critical of what has been added, to be clear.)

Rather than looking at languages which have done things completely differently,

"Completely" here is a leading word used in that context.

There is nothing "completely" different about JavaScript, or Go for that matter. All three of JS, Go, and PHP are descendants of C.

We are not talking about APL, Whitespace, Befunge, or Intercal, after all.

I think it would be more useful to look for inspiration for ones which are similar to PHP's approach, but have extra features.

so there might be good and bad experiences we can learn from there, as well. And I'm sure there are others that are much less alien than JS or Go.

I would argue JS and maybe Go is a lot more similar to PHP than Java or C#. But then the alienness is in the eye of the beholder.

You claimed you don't know JS or Go, but I don't know Java or C#, at least not enough to be proficient in them.

That said, I really don't think gatekeeping based on the genetics of a language is the path to improving it. Instead I think objectively evaluating the specifics of the proposed features is the better path. And to me each of those things I mentioned stand on their own and can be justified, as needed.

-Mike

4 months ago by Rob Landers — view source

unread

Most of it sounds completely impractical to apply in retrospect to an existing one with millions of users - apart from the bits we actually already have, like points 3 and 6.

You say it is impractical, you claim millions of users, but you don't address why the specific features are impractical.

They are no more impractical than any other new language features PHP has added in recent years (and I am not being critical of what has been added, to be clear.)

So far, nobody has shown how it is practical -- that is on the person proposing the RFC. Ideally, this would be it, you show why it is useful, how to use it, etc. But it is also political. You need to show why people would use it, why people would rewrite their entire application to use it (if the RFC calls for it), and so far, nobody has shown that other than "there are packages!"

Rather than looking at languages which have done things completely differently,

"Completely" here is a leading word used in that context.

There is nothing "completely" different about JavaScript, or Go for that matter. All three of JS, Go, and PHP are descendants of C.

I cringed at this. There is no direct lineage though they borrow come syntax from C, and if you want to push it, you might as well say they're descendants of B which borrowed syntax from BCPL which borrowed syntax from CPL which borrowed it's syntax from ALGOL... eh, no, these languages are not related to each other. Inspired, maybe.

We are not talking about APL, Whitespace, Befunge, or Intercal, after all.

I think it would be more useful to look for inspiration for ones which are similar to PHP's approach, but have extra features.

so there might be good and bad experiences we can learn from there, as well. And I'm sure there are others that are much less alien than JS or Go.

I would argue JS and maybe Go is a lot more similar to PHP than Java or C#. But then the alienness is in the eye of the beholder.

No, PHP and Go are nothing like each other. With a bit of finangling, you can actually port JavaScript line-for-line to PHP, but not the other way around. If anything, JavaScript is more like PHP than PHP is more like JavaScript.

You claimed you don't know JS or Go, but I don't know Java or C#, at least not enough to be proficient in them.

That said, I really don't think gatekeeping based on the genetics of a language is the path to improving it. Instead I think objectively evaluating the specifics of the proposed features is the better path. And to me each of those things I mentioned stand on their own and can be justified, as needed.

I don't see any gate-keeping here, just people challenging assumptions and pushing for the feature to be better than it is currently being proposed.

— Rob

4 months ago by Mike Schinkel — view source

unread

You say it is impractical, you claim millions of users, but you don't address why the specific features are impractical.

They are no more impractical than any other new language features PHP has added in recent years (and I am not being critical of what has been added, to be clear.)

So far, nobody has shown how it is practical -- that is on the person proposing the RFC. Ideally, this would be it, you show why it is useful, how to use it, etc. But it is also political. You need to show why people would use it, why people would rewrite their entire application to use it (if the RFC calls for it), and so far, nobody has shown that other than "there are packages!"

The problem with your assertion is that "impractical" is not a criticism that can be objectively determined to be true or false. It is just a pejorative used to stifle discussion which is why I responded to it as a did.

Yes I agree that it is no proposers to show people why to use it, but it is unfair to proposers to give criticism that can only be classified as opinion.

You need to show why people would use it, why people would rewrite their entire application to use it (if the RFC calls for it), and so far, nobody has shown that other than "there are packages!"

It seems you have not read any of the several other emails I have written to this list in the past several days that do far more than say "there are packages!"

Please read them in full before making such further equivalently dismissive claims.

I cringed at this. There is no direct lineage though they borrow come syntax from C, and if you want to push it, you might as well say they're descendants of B which borrowed syntax from BCPL which borrowed syntax from CPL which borrowed it's syntax from ALGOL... eh, no, these languages are not related to each other. Inspired, maybe.

Aside from your cringing, how does your pedanticism here move the discussion forward in a positive manner?

No, PHP and Go are nothing like each other. With a bit of finangling, you can actually port JavaScript line-for-line to PHP, but not the other way around. If anything, JavaScript is more like PHP than PHP is more like JavaScript.

Again, you are making a statement that cannot be objectively proven true or false, and frankly I cannot see any way in which your argument here matters to discussion of modules.

I don't see any gate-keeping here,

Those who are inside the gates never do.

I called out gatekeeping because he argued the genetic fallacy[1] for dismissing the proposed ideas rather than using objective criticism of the features proposed.

just people challenging assumptions and pushing for the feature to be better than it is currently being proposed.

Yet the challenges are premised on opinions and fallacies instead of objectively challenging the proposed features.

I am happy to defend against proposal against arguments that can be objectively evaluated, but having my arguments challenged "because they come from a language I don't know" means that my assumptions are not actually being challenged and the criticisms made are based on the challenger's pre-existing lack of comfort with the assumptions while making it appear readers the criticism is objective.

And that IMO is no way to improve a language.

-Mike
[1] https://en.wikipedia.org/wiki/Genetic_fallacy

4 months ago by Rob Landers — view source

unread

You say it is impractical, you claim millions of users, but you don't address why the specific features are impractical.

They are no more impractical than any other new language features PHP has added in recent years (and I am not being critical of what has been added, to be clear.)

So far, nobody has shown how it is practical -- that is on the person proposing the RFC. Ideally, this would be it, you show why it is useful, how to use it, etc. But it is also political. You need to show why people would use it, why people would rewrite their entire application to use it (if the RFC calls for it), and so far, nobody has shown that other than "there are packages!"

The problem with your assertion is that "impractical" is not a criticism that can be objectively determined to be true or false. It is just a pejorative used to stifle discussion which is why I responded to it as a did.

Yes I agree that it is no proposers to show people why to use it, but it is unfair to proposers to give criticism that can only be classified as opinion.

The RFC process is people problems, not technical ones. Thus they can only be solved by swaying people's opinions which sometimes involves technicalities. People have and will decline RFCs simply because they don't like it. It's that simple.

You need to show why people would use it, why people would rewrite their entire application to use it (if the RFC calls for it), and so far, nobody has shown that other than "there are packages!"

It seems you have not read any of the several other emails I have written to this list in the past several days that do far more than say "there are packages!"

Please read them in full before making such further equivalently dismissive claims.

My apologies if I've missed it, but I find your emails extremely hard to read. The extra indentation you do on your replies makes it show up as quoted text that I have to expand in my email reader. It may be that my email reader has hidden entire replies from you and I wouldn't even know it.

I cringed at this. There is no direct lineage though they borrow come syntax from C, and if you want to push it, you might as well say they're descendants of B which borrowed syntax from BCPL which borrowed syntax from CPL which borrowed it's syntax from ALGOL... eh, no, these languages are not related to each other. Inspired, maybe.

Aside from your cringing, how does your pedanticism here move the discussion forward in a positive manner?

This isn't pedanticism, it's just plainly incorrect. There's been a lot of that in this thread (I haven't been keeping track of who said what per-se), to the point where some of it can't be taken seriously, like composer taking the lock file idea from npm. Like, sure, let's just go about rewriting history in this thread too. Most of these assertions can be checked by simply doing a quick search before sending the email, but arguments based on lies/incorrect facts are not valid arguments. That is why I am pointing it out, so that you (or whomever) can come back with a valid argument.

No, PHP and Go are nothing like each other. With a bit of finangling, you can actually port JavaScript line-for-line to PHP, but not the other way around. If anything, JavaScript is more like PHP than PHP is more like JavaScript.

Again, you are making a statement that cannot be objectively proven true or false, and frankly I cannot see any way in which your argument here matters to discussion of modules.

As someone who used to make a living porting things from one language to another, I can say, quite frankly, that this is objectively true.

I don't see any gate-keeping here,

Those who are inside the gates never do.

I called out gatekeeping because he argued the genetic fallacy[1] for dismissing the proposed ideas rather than using objective criticism of the features proposed.

I'm very much not "inside the gate." I am not a voter, I just like PHP, trying to make php even better by proposing RFCs and helping out other people with RFCs. I'm not paid to be here, I'm here because I want to be. I have very limited time to spend here, so I'm not consistently involved. In fact, some of my ideas are "against the grain" of the current voters as well; this is fine. Success isn't the only way to make progress.

just people challenging assumptions and pushing for the feature to be better than it is currently being proposed.

Yet the challenges are premised on opinions and fallacies instead of objectively challenging the proposed features.

I am happy to defend against proposal against arguments that can be objectively evaluated, but having my arguments challenged *"because they come from a language I don't know" *means that my assumptions are not actually being challenged and the criticisms made are based on the challenger's pre-existing lack of comfort with the assumptions while making it appear readers the criticism is objective.

And that IMO is no way to improve a language.

There is nothing objective about the RFC process...

If you go create an RFC right now, you're faced with the following guideline in the template, before you even write a word:

Quoting [[http://news.php.net/php.internals/71525|Rasmus]]:

PHP is and should remain:

a pragmatic web-focused language

a loosely typed language

a language which caters to the skill-levels and platforms of a wide range of users

Your RFC should move PHP forward following his vision. As [[http://news.php.net/php.internals/66065|said by Zeev Suraski]] "Consider only features which have significant traction to a
large chunk of our userbase, and not something that could be useful in some
extremely specialized edge cases [...] Make sure you think about the full context, the huge audience out there, the consequences of making the learning curve steeper with
every new feature, and the scope of the goodness that those new features bring."

The reason people are challenging this so hard is that last sentence: "Make sure you think about the full context, the huge audience out there, the consequences of making the learning curve steeper with every new feature[...]". This objectively WILL make the learning curve steeper with two different execution modes. People are asking you if it is "worth it" to learn two different modes, so prove it is worth it. People are asking you if it is "worth it" to rewrite billions of lines of code, so prove it. Or ... pivot and think about how you can change your feature to work within the current syntax.

— Rob

4 months ago by Mike Schinkel — view source

unread

You say it is impractical, you claim millions of users, but you don't address why the specific features are impractical.

They are no more impractical than any other new language features PHP has added in recent years (and I am not being critical of what has been added, to be clear.)

So far, nobody has shown how it is practical -- that is on the person proposing the RFC. Ideally, this would be it, you show why it is useful, how to use it, etc. But it is also political. You need to show why people would use it, why people would rewrite their entire application to use it (if the RFC calls for it), and so far, nobody has shown that other than "there are packages!"

The problem with your assertion is that "impractical" is not a criticism that can be objectively determined to be true or false. It is just a pejorative used to stifle discussion which is why I responded to it as a did.

Yes I agree that it is no proposers to show people why to use it, but it is unfair to proposers to give criticism that can only be classified as opinion.

The RFC process is people problems, not technical ones. Thus they can only be solved by swaying people's opinions which sometimes involves technicalities. People have and will decline RFCs simply because they don't like it. It's that simple.

Absolutely.

But that argument encourages a focus on feeling and not technical objectivity.

If a proposer convinces everyone that their idea is great but ignores objective technical factors they were get an RFC passed that either cannot be implemented or worse actively harms the language.

I argue it is incumbent on those discussing RFCs to remain within the realm of the objectively quantifiable and to also expect to be challenged back when their challenges are not objectively quantifiabl,e such as when the challenge is in the form of an opinion-based characterization (where "impractical" is an opinion-based characterization without objective criteria for any proposer to address. Rowan even acknowledged that his question might have been poorly worded.)

You need to show why people would use it, why people would rewrite their entire application to use it (if the RFC calls for it), and so far, nobody has shown that other than "there are packages!"

It seems you have not read any of the several other emails I have written to this list in the past several days that do far more than say "there are packages!"

Please read them in full before making such further equivalently dismissive claims.

My apologies if I've missed it, but I find your emails extremely hard to read. The extra indentation you do on your replies makes it show up as quoted text that I have to expand in my email reader. It may be that my email reader has hidden entire replies from you and I wouldn't even know it.

Interesting. My email style has always been to try to make my emails as scannable as possible and I have used intention for that. I never suspected that indented would have the opposite effect I intended.

I would never know that unless someone called it out, which you and Rowan have mentioned.

Thank you and I will try my best to avoid indentions in the future emails to this list.

I cringed at this. There is no direct lineage though they borrow come syntax from C, and if you want to push it, you might as well say they're descendants of B which borrowed syntax from BCPL which borrowed syntax from CPL which borrowed it's syntax from ALGOL... eh, no, these languages are not related to each other. Inspired, maybe.

Aside from your cringing, how does your pedanticism here move the discussion forward in a positive manner?

This isn't pedanticism, it's just plainly incorrect. There's been a lot of that in this thread (I haven't been keeping track of who said what per-se), to the point where some of it can't be taken seriously, like composer taking the lock file idea from npm. Like, sure, let's just go about rewriting history in this thread too. Most of these assertions can be checked by simply doing a quick search before sending the email, but arguments based on lies/incorrect facts are not valid arguments. That is why I am pointing it out, so that you (or whomever) can come back with a valid argument.

It is not "incorrect" and these are not "lies." We three were debating a characterization and characterizations are by-nature derived from opinion thus cannot be objectively judged to be correct or incorrect nor accurately designated as "lies."

To which I will restate: "How is your characterization of the relationship between Go and PHP vs. my characterization really relevant to this discussion, and how does it make positive impact on the debate?"

Again, you are making a statement that cannot be objectively proven true or false, and frankly I cannot see any way in which your argument here matters to discussion of modules.

As someone who used to make a living porting things from one language to another, I can say, quite frankly, that this is objectively true.

<sigh>

I asked ChatGPT:

"If someone says "X and Y are alike" and someone else says "No, X and Y are not alike" and follows it up saying based on their experience that they know "X and Y are not alike" is objectively true, is it possible for them to be correct in their assertion that their claim is objective truth? Why or why not?"

ChatGPT responded — in part — with this:

"If the claim that "X and Y are not alike" is based solely on personal experience without clear, objective criteria or evidence, then the claim is more subjective. Personal experiences can inform perceptions, but they are not sufficient to establish objective truth without verifiable evidence."

And this:

"Conclusion

It is possible for someone to be correct in their assertion that their claim is objectively true if:

• There are clear, agreed-upon criteria for what makes X and Y alike.
• There is verifiable evidence supporting the claim that X and Y do not meet these criteria.

If these conditions are met, then the claim that "X and Y are not alike" can be objectively true. Otherwise, if the criteria are ambiguous or the claim is based solely on subjective experience, it cannot be considered an objective truth."

Full reply here: https://chatgpt.com/share/b8ae223c-5d53-4e84-8353-79d2ac15dd6a

I see no "clear, agreed-upon criteria for what makes X and Y alike" nor "verifiable evidence supporting the claim that X and Y do not meet these criteria."

As such, given these criteria, no, it is NOT objectively true.

Still, once again, "How is your claim of being the exclusive holder of objective truth between you and me really relevant to this discussion, and how does it make positive impact on the debate?"

I'm very much not "inside the gate."

Again, you debate irrelevant characterizations.

I am not a voter, I just like PHP, trying to make php even better by proposing RFCs and helping out other people with RFCs. I'm not paid to be here, I'm here because I want to be. I have very limited time to spend here, so I'm not consistently involved. In fact, some of my ideas are "against the grain" of the current voters as well; this is fine. Success isn't the only way to make progress.

For a third time, "How does your claim of not being a voter make positive impact on the debate?"

There is nothing objective about the RFC process...

Glad to understand that you do not see any value in focusing on objectivity quantifiable aspects of a technical debates. Noted.

If you go create an RFC right now, you're faced with the following guideline in the template, before you even write a word:

Quoting [[http://news.php.net/php.internals/71525|Rasmus]]:

PHP is and should remain:

a pragmatic web-focused language

a loosely typed language

a language which caters to the skill-levels and platforms of a wide range of users
Your RFC should move PHP forward following his vision. As [[http://news.php.net/php.internals/66065|said by Zeev Suraski]] "Consider only features which have significant traction to a
large chunk of our userbase, and not something that could be useful in some
extremely specialized edge cases [...] Make sure you think about the full context, the huge audience out there, the consequences of making the learning curve steeper with
every new feature, and the scope of the goodness that those new features bring."

Per my characterization I see that everything I am proposing fits into that classification.

However, based on my recent experience with your propensity to argue against the characterizations made by others I feel certain you will tell me that my characterization "wrong" and that you are the only one between the two of us who could possibly be "correct."

Such is life I guess. 🤦‍♂️

The reason people are challenging this so hard is that last sentence: "Make sure you think about the full context, the huge audience out there, the consequences of making the learning curve steeper with every new feature[...]". This objectively WILL make the learning curve steeper with two different execution modes. People are asking you if it is "worth it" to learn two different modes, so prove it is worth it. People are asking you if it is "worth it" to rewrite billions of lines of code, so prove it. Or ... pivot and think about how you can change your feature to work within the current syntax.

Are you done? Have you finished mischaracterizing my arguments, e.g. "(having to) rewrite billions of lines of code?" And are we free now to objectively discuss a proposed feature set?

Or do we need to continue to debate characterizations that are irrelevant and orthogonal to any potential proposal?

-Mike

4 months ago by Rob Landers — view source

unread

You say it is impractical, you claim millions of users, but you don't address why the specific features are impractical.

They are no more impractical than any other new language features PHP has added in recent years (and I am not being critical of what has been added, to be clear.)

So far, nobody has shown how it is practical -- that is on the person proposing the RFC. Ideally, this would be it, you show why it is useful, how to use it, etc. But it is also political. You need to show why people would use it, why people would rewrite their entire application to use it (if the RFC calls for it), and so far, nobody has shown that other than "there are packages!"

The problem with your assertion is that "impractical" is not a criticism that can be objectively determined to be true or false. It is just a pejorative used to stifle discussion which is why I responded to it as a did.

Yes I agree that it is no proposers to show people why to use it, but it is unfair to proposers to give criticism that can only be classified as opinion.

The RFC process is people problems, not technical ones. Thus they can only be solved by swaying people's opinions which sometimes involves technicalities. People have and will decline RFCs simply because they don't like it. It's that simple.

Absolutely.

But that argument encourages a focus on feeling and not technical objectivity.

I get it man. I really do. I have gone off on rants on this list a couple of times because people refuse to help make things better if they don't like it. They just say "I don't like it, good luck" without much constructive feedback like I've (hopefully) been trying to give here. I don't like this feature as-proposed, but maybe if I nudge you and whomever in a different direction, you'll come up with something I do like. But if I don't participate, it is literally a waste of everyone's time.

I'm actually on your side. I do want modules, but the devil is in the details....

If a proposer convinces everyone that their idea is great but ignores objective technical factors they were get an RFC passed that either cannot be implemented or worse actively harms the language.

This has happened a few times now...

I argue it is incumbent on those discussing RFCs to remain within the realm of the objectively quantifiable and to also expect to be challenged back when their challenges are not objectively quantifiabl,e such as when the challenge is in the form of an opinion-based characterization (where "impractical" is an opinion-based characterization without objective criteria for any proposer to address. Rowan even acknowledged that his question might have been poorly worded.)

Yep. This happens, too. See my operator overrides RFC right now. I'm actively trying to cater to all the people who voted "NO" on the previous operator overrides RFC -- which means doing the opposite of well-researched best-practices, but whatever. I can firmly say that the lack of constructive criticism has been interesting. I think people want it to fail, but it's so "php-like" it might actually pass (insert scream emoji here). I think constructive criticism is the way to go, and I DO forget this from time to time, as does everyone.

You need to show why people would use it, why people would rewrite their entire application to use it (if the RFC calls for it), and so far, nobody has shown that other than "there are packages!"

It seems you have not read any of the several other emails I have written to this list in the past several days that do far more than say "there are packages!"

Please read them in full before making such further equivalently dismissive claims.

My apologies if I've missed it, but I find your emails extremely hard to read. The extra indentation you do on your replies makes it show up as quoted text that I have to expand in my email reader. It may be that my email reader has hidden entire replies from you and I wouldn't even know it.

Interesting. My email style has always been to try to make my emails as scannable as possible and I have used intention for that. I never suspected that indented would have the opposite effect I intended.

I would never know that unless someone called it out, which you and Rowan have mentioned.

Thank you and I will try my best to avoid indentions in the future emails to this list.

:thumbs-up: Thank you. I didn't think it was on purpose, but to be honest, it took a while to figure out what was even going on. :D

I cringed at this. There is no direct lineage though they borrow come syntax from C, and if you want to push it, you might as well say they're descendants of B which borrowed syntax from BCPL which borrowed syntax from CPL which borrowed it's syntax from ALGOL... eh, no, these languages are not related to each other. Inspired, maybe.

Aside from your cringing, how does your pedanticism here move the discussion forward in a positive manner?

This isn't pedanticism, it's just plainly incorrect. There's been a lot of that in this thread (I haven't been keeping track of who said what per-se), to the point where some of it can't be taken seriously, like composer taking the lock file idea from npm. Like, sure, let's just go about rewriting history in this thread too. Most of these assertions can be checked by simply doing a quick search before sending the email, but arguments based on lies/incorrect facts are not valid arguments. That is why I am pointing it out, so that you (or whomever) can come back with a valid argument.

It is not "incorrect" and these are not "lies." We three were debating a characterization and characterizations are by-nature derived from opinion thus cannot be objectively judged to be correct or incorrect nor accurately designated as "lies."

Ah, sorry, I didn't mean to intend that you (or anyone) were lying! My intention was to simply add that as a possible state of the facts, not that anyone was doing it actively!

To which I will restate: "How is your characterization of the relationship between Go and PHP vs. my characterization really relevant to this discussion, and how does it make positive impact on the debate?"

Fair enough, but I didn't know if you would reuse that fact in a rebuttal. Ergo, it is better to correct it immediately if we want to build on the original premise.

Again, you are making a statement that cannot be objectively proven true or false, and frankly I cannot see any way in which your argument here matters to discussion of modules.

As someone who used to make a living porting things from one language to another, I can say, quite frankly, that this is objectively true.

<sigh>
I asked ChatGPT:

"If someone says "X and Y are alike" and someone else says "No, X and Y are not alike" and follows it up saying based on their experience that they know "X and Y are not alike" is objectively true, is it possible for them to be correct in their assertion that their claim is objective truth? Why or why not?"

ChatGPT responded — in part — with this:

"If the claim that "X and Y are not alike" is based solely on personal experience without clear, objective criteria or evidence, then the claim is more subjective. Personal experiences can inform perceptions, but they are not sufficient to establish objective truth without verifiable evidence."

And this:

"Conclusion

It is possible for someone to be correct in their assertion that their claim is objectively true if:

• There are clear, agreed-upon criteria for what makes X and Y alike.
• There is verifiable evidence supporting the claim that X and Y do not meet these criteria.

If these conditions are met, then the claim that "X and Y are not alike" can be objectively true. Otherwise, if the criteria are ambiguous or the claim is based solely on subjective experience, it cannot be considered an objective truth."

Full reply here: https://chatgpt.com/share/b8ae223c-5d53-4e84-8353-79d2ac15dd6a

I see no "clear, agreed-upon criteria for what makes X and Y alike" nor "verifiable evidence supporting the claim that X and Y do not meet these criteria."

As such, given these criteria, no, it is NOT objectively true.

Still, once again, "How is your claim of being the exclusive holder of objective truth between you and me really relevant to this discussion, and how does it make positive impact on the debate?"

I don't remember what we were talking about, but it seems to me that this was an off-topic part of the discussion. If I may throw this back: what does any of this email have to do with modules? :p

If you go create an RFC right now, you're faced with the following guideline in the template, before you even write a word:

Quoting [[http://news.php.net/php.internals/71525|Rasmus]]:

PHP is and should remain:

a pragmatic web-focused language

a loosely typed language

a language which caters to the skill-levels and platforms of a wide range of users
Your RFC should move PHP forward following his vision. As [[http://news.php.net/php.internals/66065|said by Zeev Suraski]] "Consider only features which have significant traction to a
large chunk of our userbase, and not something that could be useful in some
extremely specialized edge cases [...] Make sure you think about the full context, the huge audience out there, the consequences of making the learning curve steeper with
every new feature, and the scope of the goodness that those new features bring."

Per my characterization I see that everything I am proposing fits into that classification.

However, based on my recent experience with your propensity to argue against the characterizations made by others I feel certain you will tell me that my characterization "wrong" and that you are the only one between the two of us who could possibly be "correct."

I don't think you're wrong and I'm sorry if I've made you feel that way. I am merely asking you to explain why you think you're right and you might have to explain it different ways, several times, to several people before "it clicks."

— Rob

4 months ago by Michael Morris — view source

unread

With a bit of finangling, you can actually port JavaScript line-for-line
to PHP, but not the other way around.

JavaScript uses prototypical inheritance, and any program that leverages
that aspect of it will be IMPOSSIBLE to port to PHP line for line without a
massive rewrite and restructure that amounts to a hell of a lot more than
"a bit of finangling".

As someone proficient in both languages I find that claim hilarious.

Now granted, there's a lot of JavaScript out there written by programmers
coming from a classical inheritance background (i.e. PHP, C#, Java) who
therefore never leverage prototypical inheritance at all, and those
programs are trivial to port between the languages, but that isn't all
there is to JavaScript.

4 months ago by Rob Landers — view source

unread

__

With a bit of finangling, you can actually port JavaScript line-for-line to PHP, but not the other way around.

JavaScript uses prototypical inheritance, and any program that leverages that aspect of it will be IMPOSSIBLE to port to PHP line for line without a massive rewrite and restructure that amounts to a hell of a lot more than "a bit of finangling".

This is getting a bit off-topic, but "it depends" on how it gets used. Sometimes you can use static classes and composition, sometimes you can use traits, and sometimes you can just put the behavior right on the object because it is only used "locally". There are some basic patterns of when to use each and how. It's actually pretty straightforward.

It gets weird when people modify the protypes of arrays, strings, and other base-types, but people (mostly) stopped doing that ~10-15 years ago.

As someone proficient in both languages I find that claim hilarious.

But how many projects have you ported? :p

Now granted, there's a lot of JavaScript out there written by programmers coming from a classical inheritance background (i.e. PHP, C#, Java) who therefore never leverage prototypical inheritance at all, and those programs are trivial to port between the languages, but that isn't all there is to JavaScript.

Yes, this is indeed the easiest.

— Rob

4 months ago by Rowan Tommins [IMSoP] — view source

unread

That list is just package-specific, nothing about syntax, data types, control structures, package management, etc. etc.

It includes fundamental design decisions like "what does a class name look like", and "how are classes identified across boundaries". If names aren't universal, what does ::class return? How does resolution work in a DI container? Etc etc etc.

I'm sure Go has answers to all those questions, but so does PHP, and I've not seen any convincing argument why we should throw it all away and start again.

Rather than looking at languages which have done things completely differently,

There is nothing "completely" different about JavaScript, or Go for that matter. All three of JS, Go, and PHP are descendants of C.

You have misread what I wrote. I didn't say the languages are different, I said the decisions they have made around namespaces and packages are different.

There is no "genetic fallacy" or "gatekeeping" involved, I'm saying it will be easier to apply a design that shares some characteristics with what we have, than to rewrite the language to fit a design which shares none.

The descriptions of the design of packages in JS and Go make me think they don't have enough in common with PHP to be easy to apply, so I'm suggesting we look at other designs.

Rowan Tommins
[IMSoP]

4 months ago by Mike Schinkel — view source

unread

That list is just package-specific, nothing about syntax, data types, control structures, package management, etc. etc.

It includes fundamental design decisions like "what does a class name look like", and "how are classes identified across boundaries". If names aren't universal, what does ::class return? How does resolution work in a DI container? Etc etc etc.

I'm sure Go has answers to all those questions, but so does PHP, and I've not seen any convincing argument why we should throw it all away and start again.

That comment sounds like you think that I am saying to do what Go does for PHP. That is not what I was saying.

Instead, I am saying "let us look at these aspects of Go for inspiration for features that would be beneficial for PHP."

Anyway, I have started a repo to put thoughts down, so continuing this discussion is probably premature before I have something more to show/discuss.

Rather than looking at languages which have done things completely differently,

There is nothing "completely" different about JavaScript, or Go for that matter. All three of JS, Go, and PHP are descendants of C.

You have misread what I wrote. I didn't say the languages are different, I said the decisions they have made around namespaces and packages are different.

There is no "genetic fallacy" or "gatekeeping" involved, I'm saying it will be easier to apply a design that shares some characteristics with what we have, than to rewrite the language to fit a design which shares none.

Fair point.

But let us not dismiss ideas that come from a language that you admitted are not that familiar with — just because it comes from that other language — before fully understanding what is being proposed.

The descriptions of the design of packages in JS and Go make me think they don't have enough in common with PHP to be easy to apply, so I'm suggesting we look at other designs.

And I am suggesting that maybe those designs will benefit PHP more than thinking inside the box.

That said, I will applaud you bringing specific concepts to the table from any other languages.

-Mike

P.S. What I am working on at the moment — after one tweak of that list of ten things to get inspired about from Go — is a lot more like PHP than you are probably currently envisioning and can possibly be implemented with much less of a production than anyone is likely assuming.

4 months ago by Rowan Tommins [IMSoP] — view source

unread

If you got this far, thank you. This overall idea to take one of the
better things to happen to JavaScript in the last decade and
incorporate it into PHP has been bothering me for awhile so I figured
I'd share. I don't know how much merit there is to this though.

Thank you for sharing. I think it's valuable to explore radical ideas
sometimes.

I do think PHP badly needs a native concept of "module" or "package" -
in fact, I'm increasingly convinced it's the inevitable path we'll end
up on at some point. BUT I think any such concept needs to be built on
top of what we have right now. That means:

It should build on or work in harmony with namespaces, not ignore or
replace them
It should be compatible with Composer, but not dependent on it
It should be easy to take existing code, and convert it to a
module/package
It should be easy to carry on using that module/package after it's
been converted

If we can learn from other languages while we do that, I'm all for it;
but we have to remember that those languages had a completely different
set of constraints to work with.

For instance, JS has no concept of "namespaces", but does treat function
names as dynamically scoped alongside variables. So the module system
needed to give a way of managing how you imported names from one scope
to another. That's not something PHP needs, because it treats all names
as global, and namespaces have proved an extremely successful way of
sharing code without those names colliding.

Other parts of your e-mail are essentially an unrelated idea, to have
some new "PHP++" dialect, where a bunch of "bad" things are removed.
You're not the first person to be tempted by this, but I think the
history HHVM and Hack is educational here: initially, PHP and Hack were
designed to interoperate on one run-time, but the more they tried to
optimise for Hack, the harder it became to support PHP, and now Hack is
a completely independent language.

I'm not even sure what problem some of your ideas are intended to solve,
e.g. using "." instead of "::" and "->". At that point, it really does
feel like you just want to create a new language, mixing ideas from PHP
and JS, but incompatible with either of them.

I do hope this discussion can inspire some interesting ideas, but I
don't think what you've presented is the right way to go.

Regards,

--
Rowan Tommins
[IMSoP]

4 months ago by Larry Garfield — view source

unread

If you got this far, thank you. This overall idea to take one of the
better things to happen to JavaScript in the last decade and
incorporate it into PHP has been bothering me for awhile so I figured
I'd share. I don't know how much merit there is to this though.

There's a lot to chew on here, and some interesting ideas. However, reading through it, there's one key question that sticks in my mind:

What problem is this trying to solve?

What problem would packages/modules/whatever be solving that isn't already adequately solved?

There seems to be a bunch of stuff kinda-sorta being addressed in this proposal, but no clear picture of the problem being solved, and how it gets solved.

Before we get anywhere close to weeds, there's high-level questions that need to be answered.

Which of these are we trying to solve? (Solving all of them at once is unlikely, and some are mutually-incompatible.)

Adding a "strict pedantic mode" without messing with existing code?
Package-level visibility (public, package, protected, private)?
Avoid name clashes?
Improved information for autoloaders and preloading, possibly making class-per-file unnecessary in many cases?
A larger scope for the compiler to analyze in order to make optimizations?
Package-level declares, inherited by all files in the package?
Something else?

We need to know exactly what we're solving for to be able to determine if a proposal is any good at solving it.

For me personally, 2 and 4 would be the main things to address, and if someone with more compiler knowledge than me could do something on 5, that would be neat. 3 is, as Tim noted, a solved problem at this point. 1 we already are working on in stages via deprecations. 6 is potentially unwise, unless we had a good set of things that made sense to specify at a package level.

Once we know what we're trying to solve, we need to ask about constraints. The major one being the relationship with namespaces.

Do we want:

Packages and namespaces are synonymous? (This is roughly how JVM languages work, I believe.)
Packages and files are synonymous? (This is how Python and Javascript work.)
All packages correspond to a namespace, but not all namespaces are a package?

And given the near-universality of PSR-4 file structure, what impact would each of those have in practice? (Even if packages open up some new autoloading options and FIG publishes a new PSR for how to use them, there's only a billion or so PSR-4 class files in the wild that aren't going away any time soon.) My gut feeling is we want 3, but I'm sure there's a debate to be had there.

All the other stuff about different operators and file name extensions and stuff is completely irrelevant until there is a solid consensus on these basic questions. For something of this scale, to coin a phrase, "bring me problems, not solutions."

--Larry Garfield

4 months ago by Jordi Boggiano — view source

unread

First of all a quick note for the OP: I am all for it in general, but I
don't think copying the entire JS module system one to one makes sense.
It contains a lot of compromises and mistakes that we should absolutely
learn from as well as the good things they did.

Which of these are we trying to solve? (Solving all of them at once is
unlikely, and some are mutually-incompatible.)

Adding a "strict pedantic mode" without messing with existing code?

Package-level visibility (public, package, protected, private)?

Avoid name clashes?

Improved information for autoloaders and preloading, possibly making class-per-file unnecessary in many cases?

A larger scope for the compiler to analyze in order to make optimizations?

Package-level declares, inherited by all files in the package?

Something else?

I agree with most of your analysis, and IMO Package-level visibility is
the main direct win, with a larger scope for JIT optimization coming later.

It would however be very tempting to bake in 1, and remove a bunch of
things which are not removable from the language at large due to BC, as
that might be a once in a lifetime opportunity. Some features make JIT
optimizations nearly impossible (Nikita had a list somewhere.. but the
main one if probably killing references).

The autoloader information to be honest I am not sure how important this
is. For everyone not wanting to do class-per-file, note that you can
just use "classmap" autoloading in Composer. It is anyway the most
performant option at runtime [1]. The only catch is you have to re-dump
the autoloader when adding new classes/files to make them discoverable.
But I think everyone's kinda too stuck on PSR-4 because it is a standard.

Do we want:

Packages and namespaces are synonymous? (This is roughly how JVM languages work, I believe.)

Packages and files are synonymous? (This is how Python and Javascript work.)

All packages correspond to a namespace, but not all namespaces are a package?

And given the near-universality of PSR-4 file structure, what impact would each of those have in practice? (Even if packages open up some new autoloading options and FIG publishes a new PSR for how to use them, there's only a billion or so PSR-4 class files in the wild that aren't going away any time soon.) My gut feeling is we want 3, but I'm sure there's a debate to be had there.

I'd go for 3 as well. Every package having a single root namespace is
probably true of 99% of packages due to the PSR-4 autoload root.
Sub-namespaces are discretionary.

[1] https://getcomposer.org/doc/articles/autoloader-optimization.md

--
Jordi Boggiano
@seldaek -https://seld.be

4 months ago by David Gebler — view source

unread

Hello all. This is a ramble of an idea that's managed to run around my
head for a few days now. It isn't fully formed, but I've ran the thought
experiment as far as I can on my own and want to share it with all of you.

If you got this far, thank you. This overall idea to take one of the
better things to happen to JavaScript in the last decade and incorporate it
into PHP has been bothering me for awhile so I figured I'd share. I don't
know how much merit there is to this though.

I don't think PHP needs to take many if any lessons from JS to be honest.
JS evolved the way it did to solve its own set of problems, and likewise so
has PHP. I'm not really clear from reading the whole thing what problems
this loose bag of ideas is even intended to solve, really, but from what I
have gathered it seems to me more like what you want is a transpiler a la
Typescript and that this would achieve whatever you're trying to achieve.
Personal take, but if there's one thing I don't want for PHP it's to become
[any more] of a garbled mess of different styles and different ways of
achieving the same fundamental abstractions and results. Composer and PSR-4
have become de facto standards, they're very good standards, we've achieved
remarkable near unity in their adoption and whatever few inconveniences
remain through the application of those standards in certain cases are
minor trade-offs (not having multiple interfaces in the same file, etc.)

A JS-inspired subset of PHP within PHP, supported by invocation through new
keywords and handed off to a separate parser seems like it's asking for
trouble at every level from implementation to maintenance to userland to
dependency management...

4 months ago by Peter Bowyer — view source

unread

import foo from "foo.php"

I'd strongly recommend the autocomplete-friendly order instead:

from "foo.php" import foo

Overall I am keen on module/package support of some kind that allows for
visibility control at the boundary, so I can hide implementation classes
within the module and control the public interface.

Peter

4 months ago by Michael Morris — view source

unread

On Thu, Jun 27, 2024 at 2:29 PM Jordan LeDoux jordan.ledoux@gmail.com
wrote:

Who would build it is an extremely key aspect of making changes to PHP.
Ideas are hard enough to survive the RFC process when there's already an
implementation. Finding a sponsor to work on this would be the first step.

...

I like the idea but I'm a bit skeptical until we have some buy-in from

someone that could actually get this implemented.

--
Marco Deleu

Perhaps, though a conversation like this is helpful. Some rather
complicated RFCs do get approved/voted on before an implementation is done
when contributors who are familiar with the Zend engine get on board early.
Conversely, there are some extremely thoroughly implemented complicated
RFCs that get rejected because most voters don't participate in discussion
until voting is actually started. Something as broad as this probably
requires an off-list discussion with key active contributors, because
participation on list is so hit-and-miss.

Jordan

Agreed. I've seen both of those occur so I want to avoid both. Even if
large sections of the discussion are eventually done off list to avoid
getting lost in bike-shed issues, the members of the list, or at least the
voting members of the list should be kept updated periodically to prevent
the project from going down an unpopular path.

The only thing I'm truly sure about with this is that it will be profoundly
difficult to do. But it needs to be addressed because PHP is starting to
lag behind in this area.

4 months ago by Michael Morris — view source

unread

Who would build it is an extremely key aspect of making changes to PHP.
Ideas are hard enough to survive the RFC process when there's already an
implementation. Finding a sponsor to work on this would be the first step.

Agreed.

Given that ini settings are frowned upon nowadays, I think having a <?php declare(modules=1); for the initial file might make the idea more likely
to pass a vote? Or maybe I'd even try to go one step further and say that
whatever file is being executed by SAPI (the first PHP file) could be
interpreted with a dumb lookahead. If the file has import / export syntax,
treat it like PHP Module, otherwise fallback.

MKS Archive already pointed out that even this is unecessary. Just let the
landing file import the modules, even if that means it's a one line file.

I'm not familiar enough with Javascript / Typescript ecosystem, but I've
only ever seen / used the ability to import using direct filepath.

In the clientside up until recently direct filepath was the only way. That
changed with import maps

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap

NodeJS has been doing something similar serverside for much longer with
CommonJS requires, which predate by a good 5 years the ES6 mechanism. As a
serverside language it now has to juggle both.

The resolution path I sketched out is based on how NodeJS works. Can that
be improved upon? Likely - it is confusing

The fact there's weird behaviors as result of trying to import a file and
suddenly a file all the way from include_paths or php_modules seems
like a no-go to me. I'd favor using only simple file path navigation and if
the file doesn't exist, error.

Perhaps if the idea gains merit, Composer could offer something similar to
Vite where we can create an alias to a specific folder and then import
things like from '@package/path/to/file.

Composer would need a massive rewrite to be a part of this since it
currently requires the file once it determines it should do so. If we do a
system where import causes the parser to act differently then that alone
means imports can't be dealt with in the same manner as other autoloads.

This will of course require a package manager similar to composer to
become part of core. However, composer will not be eclipsed as the import
package manager (phppm?) is only concerned with user modules. These modules
must explicitly export any symbols being fetched from them, whereas
composer will continue to load files using require.

Imports can also be done against directories
import foo from "mypackage"
In this case the parser will look for "mypackage/index.php"
I'm not fond of this either.

There will need to be a way to define the entrypoint php. I think
index.php is reasonable, and if another entry point is desired it can be
called out -> "mypackage/myentry.php"

Overall, I think PHP has already reached the limit of surviving with only
PSR-4 and Composer. Single class files were a great solution to get us out
of the nightmare of require and import on top of PHP files. But more
than once I have had the desire to declare a couple of interfaces in a
single file, or a handful of Enums, etc. It seems like PHP Modules could
also address the issue with function autoloading and package-level
visibility. I like the idea but I'm a bit skeptical until we have some
buy-in from someone that could actually get this implemented.

That would be one of the larger hurdles, if not the largest.

4 months ago by Deleu — view source

unread

The fact there's weird behaviors as result of trying to import a file and
suddenly a file all the way from include_paths or php_modules seems
like a no-go to me. I'd favor using only simple file path navigation and if
the file doesn't exist, error.

Perhaps if the idea gains merit, Composer could offer something similar
to Vite where we can create an alias to a specific folder and then import
things like from '@package/path/to/file.

Composer would need a massive rewrite to be a part of this since it
currently requires the file once it determines it should do so. If we do a
system where import causes the parser to act differently then that alone
means imports can't be dealt with in the same manner as other autoloads.

Perhaps my point here wasn't so obvious. I wasn't talking about Composer
making drastic changes to accommodate this, I was only merely mentioning
Composer being used to provide something like the following:

{
    "require": {
        "php": "^8.2"
        ....
    },
    "php-modules": {
        "@packages": "./packages",
        "@utilities": "./tools/utilities"
        ....
    }
 }

Then whenever there's a import Foo from '@packages/Foo.php', the notation
@packages would be replaced by the packages folder. This is equivalent to
Vite resolve.alias
https://vitejs.dev/config/shared-options.html#resolve-alias

--
Marco Deleu

4 months ago by Michael Morris — view source

unread

Interesting to see this. Serendipitous given the email I sent on the list
in reply to Larry.

My initial thoughts:

I really like the concept of cleaning up issues that BC make impossible
to fix by introducing modules.

Thanks. The sticking point is what degree of change should be occurring.
PHP isn't as behind an 8-ball as JavaScript is since the dev can choose
their PHP version and hence deprecation works most of the time for getting
rid of old stuff. But not always. Changes that are incompatible with what
came before need a way to do things the old way during transition. Again,
see PHP 6 and unicode, which snowballed until it was clear that even if PHP
6 had been completed it wouldn't be able to run most PHP 5 code.

No need for autoloaders with modules; I assume this would be obvious,
right?

Depends largely on whether modules can include and require to get access to
old code. I also didn't discuss how they behave - do they share their
variables with includes and requires?

Not a good idea to use an ini setting; most view them to be problematic.

.htaccess îs Apache-only, so a non-starter anyway.

The first script should not be a module. If you want that, have a 1
line index.php file do an import.

I love this idea.

Modules should be directories, not .php files. Having each file be a
module makes code org really hard.

Yes, but that is how JavaScript currently handles things. It is currently
necessary when making large packages to have an index.js that exports out
the public members of the module. This entry point is configurable through
the package.json of the module.

Modules would have a symbol table metadata file generated by IDEs and
during deployment.

Node.js uses package.json and the attendant npm to do this sort of prep
work. And it's a critical part of this since modules can be versioned, and
different modules may need to run different specific versions of other
modules.

If no metadata file in directory PHP can generate one in memory during
first directory access.

.php files in modules as identified by metadata file should not be
loadable via HTTP(S).

Those are implementation details a little further down the road than we're
ready for, I think.

Having exports separate from functions and classes seems like it would
be problematic.

Again, this is how they work in JavaScript. Not saying that's the best
approach, but even if problematic it's a solved problem.

Exports could be implemented as attributes, which could be really
elegant.

Exports as attributes pairs with the symbol on the line above, and
would enable easy aliasing.

Ultimately everything in JavaScript is an object. JavaScript provides a
mechanism for handling a module file with a single default export, but it
supports multiple exports from the same file which arrive as an object best
approximated in PHP as a static class. I could hash out further, but again,
I'd like to gauge some interest and very high level feedback. I'm also
interested in learning on how other module systems out there do work. I'm
picking of JavaScript because most of the PHP community has to use it as
well for client side scripting and so most of us should have at least
passing familiarity with it.

And finally, when are you starting the RFC? :-)

It's too early for anyone to start at this moment - and while I certainly
am willing to help I'm not qualified to take the lead on this.

If adopted, this is a massive change, and the results of this conversation
won't hit for years. Let's take our time. For one, this overarching project
will need multiple coordinated RFC's, as well as figuring out what to do
and also in what order.

4 months ago by Rob Landers — view source

unread

Interesting to see this. Serendipitous given the email I sent on the list in reply to Larry.

My initial thoughts:

I really like the concept of cleaning up issues that BC make impossible to fix by introducing modules.

Thanks. The sticking point is what degree of change should be occurring. PHP isn't as behind an 8-ball as JavaScript is since the dev can choose their PHP version and hence deprecation works most of the time for getting rid of old stuff. But not always. Changes that are incompatible with what came before need a way to do things the old way during transition. Again, see PHP 6 and unicode, which snowballed until it was clear that even if PHP 6 had been completed it wouldn't be able to run most PHP 5 code.

It’s not just up to the dev, but the libraries we use and whether or not we can easily upgrade (or remove) them to upgrade the php version.

No need for autoloaders with modules; I assume this would be obvious, right?

Depends largely on whether modules can include and require to get access to old code. I also didn't discuss how they behave - do they share their variables with includes and requires?

I think it would be a mistake to exclude old code and/or prevent templating. Not only are there now decades old code in some orgs, but how would you write an email sender that sent templated emails, provide html, generate code, etc? There has to be an output from the code to be useful.

Not a good idea to use an ini setting; most view them to be problematic.

.htaccess îs Apache-only, so a non-starter anyway.

The first script should not be a module. If you want that, have a 1 line index.php file do an import.

I love this idea.

Modules should be directories, not .php files. Having each file be a module makes code org really hard.

Yes, but that is how JavaScript currently handles things. It is currently necessary when making large packages to have an index.js that exports out the public members of the module. This entry point is configurable through the package.json of the module.

I think it’s fine to use js as an inspiration, but it isn’t the only one out there. There is some precedent to consider directories as modules (go calls them “packages”) and especially in PHP where namespaces (due to PSR-4 autoloading) typically match directory structures.

For example, I’m still going to go forward with my #[Internal] attribute RFC some time in the next month or so, which will be namespace based. I have no idea if it will pass, (some people are worried about it clashing with an RFC like this one) but I think we’d have value in it for years to come until something like this gets fleshed out. We will see…

Modules would have a symbol table metadata file generated by IDEs and during deployment.

Node.js uses package.json and the attendant npm to do this sort of prep work. And it's a critical part of this since modules can be versioned, and different modules may need to run different specific versions of other modules.

Please, please, please do not make a json file a configuration language. You can’t comment in them, you can’t handle “if php version <9, load this, or if this extension is installed, use this.”

Maybe that is desirable, but doing things slightly different based on extensions loaded is def a thing.

If no metadata file in directory PHP can generate one in memory during first directory access.

.php files in modules as identified by metadata file should not be loadable via HTTP(S).

Those are implementation details a little further down the road than we're ready for, I think.

Personally, if these are going to have any special syntax, we probably shouldn’t call them .php files. Maybe .phm?

Having exports separate from functions and classes seems like it would be problematic.

Again, this is how they work in JavaScript. Not saying that's the best approach, but even if problematic it's a solved problem.

the only thing I don’t like about this import/export thing is that it reminds me of the days when we had to carefully order our require_once directives to make sure files were loaded before they were used. So, I think it is worth thinking about how loading will work and whether loading can be dynamic, hoisted out of function calls (like js), how order matters, whether packages can enrich other packages (like doctrine packages) and if so, how much they can gain access to internal state, etc. This is very much not “a solved problem.”

Exports could be implemented as attributes, which could be really elegant.

Exports as attributes pairs with the symbol on the line above, and would enable easy aliasing.

Ultimately everything in JavaScript is an object. JavaScript provides a mechanism for handling a module file with a single default export, but it supports multiple exports from the same file which arrive as an object best approximated in PHP as a static class. I could hash out further, but again, I'd like to gauge some interest and very high level feedback. I'm also interested in learning on how other module systems out there do work. I'm picking of JavaScript because most of the PHP community has to use it as well for client side scripting and so most of us should have at least passing familiarity with it.

In JavaScript, arrays are instances, in php, they are values. This is something to consider if a module exports an array of exports.

And finally, when are you starting the RFC? :-)

It's too early for anyone to start at this moment - and while I certainly am willing to help I'm not qualified to take the lead on this.

If adopted, this is a massive change, and the results of this conversation won't hit for years. Let's take our time. For one, this overarching project will need multiple coordinated RFC's, as well as figuring out what to do and also in what order.

— Rob

4 months ago by Michael Morris — view source

unread

On Thu, Jun 27, 2024 at 1:02 PM MKS Archive mikeschinkel@gmail.com
wrote:

Interesting to see this. Serendipitous given the email I sent on the list
in reply to Larry.

My initial thoughts:

I really like the concept of cleaning up issues that BC make impossible
to fix by introducing modules.

Thanks. The sticking point is what degree of change should be occurring.
PHP isn't as behind an 8-ball as JavaScript is since the dev can choose
their PHP version and hence deprecation works most of the time for getting
rid of old stuff. But not always. Changes that are incompatible with what
came before need a way to do things the old way during transition. Again,
see PHP 6 and unicode, which snowballed until it was clear that even if PHP
6 had been completed it wouldn't be able to run most PHP 5 code.

It’s not just up to the dev, but the libraries we use and whether or not
we can easily upgrade (or remove) them to upgrade the php version.

No need for autoloaders with modules; I assume this would be obvious,
right?

Depends largely on whether modules can include and require to get access
to old code. I also didn't discuss how they behave - do they share their
variables with includes and requires?

I think it would be a mistake to exclude old code and/or prevent
templating. Not only are there now decades old code in some orgs, but how
would you write an email sender that sent templated emails, provide html,
generate code, etc? There has to be an output from the code to be useful.

Not a good idea to use an ini setting; most view them to be problematic.

.htaccess îs Apache-only, so a non-starter anyway.

The first script should not be a module. If you want that, have a 1
line index.php file do an import.

I love this idea.

Going to come back to this actually.

For example, I’m still going to go forward with my #[Internal] attribute
RFC some time in the next month or so, which will be namespace based. I
have no idea if it will pass, (some people are worried about it clashing
with an RFC like this one) but I think we’d have value in it for years to
come until something like this gets fleshed out. We will see…

What about declare? I have no idea if this would work, but..

declare(importmap=[
'imports' => [
'label' : 'path',
]
]

If that is put in the initial php file then it could map out the imports.
An IDE could maintain the file as well. The other two attributes are
scopes and integrity - the latter being a hash check for the file, scopes
could be used to handle php version numbers. Multiple import maps could be
defined, with each map affecting the file and whatever it imports - the
seek order moving up.

It would be possible to let import maps affect include and require as well.
Would there be a benefit? Or just more confusion?

Modules would have a symbol table metadata file generated by IDEs and
during deployment.

Node.js uses package.json and the attendant npm to do this sort of prep
work. And it's a critical part of this since modules can be versioned, and
different modules may need to run different specific versions of other
modules.

Please, please, please do not make a json file a configuration language.
You can’t comment in them, you can’t handle “if php version <9, load this,
or if this extension is installed, use this.”

Lack of comments are a problem. NodeJS does handle engine blocks, but it's
messy. That said, I'm not a fan of json's popularity even in Javascript,
and less so in PHP where it feels foreign.

Maybe that is desirable, but doing things slightly different based on
extensions loaded is def a thing.

Keep in mind that extensions typically expose functions automatically, and
under the original proposal those functions have to be imported to be used:
import mysql_query

Perhaps PHP imports, unlike their JavaScript or even Java C# counterparts,
could be placed in try/catch blocks, with the catch resolving what to do if
the import misses.

If no metadata file in directory PHP can generate one in memory during
first directory access.

.php files in modules as identified by metadata file should not be
loadable via HTTP(S).

Those are implementation details a little further down the road than we're
ready for, I think.

Personally, if these are going to have any special syntax, we probably
shouldn’t call them .php files. Maybe .phm?

I really don't like the extension games seen in node with js, cjs and mjs,
but there's a precedent for doing it that way. In their setup if you've
set modules as the default parse method then cjs can be used to identify
files that still need to use CommonJS. And mjs can force the ES6 even in
default mode. But it is a bit of a pain and feels like it should be
avoided.

the only thing I don’t like about this import/export thing is that it
reminds me of the days when we had to carefully order our require_once
directives to make sure files were loaded before they were used. So, I
think it is worth thinking about how loading will work and whether loading
can be dynamic, hoisted out of function calls (like js), how order matters,
whether packages can enrich other packages (like doctrine packages) and if
so, how much they can gain access to internal state, etc. This is very much
not “a solved problem.”

In JavaScript import must be top of the file - you'll get an error if you
try an import following any other statement unless it's a dynamic import(),
which is a whole other Promise/Async/Kettle of fish that thankfully PHP
does not have to take into account as, until you get used to it (and even
after), async code is a headache.

In JavaScript, arrays are instances, in php, they are values. This is
something to consider if a module exports an array of exports.

import() (a different animal from import, yes, that is confusing, yay
JavaScript) returns a promise which resolves to an object. I've slammed my
head into a desk more than once over this, and it's a feature I don't want
brought in.

4 months ago by Rob Landers — view source

unread

__

Interesting to see this. Serendipitous given the email I sent on the list in reply to Larry.

My initial thoughts:

I really like the concept of cleaning up issues that BC make impossible to fix by introducing modules.

Thanks. The sticking point is what degree of change should be occurring. PHP isn't as behind an 8-ball as JavaScript is since the dev can choose their PHP version and hence deprecation works most of the time for getting rid of old stuff. But not always. Changes that are incompatible with what came before need a way to do things the old way during transition. Again, see PHP 6 and unicode, which snowballed until it was clear that even if PHP 6 had been completed it wouldn't be able to run most PHP 5 code.

It’s not just up to the dev, but the libraries we use and whether or not we can easily upgrade (or remove) them to upgrade the php version.

No need for autoloaders with modules; I assume this would be obvious, right?

Depends largely on whether modules can include and require to get access to old code. I also didn't discuss how they behave - do they share their variables with includes and requires?

I think it would be a mistake to exclude old code and/or prevent templating. Not only are there now decades old code in some orgs, but how would you write an email sender that sent templated emails, provide html, generate code, etc? There has to be an output from the code to be useful.

Not a good idea to use an ini setting; most view them to be problematic.

.htaccess îs Apache-only, so a non-starter anyway.

The first script should not be a module. If you want that, have a 1 line index.php file do an import.

I love this idea.

Going to come back to this actually.

For example, I’m still going to go forward with my #[Internal] attribute RFC some time in the next month or so, which will be namespace based. I have no idea if it will pass, (some people are worried about it clashing with an RFC like this one) but I think we’d have value in it for years to come until something like this gets fleshed out. We will see…

What about declare? I have no idea if this would work, but..

declare(importmap=[
'imports' => [
'label' : 'path',
]
]

If that is put in the initial php file then it could map out the imports. An IDE could maintain the file as well. The other two attributes are scopes and integrity - the latter being a hash check for the file, scopes could be used to handle php version numbers. Multiple import maps could be defined, with each map affecting the file and whatever it imports - the seek order moving up.

It would be possible to let import maps affect include and require as well. Would there be a benefit? Or just more confusion?

Internals has made it pretty clear: no more declare or ini entries (unless it is absolutely needed).

I personally don’t like it because it uses arrays, which are opaque, easy to typo, and hard to document/check.

Instead, maybe consider a new Reflection API?

(new ReflectionModule)->import('MyModule')->run()

From the index.php file (where “run” is an exported function and can take arguments, like $argv, request objects, globals, etc).

Inside modules, we would have the import syntax (which could arguably be compiled to the above code, more-or-less).

Modules would have a symbol table metadata file generated by IDEs and during deployment.

Node.js uses package.json and the attendant npm to do this sort of prep work. And it's a critical part of this since modules can be versioned, and different modules may need to run different specific versions of other modules.

Please, please, please do not make a json file a configuration language. You can’t comment in them, you can’t handle “if php version <9, load this, or if this extension is installed, use this.”

Lack of comments are a problem. NodeJS does handle engine blocks, but it's messy. That said, I'm not a fan of json's popularity even in Javascript, and less so in PHP where it feels foreign.

Maybe that is desirable, but doing things slightly different based on extensions loaded is def a thing.

Keep in mind that extensions typically expose functions automatically, and under the original proposal those functions have to be imported to be used: import mysql_query

they also do now, unless you either prefix them with \ or rely on the fallback resolution system. I’m honestly not sure we need a new syntax for this, but maybe just disable the global fallback system in modules?

Perhaps PHP imports, unlike their JavaScript or even Java C# counterparts, could be placed in try/catch blocks, with the catch resolving what to do if the import misses.

Right now, I usually see if(function_exists('some_func_from_extension')), so as long as imports behave as they currently do — not actually triggering any loading — then this would still work just fine.

If no metadata file in directory PHP can generate one in memory during first directory access.

.php files in modules as identified by metadata file should not be loadable via HTTP(S).

Those are implementation details a little further down the road than we're ready for, I think.

Personally, if these are going to have any special syntax, we probably shouldn’t call them .php files. Maybe .phm?

I really don't like the extension games seen in node with js, cjs and mjs, but there's a precedent for doing it that way. In their setup if you've set modules as the default parse method then cjs can be used to identify files that still need to use CommonJS. And mjs can force the ES6 even in default mode. But it is a bit of a pain and feels like it should be avoided.

I would argue that it be something seriously considered. Scanning a directory in the terminal, in production systems, while diagnosing ongoing production issues, it can be very handy to distinguish between the “old way” and “new way”, at a glance.

the only thing I don’t like about this import/export thing is that it reminds me of the days when we had to carefully order our require_once directives to make sure files were loaded before they were used. So, I think it is worth thinking about how loading will work and whether loading can be dynamic, hoisted out of function calls (like js), how order matters, whether packages can enrich other packages (like doctrine packages) and if so, how much they can gain access to internal state, etc. This is very much not “a solved problem.”

In JavaScript import must be top of the file - you'll get an error if you try an import following any other statement unless it's a dynamic import(), which is a whole other Promise/Async/Kettle of fish that thankfully PHP does not have to take into account as, until you get used to it (and even after), async code is a headache.

Are you sure? I don’t remember them removing import hoisting, but it’s probably more of a typical linting rule because it is hard to reason about.

https://www.w3schools.com/js/js_hoisting.asp

In other news, async in PHP is alive and well. Fibers are a thing and swoole just announced threading.

In JavaScript, arrays are instances, in php, they are values. This is something to consider if a module exports an array of exports.

import() (a different animal from import, yes, that is confusing, yay JavaScript) returns a promise which resolves to an object. I've slammed my head into a desk more than once over this, and it's a feature I don't want brought in.

— Rob

4 months ago by Michael Morris — view source

unread

I don't think that is correct...

Correct or not it's irrelevant trivia.

While this looks good on paper, you're going to have to standardize how

packages are accessed (API calls, etc) so they can be used in this file, or
literally anyone who wants to add a competing registry will have to create
an RFC to allow accessing their own registry, which is a ton of politics
for something that is strictly technical -- not to mention a bunch of
if-this-registry-do-that type statements scattered throughout the code,
which makes it harder to maintain.

While this is a fair point, it's a discussion of trees while we are talking
about the forest. At this early stage of sketching out this system what we
are concerned with is how the registry is set and whether multiples can be
set. Once that's hashed out the details of how registries work -
authentication and all that can be contemplated.

SAPIs are the programs that parse ALL php code and return it to the server
(ie, nginx, apache, caddy, etc) to be displayed.

I mis-spoke. What I mean is these files cannot be directly invoked. A
normal php file must load first, then the phm file, even if the php file is
a mere one liner that imports the module.

In other news, I'm not a fan of how many times I have to write "twig" just
to get Twig in the current file. The module already registers a namespace,
why can't the use-statement implicitly import the module?

Because that's not how PHP works? In this pass I took the stance of not
ditching namespaces and instead incorporating them directly into this
module system. In any event, those namespaces will continue to exist
outside the module system for years. The idea of ditching namespaces
within the modules was met with immediate resistance.

In real life, my code is going to be in a module/framework and I'm going
to need to render it there. This example of exporting a dependency also
kinda breaks encapsulation principles, and even though it is an example,
things like this end up in documentation of a feature and cause all kinds
of bad practices (like Symfony and anemic objects).

I actually thought about that after typing this and as I was going to
sleep. How this should work is the symbols imported by a module are only
visible in that module. If two modules share a dependency the resolution
must allow them to use different versions, especially if they are different
major versions as there might be a BC break in their shared dependency
which is the reason one module uses the older version

One of the first things I do in a composer.json file is remove polyfills
through the replace key. It's unnecessary, annoys me in my IDE with having
multiple classes of the same name, and hides the fact that I should
probably install an extension for better performance. How do we do that
with this new setup?

Go has replace directives in go.mod which you're welcome to look into, but
exploring that level of detail is premature.

In fact, it is worth pointing out that how would this system work with
polyfills in-general?

Not yet it isn't.

So ... if we want to round, we have to use import @math and then we can
call the global round() function?

No one said anything about removing php.ini extension directives to
globally install extensions.

Or if we want to use DateTimeImmutable we have to add import @date? That
seems like a step in the wrong direction since most people don't even know
that most (if not all) global library functions come from extensions -- and
virtually nobody knows the name of each extension and what functions they
have. Also, installing extensions is not 100% straightforward as some
environments need to use pecl, some need to use OS package managers.

I don't understand this contrarian hostility. As you say, the current
system isn't straightforward. I'm proposing something that is
straightforward and your reaction is to go on about how not straightforward
the current system like it is highly relevant to the proposal. It isn't.
What is relevant is this - composer packages that require specific
extensions aren't as portable as those without such requirements as their
incorporation requires jumping through extra hoops. Wouldn't it just be
better if a module could call out its extension dependencies and have PHP
be able to get them installed WITHOUT asking the user to fire up PECL or
modify the php.ini file?

4 months ago by Michael Morris — view source

unread

However, be aware that in a Go project repo you are likely to have only
one go.mod — or multiple if you have numerous CLI apps being generated —
whereas every directory with Go code is a package (which I think is
equivalent to what you are calling "module."

In my examples I have a local developed module being consumed by a project
(the index.php file). Trying to keep it simple in this early sketch out.

So I think your use of them here is conflating the two concepts. One is a
project-wide concept and the other is a "package" concept.

I may well be. I'm looking for something that makes sense in PHP.
Namespaces, for good or ill, are a part of php, which is why the php.mod in
my example declares a namespace, not a package.

Also, it is problematic to have php.mod and php.sum because web
servers would serve them if not carefully configured hence why I went with
a leading dot, e.g. .php.module

This is a tree detail. Working on the forest overall right now. Not that
it's wrong, but leading dots to hide files is a .nix feature that doesn't
work on Windows (though applications ported from .nix to windows often
continue to honor the convention).

Aside from being familiar per Javascript, what is the argument to
requiring the import of specific symbols vs just a package import, e.g.:

<?php

import "./src/mymodule"

mymodule->twig->render('index', ['name' => 'World']);

To me is seems to just add to boilerplate required. Note that having mymodule everywhere you reference twig makes code a lot more
self-documenting, especially on line 999 of a PHP file. 🙂

PHP's variable table and symbol table are entirely separate for historical
reasons. Plenty of people on this list can explain how and why, but suffice
to say namespace declarations have no effect on variables, and variables
declared outside functions go into the global scope - which is a real
trainwreck of a place in long lived applications. Wordpress, for example,
has a FRIGHTENING number of global variables, and they aren't namespaced
(they are prefixed, but that only goes so far).

Modules have their own variable scope. They don't affect the global scope
at all and I don't think they should be able to import globals at all with
the global keyword, but that sort of thing can be discussed later. They
are also going to need their own symbol scope in case one module needs to
run an older version of a dependency it would otherwise share with another
module in the same project because there is a BC break between the two
dependencies.

That said, I wonder if incorporating versioning does not make the scope of
modules too big to complete?

In my experience it's best to get a roadmap in place - which is what we're
doing here - and THEN scope out the roadmap and determine what pieces go in
over multiple versions

I don't think it is wise to intertwine this concept of modules with
namespaces like that, but I am replied out for the night. :-)

I'm not sure we can completely abandon the concept of namespaces so in this
version of the proposal I incorporated them since, in the initial ramble I
ignored them. Where they land is as of yet an open question.

4 months ago by Michael Morris — view source

unread

Hello all. This is a ramble of an idea that's managed to run around my
head for a few days now. It isn't fully formed, but I've ran the thought
experiment as far as I can on my own and want to share it with all of you.

If you got this far, thank you. This overall idea to take one of the
better things to happen to JavaScript in the last decade and incorporate it
into PHP has been bothering me for awhile so I figured I'd share. I don't
know how much merit there is to this though.

I don't think PHP needs to take many if any lessons from JS to be honest.
JS evolved the way it did to solve its own set of problems, and likewise so
has PHP. I'm not really clear from reading the whole thing what problems
this loose bag of ideas is even intended to solve, really, but from what I
have gathered it seems to me more like what you want is a transpiler a la
Typescript and that this would achieve whatever you're trying to achieve.
Personal take, but if there's one thing I don't want for PHP it's to become
[any more] of a garbled mess of different styles and different ways of
achieving the same fundamental abstractions and results. Composer and PSR-4
have become de facto standards, they're very good standards, we've achieved
remarkable near unity in their adoption and whatever few inconveniences
remain through the application of those standards in certain cases are
minor trade-offs (not having multiple interfaces in the same file, etc.)

Near universal unity?? You're forgetting Wordpress, which has massive PHP
market share (more than 50% of PHP backed websites - well more than
depending on which survey you cite) and DOES NOT USE COMPOSER. And it DOES
NOT USE PSR-4 either.

Composer is wonderful as a userland solution to a problem the Internals
team has failed to solve, but such a critical problem as package
management being mostly solved in userland using a configuration file
(composer.json) written in another programming language entirely is frankly
an embarrassment in my opinion.

A JS-inspired subset of PHP within PHP, supported by invocation through
new keywords and handed off to a separate parser seems like it's asking for
trouble at every level from implementation to maintenance to userland to
dependency management...

What is asking for more trouble is to stagnate, sit-on-hands, and twenty
years from now PHP will be where COBOL is today - a niche programming
language that was once widely used, but reviled, hated, and the current
generation of programmers working feverishly to remove it entirely. Not to
mention the butt of an awful lot of jokes.

No one is going to post to this list with a perfect plan to solve package
management in CORE. Not in userland - we've got that - in CORE. That is
a feature PHP lacks that is present in Java, NodeJS (not clientside JS, a
distinction needs to be made), C#, Python and Go (likely others, but those
are the ones I've used and whose implementations I am familiar with on at
least a cursory level.)

And yes, JavaScript has a carnival of problems. Most of those problems do
not apply to PHP. But I'm starting from there because that's what I would
expect the majority of the PHP community to be familiar with - especially
when you include the large forgotten WordPress legion who don't have a clue
what your beloved composer is.

And I also don't appreciate the notion that any solution proposed here
should be 100% perfect and ready for implementation day one. That's never
going to happen. But I'm not willing to join several talented programmers
that have spent a month to a year getting a working implementation into
place before submitting an RFC only to have it shot down after work has
been done. Cause frankly, I think some of y'all are only here to shoot
ideas down and never come up with new ones.

And while I'm no expert on the underlying engine there are problems with it
that have risen to the surface that may never be solvable except to move to
a subset language. Like, why in Hell does PHP need THREE scope resolution
operators when every other damn programming language I've ever seen gets by
with ONE. And then there's having variables be required to have a $ prefix
and exist on their own symbol table unaffected by namespace. And I could go
on from there, but I don't want to belabor such points like the haters of
the language do because I am willing to live with them if they must be
there. I want to improve things if I can though.

Some of you just want to furiously defend the status quo with religious
zeal. And that is not helpful. That just drives people off.

4 months ago by David Gebler — view source

unread

Near universal unity?? You're forgetting Wordpress, which has massive PHP
market share (more than 50% of PHP backed websites - well more than
depending on which survey you cite) and DOES NOT USE COMPOSER. And it DOES
NOT USE PSR-4 either.

Composer is wonderful as a userland solution to a problem the Internals
team has failed to solve, but such a critical problem as package
management being mostly solved in userland using a configuration file
(composer.json) written in another programming language entirely is frankly
an embarrassment in my opinion.

Given WP has yet to adopt Composer or PSR-4 standards, how likely do you
think it is that this particular project will be quicker to adopt any, say,
PHP 10 "user modules" feature along the lines of what you've proposed?

What is asking for more trouble is to stagnate, sit-on-hands, and twenty
years from now PHP will be where COBOL is today - a niche programming
language that was once widely used, but reviled, hated, and the current
generation of programmers working feverishly to remove it entirely. Not to
mention the butt of an awful lot of jokes.

Mmm, been hearing that one for the last twenty years, yet here we are. And
the improvements to the language in that time have been innumerable, but
each one has served as a solution to a clear problem statement.

No one is going to post to this list with a perfect plan to solve package
management in CORE. Not in userland - we've got that - in CORE. That is
a feature PHP lacks that is present in Java, NodeJS (not clientside JS, a
distinction needs to be made), C#, Python and Go (likely others, but those
are the ones I've used and whose implementations I am familiar with on at
least a cursory level.)

None of those languages have better inherent support for packages than PHP,
just different ways of doing it. PHP's way is namespaces and autoloading
and while there's a good case that if we were designing the language from
scratch today, these might be a couple of just many things all of us might
want to do differently, ultimately all these things are just variants of
loading code symbols into scope. These languages all have separate tools to
manage third party dependency libraries (more than one competing with each
other, in some cases). Composer compares more to pip or npm or maven, not
Java packages or modules in Python, JS, etc.

For me, bottom line is I don't have any problem today managing or
installing versioned vendor code in my projects, I don't have a problem
breaking my project file structure down into clear modules and I don't have
a problem referencing those modules from other modules. Nor is the way I do
those things different to how any other PHP project does those things (be
that via Composer, another custom autoloading function, a series of require
statements, or whatever).

And I also don't appreciate the notion that any solution proposed here
should be 100% perfect and ready for implementation day one. That's never
going to happen. But I'm not willing to join several talented programmers
that have spent a month to a year getting a working implementation into
place before submitting an RFC only to have it shot down after work has
been done. Cause frankly, I think some of y'all are only here to shoot
ideas down and never come up with new ones.

What I'm saying to you is that you need something much more comprehensive,
well thought out and well justified to be in a position to even have a
conceptual, ready-to-consider RFC. You don't need to have a working
implementation on day one but you need to be proposing something coherent
and with clear benefits, which I don't think this is today.

4 months ago by Mike Schinkel — view source

unread

Near universal unity?? You're forgetting Wordpress, which has massive PHP market share (more than 50% of PHP backed websites - well more than depending on which survey you cite) and DOES NOT USE COMPOSER. And it DOES NOT USE PSR-4 either.

Composer is wonderful as a userland solution to a problem the Internals team has failed to solve, but such a critical problem as package management being mostly solved in userland using a configuration file (composer.json) written in another programming language entirely is frankly an embarrassment in my opinion.

Given WP has yet to adopt Composer or PSR-4 standards, how likely do you think it is that this particular project will be quicker to adopt any, say, PHP 10 "user modules" feature along the lines of what you've proposed?

As someone who worked with WordPress as a plugin developer for about 10 years, I would say WordPress would be HIGHLY likely to adopt modules if modules could possibly integrate into WordPress without WordPress having to make the nature of wholesale changes that you are others are objecting to related to PHP. Note they have yet to adopt namespaces in any significant way (or at least if they did it has been very recent.)

I was heavily involved in a trac discussion where people were proposing to use Composer for WordPress to manage plugins and their dependencies. I pointed out that Composer is a build-time solution for PHP where WordPress has no concept of build-time and that all configuration and 3rd party package "deployment" is done using a running WordPress install, and most often by end-users how have no clue how to resolve a conflict in dependencies (such as two plugins is same function name.) Consequently I argued that using Composer for WordPress as proposed was a non-starter.

That trac ticket was opened eight years ago, and is still open with no action (because action without rewriting WordPress is impossible IMO.): https://core.trac.wordpress.org/ticket/36335

ALL THAT SAID, I do not see WordPress using modules as the new approach for plugins. Same problems would exist as for Composer unless PHP added "un_include" and "un_require" functionality. And even then I don't know if there would be enough of an impedance match to use modules for plugins.

However, what I do see WordPress doing is adding new features using modules, and I do see plugin developers embracing modules (and especially if one module could be substituted for another module at runtime without worrying about symbol naming conflicts.) #fwiw

Mmm, been hearing that one for the last twenty years, yet here we are. And the improvements to the language in that time have been innumerable

Here is a recent article that I think is insightful to review:

https://thenewstack.io/why-php-usage-has-declined-by-40-in-just-over-2-years/

None of those languages have better inherent support for packages than PHP, just different ways of doing it.

"Better" is a subjective term.

Let me give one objective criteria:

Number of directory entries that popular projects end up managing.

I am sure each of us could (and should) come up with additional objective criteria for evaluating approaches to packages/modules (this is copied from an earlier email and I have quoted it so I can add a new paragraph after it):

And the PHP language encourages a large amount of file and directory bloat.

One only need to compare the number of files in most PHP libraries to the number of files in JS or Go package to see that the nature of a language clearly does not influence.

To bring stats vs. opinion I asked ChatGPT what the two equivalent packages are to Symphony for JS and Go respectively and it suggested ExpressJS and Gin. So I cloned them to see the number of files and directories each has. From the root of each repo:

Project Files Dirs
Symfony: 12,504 2,162
ExpressJS: 259 87
Gin(GoLang): 145 30

So, does "fewer files" make it "better?" That would be hard to prove or disprove, but it is a metric developers can consider when evaluating approaches to packages/modules. They can ask themselves if they really want to have to deal with so many files and folders when programming, or if they would prefer only working with an order of magnitude fewer files and folders.

And only each individual developer can answer that question for themselves; they cannot answer that question for others.

PHP's way is namespaces and autoloading and while there's a good case that if we were designing the language from scratch today, these might be a couple of just many things all of us might want to do differently, ultimately all these things are just variants of loading code symbols into scope. These languages all have separate tools to manage third party dependency libraries (more than one competing with each other, in some cases). Composer compares more to pip or npm or maven, not Java packages or modules in Python, JS, etc.

One of the downsides of how PHP loads code is that the loading code runs in slower userland code — and for those who use a debugger — forces the developer to step into and then step out of the autoloader every time a new class or interface is loaded.

Is that bad? It is to me, but each developer must characterize those objective facts somewhere along the spectrum of good to bad for themselves.

For me, bottom line is I don't have any problem today managing or installing versioned vendor code in my projects, I don't have a problem breaking my project file structure down into clear modules and I don't have a problem referencing those modules from other modules.

"...that you are recognizing at the moment." (If you are being honest with your characterization.)

Those problems exists, you have just acclimated to them and no longer notice them, or maybe you never did.

What I'm saying to you is that you need something much more comprehensive, well thought out and well justified to be in a position to even have a conceptual, ready-to-consider RFC. You don't need to have a working implementation on day one but you need to be proposing something coherent and with clear benefits, which I don't think this is today.

Said another way — and this I 100% agree with you on, even if I lament it — the PHP internals mailing list is not the place to flesh out ideas and collaborate with others in the PHP community. That is unless you are already one of the primary contributors to PHP, and even then the primary contributors start out working on their ideas with others they know off-list. #it_is_what_it_is

-Mike

4 months ago by Rowan Tommins [IMSoP] — view source

unread

Composer is wonderful as a userland solution to a problem the Internals
team has failed to solve, but such a critical problem as package
management being mostly solved in userland

I don't think anyone failed to solve anything. Somebody came along and built a solution, it worked well, and people adopted it. In fact, so many people adopted it that we're in the happy place of not having competing tools, outside of application-specific plugin installers.

Its release cycle is independent of PHP's, which means you can get the latest version of Composer whatever version of PHP you're on. It's written in PHP at least in part because there are a lot more PHP programmers willing to maintain PHP tools than there are C programmers willing to maintain PHP tools.

If Composer (or some new package manager) was marked as an official part of the PHP project, I doubt it would make any difference to how people find it - users looking at existing projects find it because it's the first line of the installation instructions; new PHP users learning from scratch are probably following tutorials the official project has no control over, which either mention it or not.

using a configuration file
(composer.json) written in another programming language

JSON is not a programming language. It looks a bit like JS, because it was invented by a JS developer, but was explicitly designed as a cross-language data format.

Some other package managers use other data formats, like XML, or TOML. There probably are languages which use their own source code for that configuration, but I've not come across any.

As I mentioned on another reply, it would take an extremely good sales pitch for me to pay any attention to any attempt at a replacement for Composer, especially one that wasn't compatible with its huge library of existing packages.

If the sales pitch is "JSON is not the best format for package configuration", count me out.

The same goes for a lot of other suggestions in this thread: I'm going to need a pretty strong sales pitch to change my hierarchical namespaces to flat module imports - and more importantly, projects like Symfony and Laravel are going to need that sales pitch, before they break compatibility for thousands of applications built on their current packages.

Other parts of this thread are just random rants about things people don't like in PHP, which I have zero interest in, and have nothing whatsoever to do with modules or package management.

Rowan Tommins
[IMSoP]

4 months ago by Mike Schinkel — view source

unread

However, be aware that in a Go project repo you are likely to have only one go.mod — or multiple if you have numerous CLI apps being generated — whereas every directory with Go code is a package (which I think is equivalent to what you are calling "module."
In my examples I have a local developed module being consumed by a project (the index.php file). Trying to keep it simple in this early sketch out.

So I think your use of them here is conflating the two concepts. One is a project-wide concept and the other is a "package" concept.

I may well be. I'm looking for something that makes sense in PHP. Namespaces, for good or ill, are a part of php, which is why the php.mod in my example declares a namespace, not a package.

Also, it is problematic to have php.mod and php.sum because web servers would serve them if not carefully configured hence why I went with a leading dot, e.g. .php.module

This is a tree detail. Working on the forest overall right now. Not that it's wrong, but leading dots to hide files is a .nix feature that doesn't work on Windows (though applications ported from .nix to windows often continue to honor the convention).

Aside from being familiar per Javascript, what is the argument to requiring the import of specific symbols vs just a package import, e.g.:

<?php
import "./src/mymodule"

mymodule->twig->render('index', ['name' => 'World']);

To me is seems to just add to boilerplate required. Note that having mymodule everywhere you reference twig makes code a lot more self-documenting, especially on line 999 of a PHP file. 🙂

PHP's variable table and symbol table are entirely separate for historical reasons. Plenty of people on this list can explain how and why, but suffice to say namespace declarations have no effect on variables, and variables declared outside functions go into the global scope - which is a real trainwreck of a place in long lived applications. Wordpress, for example, has a FRIGHTENING number of global variables, and they aren't namespaced (they are prefixed, but that only goes so far).

Modules have their own variable scope. They don't affect the global scope at all and I don't think they should be able to import globals at all with the global keyword, but that sort of thing can be discussed later. They are also going to need their own symbol scope in case one module needs to run an older version of a dependency it would otherwise share with another module in the same project because there is a BC break between the two dependencies.

That said, I wonder if incorporating versioning does not make the scope of modules too big to complete?

In my experience it's best to get a roadmap in place - which is what we're doing here - and THEN scope out the roadmap and determine what pieces go in over multiple versions

I don't think it is wise to intertwine this concept of modules with namespaces like that, but I am replied out for the night. :-)

I'm not sure we can completely abandon the concept of namespaces so in this version of the proposal I incorporated them since, in the initial ramble I ignored them. Where they land is as of yet an open question.

After some private time spent documenting my thoughts on modules I realized my thoughts have diverged from your proposal, so rather than challenge any of your arguments I will just demure and work on my own contribution.

I discovered the need for a small new feature that would generally be incredibly useful but also could empower userland to create their own form of modules, and that feature proposal will be much smaller in scope compared to one for your concept of modules. It may or may not be orthogonal to the discussion you are leading.

-Mike

4 months ago by Michael Morris — view source

unread

I have no proposal. I'm brainstorming. Please don't step out of this
conversation as it has been enormously helpful.

On Sat, Jun 29, 2024 at 5:40 AM Mike Schinkel mike@newclarity.net
wrote:

However, be aware that in a Go project repo you are likely to have only
one go.mod — or multiple if you have numerous CLI apps being generated —
whereas every directory with Go code is a package (which I think is
equivalent to what you are calling "module."
In my examples I have a local developed module being consumed by a
project (the index.php file). Trying to keep it simple in this early sketch
out.

So I think your use of them here is conflating the two concepts. One is
a project-wide concept and the other is a "package" concept.

I may well be. I'm looking for something that makes sense in PHP.
Namespaces, for good or ill, are a part of php, which is why the php.mod in
my example declares a namespace, not a package.

Also, it is problematic to have php.mod and php.sum because web
servers would serve them if not carefully configured hence why I went with
a leading dot, e.g. .php.module

This is a tree detail. Working on the forest overall right now. Not that
it's wrong, but leading dots to hide files is a .nix feature that doesn't
work on Windows (though applications ported from .nix to windows often
continue to honor the convention).

Aside from being familiar per Javascript, what is the argument to
requiring the import of specific symbols vs just a package import, e.g.:

<?php
import "./src/mymodule"

mymodule->twig->render('index', ['name' => 'World']);

To me is seems to just add to boilerplate required. Note that having
mymodule everywhere you reference twig makes code a lot more
self-documenting, especially on line 999 of a PHP file. 🙂

PHP's variable table and symbol table are entirely separate for
historical reasons. Plenty of people on this list can explain how and why,
but suffice to say namespace declarations have no effect on variables, and
variables declared outside functions go into the global scope - which is a
real trainwreck of a place in long lived applications. Wordpress, for
example, has a FRIGHTENING number of global variables, and they aren't
namespaced (they are prefixed, but that only goes so far).

Modules have their own variable scope. They don't affect the global
scope at all and I don't think they should be able to import globals at all
with the global keyword, but that sort of thing can be discussed later.
They are also going to need their own symbol scope in case one module needs
to run an older version of a dependency it would otherwise share with
another module in the same project because there is a BC break between the
two dependencies.

That said, I wonder if incorporating versioning does not make the scope
of modules too big to complete?

In my experience it's best to get a roadmap in place - which is what
we're doing here - and THEN scope out the roadmap and determine what pieces
go in over multiple versions

I don't think it is wise to intertwine this concept of modules with
namespaces like that, but I am replied out for the night. :-)

I'm not sure we can completely abandon the concept of namespaces so in
this version of the proposal I incorporated them since, in the initial
ramble I ignored them. Where they land is as of yet an open question.

After some private time spent documenting my thoughts on modules I
realized my thoughts have diverged from your proposal, so rather than
challenge any of your arguments I will just demure and work on my own
contribution.

I discovered the need for a small new feature that would generally be
incredibly useful but also could empower userland to create their own form
of modules, and that feature proposal will be much smaller in scope
compared to one for your concept of modules. It may or may not be
orthogonal to the discussion you are leading.

-Mike

4 months ago by Michael Morris — view source

unread

So let's take another crack at this based on all the points raised in the
thread. This should also underline why I don't consider this an RFC - I am
iterating until we arrive at something that may be refinable into an RFC.
And I say we because without the aid of those in this conversation I would
not have arrived at what will follow.

Before I continue I would like to apologize for being somewhat irritable.
We're all here because we enjoy using this language and want to see it
improved and prevent bad changes. Opinions will differ on this and in the
heat of the moment of arguing a point things can get borderline.

Returning to a point I made earlier, Composer isn't used on Wordpress. I
went over to the Wordpress discussion list and read over why, because that
discussion provides clues to what kind of package management may be
adoptable. I think the largest point is that Wordpress should be usable
without ever resorting to using the command line. Yes, it does have a
command line tool - wp-cli - and it is powerful, but using it as an
administrator of a Wordpress site is not required.

The largest block to composer's inclusion in Wordpress is the inability to
run multiple versions of a module. Yes, it's a mess when this happens, but
if you're an end user, you just want your plugins to work. If one plugin
that no one has updated in a year that you're using is consuming version 2
of a package, you're gonna be annoyed at best if the module stops working
when you install a new plugin that is using version 3 of the same package
and has a BC break in it. Composer can't resolve this easily.

There are WordPress plugins that use composer - I have a couple in the
website I'm working on. But they accomplish the inclusion of composer by
redistributing the packages, and using a utility
called brianhenryie/strauss to monkey type the entire included package into
the plugin, changing the namespace of the entire package to something
different. The approach works, but it's ugly. In any event, the plugin
that results from this carries a copy of the code from packagist rather
than sourcing the code from packagist.

-- IMPORT --

The import statement is for bringing in packages. It needs to be able to
deal with:

Extensions - the existing and oldest of packages for PHP
PECL Extensions
Phar Packages
Composer Packages
PHP Modules - this is the new module system that has dominated the
conversation, but in this iteration it's going to be broken away from
import to some degree in this iteration.

Today we'll look just at composer.

Now import needs to load packages in a manner that allows different
versions to be run concurrently. A PHP application such as Wordpress should
be distributable without needing to use the command line. That is, if
WordPress leverages this in any way, they don't have to give up their
famous 10 minute quick install.

Some terms here to keep myself from getting lost (let alone anyone trying
to read this).

APPLICATION - This is the overall application - WordPress, Drupal,
BobbysFirstFramework, etc. - that is doing the import. This code is on the
root scope.
ROOT SCOPE - This is where the global variables and the namespaces as we
know them exist. Contrast this with
PACKAGE SCOPE - Each package brought in with import gets its own package
scope. This is a distinct behavior from Include/Require. I think each
package scope will need to be on its own request thread, but this is an
implementation detail I can't speak to with any authority. The goal is
whatever happens in a package stays in the package. If two different
packages want to define /foo(), they can.

When a package is imported the parser will look for a .php.mod file at
the root of the package. Among other things, this file details what type of
package it is and where to mount it by default in the namespace of the ROOT
SCOPE. So,

GIVEN a package with this .php.mod file

package MyModule

WHEN I issue this import in an application

import "MyModule";

THEN I should be able to access a method in that module with this code

\MyModule\foo();

Aliasing is an option - import "MyModule" as BeerModule will make the
methods accessible in the root with \BeerModule\foo();

Unlike require/include import is sensitive to the namespace it is in for
mounting. So

namespace Trees;

import "MyModule";

MyModule\foo(); // works

\Trees\MyModule\foo(); // needed from another namespace.

That said, with aliasing an absolute namespace for the module can be
assigned.

namespace Trees;

import "MyModule" as \MyModule;

MyModule\foo(); // works if my understanding of existing namespace
resolution rules is correct.
\MyModule\foo(); // also works.

Now, with that in place, let's crack a tougher nut - handling a composer
package. By default composer is designed to set up an autoloader, then
resolve symbol references as they come up. This works until you have two
packages that want the same symbol reference - which will most frequently
occur with incompatible versions of the same package. So our puzzle here
is how to allow composer to do its thing without rewriting it. We'll deal
with admittedly the hardest case first - importing a package whose
maintainers have taken no action to make it compatible with this new system.

import "composer://packagist.org/packages/twig/twig#v3.10.3"; as
TemplateEngine

The reason for that alias and not "Twig" is because the mounting point
comes before the internal namespace of the file. This is unavoidable with
this scheme

The URL there is "loader://package_url". PHP by default will know what the
composer loader is. It will look to see if the user has globally installed
composer already and use that, otherwise it will locally install composer
for the project, initialize it, download the package and have composer
resolve the package ending in setting up an autoloader that is only invoked
within that package.

Application configuration can make a lot of this go away. So let's step
away from the import statement itself to look at that.

-- APPLICATIONS --
Applications can configure how they store their packages, but in the
absence of such PHP will use some logical default behaviors. We've already
looked at one, the loading of a composer package, but we have a long ugly
import as a result.

Most PHP applications have a single point of entry, and part of that is the
establishment of a cwd (current working directory). When PHP loads a file
it will look for a .php-packages directory in the current working
directory and if one doesn't exist it will make one the first time the
import statement is invoked (so code not using import will not have this
directory created). It is here that the package downloads land. We can
also choose to go ahead and make this directory ourselves and place
.php.mod in that directory. Let's look at what one might look like for
Drupal, which already uses composer.

package Drupal;

php: 10

registry packagist.org/packages composer

imports (
phar://getcomposer.org/composer.phar
)

init (
composer install
)

require (
./vendor/autoload.php
)

Now, composer is a known and popular quantity, so the imports, init and
require directives can probably be baked into PHP, but if they ever change

or if a competitor to composer shows up like yarn did to npm then there
needs to be a way to set it up.

Also, for the moment I'm using go.mod's format because it feels
the cleanest. The exact format of this file - whether it's yaml, json,
toml, .ini or whatever else, is a discussion for another day. Key in on the
type of information that needs to be relayed here, not how it's relayed.

Importantly, because this .php.mod file is at the top level of the
application's .php-packages directory it affects the behavior of the ROOT
SCOPE.

The php directive gives the minimum php version for the application.

The registry directive sets the registry to packagist and sets the loader
for that registry to composer. Multiple registries can have separate
loaders.

The imports directive loads composer using the default phar loader. We use
an absolute path because we don't have a phar registry. This particular
call could be baked in due to composer's popularity.

The init directive runs the first time the application runs, just before
any file in the application is parsed. The .php.sum file will bookmark the
last time the init has run and there will likely need to be a mechanism to
force it's rerun.

The require directive requires these files once before starting the
application's first file.

Assume we move our existing composer.json file into .php-packages, what
then? We gain the following:

Composer install will be ran for us, without using the cli.
The autoloader will be set up for us without explicitly requiring it
anywhere.
We can install an alternate version of packages into their own package
scope

import "twig/twig#v2.5" as OldTwig;

Or again, if we are in a namespace we can import to that namespace.

I'll stop here cause this is a healthy chunk to absorb and I just spent 4
hours thinking on this as I wrote it out and I'm tired. If this was in the
language today, Wordpress plugins could start using it without fear of
mucking each other up or messing with Wordpress core. But that's a
discussion for later.

4 months ago by Rob Landers — view source

unread

So let's take another crack at this based on all the points raised in the thread. This should also underline why I don't consider this an RFC - I am iterating until we arrive at something that may be refinable into an RFC. And I say we because without the aid of those in this conversation I would not have arrived at what will follow.

Before I continue I would like to apologize for being somewhat irritable. We're all here because we enjoy using this language and want to see it improved and prevent bad changes. Opinions will differ on this and in the heat of the moment of arguing a point things can get borderline.

Returning to a point I made earlier, Composer isn't used on Wordpress. I went over to the Wordpress discussion list and read over why, because that discussion provides clues to what kind of package management may be adoptable. I think the largest point is that Wordpress should be usable without ever resorting to using the command line. Yes, it does have a command line tool - wp-cli - and it is powerful, but using it as an administrator of a Wordpress site is not required.

The largest block to composer's inclusion in Wordpress is the inability to run multiple versions of a module. Yes, it's a mess when this happens, but if you're an end user, you just want your plugins to work. If one plugin that no one has updated in a year that you're using is consuming version 2 of a package, you're gonna be annoyed at best if the module stops working when you install a new plugin that is using version 3 of the same package and has a BC break in it. Composer can't resolve this easily.

There are WordPress plugins that use composer - I have a couple in the website I'm working on. But they accomplish the inclusion of composer by redistributing the packages, and using a utility called brianhenryie/strauss to monkey type the entire included package into the plugin, changing the namespace of the entire package to something different. The approach works, but it's ugly. In any event, the plugin that results from this carries a copy of the code from packagist rather than sourcing the code from packagist.

-- IMPORT --

The import statement is for bringing in packages. It needs to be able to deal with:

Extensions - the existing and oldest of packages for PHP

PECL Extensions

Phar Packages

Composer Packages

PHP Modules - this is the new module system that has dominated the conversation, but in this iteration it's going to be broken away from import to some degree in this iteration.

Today we'll look just at composer.

Now import needs to load packages in a manner that allows different versions to be run concurrently. A PHP application such as Wordpress should be distributable without needing to use the command line. That is, if WordPress leverages this in any way, they don't have to give up their famous 10 minute quick install.

Some terms here to keep myself from getting lost (let alone anyone trying to read this).

APPLICATION - This is the overall application - WordPress, Drupal, BobbysFirstFramework, etc. - that is doing the import. This code is on the root scope.

ROOT SCOPE - This is where the global variables and the namespaces as we know them exist. Contrast this with

PACKAGE SCOPE - Each package brought in with import gets its own package scope. This is a distinct behavior from Include/Require. I think each package scope will need to be on its own request thread, but this is an implementation detail I can't speak to with any authority. The goal is whatever happens in a package stays in the package. If two different packages want to define /foo(), they can.

When a package is imported the parser will look for a .php.mod file at the root of the package. Among other things, this file details what type of package it is and where to mount it by default in the namespace of the ROOT SCOPE. So,

GIVEN a package with this .php.mod file

package MyModule

WHEN I issue this import in an application

import "MyModule";

THEN I should be able to access a method in that module with this code

\MyModule\foo();

Aliasing is an option - import "MyModule" as BeerModule will make the methods accessible in the root with \BeerModule\foo();

Just to challenge you a bit here. The language already has use for this. Does use stay or is it replaced with import and why not just change the meaning of use in a package context?

Unlike require/include import is sensitive to the namespace it is in for mounting. So

namespace Trees;
import "MyModule";

MyModule\foo(); // works

\Trees\MyModule\foo(); // needed from another namespace.

That said, with aliasing an absolute namespace for the module can be assigned.

namespace Trees;
import "MyModule" as \MyModule;

MyModule\foo(); // works if my understanding of existing namespace resolution rules is correct.
\MyModule\foo(); // also works.

Now, with that in place, let's crack a tougher nut - handling a composer package. By default composer is designed to set up an autoloader, then resolve symbol references as they come up. This works until you have two packages that want the same symbol reference - which will most frequently occur with incompatible versions of the same package. So our puzzle here is how to allow composer to do its thing without rewriting it. We'll deal with admittedly the hardest case first - importing a package whose maintainers have taken no action to make it compatible with this new system.

import "composer://packagist.org/packages/twig/twig#v3.10.3"; as TemplateEngine

The reason for that alias and not "Twig" is because the mounting point comes before the internal namespace of the file. This is unavoidable with this scheme

The URL there is "loader://package_url". PHP by default will know what the composer loader is. It will look to see if the user has globally installed composer already and use that, otherwise it will locally install composer for the project, initialize it, download the package and have composer resolve the package ending in setting up an autoloader that is only invoked within that package.

Adding this after I just wrote the below book. It started out simple enough and it drives me crazy when people crash my proposal with a counter-proposal. So, by all means, take it with a grain of salt ... I kinda went overboard thinking about it.

I think composer and friends are a moot point. If we go a bespoke way for everything, we end up with a mess. What about creating "hooks" that things like composer can "register" an installer at? For example, we could define a "WELL_KNOWN/installers/composer/hooks.json" (I'm gonna steal a bunch of ideas from kubernetes from here on), where WELL_KNOWN is some engine-specific directory (like where the php.ini file is). Basically, any installer can register an installer by creating a directory in WELL_KNOWN/installers of which an installer might look like the following for composer:

WELL_KNOWN/
installers/
composer/
hooks.json
composer.phar
cache

and hooks.json could have a schema of something like:

{
"name": "composer",
"version": "4.5",
"executable": "./composer.phar",
"scheme": "composer",
"command": "install-package-module"
}

Then the engine can scan these directories and read each hooks.json. Then when it gets to your example above, it sees the scheme "composer" in the URL, looks for an installer with that name, and calls the executable with some arguments (the command, the "URL" aka the package, current directory, etc). So, it might call composer with something like:

/WELL_KNOWN/installers/composer/composer.phar install-package-module packagist.org/packages/twig/twig#v3.10.3 /app/public

The command is expected to dump the file to stdout -- which PHP then pipes to wherever it is supposed to go (as you mention below).

At this point, the user may not know a single thing about composer or how it works, or anything, really. As far as they are concerned, they said they wanted an import and they got one. However, there still exists a bunch of composer-specific scripts, etc. In this case, we make new WELL_KNOWN types. For example, we can have "script-runners" that can be registered so you can just do "php run composer test" and it will run composer test for you. If you want something shorter, you can just add alias composer="php run composer" to your shell and bob's your uncle.

This is basically how kubernetes handles networking, disks, etc. so that the entire thing is completely swappable and extendable. So, we know the idea is sound and "just works," we just need to customize it for php.

— Rob

4 months ago by Mike Schinkel — view source

unread

I think composer and friends are a moot point. If we go a bespoke way for everything, we end up with a mess. What about creating "hooks" that things like composer can "register" an installer at? For example, we could define a "WELL_KNOWN/installers/composer/hooks.json" (I'm gonna steal a bunch of ideas from kubernetes from here on), where WELL_KNOWN is some engine-specific directory (like where the php.ini file is). Basically, any installer can register an installer by creating a directory in WELL_KNOWN/installers of which an installer might look like the following for composer:

There is such a thing as a /.well-known/ URI thanks to RFC 8615:

https://datatracker.ietf.org/doc/html/rfc8615
https://en.wikipedia.org/wiki/Well-known_URI

Not sure if you are envisioning a web-accessible hooks.json or not.

#justfyi

-Mike

4 months ago by Mike Schinkel — view source

unread

So let's take another crack at this based on all the points raised in the thread. This should also underline why I don't consider this an RFC - I am iterating until we arrive at something that may be refinable into an RFC. And I say we because without the aid of those in this conversation I would not have arrived at what will follow.

Before I continue I would like to apologize for being somewhat irritable. We're all here because we enjoy using this language and want to see it improved and prevent bad changes. Opinions will differ on this and in the heat of the moment of arguing a point things can get borderline.

Returning to a point I made earlier, Composer isn't used on Wordpress. I went over to the Wordpress discussion list

What?!? No good WordPressista would be caught dead using an uncapitalized "p."

https://developer.wordpress.org/reference/functions/capital_p_dangit/

Have you no shame? ;-)

and read over why, because that discussion provides clues to what kind of package management may be adoptable. I think the largest point is that Wordpress should be usable without ever resorting to using the command line. Yes, it does have a command line tool - wp-cli - and it is powerful, but using it as an administrator of a Wordpress site is not required.

But seriously, I think the issue is not the CLI usage as Composer could be included as library in PHP if it made sense. The issue is different plugins loading different versions of the same dependencies, something that Composer delegates responsibility to the developer to resolve because PHP does not provide package-level scoping...

PACKAGE SCOPE - Each package brought in with import gets its own package scope. This is a distinct behavior from Include/Require. I think each package scope will need to be on its own request thread, but this is an implementation detail I can't speak to with any authority. The goal is whatever happens in a package stays in the package. If two different packages want to define /foo(), they can.

Yes, package-level scoping would be useful.

However, maybe we could first focus on how to get "package" scope as a much smaller scope as a starting point? No pun intended on "scope." :-)

Constrained scoped should be orthogonal to namespaces, and also thus support namespaces as both Rowan and Larry have explicitly said they want.

When a package is imported the parser will look for a .php.mod file at the root of the package. Among other things, this file details what type of package it is and where to mount it by default in the namespace of the ROOT SCOPE.

I (obviously) like the idea of a .php.mod or differently named file that PHP would get site and/or directory information from. Such as pre-compiled equivalent to a `.phar file, but then it would not be a text file as you have proposed. I do think having the version of PHP expected would be interesting which would define the author's intent — assuming they maintained it — and could allow multiple versions of PHP running on the same server.

However, I do agree with Rowan that trying to create new functionality that differs from Composer is likely destined to failure. And if PHP allowed package-level scoping then Composer could likely resolve the issues that WordPress faces without needing a new way to manage packages.

(I really wish Composer supported direct package reference via URL rather than only supporting registries. But I digress.)

-Mike

4 months ago by Arvids Godjuks — view source

unread

So let's take another crack at this based on all the points raised in the
thread. This should also underline why I don't consider this an RFC - I am
iterating until we arrive at something that may be refinable into an RFC.
And I say we because without the aid of those in this conversation I would
not have arrived at what will follow.

snip

TL;DR: As a userland developer, in my opinion, this is just a downgrade
from what we have now. Enhance namespaces to have the ability to have
internal/private classes, interfaces, enums and constants. That's about it.

Autoloading is one of the best killer features of PHP - love it or hate it

it's your personal preference. I've seen a sizeable chunk of developers
that come from other languages discover PHP's autoloading and their minds
just get blown. Performance has not been an issue for a long time due to
opcache and all the optimizations that have been done to it and ability to
preload bytecode. Then there are things like FrankenPHP, Swoole, ReactPHP
and others that entirely sidestep that issue. And then there's the active
development of JIT engine - just let the people working on the
implementation time to cook.
It works, worked for a long time and there are not so many things wrong
with it to entirely upend the whole ecosystem and split the language.
Here's your HARD REMINDER about Python 2 => Python 3 and how that went and
is still somewhat ongoing. Sometimes copying things from other places is
just wrong and does not fit the ethos of the language you want to change.
PHP always was and still is "doing it's thing". Practicality is on the high
list of priorities here and that is why many of us have chosen the language
and stuck with it 2 decades in.

The only thing I want to be added to namespaces is the ability to define
internal classes, interfaces, enums and constants so that when you write
application code, the internals of packages and your own code's "private"
details do not leak to the business level layer via autocomplete
suggestions and in general reducing the cognitive load of that aspect. And
there are obvious advantages of locking down the internals of a library so
people do not abuse it and then complain when things get broken.

And i don't even want to comment how you want to bring composer
functionality into PHP code and make it a weird combo of PHP, composer,
docker-compose.yml - insert the "WHY?!" meme here.
What I love about PHP ecosystem is for the most part, every tool has its
main job and it does that job exceedingly well. PHP core is developing the
engine and doing a great job. Composer is doing its job and it is the best
package management tool out there. And so on - we have one of the best
quality ecosystems around because things are not jammed into each other and
trying to create a "supertool". There is something to be said about
Internals refusing to if not endorse, at least acknowledge the popular
tooling and lean on it more (like rector for upgrades and so on), but that
is a completely different discussion. The PECL is being worked on to be
replaced, so we can't even raise that point any more - PHP Foundation is
taking care of it.

Arvīds Godjuks
+371 26 851 664
arvids.godjuks@gmail.com
Telegram: @psihius https://t.me/psihius

4 months ago by Mike Schinkel — view source

unread

TL;DR: As a userland developer, in my opinion, this is just a downgrade from what we have now. Enhance namespaces to have the ability to have internal/private classes, interfaces, enums and constants. That's about it.

Please note my comments that follow do not mean I am in support of this package proposal as presented.

Autoloading is one of the best killer features of PHP - love it or hate it - it's your personal preference.

Two really solid reasons to hate autoloading as implemented in PHP:

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders, it means there can be buggy autoloaders, and it means that when using XDEBUG every time a new symbol is found when the developer is single-step debugging the developer will be dropped into the autoloader and then best case they then immediately trace out. All of these aspects a major PITA and time waster and make debugging more exhausting than it needs to be.
Autoloading effectively necessitates that every symbol be in its own separate file. This needlessly bloats number of files and directories by more than an order of magnitude — see my numbers from recent discussion — and that also mean related code is located farther away from other related code. This can be worked around but the workarounds I've seen are all fragile and unable to be generic, and few 3rd party packages do this.

I've seen a sizeable chunk of developers that come from other languages discover PHP's autoloading and their minds just get blown.

It is unclear to me if by saying their "minds just get blown" if that means you think they see it as a positive or negative?

As a developer who spent a decade in PHP and then branched out and added Go to my repertoire I can tell you one of the nicest differences I experienced was not having to deal with an autoloader during debugging, and not being so constrained was to have to create a new file for every symbol. Go projects need an order of magnitude fewer files. It is just so much easier to grok the source code of a Go project compared to a PHP because of this one simple fact. Now when I program in PHP I find myself constantly cursing the fact that I have to deal with the autoloader.

BTW, I know Go is a pre-compiled language unlike PHP, but that does not necessarily preclude PHP from having a better solution for code loading and organization.

Performance has not been an issue for a long time due to opcache and all the optimizations that have been done to it and ability to preload bytecode. Then there are things like FrankenPHP, Swoole, ReactPHP and others that entirely sidestep that issue. And then there's the active development of JIT engine - just let the people working on the implementation time to cook.

This reads to me like Stockholm syndrome, e.g. "My captors still hold me captive, but they no longer beat me every day."

It works, worked for a long time and there are not so many things wrong with it to entirely upend the whole ecosystem and split the language. Here's your HARD REMINDER about Python 2 => Python 3 and how that went and is still somewhat ongoing.

Totally agree on that.

-Mike

4 months ago by Dusk — view source

unread

Autoloading effectively necessitates that every symbol be in its own separate file.

How so? While Composer "recommends" PSR4 autoloading with one class per file, other configurations are entirely possible, either with Composer's autoload.classmap (which uses a precomputed class -> filename table), or with a custom autoloader.

4 months ago by Mike Schinkel — view source

unread

Autoloading effectively necessitates that every symbol be in its own separate file.

How so? While Composer "recommends" PSR4 autoloading with one class per file, other configurations are entirely possible,

I spoke inartfully, sorry.

The culture of projects throughout the PHP ecosystem has resulted in most projects being one class per file, and there have not evolved any defecto-standard tools for creating those classmaps further ensuring PSR4 ubiquity.

either with Composer's autoload.classmap (which uses a precomputed class -> filename table), or with a custom autoloader.

And, refer back to reason #1 to hate autoloading in PHP.

But that is why interoperability in PHP world is so high. When it was introduced, it allowed enabling autoloading for most code bases out there regardless of what their structure was and still is. Sure, you have to be careful how you do it, but that is also not really a userland concern - most of those have been implemented once and then almost never touched :) The community has settled on a general approach since naturally and that's how vast majority of people write their code. Buggy code is buggy code, it really has not much to do with the autoloading and with how people write buggy code :) You are blaming the hammer for user trying to nail the screw into wood :)

Two things can be true at the same time. Something can have a historical benefit but not be ideal moving forward when compared to potential future alternatives.

The major benefit that autoloading has today is inertia. Yes, that's a huge benefit. But that does not make autoloading beneficial.

You can already sidestep autoloader by adding a require statement to any file and loading everything without triggering autoload.

There is not a standard way to sidestep the autoloader that has majority support in 3rd party packages. It's a roll-your-own thing.

It's far more powerful in userland because it allows people to do whatever they want with it if they do not like the standard autoloader(s).

The only reason people think about how they want autoloading to work is because they have to. Technology is supposed to evolve to eliminate rote and low-value add tasks.

By the same token I could say we should introduce the ability in userland for people to modify the implementations of standard library functions so people could do whatever they want with it if they do not like the standard implementations.

But clearly that latter would be crazy talk.

That's just how PSR-4 standard was written and works, and it's a good default for a reason. On small projects - sure, I can see it being overkill. But I haven't worked on a small project in a decade and I rarely have that little code in one file that I would want to stuff all together in one file. I do use PHPStorm, so it is very adapted to how most PHP projects are and provides excellent navigation abilities that fit the PSR-4 and that way of structuring projects. Vast part of the community uses it. It's a standard in a lot of companies for a reason.

Again, I will say this is Stockholm syndrome.

See my reply to autoloading things above - you can eliminate the autoloader triggering easily in probably 15-30 minutes flat.

But to do so there is not one single way that is reality standard. So I have to own maintenance of that low-value code.

There are aspects to this that go beyond just technical aspects. Wherever when implementing autoloading people were secret geniuses, stumbled into it accidentally or just had practical needs themselves that they just implemented in this way - it has been a major turning point in PHP's life and has transformed the ecosystem into what it is today together with the rise of the composer package manager. I have seen people talk about visibility in namespaces, package concept and all that, but every time it was building upon existing autoloading mechanics - some adding capabilities to them, modifying them, having new type of autoloader. But I have never encountered anyone in this community in 20 years i have been an active part of it to propose a radical change like this

That is probably because you all have been working mostly in PHP and not experiencing better alternatives. BTW, NodeJS/Javascript is NOT a better alternative, IMO.

Besides, what I proposed earlier was not a "radical change." Characterizing something as "radical" is just a rhetorical tactic used to discredit said thing. What I proposed was very incremental and would allow all existing code including autoloaders to co-exist with a module/package concept. Not radical.

and I'm fairly certain after keeping up with the thread that it is almost universally not what people want. Most people just want the toolbox be "finished" so to speak, not get a completely new one in addition that has no compatibility with the old one.

I get it. I am no longer proposing an alternative to the autoloader. PHP developers are comfortable with autoloading and that is that.

But that does not mean that I cannot tell you and others the emperor has no clothes in hopes that people eventually see that there can be better alternatives.

-Mike

4 months ago by Jordan LeDoux — view source

unread

and I'm fairly certain after keeping up with the thread that it is
almost universally not what people want. Most people just want the toolbox
be "finished" so to speak, not get a completely new one in addition that
has no compatibility with the old one.

I get it. I am no longer proposing an alternative to the autoloader. PHP
developers are comfortable with autoloading and that is that.

But that does not mean that I cannot tell you and others the emperor has
no clothes in hopes that people eventually see that there can be better
alternatives.

-Mike

I'm not sure that constantly reiterating a point that everyone already
knows but simply disagrees with is productive for the list, considering
that the objection boils down to "but I don't like it" instead of "here are
the concrete technical drawbacks". All of the objections you had seemed to
be from the perspective "but what if the developer is only allowed 100
files on disk and only uses notepad to edit the code?" I don't think those
are technical drawbacks personally, I think those are developers that need
to at least start programming like they are living in 2005.

Jordan

4 months ago by Mike Schinkel — view source

unread

and I'm fairly certain after keeping up with the thread that it is almost universally not what people want. Most people just want the toolbox be "finished" so to speak, not get a completely new one in addition that has no compatibility with the old one.

I get it. I am no longer proposing an alternative to the autoloader. PHP developers are comfortable with autoloading and that is that.

But that does not mean that I cannot tell you and others the emperor has no clothes in hopes that people eventually see that there can be better alternatives.

-Mike

I'm not sure that constantly reiterating a point that everyone already knows but simply disagrees with is productive for the list, considering that the objection boils down to "but I don't like it" instead of "here are the concrete technical drawbacks". All of the objections you had seemed to be from the perspective "but what if the developer is only allowed 100 files on disk and only uses notepad to edit the code?" I don't think those are technical drawbacks personally, I think those are developers that need to at least start programming like they are living in 2005.

Given the paragraph I wrote that started with "I get it..." I was planning on not making comment on topic.

All of the objections you had seemed to be from the perspective "but what if the developer is only allowed 100 files on disk and only uses notepad to edit the code?" I don't think those are technical drawbacks personally, I think those are developers that need to at least start programming like they are living in 2005.

However, since you GROSSLY mischaracterized my argument, I am going to call you out on that bit of attempted demonization of my argument on your part.

But I won't repeat the argument because, why bother?

-Mike

4 months ago by Richard Miles — view source

unread

Howdy people,

Autoloading effectively necessitates that every symbol be in its own separate file.

How so? While Composer "recommends" PSR4 autoloading with one class per file, other configurations are entirely possible, either with Composer's autoload.classmap (which uses a precomputed class -> filename table), or with a custom autoloader.

If I'm understanding the main issue, we need package-level scope.
Rather than build a whole new thing, what if we solve this problem?

Thought:
What if php implicitly prepended namespaces with the composer package version?
If two incompatible versions are required, the composer downloads both. Enabled/Disabled with flag?

Maybe: org/package/version/

This implicit version would show in debug backtraces. So importing Example\Example; might look like this in the trace:

1.0.8\Example\Example

Packages imports by namespace could automatically resolve to the version defined in that file's local package JSON.
Reflection to know where the code is running is easy enough, but this would require interpreter-level changes.
It’s a little late, but off the top of my head, it seems fairly backward-compatible.
However, if someone relies on parsing a stack trace… hmmm (trace flag too?)

Contextual sidenote:
Mac doesn’t ship with brew. Windows doesn’t ship with Chocloaty.
Python ships with pip, and Node ships with npm.
Python didn’t start shipping with pip until version >= 3.4.

I would be happy to see PHP shipping with the composer as a default-enabled flag.

Best,
Richard Miles

4 months ago by Aleksander Machniak — view source

unread

What if php implicitly prepended namespaces with the composer package version?
If two incompatible versions are required, the composer downloads both. Enabled/Disabled with flag?

While we're brainstorming... What if something like this would be possible?

include $file with (strict_types=1, scope=$prefix);

Composer would be able to do something useful with that, I suppose.

--
Aleksander Machniak
Kolab Groupware Developer [https://kolab.org]
Roundcube Webmail Developer [https://roundcube.net]

PGP: 19359DC1 # Blog: https://kolabian.wordpress.com

4 months ago by Michael Morris — view source

unread

What if php implicitly prepended namespaces with the composer package
version?
If two incompatible versions are required, the composer downloads both.
Enabled/Disabled with flag?

While we're brainstorming... What if something like this would be possible?

include $file with (strict_types=1, scope=$prefix);

Composer would be able to do something useful with that, I suppose.

I would prefer to leave sleeping dogs where they lie - that is not change
how include, include_once, require and require_once work. For one, code
imports are never optional, so the two include statements are NEVER
appropriate. Second, symbol definitions are never to be done twice, so
require is NEVER appropriate. That leaves require_once, and having it
behave differently from its siblings is asking for confusion. Hence the
reason I'd rather dispense with them entirely and move to a new import
mechanism entirely - import. And since it's a new import mechanism it
could clean things up that cannot be cleaned any other way. I've tried to
stay away from that in the last few emails, but it is there. At a
minimum in my opinion, import should be a "code first" importer and not
require the file to be imported use <?php to get into code mode. If this
means using a separate extension for these files such as phm so that the
IDE's can figure it out (and so can users at a glance) so be it. But the
behavior of the import statement is parallel to the package issue itself -
that is import files could be nothing special and the package mechanism
would still be a feature worth targetting.

4 months ago by Arvids Godjuks — view source

unread

On Jul 1, 2024, at 7:57 AM, Arvids Godjuks arvids.godjuks@gmail.com
wrote:

TL;DR: As a userland developer, in my opinion, this is just a downgrade
from what we have now. Enhance namespaces to have the ability to have
internal/private classes, interfaces, enums and constants. That's about it.

Please note my comments that follow do not mean I am in support of this
package proposal as presented.

Autoloading is one of the best killer features of PHP - love it or hate
it - it's your personal preference.

Two really solid reasons to hate autoloading as implemented in PHP:

Autoloading runs userland code. This means it has the potential
conflict between different packages with different autoloaders, it means
there can be buggy autoloaders, and it means that when using XDEBUG every
time a new symbol is found when the developer is single-step debugging the
developer will be dropped into the autoloader and then best case they then
immediately trace out. All of these aspects a major PITA and time waster
and make debugging more exhausting than it needs to be.

But that is why interoperability in PHP world is so high. When it was
introduced, it allowed enabling autoloading for most code bases out there
regardless of what their structure was and still is. Sure, you have to be
careful how you do it, but that is also not really a userland concern -
most of those have been implemented once and then almost never touched :)
The community has settled on a general approach since naturally and that's
how vast majority of people write their code. Buggy code is buggy code, it
really has not much to do with the autoloading and with how people write
buggy code :) You are blaming the hammer for user trying to nail the screw
into wood :)

You can already sidestep autoloader by adding a require statement to any
file and loading everything without triggering autoload. You can add your
own autoloader that has a map of high-level namespaces where you want to
load it as a package and recursively include everything that way. The tools
are there, you just need to use them. If anything, since PHP now has
attributes, you can just make yourself an attribute and handler for it and
have a #[Package('name')] that can find all files with that attribute and
load it all as a package.
It's far more powerful in userland because it allows people to do whatever
they want with it if they do not like the standard autoloader(s). If
anything, combined with composer folks, PHP-FIG could come up with a
community based #[Package] tag and make packages a thing.

Autoloading effectively necessitates that every symbol be in its own
separate file. This needlessly bloats number of files and directories by
more than an order of magnitude — see my numbers from recent discussion —
and that also mean related code is located farther away from other related
code. This can be worked around but the workarounds I've seen are all
fragile and unable to be generic, and few 3rd party packages do this.

That's just how PSR-4 standard was written and works, and it's a good
default for a reason. On small projects - sure, I can see it being
overkill. But I haven't worked on a small project in a decade and I rarely
have that little code in one file that I would want to stuff all together
in one file. I do use PHPStorm, so it is very adapted to how most PHP
projects are and provides excellent navigation abilities that fit the PSR-4
and that way of structuring projects. Vast part of the community uses it.
It's a standard in a lot of companies for a reason.

I've seen a sizeable chunk of developers that come from other languages
discover PHP's autoloading and their minds just get blown.

It is unclear to me if by saying their "minds just get blown" if that
means you think they see it as a positive or negative?

In a positive manner - a lot of people love that they do not have to fiddle
with import statements and just can leave that part to the ecosystem and
IDE to figure out.

As a developer who spent a decade in PHP and then branched out and added
Go to my repertoire I can tell you one of the nicest differences I
experienced was not having to deal with an autoloader during debugging, and
not being so constrained was to have to create a new file for every symbol.
Go projects need an order of magnitude fewer files. It is just so much
easier to grok the source code of a Go project compared to a PHP because of
this one simple fact. Now when I program in PHP I find myself constantly
cursing the fact that I have to deal with the autoloader.

BTW, I know Go is a pre-compiled language unlike PHP, but that does not
necessarily preclude PHP from having a better solution for code loading and
organization.

See my reply to autoloading things above - you can eliminate the autoloader
triggering easily in probably 15-30 minutes flat.

Performance has not been an issue for a long time due to opcache and all
the optimizations that have been done to it and ability to preload
bytecode. Then there are things like FrankenPHP, Swoole, ReactPHP and
others that entirely sidestep that issue. And then there's the active
development of JIT engine - just let the people working on the
implementation time to cook.

This reads to me like Stockholm syndrome, e.g. "My captors still hold me
captive, but they no longer beat me every day."

It's not that, it's literally true. PHP is one of the fastest-interpreted
languages, butting heads with nodejs only losing to it in default simple
application implementations because PHP is not event loop first (but
Franken PHP and others have a few things to say about that and then there
have been recently people playing with fibers and stuff and got performance
results that showed PHP being faster than nodejs at higher loads. Sorry I
do not have a link handly :\ )

It works, worked for a long time and there are not so many things wrong
with it to entirely upend the whole ecosystem and split the language.
Here's your HARD REMINDER about Python 2 => Python 3 and how that went and
is still somewhat ongoing.

Totally agree on that.

-Mike

There are aspects to this that go beyond just technical aspects. Wherever
when implementing autoloading people were secret geniuses, stumbled into it
accidentally or just had practical needs themselves that they just
implemented in this way - it has been a major turning point in PHP's life
and has transformed the ecosystem into what it is today together with the
rise of the composer package manager. I have seen people talk about
visibility in namespaces, package concept and all that, but every time it
was building upon existing autoloading mechanics - some adding capabilities
to them, modifying them, having new type of autoloader. But I have never
encountered anyone in this community in 20 years i have been an active part
of it to propose a radical change like this and I'm fairly certain after
keeping up with the thread that it is almost universally not what people
want. Most people just want the toolbox be "finished" so to speak, not get
a completely new one in addition that has no compatibility with the old one.
To be frank, the PHP ecosystem just does not have the resources to eat a
change on that level and support more than one implementation. Unlike many
other languages, PHP does not really get support and investment from the
likes of google, Microsoft, meta and so on. PHP Foundation is great, but
it's not google that can singlehandedly throw 1000 devs at supporting a
language and not even really feel it.

Arvīds Godjuks
+371 26 851 664
arvids.godjuks@gmail.com
Telegram: @psihius https://t.me/psihius

4 months ago by Rob Landers — view source

unread

TL;DR: As a userland developer, in my opinion, this is just a downgrade from what we have now. Enhance namespaces to have the ability to have internal/private classes, interfaces, enums and constants. That's about it.

Please note my comments that follow do not mean I am in support of this package proposal as presented.

Autoloading is one of the best killer features of PHP - love it or hate it - it's your personal preference.

Two really solid reasons to hate autoloading as implemented in PHP:

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders, it means there can be buggy autoloaders, and it means that when using XDEBUG every time a new symbol is found when the developer is single-step debugging the developer will be dropped into the autoloader and then best case they then immediately trace out. All of these aspects a major PITA and time waster and make debugging more exhausting than it needs to be.

FWIW, (in Intellij at least), you can set it to skip those files.

— Rob

4 months ago by Mike Schinkel — view source

unread

TL;DR: As a userland developer, in my opinion, this is just a downgrade from what we have now. Enhance namespaces to have the ability to have internal/private classes, interfaces, enums and constants. That's about it.

Please note my comments that follow do not mean I am in support of this package proposal as presented.

Autoloading is one of the best killer features of PHP - love it or hate it - it's your personal preference.

Two really solid reasons to hate autoloading as implemented in PHP:

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders, it means there can be buggy autoloaders, and it means that when using XDEBUG every time a new symbol is found when the developer is single-step debugging the developer will be dropped into the autoloader and then best case they then immediately trace out. All of these aspects a major PITA and time waster and make debugging more exhausting than it needs to be.

FWIW, (in Intellij at least), you can set it to skip those files.

I just went and looked again, and after having requested the feature in PhpStorm over a decade ago, it appears they finally have added it. I had given up that they ever would, and must have missed it when they added it. Thanks for prodding me to look for it.

Of course that doesn't help if the source of the bug is in the userland autoloader code, but is an improvement much of the time otherwise.

-Mike

4 months ago by Stephen Reay — view source

unread

Sent from my iPhone

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders

Can run userland code. It doesn't have to; FYI spl_autoload (https://www.php.net/manual/en/function.spl-autoload.php) has existed since php5.1 and works amazingly well.

That "standards" like psr-whatever can't (read: choose not to) use it says more about people and maintaining their little fiefdoms than anything else.

4 months ago by Vincent de Lau — view source

unread

From: Stephen Reay php-lists@koalephant.com
Sent: Wednesday, July 3, 2024 1:17 PM

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders

Can run userland code. It doesn't have to; FYI spl_autoload (https://www.php.net/manual/en/function.spl-autoload.php) has existed since php5.1 and works amazingly well.

That "standards" like psr-whatever can't (read: choose not to) use it says more about people and maintaining their little fiefdoms than anything else.

As a PHP-FIG Core Committee member, I find this characterisation of people involved in the FIG offensive. My contribution, however big or small, is intended to help the PHP community at large.

Accusing people of 'maintaining their feifdom', especially in the light of a discussion on autoloading is completely uncalled for. PSR-0 and PSR-4 are the product of (at the time) big relevant userland projects unifying and codifying a common way of doing things, to avoid conflicts and improve interoperability. Even though it is a recommendation, I believe it is fair to say it has become a de-facto standard in the eco-system, like it or not. Still, nobody forces anyone to adhere to it, not even Composer.

Composer has various autolading strategies 1, PSR-0 and PSR-4 are only two of them. It should be possible to add a 'package' or 'module' strategy when needed, if needed. When the module implementation requires significant userland work to make it practical, people will probably rely on Composer for autoloading, which would probaly also have some requirements on how the source package is organised. That would probably also trigger an update or addendum to PSR-4, in collaboration with anyone that wants to.

Likewise, when PHP makes changes to it's autoloading infrastructure I would expect Composer to leverage those to improve any of the strategies. If for instance a built-in classmap autoloader would exist, I would expect Composer to use that instead of a userland method.

To come back to spl_autoload: That function pre-dates namespaces and is highly opinionated on how to organise code. All lower-case filenames, class per-file, files in include_path, full namespace in path, you name it. If that is what projects wanted at the time, or even now, PSR-0 and the PHP-FIG would possibly not even exist.

With PHP most of the time not making strong recommendations or even enforcing certain patterns, the community will seek guidance elsewere. For some topics this leads to competing interpretations, sometimes they lead to de-facto standards. In the end, nothing and nobody is perfect. PHP, internals, the FIG, the PSRs, all are made by, or consist of, people trying their best.

--
Vincent de Lau

4 months ago by Stephen Reay — view source

unread

From: Stephen Reay php-lists@koalephant.com
Sent: Wednesday, July 3, 2024 1:17 PM

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders

Can run userland code. It doesn't have to; FYI spl_autoload (https://www.php.net/manual/en/function.spl-autoload.php) has existed since php5.1 and works amazingly well.

That "standards" like psr-whatever can't (read: choose not to) use it says more about people and maintaining their little fiefdoms than anything else.

As a PHP-FIG Core Committee member, I find this characterisation of people involved in the FIG offensive. My contribution, however big or small, is intended to help the PHP community at large.

If you choose to be offended by my opinion, I can't really help that.

To come back to spl_autoload: That function pre-dates namespaces and is highly opinionated on how to organise code. All lower-case filenames, class per-file, files in include_path, full namespace in path, you name it. If that is what projects wanted at the time, or even now, PSR-0 and the PHP-FIG would possibly not even exist.

It's less highly opinionated than either PSR, but that's my whole point: it's someone else's opinion, hence it's opposed by FIG.

Neither of which is the point I was making - someone claimed that autoloaders are implicitly userland code. The point is they don't have to be, and there is a perfectly useable one built in to the SPL extension; if it's "too opinionated" (or the opinions are ones you don't like), it's hardly the most in-depth of functions, and it already has configurable parts, so adding in more control shouldn't exactly require a rocket scientist to add, for example, the ability to use the original case of the class name.

4 months ago by Rob Landers — view source

unread

To come back to spl_autoload: That function pre-dates namespaces and is highly opinionated on how to organise code. All lower-case filenames, class per-file, files in include_path, full namespace in path, you name it. If that is what projects wanted at the time, or even now, PSR-0 and the PHP-FIG would possibly not even exist.

It's less highly opinionated than either PSR, but that's my whole point: it's someone else's opinion, hence it's opposed by FIG.

Neither of which is the point I was making - someone claimed that autoloaders are implicitly userland code. The point is they don't have to be, and there is a perfectly useable one built in to the SPL extension; if it's "too opinionated" (or the opinions are ones you don't like), it's hardly the most in-depth of functions, and it already has configurable parts, so adding in more control shouldn't exactly require a rocket scientist to add, for example, the ability to use the original case of the class name.

To be fair, I didn't know this was a thing (you learn something new every day) and I'd rather have all-lower-case filenames, so I may actually look into this. I wonder if composer exposes this as an option?

— Rob

4 months ago by Matthew Weier O'Phinney — view source

unread

On Wed, Jul 3, 2024 at 9:50 AM Stephen Reay php-lists@koalephant.com
wrote:

From: Stephen Reay php-lists@koalephant.com
Sent: Wednesday, July 3, 2024 1:17 PM

Autoloading runs userland code. This means it has the potential conflict
between different packages with different autoloaders

Can run userland code. It doesn't have to; FYI spl_autoload (
https://www.php.net/manual/en/function.spl-autoload.php) has existed
since php5.1 and works amazingly well.

That "standards" like psr-whatever can't (read: choose not to) use it says
more about people and maintaining their little fiefdoms than anything else.

As a PHP-FIG Core Committee member, I find this characterisation of people
involved in the FIG offensive. My contribution, however big or small, is
intended to help the PHP community at large.

If you choose to be offended by my opinion, I can't really help that.

No, but you also don't need to air your personal grievances on the mailing
list. If you don't like what FIG or any other entity in the PHP ecosystem
is doing, this is NOT the place to air that grievance. Internals is for
discussing changes to the runtime. Calling out entities like this here is
bound to alienate folks who want to work on the engine, and who are also
parts of those groups.

It also doesn't help your argument when you're stating things that are flat
out wrong as facts. You can absolutely use spl_autoload() alongside the PSR
recommendations or Composer; see more below.

To come back to spl_autoload: That function pre-dates namespaces and is
highly opinionated on how to organise code. All lower-case filenames, class
per-file, files in include_path, full namespace in path, you name it. If
that is what projects wanted at the time, or even now, PSR-0 and the
PHP-FIG would possibly not even exist.

It's less highly opinionated than either PSR, but that's my whole point:
it's someone else's opinion, hence it's opposed by FIG.

That's a gross mischaracterization.

In point of fact, most frameworks that joined FIG in the beginning were
leveraging spl_autoload_register(), which provides a stack of autoloaders
that each provide their own logic for how to map classes to where on the
filesystem they live. spl_autoload_register() came after spl_autoload(),
and was introduced to add flexibility to the language, as spl_autoload is
proscriptive and only allows a single approach to autoloading, and it
wasn't even one that was widely used at the time it was introduced. It's
not about opinions, it's about recognizing that different approaches
might have merit. (Some might give better performance, some might allow
pulling items out of a phar or tarball, etc.)

PSR-0 was created because a large number of projects were writing their own
autoloaders that were doing similar things, and most of them were doing
things differently than spl_autoload() due to limitations of that
approach, and all were using spl_autoload_register(). Creating a standard
approach allowed users of these projects to use a single autoloader to load
code from each within their application, which helped improve performance
and reduced autoloading conflicts. PSR-4 extended the concept, while
keeping some of the core ideas in place. And, again, YOU DO NOT NEED TO
FOLLOW either one.

Why?

Because Composer uses spl_autoload_register() internally, and enables
multiple autoloading approaches (PSR-0, PSR-4, classmap, file, etc.) out of
the box. And if you don't want to use those for your own code... you can
add another autoloader to the stack using spl_autoload_register(). You can
even add your own before invoking the Composer autoloader to ensure it
gets precedence. Composer's then becomes primarily a tool for loading the
third-party code your application depends on.

Neither of which is the point I was making - someone claimed that
autoloaders are implicitly userland code. The point is they don't have to
be, and there is a perfectly useable one built in to the SPL extension; if
it's "too opinionated" (or the opinions are ones you don't like), it's
hardly the most in-depth of functions, and it already has configurable
parts, so adding in more control shouldn't exactly require a rocket
scientist to add, for example, the ability to use the original case of the
class name.

The configurable part for autoloading in the language is
spl_autoload_register(), full stop. And this does require userland code.
Yes, you can register spl_autoload() with it, and this is part of the
engine, but that's the only language-level autoloader at this time. I'd
argue we shouldn't add any more to the engine; the stack approach of
spl_autoload_register() ensures we can reduce engine complexity and
maintenance by offloading it to something that can evolve at a faster pace
than the language.

I'm following the packaging threads closely, and the one thing I've failed
to see a solid argument for is what problems the current approach of
using namespaced code doesn't address. I can definitely see a need for
marking things as package private (i.e., not part of the publicly
consumable API), but that also feels like something we could address in
other ways. I know Larry has asked this same question before, and it's
really what I want to see answered, because packages might be the solution,
but there may be other approaches we could take that also accomplish those
goals.

--
Matthew Weier O'Phinney
mweierophinney@gmail.com
https://mwop.net/
he/him

4 months ago by Michael Morris — view source

unread

On Wed, Jul 3, 2024 at 12:52 PM Matthew Weier O'Phinney <
mweierophinney@gmail.com> wrote:

I'm following the packaging threads closely, and the one thing I've failed
to see a solid argument for is what problems the current approach of
using namespaced code doesn't address.

Running multiple versions of the same code. Say you're writing an
extension for Drupal and you want to use a nifty new feature of the newly
released Twig 4. Under the current system you're outta luck until the
Drupal project moves to 4, and that might take awhile, ESPECIALLY if 4 has
BC breaks.

You can monkey type the twig files with something like
the brianhenryie/strauss package for WordPress, programmatically changing
the namespace declaration of every file in the package to something of your
choosing, but that feels a bit hackish at best.

4 months ago by Rob Landers — view source

unread

I'm following the packaging threads closely, and the one thing I've failed to see a solid argument for is what problems the current approach of using namespaced code doesn't address.

Running multiple versions of the same code. Say you're writing an extension for Drupal and you want to use a nifty new feature of the newly released Twig 4. Under the current system you're outta luck until the Drupal project moves to 4, and that might take awhile, ESPECIALLY if 4 has BC breaks.

So, if v4 has BC breaks ... how would Drupal not crash? If you allow multiple versions, how would you use both versions? I'm not even sure that is a logical possibility.

— Rob

4 months ago by Michael Morris — view source

unread

So, if v4 has BC breaks ... how would Drupal not crash? If you allow
multiple versions, how would you use both versions? I'm not even sure that
is a logical possibility.

Twig in Drupal will be installed the old way and find itself bound at \Twig\

import 'twig/twig v4' as NewTwig

That aliases the new version to \NewTwig\

And you can work with it by addressing the new library at the new
namespace. I'll go through this in more detail in the 4th iteration post
for this which I'll work on tonight.

4 months ago by Michael Morris — view source

unread

Hello all. Hitting reset again as the primary problem at hand has become
clear. Let's recap it.

Autoloading is great for loading packages, but it can't load different
versions of the same package at the same time. Why would you want to do
that?

When you don't have full control of the code.

For example, consider Drupal. It is running Twig at some version of 3 at
the moment. Suppose Twig 4 is introduced with significant backward
compatibility breaks (Not saying the authors would do such a thing) but
also wonderful features.

If you're writing a Drupal extension you might want to use this new Twig.
This is possible if you are willing to monkey-type the package - that is,
have a code package traverse over the entire package and change all
instances of namespace Twig in the files to namespace NewTwig. You can
then use the package at the namespace of \NewTwig.

This is painful, but the pain factor increases if multiple extension
developers choose to do the same thing. Each extension using its own Twig
library is going to incur a performance hit.

One upshot of this is I've noted that major package distributors, like
Symfony, take BC into account with major releases - and may not develop new
features or change things in those releases out of fear of people not
wanting to upgrade.

Now don't get me wrong, changing things just because is a bad thing. If a
BC can be avoided it should be. But having a mechanism to move forward is
important.

In some ways versioning packages is like static typing variables. It
doesn't seem important at all until you are faced with a problem only it
can solve, or faced with a problem created by dynamic typing of variables.

What can be done in the engine?

Well first off, recognize that autoloading isn't going to work with a
versioned package scheme. Autoloaders, regardless of their resolution
schema be it PSR-0, PSR-4, or BobbysFirstAutoloader-Scheme can only have
one symbol per package, set by the namespace.

Can PHP support multiple packages without rewriting the whole engine? I
think so, but it isn't trivial, and the side effects need to be cordoned
off so that those who need this complexity can have it while the beginning
and intermediate coders can ignore it just like they ignore strict
comparison operators and strict typing unless a library they are trying to
use foists it on them.

This is why I advocate a new keyword for this - import. Import's behavior
is most similar to require_once, but it doesn't have to be the same. Since
it is a new entrypoint into the engine the way the engine considers the
code can be different - whether slightly different or radically different
is a debate for another time. I'm going to stick with only those changes
that make sense in the context of package links.

Let's start with the simplest problem, importing this file.

namespace A;
function foo() { echo 'Hi'; }

To review, if we require_once this file we'll find the function at
\A\foo(). If our current file uses the same namespace we can just use foo()

At its root import would do the same. import "file.php" would do the same
as a require_once assuming there's no difference between the file structure
rules for import - again there is opportunity here, but it's not a
requirement.

If that's all it does, it's pointless. However, import can alias.

import 'file.php' as B;

Now we have \B\foo(); This makes it relatively easy to have two different
versions of the package running since in our own code we can always
reference the foo in the B namespace. But while that allows limited package
versioning, it doesn't solve the multiple extensions wanting to use the new
stuff problem outlined above.

So we have to call out the version in code, like so.

import 'file.php v1.0.0';

A simple space separates the version from the file. If the filename has a
space, well \ characters aren't just for namespaces.

Now for the first real behavior difference between import and require_once,
even if we aren't doing anything fancy. Import cares about the namespace
it's invoked from. Require_once does not. To illustrate this behavior
he's some pseudocode - we are including the file.php given earlier

namespace D;
require_once 'file.php';

\A\foo(); // Hi.

import 'file.php';

\D\A\foo(); // Hi.

See that? The namespace of the calling file is prepended to the namespace
contained in the import.

Why? What's the value here? I'll explain.

Now, let's suppose we do have two versions of file.php. So in addition to
the above, elsewhere in the code this happens

namespace C;
import 'file.php v2.0.0'

A\foo(); // Welcome, since version 2 echoes welcome. Remember your
namespace resolution rules - this import is actually at:
\C\A\foo(); Welcome, as this is the absolute path to the code we just
imported.
\A\foo(); // Hi, as the package at root was brought in by require_once()
\D\A\foo(); Hi, as that's what was imported into the D namespace.

Now for the kicker

namespace E;
import 'file.php';

A\foo(); // Hi.

The engine can be left as is and this would work, but if the engine is
altered to support symbolic links on the symbol table then the performance
hit might be avoided. That is, when a redundant import occurs that would
pull the same package the engine just quickly links up the new namespace.
Hence \E\A\foo() quietly points to \D\A\foo() as it was declared first.

What hasn't been discussed in this iteration are the following critical
points:

How the package path gets resolved in the first place. Does it work like
require and check locally then check the PHP include paths?
When does the code get downloaded from where it is downloaded?
Is a registry used like composer and npm, or are repos directly invoked
as in go (I don't remember how Python does it, but someone providing that
example might be useful)
The huge ball of wax that is the package definition file. Just look at
the properties of composer.json and package.json to get an idea of that
scope. How much of if any of this should PHP deal with.
Is import to be locked into loading other PHP files, or could it deal
with .so (Unix) or .dll (Windows) files? Phar files?

It's not like I'm not interested in any of these questions, but too many
questions at once is too much so I'd like to leave them aside for now.

And there are yet more questions as well raised in previous iterations, but
I've again left those out because they touched off controversy. While I'm
not afraid of such, I'm inclined to avoid it if possible.

A quick thank you to everyone who has participated in the thread, even the
torpedo tossers because it's forcing me to think this through entirely. And
I'm trying to take as much into consideration as possible. And yes, this
remains a brainstorm for now, but each successive brainstorm is more tight
than the one before it.

4 months ago by Mike Schinkel — view source

unread

Can PHP support multiple packages without rewriting the whole engine? I think so, but it isn't trivial, and the side effects need to be cordoned off so that those who need this complexity can have it while the beginning and intermediate coders can ignore it just like they ignore strict comparison operators and strict typing unless a library they are trying to use foists it on them.

This is why I advocate a new keyword for this - import.

There are ~6300 uses of the keyword import on GitHub:

https://github.com/search?q=import+language%3APHP+symbol%3A%2F%5Eimport%24%2F&type=code https://github.com/search?q=import+language:PHP+symbol:/^import$/&type=code

That's a lot of BC breakage for some people.

For this proposal, would one of these not be acceptable instead (assuming that the compiler could handle import in this way w/o it being a new reserved word)?:

include "file.php" as import
use import "file.php"

Import's behavior is most similar to require_once, but it doesn't have to be the same. Since it is a new entrypoint into the engine the way the engine considers the code can be different - whether slightly different or radically different is a debate for another time. I'm going to stick with only those changes that make sense in the context of package links.

When you first proposed modules I was understood (wrongly?) that you were proposing imports to be a statement that imported into file scope and then at the end of loading the file PHP would flush those symbols just like how (I think?) JavaScript imports work (I am ignoring the complexity closures add to simplify our discussion here.)

This sounds like you are saying import would (instead?) be dynamic like include* and require* and any symbols loaded with import would continue their lifetime until the program is finished or the page is loaded, depending on how the program is run?

I ask because when I was envisioning page scope being added to PHP I was also envisioning that PHP could perform more optimizations if the new symbols only affected the currently loaded page. If you are proposing beyond-page lifetime then that precludes this optimizations which is not a deal killer but is a disappointment.

Now we have \B\foo(); This makes it relatively easy to have two different versions of the package running since in our own code we can always reference the foo in the B namespace. But while that allows limited package versioning, it doesn't solve the multiple extensions wanting to use the new stuff problem outlined above.

Consider the following parts of an application:

Bespoke app
"Enterprise Reports" library
Twig v3 used by "Enterprise Reports"
"ProductsPro" library
Twig v4 used by "ProductsPro"
"PP2ER Connector" library

Given your scenario I guess you assume Enterprise Reports would import Twig v3 as maybe ER\Twig and ProductsPro would import Twig v4 as maybe PP\Twig, right?

How does the PP2ER Connector use Twig? Does it create it own PP2ER\Twig? What if the connector needs to use the ER\Twig\Environment from ProductsPro with the Twig\Loader\FilesystemLoader from Enterprise Reports where those classes have private and protected properties that are simple not composable, and especially not across versions.

Or what it he app itself needs to use the functionality of both in a way that requires access to private and/or protected property values or methods across the two versions?

So we have to call out the version in code, like so.

import 'file.php v1.0.0';

Where will PHP be able to get the version number in a performant manner, remembering that the problem to be solved is dependencies of dependencies so you cannot rely on a strict directory structure with version numbers unless a non-PSR4 autoloader format is introduced and widely adopted?

Will packages need to ship composer.lock and developers deploy them? Will that be performant and secure enough?

What about libraries and packages that do not use Composer? How will WordPress handle this with plugin dependencies?

-Mike

4 months ago by Michael Morris — view source

unread

There are ~6300 uses of the keyword import on GitHub:

https://github.com/search?q=import+language%3APHP+symbol%3A%2F%5Eimport%24%2F&type=code
https://github.com/search?q=import+language:PHP+symbol:/%5Eimport$/&type=code

That's a lot of BC breakage for some people.

No worse than PHP 7's keyword introductions.

Import's behavior is most similar to require_once, but it doesn't have to
be the same. Since it is a new entrypoint into the engine the way the
engine considers the code can be different - whether slightly different or
radically different is a debate for another time. I'm going to stick with
only those changes that make sense in the context of package links.

When you first proposed modules I was understood (wrongly?) that you
were proposing imports to be a statement that imported into file scope and
then at the end of loading the file PHP would flush those symbols just like
how (I think?) JavaScript imports work (I am ignoring the complexity
closures add to simplify our discussion here.)

That was the first iteration, yes. I am adjusting to the feedback on the
list. JavaScript does imports the way it does because of how files scope,
and how the module system itself scopes, which isn't readily retrofitted
onto PHP. Also, at the time I was toying around with the format and had yet
to hit upon the situation where it could be useful, that being versioned
files.

This sounds like you are saying import would (instead?) be dynamic
like include* and require* and any symbols loaded with import would
continue their lifetime until the program is finished or the page is
loaded, depending on how the program is run?

Yes, because that is how PHP itself does work under the hood, at least for
php file includes. How it would go about doing this when resolving .so or
.dll extensions is another matter. Does it have to be this way? That's the
hint I've gotten from the feedback but only a core contributor with
experience working on the engine could say for sure.

I ask because when I was envisioning page scope being added to PHP I was
also envisioning that PHP could perform more optimizations if the new
symbols only affected the currently loaded page. If you are proposing
beyond-page lifetime then that precludes this optimizations which is not a
deal killer but is a disappointment.

Whether the optimizations affect the file on load depends on what's being
optimized to be honest. There is still an opportunity here.

Now we have \B\foo(); This makes it relatively easy to have two different
versions of the package running since in our own code we can always
reference the foo in the B namespace. But while that allows limited package
versioning, it doesn't solve the multiple extensions wanting to use the new
stuff problem outlined above.

Consider the following parts of an application:

Bespoke app

"Enterprise Reports" library

Twig v3 used by "Enterprise Reports"

"ProductsPro" library

Twig v4 used by "ProductsPro"

"PP2ER Connector" library

Given your scenario I guess you assume Enterprise Reports would import
Twig v3 as maybe ER\Twig and ProductsPro would import Twig v4 as maybe
PP\Twig, right?

Correct.

How does the PP2ER Connector use Twig?

Depends on which one it wishes to use, \ER\Twig or \PP\Twig

Does it create it own PP2ER\Twig? What if the connector needs to use
the ER\Twig\Environment from ProductsPro with the
Twig\Loader\FilesystemLoader from Enterprise Reports where those classes
have private and protected properties that are simple not composable, and
especially not across versions.

I've never seen cross version mixing like you're describing so I didn't
take it into account. That said, the extant copies of those classes will be
variables, and hopefully not global variables.

Or what it he app itself needs to use the functionality of both in a way
that requires access to private and/or protected property values or
methods across the two versions?

That isn't in scope for this discussion. The whole point of private and
protected scope modifiers is to restrict access by outside code. Breaking
through that can be done with the Reflection API in some cases, but it
isn't easy.

So we have to call out the version in code, like so.

import 'file.php v1.0.0';

Where will PHP be able to get the version number in a performant manner?

A question for another day. I'm not going to touch on it yet as I want
feedback on the rest of what I've written in this iteration first and,
honestly, I want to mull it over in my head a little more. It may be that
the code doesn't have the version number but the package declaration file
does. There definitely would be advantages to that, but it may still be
desirable to have the version called out in the import statement in code.

4 months ago by Mike Schinkel — view source

unread

There are ~6300 uses of the keyword import on GitHub:

https://github.com/search?q=import+language%3APHP+symbol%3A%2F%5Eimport%24%2F&type=code

That's a lot of BC breakage for some people.

No worse than PHP 7's keyword introductions.

True.

OTOH, if you don't actually have to break BC for discretionary changes, then it would be better not to.

Import's behavior is most similar to require_once, but it doesn't have to be the same. Since it is a new entrypoint into the engine the way the engine considers the code can be different - whether slightly different or radically different is a debate for another time. I'm going to stick with only those changes that make sense in the context of package links.

When you first proposed modules I was understood (wrongly?) that you were proposing imports to be a statement that imported into file scope and then at the end of loading the file PHP would flush those symbols just like how (I think?) JavaScript imports work (I am ignoring the complexity closures add to simplify our discussion here.)

That was the first iteration, yes. I am adjusting to the feedback on the list. JavaScript does imports the way it does because of how files scope, and how the module system itself scopes, which isn't readily retrofitted onto PHP.

I think it can be retrofitted to PHP, but that is a different issue and I won't derail your discussion to suggest it in this thread.

Also, at the time I was toying around with the format and had yet to hit upon the situation where it could be useful, that being versioned files.

This sounds like you are saying import would (instead?) be dynamic like include* and require* and any symbols loaded with import would continue their lifetime until the program is finished or the page is loaded, depending on how the program is run?

Yes, because that is how PHP itself does work under the hood, at least for php file includes. How it would go about doing this when resolving .so or .dll extensions is another matter. Does it have to be this way? That's the hint I've gotten from the feedback but only a core contributor with experience working on the engine could say for sure.

I ask because when I was envisioning page scope being added to PHP I was also envisioning that PHP could perform more optimizations if the new symbols only affected the currently loaded page. If you are proposing beyond-page lifetime then that precludes this optimizations which is not a deal killer but is a disappointment.

Whether the optimizations affect the file on load depends on what's being optimized to be honest. There is still an opportunity here.

Where do you see opportunity for optimization — assuming your vision of imports — that is not already a potential for optimization?

Now we have \B\foo(); This makes it relatively easy to have two different versions of the package running since in our own code we can always reference the foo in the B namespace. But while that allows limited package versioning, it doesn't solve the multiple extensions wanting to use the new stuff problem outlined above.

Consider the following parts of an application:

Bespoke app

"Enterprise Reports" library

Twig v3 used by "Enterprise Reports"

"ProductsPro" library

Twig v4 used by "ProductsPro"

"PP2ER Connector" library

Given your scenario I guess you assume Enterprise Reports would import Twig v3 as maybe ER\Twig and ProductsPro would import Twig v4 as maybe PP\Twig, right?

Correct.

How does the PP2ER Connector use Twig?

Depends on which one it wishes to use, \ER\Twig or \PP\Twig

Does it create it own PP2ER\Twig? What if the connector needs to use the ER\Twig\Environment from ProductsPro with the Twig\Loader\FilesystemLoader from Enterprise Reports where those classes have private and protected properties that are simple not composable, and especially not across versions.

I've never seen cross version mixing like you're describing so I didn't take it into account. That said, the extant copies of those classes will be variables, and hopefully not global variables.

I have with WordPress plugins.

I wish I could say exactly what, where and when, but sadly that knowledge was lost to the mists of time.

Or what it he app itself needs to use the functionality of both in a way that requires access to private and/or protected property values or methods across the two versions?

That isn't in scope for this discussion. The whole point of private and protected scope modifiers is to restrict access by outside code. Breaking through that can be done with the Reflection API in some cases, but it isn't easy.

I think you misunderstood me. I was not advocating reaching into private or protected.

What I was saying is if one indirect dependency used v3 and another indirect dependency used v4 then when there is private state the caller is still unable to get them to interoperate.

Or more simply, indirect dependencies are likely to cause problems when interoperability is needed that are not directly in the dependency chain.

OTOH, it may be rare enough that we can say "Sucks to be you" if someone needs that interoperability. ¯_(ツ)_/¯

So we have to call out the version in code, like so.

import 'file.php v1.0.0';

Where will PHP be able to get the version number in a performant manner?

A question for another day.

Frankly if your proposal hinges on using version numbers to differentiate then I think it is not something you can postpone answering.

If there is not a good answer then the approach you are exploring is moot, at least as far as I can see.

-Mike

4 months ago by Michael Morris — view source

unread

import 'file.php v1.0.0';

Where will PHP be able to get the version number in a performant manner?

A question for another day.

Frankly if your proposal hinges on using version numbers to differentiate
then I think it is not something you can postpone answering.

If there is not a good answer then the approach you are exploring is moot,
at least as far as I can see.

So I've had more time to mull this over, and some research, and I think I
have an approach.

First, instead of 'import', use 'require_module'. The parsing rules for
require_module differ from require how the file is parsed, a subject for
another time. Also, it's parallel to what is to follow.

Speaking of new functions, let's start with these

spl_set_include_ini_map('importmap.ini');
spl_set_include_json_map('importmap.json")

The json file is pretty much identical to the JavaScript importmaps. The
ini file looks like this

root = "/absolute/path/to/application/root"

[imports]
square = "./path/to/square.js"
circle = "./path/to/circle.js"
other/ = "./path/to/other/"

[scopes]
\A[square] = './path/to/square/in/namespace/A/a.js'

Whichever format is used is a matter of personal preference. The file can
be, and likely should be, written by composer or some future package
manager.

The root attribute in the map sets the root for all relative paths given in
the map.

Imports are the standard imports for the project. The token on the left
maps to the target on the right.
Import maps affect all includes. Import map tokens are considered before
anything use on the include resolution rules. So include 'square' would
bring in '/absolute/path/to/application/root/path/to/square.js' given the
ini file above.

An import token ending in / is a prepend and the path it maps to must also
end in a slash. So `require_module 'other/triangle.php' will map to
'/absolute/path/to/application/root/path/to/other/triangle.php' given the
ini file above.

Scopes have a namespace followed by the token in brackets. Scopes only
affect require_module as the other include mechanisms do not pay attention
to namespaces. When in that namespace the specified file will be loaded
instead of the default outlined in imports.

The import map system is inspired by but not exactly like JavaScript's
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap

This approach gets whatever RFC that comes out of this proposal out of the
business of trying to design a package manager.

4 months ago by Mike Schinkel — view source

unread

So I've had more time to mull this over, and some research, and I think I have an approach.

First, instead of 'import', use 'require_module'. The parsing rules for require_module differ from require how the file is parsed, a subject for another time. Also, it's parallel to what is to follow.

Speaking of new functions, let's start with these

spl_set_include_ini_map('importmap.ini');
spl_set_include_json_map('importmap.json")

Those are a mouthful!

The json file is pretty much identical to the JavaScript importmaps. The ini file looks like this

root = "/absolute/path/to/application/root"

[imports]
square = "./path/to/square.js"
circle = "./path/to/circle.js"
other/ = "./path/to/other/"

[scopes]
\A[square] = './path/to/square/in/namespace/A/a.js'

I assume rather than .js you mean .php files in your example?

Also, I am not following how these imports and scopes will relate to the actual PHP code that would be affected/using packages.

Whichever format is used is a matter of personal preference. The file can be, and likely should be, written by composer or some future package manager.

Part of me likes the flexibility of two formats. The other more pragmatic part of me says stick with one format for fewer related bugs and to reduce the effort to support it for internal code and by 3rd parties.

Unless it can be fully cached by opcache I would think it would need to be the format that can be parsed the fastest, which could be binary like a Protobuf file.

The root attribute in the map sets the root for all relative paths given in the map.

Are you saying that a publisher of a package would need to write the absolute path to their importmap.* file. How will that work? Even for a bespoke app Composer may not have enough access to the server to know this, and 3rd party packages will definitely not know it for their users. At least I don't think so.

Also, at this point trying to keep track of all your ideas is impossible, at least for me. Have you reconsidered putting it in a repo or at least a Gist yet so it is easier to see the scope of your current ideas about packages?

-Mike

4 months ago by Michael Morris — view source

unread

So I've had more time to mull this over, and some research, and I think I
have an approach.

First, instead of 'import', use 'require_module'. The parsing rules for
require_module differ from require how the file is parsed, a subject for
another time. Also, it's parallel to what is to follow.

+1

Speaking of new functions, let's start with these

spl_set_include_ini_map('importmap.ini');
spl_set_include_json_map('importmap.json")

Those are a mouthful!

The json file is pretty much identical to the JavaScript importmaps. The
ini file looks like this

root = "/absolute/path/to/application/root"

[imports]
square = "./path/to/square.js"
circle = "./path/to/circle.js"
other/ = "./path/to/other/"

[scopes]
\A[square] = './path/to/square/in/namespace/A/a.js'

I assume rather than .js you mean .php files in your example?

Also, I am not following how these imports and scopes will relate to the
actual PHP code that would be affected/using packages.

Whichever format is used is a matter of personal preference. The file can
be, and likely should be, written by composer or some future package
manager.

Part of me likes the flexibility of two formats. The other more pragmatic
part of me says stick with one format for fewer related bugs and to reduce
the effort to support it for internal code and by 3rd parties.

Unless it can be fully cached by opcache I would think it would need to be
the format that can be parsed the fastest, which could be binary like a
Protobuf file.

The root attribute in the map sets the root for all relative paths given
in the map.

Are you saying that a publisher of a package would need to write the
absolute path to their importmap.* file. How will that work? Even for a
bespoke app Composer may not have enough access to the server to know this,
and 3rd party packages will definitely not know it for their users. At
least I don't think so.

Also, at this point trying to keep track of all your ideas is impossible,
at least for me. Have you reconsidered putting it in a repo or at least a
Gist yet so it is easier to see the scope of your current ideas about
packages?

I'm getting there. I'm trying to boil things down at this point to
something that can be put into such.

I went to sleep thinking about this post, on import maps in general and how
Composer works, specifically when you use a class map instead of the PSR-0
or PSR-4 schemes. In that mode, Composer does pretty much what I've
described. This got me to thinking, could setting an import map be an
alternative to setting an autoload function? Would having the PHP runtime
load the file be faster than mucking with some userland code to do the
same? And would the engine be able to do anything that can't be done in
userland? I think so.

So first, I agree that supporting two formats, while convenient, increases
the maintenance burden, so let's just go with ini. As far as the
installing function - a better name is this.

spl_autoload_map( string $filepath );

This function will register an autoload map wrapped with an internal
autoloader to work with it. If called multiple times the maps will resolve
in the order they are called. If called along with the existing
spl_autoload_register function then the maps and autoload functions will be
used in the order of their declaration. Packages are expected to carry
their own autoload maps. Autoload maps can only be loaded once - attempts
to load the same map multiple times will raise a E_USER_WARNING

The contents of the file along with comments as to what they do.

; The root directive affects the relative paths in the map. If set the
location specified becomes
; the root. If not set all relative paths are relative to THIS FILE.
;root = '/some/path'

; If a package is declared, that is the root namespace for all the
includes that follow.
; As this example is for a root autoloader it is commented out, as the
root autoload function of
; an app doesn't need a package name.
;package =

; A map relates symbols to files. This one is for the existing
include/require system. Actual
; file loading is performed by require_once.
[includes]
A\B\Cat = './path/to/Cat.php'

; A symbol can invoke the loading of multiple files. This will be useful
if the package manager that
; prepares this file determines that a polyfill will be needed. Files
will be loaded in order given.
A\TestClass[] = './path/to/polyfill.php'
A\TestClass[] = './path/to/TestClass.php'

; A wildcard can be used if there is a desire to pack multiple symbols
together
; Note the path with the most specific match wins here, so \A\foo() will
invoke the autoload below,
; but \A\TestClass or \A\B\Cat will trigger their respective definitions
above.
A* = './path/to/A/Namespaces/functions.php'

; A path fragment can be used, in which case PSR-4 will be used to map
the rest of the symbol to the filename.
; Pay attention to the direction of the slash at the tail - if the symbol
key has this the value MUST also have this.
B/ = './path/to/B/'

; A package is declared with a @ and maps the package namespace to its
autoload file.
; If the package name here doesn't match what the package calls itself
then the symbol
; given here takes precedence, acting as an alias.
@C = './path/to/C/autoload.ini'

; An import into a package can be done like so
; Twig will load into \C\Twig and that use will need to be used by any
code outside the C package.
@C\Twig/ = './path/to/Twig/'

; The same library can be loaded into a different package, but a symbolic
link is used internally in the engine to optimize
@D\Twig/ = './path/to/Twig/'

; Nothing stops a different package from loading a different version now.
@E\Twig/ = './path/to/Twig/Version4/'

; Modules are loaded with require_module but their maps work the same.
[modules]

And, honestly, I think that's it. This autoloader has some tricks up its
sleeve a userland autoloader does not

Can deal with functions and constants.
Can deal with different versions of the same package being loaded.
Should be able to run faster than its equivalent userland code.

As can be inferred from the above, a full autoload map can get complex in a
hurry. However, it is also meant to be created by Composer or another
package manager at some point. The advantage is a project like WordPress
can ship with this autoload map and not require the end user to install
composer unless they want to create their own. Any plugins can register
their own autoloaders to handle their needs.

4 months ago by Mike Schinkel — view source

unread

I went to sleep thinking about this post, on import maps in general and how Composer works, specifically when you use a class map instead of the PSR-0 or PSR-4 schemes. In that mode, Composer does pretty much what I've described. This got me to thinking, could setting an import map be an alternative to setting an autoload function? Would having the PHP runtime load the file be faster than mucking with some userland code to do the same? And would the engine be able to do anything that can't be done in userland? I think so.

I very much like this direction of thought.

For context, when I worked with WordPress for about a decade I only ever used only Composer for websites but never for my own plugins, and I almost never used namespaces nor PSR-4 autoloaders for anything except when a plugin used them. I almost exclusively used naming convensions for "namespacing" and classmaps for autoloading.

Why? Was it because I was a "bad" programmer? No, it was because when in Rome, you do as the Romans do. And also because when I tried to use Composer and PSR-4 I was always fighting when them to do things in the way that worked best for WordPress.

Or to use a more modern analogy to PHP and WordPress, even though you may be landlocked by the country of Italy, if you are within the borders of Vatican City you follow the laws and conventions of Vatican City when they conflict with those of Italy.

So first, I agree that supporting two formats, while convenient, increases the maintenance burden, so let's just go with ini. As far as the installing function - a better name is this.

Obviously I agree with having only one format, but not sure I concur with the use of .ini. However, me debating against .ini would be me bikeshedding and thus I will demure here.

spl_autoload_map( string $filepath );

Adding an spl_autoload_map() function and feature really resonates with me. As I said, I (almost?) always used class maps with WordPress so if I were to build another WordPress site with a future PHP that made an spl_autoload_map() available, I would TOTALLY use it.

Reading this however caused me to ponder things certain people has said recently — and many people have said for years on this list — and I think I am recognizing something that I have always known but never put the pieces together before.

Many (most?) people on PHP Internals view WordPress coding standards as bad and some even view addressing WordPress developers needs as bad for PHP. And in general I concur that those people are reasonably justified in their belief WordPress' coding standards are not the standards that PHP developer who want to do professional level software engineering should aspire.

And since many (most?) PHP Internals members generally do not experience the issues that WordPress developers have they do not recognize that they are issues; IOW, "out of sight, out of mind."

I also think some list members tend to dismiss WordPress developers pains as unimportant and/or think that addressing those pains have will harm PHP.

(BTW, I recently had a dialog off-list with someone who wrote in an email that "Wordpress is an exception, but nobody these days treats WordPress as a valid example to do anything. It is an ancient piece of legacy code that has no bearing on modern situation and it's their problem to deal with." So I am not just erecting a straw man here.)

But I think what most may not consciously recognize is that WordPress is a different type of web app than an app build using Symfony or Laravel and deployed by its developers, or by some other professional developer.

WordPress differs from the apps many (most?) developers on PHP Internals work with in the following way:

WordPress = User-managed app
Most = Developer-managed apps

In a Developer-Managed app developers choose which 3rd party functionality will be incorporated into their sites whereas with a User-managed app users choose which 3rd party functionality will be incorporated into their site. And that is the KEY difference.

So I am wondering if we can get people on this PHP Internals list who dismiss the needs of WordPress developer BECAUSE it is WordPress to recognize that User-Managed apps ARE a class of PHP applications have needs that deserve to be addressed?

Two (2) unmet needs of User-Managed apps that "standard" PHP currently does not address come to mind:

User-managed apps needs to be able to handle both:

User-added add-ons ("plugins" in WordPress, "modules" in Drupal) that have conflicting dependencies, and
Add-on directory structures that do not follow a PSR-4 directory hierarchy.

As for #2, even if those apps could rearchitect their existing directory structure they cannot realistically be expected to do with because of the huge BC issues their users would experience.

And newly created User-managed apps may still find that a PSR-4 directory structure is not in the best interest of their project or their users. To elaborate, PSR-4 generally assumes that ALL code goes into ONE hierarchy and that any and all code that will be autoload gets placed in that hierarchy.

But with add-ons it makes a lot more sense to have the entire add-on contained in its own add-on directory. This is exactly where PSR-4 breaks down with respect to User-managed apps.

Sure, you can have multiple PSR-4 autoloader root directories, but that does not scale well to websites with a large number of add-ons as many WordPress sites I worked on used. Some had over 100 plugins. With a hierarchy of autoloader maps that Michael Morris is proposing WordPress could collect up all the maps and create one map every time a plugin is added, updated or deleted.

</epiphany>

Based on my above labeled epiphany, I think MOST of what you recently proposed could address those unmet needs of User-managed apps written in PHP, with a few caveats and improvements. Read on.

; A path fragment can be used, in which case PSR-4 will be used to map the rest of the symbol to the filename.

; Pay attention to the direction of the slash at the tail - if the symbol key has this the value MUST also have this.
B/ = './path/to/B/'

It is not clear to me what a trailing slash means, and especially why it is needed on the left-hand side? And why slash here when namespaces use backslash?

Also, as someone raised on DOS and then Windows only "converting" in 2009, I still get confused in *nix when to use a trailing slash and when to not, so this trailing slash worries me, if only for that reason alone.

; A package is declared with a @ and maps the package namespace to its autoload file.
; If the package name here doesn't match what the package calls itself then the symbol
; given here takes precedence, acting as an alias.
@C = './path/to/C/autoload.ini'

Using the @ here feels cryptic, and hard to discover and remember.

I think this would be infinitely easier to follow if packages were just included in a [packages] section.

Your comments also confuse me a bit.

Is this saying that your hypothetical app — which you stated this .ini file is for — needs to use a package named C use "definition" is located at './path/to/C/autoload.ini' then it would use this syntax, and that in the app its components would be accessed at namespace \C?

And I were to have:

@Foo\Bar\Baz = './path/to/Foo/Bar/Baz/autoload.ini'

Then in the app its components would be accessed at namespace \Foo\Bar\Baz?

I think if your examples used hypothetical "real-world" symbols it would be easier to follow than A, B, C, D, etc.

; An import into a package can be done like so
; Twig will load into \C\Twig and that use will need to be used by any code outside the C package.
@C\Twig/ = './path/to/Twig/'

; The same library can be loaded into a different package, but a symbolic link is used internally in the engine to optimize
@D\Twig/ = './path/to/Twig/'

; Nothing stops a different package from loading a different version now.
@E\Twig/ = './path/to/Twig/Version4/'

Okay, this makes sense. OTOH, this is the part that of your proposal that is incomplete for the needs of User-managed apps IMO.

I think you are implying a necessary "best practice" that whenever any PHP library, or package would include code they would need to prefix the namespace of package when importing it and then when using it. Given an org named ACME that released a library called Widgets then if it were to use Twig it should import and use Twig like this (did I understand your intent correctly?):

@ACME\Widgets\Twig/ = './path/to/Twig/'

And in PHP code?:

use \ACME\Widgets\Twig;

I think that would work well for newer libraries and packages authored and used by developers of Developer-managed apps. OTOH I do not think it would be sufficient for any existing libraries or frameworks, nor for non-professional developers scratching their own itch on a User-managed apps and then deciding to publish it for others to use (which happens a lot with User-managed apps.)

The problem would be that most (all?) of those would not be namespace-prefixing Twig but instead using it directly. I believe you need an ADDITIONAL replace sectionS that allowed an app/website developer to indicate that namespace A should instead be replaced in use statements and direct references with B\A for code that exists in directory(s) C but not in directories C\D where C and D can be globs.

To illustrate I created a completely hypothetical .ini that the WordPress plugin admin page could create any time a user would install WordPress and any time the user would add/edit/delete plugins or themes (I did try to modifying your A/B/C example but couldn't come up with anything that could illustrate the use-case):

[default]
root = '/wp-content/'

[includes]
UpdraftPlus = './plugins/updraft-plus/index.php' # Uses Twig v4
Automattic\JetPack = './plugins/jetpack/jetpack.php' # Uses Twig
Elementor = './plugins/elementor/elementor.php' # Uses Twig v4

[packages]
Yoast\SEO = './plugins/yoast-seo/autoload.ini' # Uses Twig v4 as Yoast\Twig
WPForms = './plugins/wp-forms/autoload.ini' # Uses Twig as WPForms\Twig

[replace]
Twig[UpdraftPlus] = 'Twig_edaf27eb'
Twig[Elementor] = 'Twig_edaf27eb'

In the above example [packages] are ones that have gotten religion and have delivered a best-practices package where they have namespaced Twig. We can ignore them for now as they follow your best-practices.

The [includes] are ones that have paid no attention to newer best practice and/or simply have not been updated by their authors. They were implemented to load and use Twig as simply \Twig.

The [replace] section tells PHP that when \Twig is found included or used within the [includes] files denoted by those namespaces referenced such as UpdraftPlus it should instead use the namespace of \Twig_edaf27eb (dynamically generated by WordPress), and the same goes for any includes and uses by Elementor.

My hypothetical design may not survive a fully-working implementation, but I hope it illustrates that we need to:

1.) Handle those who are NOT following best practices, AND
2.) Alias namespaces when used IN ADDITION TO when imported.

Of course if the code in any of the three plugins included expect the namespaces to be exact via reflection then they would break, but I think it would be a reasonable breakage as most plugins won't do this and most plugins that break could either be updated by their authors or disabled for installs on newer PHP by the WordPress plugin repo.

BTW, Go uses replace in go.mod albeit as a compiled language its use is not a one-to-one analogue to the example above. If you are interested in seeing them in the wild here is what the use of replace looks like for Kubernetes: https://github.com/kubernetes/kubernetes/blob/master/go.mod#L227-L258

As an "optimization", WordPress could recognize that Twig and Twig4 are being used not only by the includes but also by the packages and could generate this optimization instead:

[default]
root = '/wp-content/'

[packages]
Yoast\SEO = './plugins/yoast-seo/autoload.ini' # Uses Twig v4 as Yoast\Twig
WPForms = './plugins/wp-forms/autoload.ini' # Uses Twig as WPForms\Twig

[replace]
Twig[UpdraftPlus] = 'Twig_edaf27eb'
Twig[Elementor] = 'Twig_edaf27eb'
Twig[Yoast\SEO] = 'Twig_edaf27eb'

Twig[Automattic\JetPack] = 'Twig_2ba3f91f'
Twig[WPForms] = 'Twig_2ba3f91f'
As a further optimization, WordPress could reach into all the .ini files recursively and create a SINGLE autoload map, but to do that we would need an additional section: [ignore]:
[ignore]
Yoast\SEO = './plugins/yoast-seo/autoload.ini'
WPForms = './plugins/wp-forms/autoload.ini'
The above assumes that WordPress already generated everything that was needed to no longer need to load the autoload.ini files for either the Yoast\SEO or WPForms namespaces and thus the ignore tells spl_autoload_map() to ignore any calls to these autoload.ini files if called later. (I would have created a complete example but at this point I am too tired for that.)

And lastly, because WordPress would need to generate this and having a web app write to a file is a modern security no-no, then spl_autoload_map() should accept multiple different valid values:

spl_autoload_map( string|array|\PHP\AutoloadMap $map);

String would be the .ini file path
Array would be the format returned by parse_ini_file() for parsing an applicable .ini file
\PHP\AutoloadMap could be a new class containing the required values in object format. (Hopefully adding such a class as a third option would not be controversial to the list members who criticize those developers still wanting to use arrays as hash maps?)

And that is about it for my feedback today.

-Mike

4 months ago by Michael Morris — view source

unread

I went to sleep thinking about this post, on import maps in general and
how Composer works, specifically when you use a class map instead of the
PSR-0 or PSR-4 schemes. In that mode, Composer does pretty much what I've
described. This got me to thinking, could setting an import map be an
alternative to setting an autoload function? Would having the PHP runtime
load the file be faster than mucking with some userland code to do the
same? And would the engine be able to do anything that can't be done in
userland? I think so.

I very much like this direction of thought.

For context, when I worked with WordPress for about a decade I only ever
used only Composer for websites but never for my own plugins, and I almost
never used namespaces nor PSR-4 autoloaders for anything except when a
plugin used them. I almost exclusively used naming convensions for
"namespacing" and classmaps for autoloading.

Some context from where I'm coming from. I have been working exclusively
in React, NodeJS and Go up till about a year ago, and in Drupal before that

it being 10 years since the last time I looked at WordPress. I need work
though, and I was offered a job where about two-thirds of my workload is a
massive WordPress site that has grown well outside the scope WordPress does
best. So I've been getting used to it again and added some ulcers along
the way.

Obviously I agree with having only one format, but not sure I concur with
the use of .ini. However, me debating against .ini would be me
bikeshedding and thus I will demure here.

I'm not married to ini either, but it will work for illustration. Adopting
another format will be one of the first decisions to make come
implementation.

<epiphany> ... Many (most?) people on PHP Internals view WordPress coding standards as bad and some even view addressing WordPress developers needs as bad for PHP....

I really don't want to get into that crossfire. WordPress is the 800lb.
gorilla of the PHP app world having more server market share that is
dominant - how dominant depends on who you ask. The most conservative
estimate I've seen is about half of PHP sites are running WordPress, and
the most pro-WP quote I saw claimed it has around 80% share.

; A path fragment can be used, in which case PSR-4 will be used to map
the rest of the symbol to the filename.

; Pay attention to the direction of the slash at the tail - if the
symbol key has this the value MUST also have this.
B/ = './path/to/B/'

It is not clear to me what a trailing slash means, and especially why it
is needed on the left-hand side?

This is taken from the JavaScript importmap specification, but it exists to
cordon off direct file includes from directory includes. Original spec
here:
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap

And why slash here when namespaces use backslash?

Because it's the end of a path and unless you're on Windows directories are
delimited with /. They aren't the same thing

Also, as someone raised on DOS and then Windows only "converting" in 2009,
I still get confused in *nix when to use a trailing slash and when to not,
so this trailing slash worries me, if only for that reason alone.

I understand. Here it's to call out that the key references a directory,
not a file. The value must explicitly be a directory, not a file, and a
trailing slash is the way that is done. Files are not required to have
extensions after all.

Using the @ here feels cryptic, and hard to discover and remember.

Perhaps, but this file is meant to be assembled by composer or some other
automated means - not by hand. @ as a package operator is used in Node.js
to mark package namespaces - essentially the vendor of the package is
marked with @.

I think this would be infinitely easier to follow if packages were just
included in a [packages] section.

Packages will need to have some special handling though. And the questions
I'm getting make that point increasingly clear.

If, as Larry Gafield has stated, the engine cannot be made to maintain
multiple symbol tables, then the packages must be bound onto the master
symbol table. This presents a problem if multiple packages with different
versions are to be installed. One way to resolve this is some chicanery.
If PHP is instructed to install a file as a "package" it could load it onto
the symbol table in an auto-generated namespace - say \pkg_hash\Twig - and
then symbolic link to the package from other namespaces that are
referencing it. Packages can (must) include other packages as their
dependencies. If the symlinks are done right the end programmers do not
need to concern themselves with this implementation detail.

But the legacy code in the package itself needs to believe it is where it
expects to be - on root. When a Twig file calls to another Twig file it
calls use Twig\Class in some form or another.

Monkey patching such as what happens in
https://packagist.org/packages/brianhenryie/strauss file accomplishes this
in userland by rewriting the files. The whole of the namespace resolution
system is on the fly file renaming, so doing this in the engine should be
possible.

; An import into a package can be done like so
; Twig will load into \C\Twig and that use will need to be used by any
code outside the C package.
@C\Twig/ = './path/to/Twig/'

Your comments also confuse me a bit.

Is this saying that your hypothetical app — which you stated this .ini
file is for — needs to use a package named C use "definition" is located
at './path/to/C/autoload.ini' then it would use this syntax, and that in
the app its components would be accessed at namespace \C?

Packages can have an ini file, but they are not REQUIRED to have one.
Otherwise, the existing library of packages will not be loadable because
they do not have such files. Without the @ operator the library wouldn't
load correctly because all the files in Twig, parsed normally, will load to
\Twig. The @C\ tells PHP to prefix C to every namespace declaration loaded
in that directory and each use statement. This is a new behavior entirely
and needs to be carefully considered.

And I were to have:

@Foo\Bar\Baz = './path/to/Foo/Bar/Baz/autoload.ini'

Then in the app its components would be accessed at namespace
\Foo\Bar\Baz?

Yes.

Okay, this makes sense. OTOH, this is the part that of your proposal that
is incomplete for the needs of User-managed apps IMO.

This is the part that needs the most scrutiny overall.

I think you are implying a necessary "best practice" that whenever any PHP
library, or package would include code they would need to prefix the
namespace of package when importing it and then when using it. Given an org
named ACME that released a library called Widgets then if it were to use
Twig it should import and use Twig like this* (did I understand your
intent correctly?):*

@ACME\Widgets\Twig/ = './path/to/Twig/'

And in PHP code?:

use \ACME\Widgets\Twig;

namespace ACME\Widgets;
use Twig\Extension;

Namespace resolution rules can be tricky.

I think that would work well for newer libraries and packages authored and
used by developers of Developer-managed apps. OTOH I do not think it would
be sufficient for any existing libraries or frameworks, nor for
non-professional developers scratching their own itch on a User-managed
apps and then deciding to publish it for others to use (which happens a
lot with User-managed apps.)

The problem would be that most (all?) of those would not be
namespace-prefixing Twig but instead using it directly. I believe you need
an ADDITIONAL replace sectionS that allowed an app/website developer to
indicate that namespace A should instead be replaced in use statements
and direct references with B\A for code that exists in directory(s) C
but not in directories C\D where C and D can be globs.

...

[replace]
Twig[UpdraftPlus] = 'Twig_edaf27eb'
Twig[Elementor] = 'Twig_edaf27eb'

No, PHP should handle this quietly under the hood without needing the
autocomplete author to do this whether it is a person or a userland program
like Composer.

And lastly, because WordPress would need to generate this and having a web
app write to a file is a modern security no-no, then spl_autoload_map()
should accept multiple different valid values:

spl_autoload_map( string|array|\PHP\AutoloadMap $map);

String would be the .ini file path

Array would be the format returned by parse_ini_file() for parsing an
applicable .ini file

\PHP\AutoloadMap could be a new class containing the required values in
object format. (Hopefully adding such a class as a third option would
not be controversial to the list members who criticize those developers
still wanting to use arrays as hash maps?)

And that is about it for my feedback today.

I'm not opposed to that. Indeed, doing that would allow for additional
formats without the overhead of directly maintaining them

spl_autoload_map(json_decode('autoload.json'))

Now earlier today I had a thought that shook me. The entry point of the
application in this example will likely call this function before anything
else, just as applications like Drupal call the autoload require before
anything else.

So, why not call the map first? then let the map designate the entry point?

A HUGE Can of opportunity worms opens. This is also a wild tangent, but
again this is a brainstorm thread.

; First this autoload map needs to tell PHP what it is. A Package, or an
application.
map_type = application

; The Setup files are those which always need to be run before the
application is
; ready to do anything. These files will be loaded with NO ACCESS to
globals or
; or super globals. The engine's state after running these files will be
op cached and
; each subsequent request starts from here. This allows the considerable
setup work
; that applications like Drupal do to be resolved only once and then
start from that
; cache. This is also the reason why superglobals aren't visible to these
files - the
; state cannot make decisions about any specific request because they'd
bleed into
; subsequent requests - a potential security nightmare.
;
; Note the autoloader of this script will be setup in full before these
files parse.
[setup]
[] = 'path/to/first/setup/file.php'
[] = 'path/to/second/setup/file.php'

Just a thought.

4 months ago by Arvids Godjuks — view source

unread

<epiphany> ... Many (most?) people on PHP Internals view WordPress coding standards as bad and some even view addressing WordPress developers needs as bad for PHP....

I really don't want to get into that crossfire. WordPress is the 800lb.
gorilla of the PHP app world having more server market share that is
dominant - how dominant depends on who you ask. The most conservative
estimate I've seen is about half of PHP sites are running WordPress, and
the most pro-WP quote I saw claimed it has around 80% share.

Automattic has chosen on purpose to be the way it is and they chosen to
rewrite everything into NodeJS, not that it will really help them anyway :)

4 months ago by Mike Schinkel — view source

unread

Some context from where I'm coming from. I have been working exclusively in React, NodeJS and Go up till about a year ago, and in Drupal before that - it being 10 years since the last time I looked at WordPress. I need work though, and I was offered a job where about two-thirds of my workload is a massive WordPress site that has grown well outside the scope WordPress does best. So I've been getting used to it again and added some ulcers along the way.

Yes, a university semester could be filled with the dynamics of open-source when a project's user base and those overseeing its governance diverge. #fwiw

BTW, you don't happen to be working for my last WordPress client, are you? 😲

<epiphany> ... Many (most?) people on PHP Internals view WordPress coding standards as bad and some even view addressing WordPress developers needs as bad for PHP....
I really don't want to get into that crossfire. WordPress is the 800lb. gorilla of the PHP app world having more server market share that is dominant - how dominant depends on who you ask. The most conservative estimate I've seen is about half of PHP sites are running WordPress, and the most pro-WP quote I saw claimed it has around 80% share.

The reality is it does not matter if you want to or not, by arguing for a feature on PHP Internals that is needed more by user-managed apps like WordPress than developer-managed apps, you have stepped into said crossfire.

You do not get to pick your reality, your reality picks you.

B/ = './path/to/B/'

It is not clear to me what a trailing slash means, and especially why it is needed on the left-hand side?

This is taken from the JavaScript importmap specification, but it exists to cordon off direct file includes from directory includes. Original spec here: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap

And why slash here when namespaces use backslash?

Because it's the end of a path and unless you're on Windows directories are delimited with /. They aren't the same thing

Also, as someone raised on DOS and then Windows only "converting" in 2009, I still get confused in *nix when to use a trailing slash and when to not, so this trailing slash worries me, if only for that reason alone.

I understand. Here it's to call out that the key references a directory, not a file. The value must explicitly be a directory, not a file, and a trailing slash is the way that is done. Files are not required to have extensions after all.

Using the @ here feels cryptic, and hard to discover and remember.

Perhaps, but this file is meant to be assembled by composer or some other automated means - not by hand. @ as a package operator is used in Node.js to mark package namespaces - essentially the vendor of the package is marked with @.

I can tell your design here have been heavily influenced by your stated extensive experience with Node.js. Node.js' conventions probably seem second nature to you and thus easy for you to understand.

But for those us not steeped in Node.js, they are rather cryptic. Any time language designers chooses cryptic over obvious they place a large burden on learners and relatedly also on the experienced to deal with, correct and educate learners. For the languages targeting larger user bases like PHP — vs, say Haskell — the negative impact of cryptic can be widespread and costly in time spent and user base lost.

So even if Node.js made the regrettable decision to choose cryptic over obvious I would implore anyone considering additions to PHP to side with obvious over cryptic. Even when said additions will often be automated, unless they will always be automated, such as the OpCache.

What does "obvious" mean here? Assuming .ini files are use, then sections names that define usage, not special sigils.

I think this would be infinitely easier to follow if packages were just included in a [packages] section.

Packages will need to have some special handling though.

Which does not preclude a special section for [packages].

If, as Larry Gafield has stated, the engine cannot be made to maintain multiple symbol tables, then the packages must be bound onto the master symbol table.

I have heard that repeatedly, but I have yet to hear why.

PHP is just a C program, and clearly a C program can have multiple data structures. Linux had a similar problem with filesystems and thus UnionFS was created. I'd really like to know why an equivalent architecture cannot be achieved with PHP's symbol table.

(If the reason why PHP "cannot be made to maintain multiple symbol tables" has been stated already, I could have easily missed it. If so and someone has a link to externals.io with the explanation, it would really appreciate that link.)

But the legacy code in the package itself needs to believe it is where it expects to be - on root. When a Twig file calls to another Twig file it calls use Twig\Class in some form or another.

Monkey patching such as what happens in https://packagist.org/packages/brianhenryie/strauss file accomplishes this in userland by rewriting the files. The whole of the namespace resolution system is on the fly file renaming, so doing this in the engine should be possible.

; An import into a package can be done like so
; Twig will load into \C\Twig and that use will need to be used by any code outside the C package.
@C\Twig/ = './path/to/Twig/'

Your comments also confuse me a bit.

Is this saying that your hypothetical app — which you stated this .ini file is for — needs to use a package named C use "definition" is located at './path/to/C/autoload.ini' then it would use this syntax, and that in the app its components would be accessed at namespace \C?

Packages can have an ini file, but they are not REQUIRED to have one. Otherwise, the existing library of packages will not be loadable because they do not have such files.

Yes, hopefully that would be obvious to everyone.

Without the @ operator the library wouldn't load correctly because all the files in Twig, parsed normally, will load to \Twig. The @C\ tells PHP to prefix C to every namespace declaration loaded in that directory and each use statement. This is a new behavior entirely and needs to be carefully considered.

Again, cryptic.

Certainly there is more than one way to indicate this required behavior besides a cryptic sigil.

I think that would work well for newer libraries and packages authored and used by developers of Developer-managed apps. OTOH I do not think it would be sufficient for any existing libraries or frameworks, nor for non-professional developers scratching their own itch on a User-managed apps and then deciding to publish it for others to use (which happens a lot with User-managed apps.)

The problem would be that most (all?) of those would not be namespace-prefixing Twig but instead using it directly. I believe you need an ADDITIONAL replace sectionS that allowed an app/website developer to indicate that namespace A should instead be replaced in use statements and direct references with B\A for code that exists in directory(s) C but not in directories C\D where C and D can be globs.

...
[replace]
Twig[UpdraftPlus] = 'Twig_edaf27eb'
Twig[Elementor] = 'Twig_edaf27eb'

No, PHP should handle this quietly under the hood without needing the autocomplete author to do this whether it is a person or a userland program like Composer.

I am pretty sure having PHP "handle it quietly under the hood" could never be performant enough for real-world production use on medium-to-high traffic websites. That is why I proposed it be prepared in advance by the actor that "builds" the app's source code.

For developer-managed apps that actor is typically Composer. But for user-managed apps that actor is the user-managed app itself, i.e. WordPress and the code in its plugin admin page. WordPress would store than info in its MySQL database which is why — for others reading this since you already agreed — spl_autoload_map() would need to be able to get the map as an array vs. as just a file.

But I may be wrong. However, one thing is certain; if I am correct that it could not be done performantly in PHP then it will never get added to PHP in that form, even if an RFC were to pass. So there is really only one way to know and that is to develop a proof-of-concept, of which the relevant part here could be implement in userland PHP.

As an side, I have already started piddling with this PoC, but I am not sure how far I can or will take it. But I might be open to collaborating on it if you are interested.

Now earlier today I had a thought that shook me. The entry point of the application in this example will likely call this function before anything else, just as applications like Drupal call the autoload require before anything else.

So, why not call the map first? then let the map designate the entry point?

A HUGE Can of opportunity worms opens. This is also a wild tangent, but again this is a brainstorm thread.

; First this autoload map needs to tell PHP what it is. A Package, or an application.
map_type = application

Actually, this seems like a pretty obvious next step once one accepts the idea of autoload maps.

; The Setup files are those which always need to be run before the application is
; ready to do anything. These files will be loaded with NO ACCESS to globals or
; or super globals. The engine's state after running these files will be op cached and
; each subsequent request starts from here. This allows the considerable setup work
; that applications like Drupal do to be resolved only once and then start from that
; cache.

Although I like the architecture of having a file define things declaratively where they can easily be processed by tooling — I tried and failed to find a way to standardize wp-config.php — I am not sure how this would empower the OpCache any more than regular code loading from index.php?

This is also the reason why superglobals aren't visible to these files - the
; state cannot make decisions about any specific request because they'd bleed into
; subsequent requests - a potential security nightmare.
;
; Note the autoloader of this script will be setup in full before these files parse.
[setup]
[] = 'path/to/first/setup/file.php'
[] = 'path/to/second/setup/file.php'

Bikeshedding a bit, I am not sure "setup" is the right word, but as it is bikeshedding I will move on.

While I like the idea of a declarative file to empowering tooling, I am not sure how introducing such as file into PHP's page load process provides enough improvement vs. simply include()ing "setup" php files in the first few lines of index.php? I think there would need to be more types of things such a file could standardize and improve performance for would need to get buy-in and support for such a change.

To figure that out what those types of things might be would IMO requiring looking at widely-used PHP apps — both user and developer managed apps — to see what things could see an actual benefit by defining them in a file like this. For example, could a database connection be kept alive between page loads on a high-traffic site? (I have zero idea if this latter would increase performance or even be a good idea, it is just the first thing that came to mind.)

-Mike

4 months ago by Larry Garfield — view source

unread

<epiphany>
Reading this however caused me to ponder things certain people has said
recently — and many people have said for years on this list — and I
think I am recognizing something that I have always known but never put
the pieces together before.

Many (most?) people on PHP Internals view WordPress coding standards as
bad and some even view addressing WordPress developers needs as bad for
PHP. And in general I concur that those people are reasonably justified
in their belief WordPress' coding standards are not the standards that
PHP developer who want to do professional level software engineering
should aspire.

And since many (most?)* *PHP Internals members generally do not
experience the issues that WordPress developers have they do not
recognize that they are issues; IOW, *"out of sight, out of mind." *

I also think some list members tend to dismiss WordPress developers
pains as unimportant and/or think that addressing those pains have will
harm* *PHP.

(BTW, I recently had a dialog off-list with someone who wrote in an
email that *"Wordpress is an exception, but nobody these days treats
WordPress as a valid example to do anything. It is an ancient piece of
legacy code that has no bearing on modern situation and it's their
problem to deal with." *So I am not just erecting a straw man here.)

But I think what most may not consciously recognize is that* WordPress
is a different type of web app* than an app build using Symfony or
Laravel and deployed by its developers, or by some other professional
developer.

WordPress differs from the apps many (most?) developers on PHP
Internals work with in the following way:

WordPress = User-managed app
Most = Developer-managed apps

In a* Developer-Managed app* developers choose which 3rd party
functionality will be incorporated into their sites whereas with a
User-managed app users choose which 3rd party functionality will be
incorporated into their site. And that is the KEY difference.

So I am wondering if we can get people on this PHP Internals list who
dismiss the needs of WordPress developer BECAUSE it is WordPress to
recognize that User-Managed apps ARE a class of PHP applications have
needs that deserve to be addressed?
*
*
Two (2)* unmet needs of User-Managed apps *that *"standard" *PHP
currently does not address come to mind:

User-managed apps needs to be able to handle both:
*
*

User-added add-ons *("plugins" in WordPress, "modules" in Drupal)
*that have conflicting dependencies, and

*Add-on directory structures *that do not follow a PSR-4 directory hierarchy.

As for #2, even if those apps could rearchitect their existing
directory structure they cannot realistically be expected to do with
because of the huge BC issues their users would experience.

And newly created User-managed apps may still find that a PSR-4
directory structure is not in the best interest of their project or
their users. To elaborate, PSR-4 generally assumes that ALL code goes
into ONE hierarchy and that any and all code that will be autoload gets
placed in that hierarchy.

But with add-ons it makes a lot more sense to have the entire add-on
contained in its own add-on directory. This is exactly where PSR-4
breaks down with respect to User-managed apps.

Sure, you can have multiple PSR-4 autoloader root directories, but that
does not scale well to websites with a large number of add-ons as many
WordPress sites I worked on used. Some had over 100 plugins. With a
hierarchy of autoloader maps that Michael Morris is proposing WordPress
could collect up all the maps and create one map every time a plugin is
added, updated or deleted.
</epiphany>

I am going to jump in here on this point specifically, because it seems to be a mix of genuinely insightful observation (though not unique) and uninformed FUD.

Some context: I haven't seriously used Wordpress in, ever. However, I was a Drupal lead developer for many years, and wrote, among other things, Drupal's DBAL, Drupal's first autoloader, Drupal's PSR-3 implementation, was involved in Drupal's file organization guidelines for Drupal 8+ (when Drupal adopted a PSR-0/4 autoloader), and led the Drupal 8 "Modernize all the things" effort. So I do have some non-trivial experience in this area.

First, you're correct that there is an architectural difference between "projects that assume the owner has CLI access" and those that do not. You are also correct that most of the Internals crowd comes from the former.

However, I don't think it's fair to say that's why Internals folks "dismiss" Wordpress generally. We dismiss Wordpress generally because

WP actively harms the PHP community by encouraging the use of ancient PHP versions with known security issues.
WP's code base actively avoids using what have been considered known best-practices (in either type of application) for 15 years.
WP's core team actively avoids being involved in Internals to collaborate on how to make the language better for them. In fact, they've made it very clear that PHP is a legacy implementation detail and Node/client-side JS is where their focus is. The only WP-affiliated person I can even think of that has been a semi-regular Internals contributor is Juliette (whose participation I very much welcome).

That said, it was recently pointed out to me that Automattic is the top contributor to the PHP Foundation (https://opencollective.com/phpfoundation), which is very much appreciated and nothing to sneeze at.

And yes, I fully agree that any module/package/thing needs to take into account the needs of both types of projects. As Rowan has repeated, that means keeping the impact of any changes minimal, so that the Composer ecosystem and WP ecosystem can TYPO3 ecosystem can build their own tooling on top of it.

You'll note I did not list Drupal there. That's because modern Drupal is composer-based, and has been for many years. I was the one that pushed hard for adopting Composer, its autoloader, and PSR-4 for Drupal 8 in the first place. While much of the transition happened after I left the project, the groundwork is over a decade old. Composer is the preferred way to use Drupal, and to install Drupal modules.

So the line is not as hard between those two models as you might think.

PSR-4 generally assumes that ALL code goes
into ONE hierarchy and that any and all code that will be autoload gets
placed in that hierarchy.

This is flatly untrue, and belies a considerable ignorance about how PSR-4 and Composer work.

PHP supports multiple autoloader callbacks, and has for over 15 years. You absolutely can register multiple if you'd like, using whatever logic you like. PHP will call each one in turn until the class is loaded.

All PSR-4 does is specify a directory structure that makes a common autoloader stupidly simple to write. It's just a few lines long. But you can already do any logic you like for an autoloader. PHP doesn't care.

However, nothing precludes you from registering multiple autoloaders, all using PSR-4, all using a different path root. That has been trivially simple to do since 2009. (OK, it was PSR-0 at the time, but the implications here are the same.) So your statement above about "all code goes into one hierarchy" is simply flat out false.

Of course, as you note, registering lots of separate autoloaders has a performance impact. That is true. Which is why I don't think anyone actually does that.

Composer, for instance, registers a single autoloader only. That autoloader internally tracks many dozens of PSR-4 roots (one for each package, sometimes two per package), as well as files that will get force-loaded when the autoloader is registered, plus generated classmaps.

Using class maps, you can put a hundred classes in one file and composer can handle that today. That has always been possible. That no one does so is a sign that there's little reason to do so in most cases.

In fact, if you use an optimized/dumped autoloader, then Composer simply builds an internal giant lookup table of what class maps to what file. PSR-4 is then completely irrelevant at runtime. It's already one giant O(1) lookup map. That can be done today. That is done today.

But what about systems like Drupal, that don't put code in /vendor/? Drupal ties directly into Composer via its API,and has done so for a decade. Drupal, a "user-managed application" as you describe it, has Composer baked in at a core level.

It looks like the integration has evolved considerably since I was last involved, but have a look at:

https://git.drupalcode.org/project/drupal/-/tree/11.x/composer

As of when I last looked at it (around 2016 or so), Drupal registers its module code roots with Composer directly, and then Composer takes over from there and integrates Drupal's code into its own indexes. There is still only one single autoloader registered with PHP. This is entirely fine.

I would encourage you to do your research before speaking pseudo-authoritatively on this topic, as you clearly are mis-stating both the problem and the tools involved today.

What Drupal does not do is address the "different dependency version" question. And neither does Wordpress. Or TYPO3. Or any other project. Because that's a core PHP limitation.

In Python, every module is really just an object with a big dictionary of the functions/classes it has. When you "import" a symbol from another module, the engine is doing little more than $this['foo'] &= $that['foo']. That's a core part of how the language works. (I'm not sure of Javascript's details, but I suspect from using it that it's similar.)

PHP works very very differently. PHP has a single global list of symbols. (Well, two, for classes and functions.) Namespaces are just syntax sugar over very-long-names, nothing more. There is no "local symbol table," so having different local symbol tables point to different code blocks using the same name is not even conceivable.

If you want to change that, and give PHP multiple local symbol tables, then autoloading... is utterly irrelevant. The question there is "how can we introduce local symbol tables in the engine without requiring 10 million developers to rewrite the file header of 1 billion PHP files across the world?" Honestly, I'm not convinced its even possible. Someone with more engine knowledge than I could be able to find away, maybe, but I am skeptical. If it's even possible, I suspect it would be an absurdly large amount of work and necessarily include many hard BC breaks.

If you'd like to prove me wrong, go for it. But that's the problem to address. Debating file paths is about four steps down the line before it's even relevant. And even then... if you can't make a PSR-4-organized package (of which there are several hundred thousand) slot into that new model comfortably with zero effort on the part of the package author, it's doomed.

So please, spare us the ill-informed descriptions of how you think autoloaders work, when you have demonstrated you do not know how they work.

Spare us the litany of complaints about PSR-4 when you have demonstrated you don't know what PSR-4 says.

Spare us the gnashing of teeth about how hard it is to use a Wordpress plugin that hasn't been updated in 10 years with a modern plugin because the former is still using a 10 year old abandoned version of some library, when that's not PHP's problem, that's a Wordpress maintenance problem.

If you want to move this effort forward, here's your todo list:

Do some research in the engine to determine if local symbol tables are even possible without rewriting the engine.
Work through the highly complex logic of handling three layer overlapping transitive dependencies in a diamond pattern with conflicting version requirements.
Investigate the performance impact of maintaining multiple versions of the same code in memory at once, when the order they get loaded will vary by request.
Think through how you'd support both composer-based and "user managed" applications with such a model, especially projects that are already architecturally a decade out of date (like Wordpress).

When you have a proven that it's even possible to have multiple local symbol tables, we can talk. Until then, please spare us.

--Larry Garfield

4 months ago by Rob Landers — view source

unread

<epiphany>
Reading this however caused me to ponder things certain people has said
recently — and many people have said for years on this list — and I
think I am recognizing something that I have always known but never put
the pieces together before.

Many (most?) people on PHP Internals view WordPress coding standards as
bad and some even view addressing WordPress developers needs as bad for
PHP. And in general I concur that those people are reasonably justified
in their belief WordPress' coding standards are not the standards that
PHP developer who want to do professional level software engineering
should aspire.

And since many (most?)* *PHP Internals members generally do not
experience the issues that WordPress developers have they do not
recognize that they are issues; IOW, *"out of sight, out of mind." *

I also think some list members tend to dismiss WordPress developers
pains as unimportant and/or think that addressing those pains have will
harm* *PHP.

(BTW, I recently had a dialog off-list with someone who wrote in an
email that *"Wordpress is an exception, but nobody these days treats
WordPress as a valid example to do anything. It is an ancient piece of
legacy code that has no bearing on modern situation and it's their
problem to deal with." *So I am not just erecting a straw man here.)

But I think what most may not consciously recognize is that* WordPress
is a different type of web app* than an app build using Symfony or
Laravel and deployed by its developers, or by some other professional
developer.

WordPress differs from the apps many (most?) developers on PHP
Internals work with in the following way:

WordPress = User-managed app
Most = Developer-managed apps

In a* Developer-Managed app* developers choose which 3rd party
functionality will be incorporated into their sites whereas with a
User-managed app users choose which 3rd party functionality will be
incorporated into their site. And that is the KEY difference.

So I am wondering if we can get people on this PHP Internals list who
dismiss the needs of WordPress developer BECAUSE it is WordPress to
recognize that User-Managed apps ARE a class of PHP applications have
needs that deserve to be addressed?
*
*
Two (2)* unmet needs of User-Managed apps *that *"standard" *PHP
currently does not address come to mind:

User-managed apps needs to be able to handle both:
*
*

User-added add-ons *("plugins" in WordPress, "modules" in Drupal)
*that have conflicting dependencies, and

*Add-on directory structures *that do not follow a PSR-4 directory hierarchy.

As for #2, even if those apps could rearchitect their existing
directory structure they cannot realistically be expected to do with
because of the huge BC issues their users would experience.

And newly created User-managed apps may still find that a PSR-4
directory structure is not in the best interest of their project or
their users. To elaborate, PSR-4 generally assumes that ALL code goes
into ONE hierarchy and that any and all code that will be autoload gets
placed in that hierarchy.

But with add-ons it makes a lot more sense to have the entire add-on
contained in its own add-on directory. This is exactly where PSR-4
breaks down with respect to User-managed apps.

Sure, you can have multiple PSR-4 autoloader root directories, but that
does not scale well to websites with a large number of add-ons as many
WordPress sites I worked on used. Some had over 100 plugins. With a
hierarchy of autoloader maps that Michael Morris is proposing WordPress
could collect up all the maps and create one map every time a plugin is
added, updated or deleted.
</epiphany>

I am going to jump in here on this point specifically, because it seems to be a mix of genuinely insightful observation (though not unique) and uninformed FUD.

Some context: I haven't seriously used Wordpress in, ever. However, I was a Drupal lead developer for many years, and wrote, among other things, Drupal's DBAL, Drupal's first autoloader, Drupal's PSR-3 implementation, was involved in Drupal's file organization guidelines for Drupal 8+ (when Drupal adopted a PSR-0/4 autoloader), and led the Drupal 8 "Modernize all the things" effort. So I do have some non-trivial experience in this area.

First, you're correct that there is an architectural difference between "projects that assume the owner has CLI access" and those that do not. You are also correct that most of the Internals crowd comes from the former.

However, I don't think it's fair to say that's why Internals folks "dismiss" Wordpress generally. We dismiss Wordpress generally because

WP actively harms the PHP community by encouraging the use of ancient PHP versions with known security issues.

WP's code base actively avoids using what have been considered known best-practices (in either type of application) for 15 years.

WP's core team actively avoids being involved in Internals to collaborate on how to make the language better for them. In fact, they've made it very clear that PHP is a legacy implementation detail and Node/client-side JS is where their focus is. The only WP-affiliated person I can even think of that has been a semi-regular Internals contributor is Juliette (whose participation I very much welcome).

That said, it was recently pointed out to me that Automattic is the top contributor to the PHP Foundation (https://opencollective.com/phpfoundation), which is very much appreciated and nothing to sneeze at.

I have some insight on this, but also, there are people here who are ex-coworkers from Automattic. I haven't worked there for a couple years now, but I did get involved in these discussions ... as much as (un)reasonably possible because I wanted to see WordPress Core (the free version) modernized. That being said, I don't speak for any of them; these are just my personal observations.

Much of the constraints there actually comes from [cheap] hosting. If they make it harder to upgrade ... people just won't upgrade. Getting people to upgrade is already challenging enough. Though, there was a lot of work fixing that problem when I left.
You can purchase backported PHP versions with all the security patches applied. There's literally an entire industry keeping old php versions running; so that argument isn't really valid. And yes, this was also one of my original arguments...
The WordPress codebase seems to have taken a different branch through history. But I wouldn't say that they avoid best-practices. Are there a lot of plugin developers that do? Yes. Yes they do. But if you have a lightweight strategy-pattern (apply_filters/do_action), you'd be dumb not use it. WordPress does use it, and they use it quite well to do a lot of things you don't really see in other CMS's/codebases because strategy-patterns are usually quite expensive.
And finally, I *do *see colleagues on here giving feedback, they just don't declare they have anything to do with Automattic or WordPress. Why would they?

— Rob

4 months ago by Mike Schinkel — view source

unread

WordPress differs from the apps many (most?) developers on PHP
Internals work with in the following way:

WordPress = User-managed app
Most = Developer-managed apps

In a* Developer-Managed app* developers choose which 3rd party
functionality will be incorporated into their sites whereas with a
User-managed app users choose which 3rd party functionality will be
incorporated into their site. And that is the KEY difference.

So I am wondering if we can get people on this PHP Internals list who
dismiss the needs of WordPress developer BECAUSE it is WordPress to
recognize that User-Managed apps ARE a class of PHP applications have
needs that deserve to be addressed?
*
*
Two (2)* unmet needs of User-Managed apps *that *"standard" *PHP
currently does not address come to mind:

User-managed apps needs to be able to handle both:
*
*

User-added add-ons *("plugins" in WordPress, "modules" in Drupal)
*that have conflicting dependencies, and

*Add-on directory structures *that do not follow a PSR-4 directory hierarchy.

As for #2, even if those apps could rearchitect their existing
directory structure they cannot realistically be expected to do with
because of the huge BC issues their users would experience.

And newly created User-managed apps may still find that a PSR-4
directory structure is not in the best interest of their project or
their users. To elaborate, PSR-4 generally assumes that ALL code goes
into ONE hierarchy and that any and all code that will be autoload gets
placed in that hierarchy.

But with add-ons it makes a lot more sense to have the entire add-on
contained in its own add-on directory. This is exactly where PSR-4
breaks down with respect to User-managed apps.

Sure, you can have multiple PSR-4 autoloader root directories, but that
does not scale well to websites with a large number of add-ons as many
WordPress sites I worked on used. Some had over 100 plugins. With a
hierarchy of autoloader maps that Michael Morris is proposing WordPress
could collect up all the maps and create one map every time a plugin is
added, updated or deleted.
</epiphany>

I am going to jump in here on this point specifically, because it seems to be a mix of genuinely insightful observation (though not unique) and uninformed FUD.

Some context: I haven't seriously used Wordpress in, ever. However, I was a Drupal lead developer for many years, and wrote, among other things, Drupal's DBAL, Drupal's first autoloader, Drupal's PSR-3 implementation, was involved in Drupal's file organization guidelines for Drupal 8+ (when Drupal adopted a PSR-0/4 autoloader), and led the Drupal 8 "Modernize all the things" effort. So I do have some non-trivial experience in this area.

Yes. I remember that about you as well, Crell.

First, you're correct that there is an architectural difference between "projects that assume the owner has CLI access" and those that do not. You are also correct that most of the Internals crowd comes from the former.

However, I don't think it's fair to say that's why Internals folks "dismiss" Wordpress generally.

No, I said you dismiss the concerns of user-managed apps because you dismiss WordPress, which we know because you just gave a list of why you dismiss WordPress.

What I was hoping was to make the distinction that user-managed apps — regardless of if they are WordPress, are some other user-managed app — have needs that are unfair to demonize simply because you have contempt for the poster-child of user-managed apps.

And yes, I fully agree that any module/package/thing needs to take into account the needs of both types of projects. As Rowan has repeated, that means keeping the impact of any changes minimal, so that the Composer ecosystem and WP ecosystem can TYPO3 ecosystem can build their own tooling on top of it.

You'll note I did not list Drupal there. That's because modern Drupal is composer-based, and has been for many years. I was the one that pushed hard for adopting Composer, its autoloader, and PSR-4 for Drupal 8 in the first place. While much of the transition happened after I left the project, the groundwork is over a decade old. Composer is the preferred way to use Drupal, and to install Drupal modules.

Interesting regarding Drupal 8 and beyond.

The ONE place where I will accept your claims that my comments have been "uninformed" and/or "ignorant" has been WRT the details of using Drupal 8+ and beyond. BTW, based on your comments about Drupal and my subsequent analysis it turns out that Drupal is now much more of a developer-managed app than a user-managed app.

I started my PHP career on Drupal in 2008 and worked with it for two years until the work dried up and I had to switch to WordPress. I later found a paying Drupal project but after two weeks I fired myself because working with Drupal was such a pain compared to working with WordPress (long story, but basically it came down to theming architectures — Drupal's was coupled vs. WordPress' being decoupled — making Drupal much harder to theme when compared to WordPress.)

Interesting that Drupal 8+ fully embraced PSR-4. I was aware of their embrace of Symfony in Drupal 8 and even wrote a blog post[1] right at 10 years ago on how that decision would result in a sizable decline in Drupal's user base over time. I drew on my experience watching the precipitous decline of Visual Basic when Microsoft transitioned from Visual Basic 6 to VB.NET — and history shows I was right about Drupal, at least in predicting the sizable market decline. Visual Basic was once the most widely-used programming language in the world. But after moving to VB.NET it fell into almost complete obscurity.

In the past decade Drupal's marketshare of CMS has fallen significantly, from around 7.2% of CMS in Jan 2013 to 1.4% in July 2024. Of course there are far too many factors to say Drupal's decline was in-fact because of their embrace of complexity but it certainly correlates. A more charitable analysis is that Drupal chose to go upmarket to the enterprise which needed that complexity, but that would also imply that the rest of the market moved on from Drupal likely because the complexity no longer met their needs. #fwiw

So the line is not as hard between those two models as you might think.

PSR-4 generally assumes that ALL code goes
into ONE hierarchy and that any and all code that will be autoload gets
placed in that hierarchy.

This is flatly untrue, and belies a considerable ignorance about how PSR-4 and Composer work.

No, It shows that you ignored my use of the word "generally."

And the only thing it belies is that you chose the least charitable characterization of my knowledge of how PSR-4 and Composer work that you could have, instead of charitably presuming that maybe I actually do have valuable insight you do not have.

I do not presume the same of you — knowing you have knowledge I do no have in various areas. I do however make exception for those areas where you both claim a lack of experience and then demonstrate a lack of knowledge, as I reveal in sections below.

Btw, if I revised my words to avoid being imprecise with "generally" I would write that PHP encourages too many autoloader callbacks, and the way most of the PHP community tries to manage that is to condense all those independent groups of code each with their own autoloader callback, down into as few PSR-4 hierarchies that with as few autoloader callbacks as possible.

Just like procedural __get and __set magic methods seemed like a good idea at the time but now declarative property hooks will supplant them so to I argue that procedural autoloaders — which seemed like a good idea at the time — have the same types of issues as procedural magic methods and that moving to a declarative form for autoloading would improve PHP significantly.

PHP supports multiple autoloader callbacks, and has for over 15 years. You absolutely can register multiple if you'd like, using whatever logic you like. PHP will call each one in turn until the class is loaded.

Of course PHP supports multiple autoloader callbacks. I know, I had to work with many autoloaders over the years of developing WordPress sites and plugins.

But PHP supporting multiple autoloader callbacks is far from the benefit you are insinuating when they proliferate.

Consider a user-managed WordPress site — and I worked on many like this after the site had gotten so slow the user had to hire a developer — where the user has 50+ active plugins on the site. And then consider that up to 51 autoloaders had been registered, or on average 25 autoloaders run EVERY new symbol load attempt which results in a performance and complexity nightmare.

So if there is ignorance in this thread, it is the blissful ignorance of the issues that too many autoloader callbacks can cause. And that ignorance from someone who directly claims "I haven't seriously used Wordpress in, ever."

So please let us both acknowledge what each of us do know vs. what we do not, and then accept that the other likely has insight into the areas the opposite person does not know.

All PSR-4 does is specify a directory structure that makes a common autoloader stupidly simple to write. It's just a few lines long. But you can already do any logic you like for an autoloader. PHP doesn't care.

However, nothing precludes you from registering multiple autoloaders, all using PSR-4, all using a different path root. That has been trivially simple to do since 2009. (OK, it was PSR-0 at the time, but the implications here are the same.) So your statement above about "all code goes into one hierarchy" is simply flat out false.

Of course, as you note, registering lots of separate autoloaders has a performance impact. That is true. Which is why I don't think anyone actually does that.

No one, well, except everyone who writes a plugin for WordPress.

And in production, except everyone who runs multiple plugins, which is practically everyone who runs a WordPress site.

Again, your admitted lack of ever using WordPress is showing your lack of experience on this topic.

Composer, for instance, registers a single autoloader only. That autoloader internally tracks many dozens of PSR-4 roots (one for each package, sometimes two per package), as well as files that will get force-loaded when the autoloader is registered, plus generated classmaps.

Using class maps, you can put a hundred classes in one file and composer can handle that today. That has always been possible. That no one does so is a sign that there's little reason to do so in most cases.

What is your evidence of your claim that "no one does so?"

I just pulled up the source for one my most recently WordPress clients — the first one I looked at since your email — and searched for spl_autoload_register(" in the /wp-content/plugins/` directory. I found 43 different autoloaders being registered. Here is a screenshot of that:

https://postimg.cc/TyWjW6Px

So clearly yes, "people do so." And many of them do. True, they are not inside the bubble where you reside, a bubble where no other residents do that either. But outside that bubble the practice is widespread as there is not a better way that is also realistic for the individual site builder to choose.

In fact, if you use an optimized/dumped autoloader, then Composer simply builds an internal giant lookup table of what class maps to what file. PSR-4 is then completely irrelevant at runtime. It's already one giant O(1) lookup map. That can be done today. That is done today.

Yes, it can be done today. It is done today. By. Developer. Managed. Apps.

But what about systems like Drupal, that don't put code in /vendor/? Drupal ties directly into Composer via its API,and has done so for a decade. Drupal, a "user-managed application" as you describe it, has Composer baked in at a core level.

It looks like the integration has evolved considerably since I was last involved, but have a look at:

https://git.drupalcode.org/project/drupal/-/tree/11.x/composer

Like I previously said, if the team managing a project does not care about BC — as Drupal apparently did not in its transition to Composer — then tying Composer into the core is not a problem.

But if they DO care about BC — as WordPress very much cares about — then fully integrating Composer into an existing user-managed app is a non-starter. And as Drupal's decline in user base over the past decade shows, doing so also which carries a high level of risk of shedding user base, too.

As of when I last looked at it (around 2016 or so), Drupal registers its module code roots with Composer directly, and then Composer takes over from there and integrates Drupal's code into its own indexes. There is still only one single autoloader registered with PHP. This is entirely fine.

Yes, as the Drupal module Ludwig[3] explains. It can be managed by a Drupal module, but even its own authors do not recommend doing so, stating (parenthetical mine): "However, please note that Composer (on the CLI) is highly recommended whenever possible!"

Which again, means current Drupal is now effectively a developer-managed app.

I would encourage you to do your research before speaking pseudo-authoritatively on this topic, as you clearly are mis-stating both the problem and the tools involved today.

Back at you; see the above.

What Drupal does not do is address the "different dependency version" question. And neither does Wordpress. Or TYPO3. Or any other project. Because that's a core PHP limitation.

That too.

PHP works very very differently. PHP has a single global list of symbols. (Well, two, for classes and functions.) Namespaces are just syntax sugar over very-long-names, nothing more. There is no "local symbol table," so having different local symbol tables point to different code blocks using the same name is not even conceivable.

If you want to change that, and give PHP multiple local symbol tables, then autoloading... is utterly irrelevant. The question there is "how can we introduce local symbol tables in the engine without requiring 10 million developers to rewrite the file header of 1 billion PHP files across the world?"

You have not shown how having a local symbol table "requires 10 million developer to rewrite header files..."
As I envision it, no such thing would be required.

However, maybe I just do not understand enough about how PHP internals currently works? How about explaining your assertion that a local symbol table requires a rewrite of all PHP file's header?

Honestly, I'm not convinced its even possible. Someone with more engine knowledge than I could be able to find away, maybe, but I am skeptical. If it's even possible, I suspect it would be an absurdly large amount of work and necessarily include many hard BC breaks.

If you'd like to prove me wrong, go for it. But that's the problem to address. Debating file paths is about four steps down the line before it's even relevant. And even then... if you can't make a PSR-4-organized package (of which there are several hundred thousand) slot into that new model comfortably with zero effort on the part of the package author, it's doomed.

Now is a good time to summarize the request that you claim is "4 steps down the line" that I (and maybe Michael Morris?) wants to get PHP to incorporate, at least as I see it.

The request is to add class maps with a PHP-standardized format into PHP core so that when a library of code needs to register classes to be autoloaded they can contribute to a cascading of class maps where ONE internal function checks the single union of all class maps for a mapped symbol before turning over control if no such symbol is found in the map to the procedural autoloaders that are currently available. That way any PHP code could register its own class map without having to get the core app to rearchitect itself to allow that to happen.

It is really as simple as that, and the duplicate symbol issue is orthogonal, but relating them can see synergy in implementation.

So please, spare us the ill-informed descriptions of how you think autoloaders work, when you have demonstrated you do not know how they work.

Spare us the litany of complaints about PSR-4 when you have demonstrated you don't know what PSR-4 says.

No, spare me your bad-faith accusations of ignorance — I demonstrated my knowledge above — and your claims that "nobody is doing" exactly what I illustrated above that they are doing.

Spare us the gnashing of teeth about how hard it is to use a Wordpress plugin that hasn't been updated in 10 years with a modern plugin because the former is still using a 10 year old abandoned version of some library, when that's not PHP's problem, that's a Wordpress maintenance problem.

Again, showing that your contempt of WordPress is allowing you to dismiss the needs of all user-manage apps as being "a maintenance problem."

If you want to move this effort forward, here's your todo list:

Do some research in the engine to determine if local symbol tables are even possible without rewriting the engine.

Will do.

Work through the highly complex logic of handling three layer overlapping transitive dependencies in a diamond pattern with conflicting version requirements.

Why? I do not see that as a problem that needs to be addressed here for PHP core to solve as it is too time-consuming to be resolved on load. It either needs to be handled by:

1.) A tool like Composer, which already throws an error when such conflicting requirements occur,

2.) A tool like WordPress' plugin manager, which will just throw an error in a sandbox and let the user or site administrator resolve the issue, or

3.) PHP itself which would just throw an error when such a conflict emerges and lets the developer figure it out.

Feels like you are just nerd-sniping here.

Investigate the performance impact of maintaining multiple versions of the same code in memory at once, when the order they get loaded will vary by request.

More nerd-sniping.

By the same token I could say to you "Investigate the performance impact of maintaining a large number of autoloaders in the same codebase" — which I illustrated is happening — except you have the status-quo on your side.

Performance impact due to "maintaining multiple versions of the same code in memory at once" is no different than the performance impact of maintaining multiple versions of equivalent code in memory at once that have had their namespaces rewritten. That is what those who have been forced to solve the same-symbol problem have been doing in userland. Most others have just given up on running two plugins when their dependencies conflict.

It is not up to PHP to address when people load too much code into memory other than for PHP to fail to continue running. Yes, if people duplicate exact same code that should be addressed — ideally by Composer or the app itself — but if different versions cause performance problems its the site builder's responsibility to address, not PHP.

Think through how you'd support both composer-based and "user managed" applications with such a model, especially projects that are already architecturally a decade out of date (like Wordpress).

Already done. I explained how an spl_autoload_map() would do exactly that above.

When you have a proven that it's even possible to have multiple local symbol tables, we can talk. Until then, please spare us.

My one useful takeaway from your email — except that I already knew that — was the need to figure out how PHP can handle multiple symbol tables. Beyond that, your take your own advice and spare us (me) from your contempt and condescension as they are not good looks on anyone.

-Mike

[1] https://mikeschinkel.com/2014/the-decline-of-drupal-or-how-to-fix-drupal-8/
[2] https://w3techs.com/technologies/history_overview/content_management/ms/y
[3] https://www.drupal.org/project/ludwig

4 months ago by Jordan LeDoux — view source

unread

[snip]
My one useful takeaway from your email — except that I already knew that —
was the need to figure out how PHP can handle multiple symbol tables.
Beyond that, your take your own advice and spare us (me) from your contempt
and condescension as they are not good looks on anyone.

While Larry was very blunt (and I don't really fault him for that either),
that should not be your only takeaway. The short version was that he was
telling you, and the other people in this thread, to stop pontificating and
to do something that resembles developing/researching a solution. A lot of
the people who actually have experience working on the engine are literally
ignoring this conversation right now (and may come back to it next month)
because it's just too much noise and nonsense right when we're near feature
freeze.

Your focus and intent to drive this discussion towards trying to duplicate
composer reduces its usefulness. I get that you do not like
PSR-4/composer/whatever. I honestly cannot wade through the 15,000-ish
words you've sent in these threads to nail down the specifics. But I know
for a fact that an attempt to redesign composer is:

Orthogonal to PHP modules (they have nothing to do with each other from
a design perspective).
Doomed to failure.

Your one useful takeaway, that Larry gave you specific steps for, is to
focus on the feature the original proposer of the thread was trying for,
instead of continuing to derail it into composer-related nonsense.

Jordan

4 months ago by Mike Schinkel — view source

unread

[snip]
My one useful takeaway from your email — except that I already knew that — was the need to figure out how PHP can handle multiple symbol tables. Beyond that, your take your own advice and spare us (me) from your contempt and condescension as they are not good looks on anyone.

While Larry was very blunt (and I don't really fault him for that either), that should not be your only takeaway. The short version was that he was telling you, and the other people in this thread, to stop pontificating and to do something that resembles developing/researching a solution.

Why support a claim that assumes I have not been developing/researching a solution when you have absolutely no knowledge of what I have been doing? (I have in-fact been researching and developing a PoC. Full time for the past week, actually.)

A lot of the people who actually have experience working on the engine are literally ignoring this conversation right now (and may come back to it next month) because it's just too much noise and nonsense right when we're near feature freeze.

Your focus and intent to drive this discussion towards trying to duplicate composer reduces its usefulness. I get that you do not like PSR-4/composer/whatever. I honestly cannot wade through the 15,000-ish words you've sent in these threads to nail down the specifics. But I know for a fact that an attempt to redesign composer is:

You claim "my focus and intent" is to "duplicate composer" and yet you claim you "honestly cannot wade through the 15,000-ish words?"

If you haven't read my email, then how exactly do you know what my focus and intent has been?

In fact, my focus and intent has not been to duplicate composer. FULL STOP.

But since I already fully explained my focus and intent then — other than the fact you didn't read it — it would do a disservice to everyone to repeat it.

Orthogonal to PHP modules (they have nothing to do with each other from a design perspective).

Doomed to failure.

Your one useful takeaway, that Larry gave you specific steps for, is to focus on the feature the original proposer of the thread was trying for, instead of continuing to derail it into composer-related nonsense.

If you wanted to quiet a thread that obviously annoys you for some reason it would seem the last thing you to do is immediately knee-jerk a reply with three (3) different straw man accusations — each of which can easily be disproven by reading my email to which you replied — when just ignoring the email was your option.

-Mike

4 months ago by Rowan Tommins [IMSoP] — view source

unread

The request is to add class maps with a PHP-standardized format into
PHP core so that when a library of code needs to register classes to be
autoloaded they can contribute to a cascading of class maps where ONE
internal function checks the single union of all class maps for a
mapped symbol before turning over control if no such symbol is found in
the map to the procedural autoloaders that are currently available.
That way any PHP code could register its own class map without having
to get the core app to rearchitect itself to allow that to happen.

Everything you've just described is possible, today, with no changes to PHP core. And the easiest way to implement it is to borrow the existing battle-tested implementation in Composer, maybe tweaking it with plugins, and automating it with its PHP API rather than using its CLI interface.

If you look at "vendor/composer/autoload_psr4.php" in a project where you've run "composer install", you'll find a mapping of namespaces to directories; PSR-4 then defines where individual classes are within each of those directories. If you tell Composer to dump an optimized autoloader, it will instead create a mapping of individual class names to file paths.

So let's step back a second, and look at the pros and cons of implementing it in core:

Pros:

Possibly a bit faster if the C code can be optimised
"Blessed" by the PHP project, which makes it a standard of sorts
No library code to copy or install into a project

Cons:

Requires a minimum PHP version to run (a huge problem for WordPress, famous for its broad version support)
Improvements can only be accessed by updating PHP
Breaking changes are nearly impossible, because users can't choose between major versions within one version of PHP
Limited ability to tweak for a particular project if the standard solution doesn't meet their needs

Things that won't change:

You still need a standard layout of files within each directory (e.g. PSR-4, or whatever layout you prefer), or to run something that analyzes your code and generates a comprehensive class map (i.e. does the same job as Composer's optimized autoload generator)
The application still needs some central code to register those directory or file lists, e.g. when you install a WordPress plugin, something has to load its configuration file
PHP still won't be able to have two classes with the same name, because that's a completely separate problem, unrelated to autoloading

Regards,

Rowan Tommins
[IMSoP]

4 months ago by Mike Schinkel — view source

unread

Preface: I am going to bow out of this conversation now (unless pulled back in) and come back after 8.4 settles.

In the mean time I'll be working on two proof-of-concepts. One is a totally userland PoC for packages being able to load same-named symbols, and the other will be working out how to use multiple symbol tables in PHP core. And as I have never modified php-src before, that is going to take some learning.

However, below are responses to Rowan and Larry.

The request is to add class maps with a PHP-standardized format into
PHP core so that when a library of code needs to register classes to be
autoloaded they can contribute to a cascading of class maps where ONE
internal function checks the single union of all class maps for a
mapped symbol before turning over control if no such symbol is found in
the map to the procedural autoloaders that are currently available.
That way any PHP code could register its own class map without having
to get the core app to rearchitect itself to allow that to happen.

Everything you've just described is possible, today, with no changes to PHP core.

I described a standardized format that would be processed by PHP,

How exactly can any one individual implement a standardized format on their own, today? One that anyone can depend on to be recognized and processed by any version of PHP after the one that first implemented it?

Rhetorical question. Of course.

So let's step back a second, and look at the pros and cons of implementing it in core:

Pros:

Possibly a bit faster if the C code can be optimised

"Blessed" by the PHP project, which makes it a standard of sorts

Or — without the blessing of a standard — as we sometimes say in the USA: "Besides that Mrs Lincoln, how was the play?"

Let me use an analogy. Envision two people on a city council of a small town. One proposes the city should implement a water, power and sewer grid so anyone who wants to build a new home or business in the city would be able to do so easily.

The other argues that anyone wanting to build can set up their own generator, dig their own well, and dig their own septic tank so why should the city provide such a grid? After all, anyone who is motivated enough can build "today," right?

Clearly that latter argument ignores the value of a central authority providing infrastructure others can just depend on existing, minimizing their cost, effort and risk. It also ignores that without that infrastructure most simply won't build there.

In PHP some bits like nice-to-have functions are easy enough to relegate to userland — although str_starts_with() and str_ends_with() sure are nice to just have like a community center in that city which all residents can use. OTOH, for infrastructure services, PHP benefits itself and others by providing them even if someone could build the functionality themselves.

To be clear, the fact something is an infrastructure service itself is not sufficient to argue it must be included in PHP core — the need for service itself should stand on its own.

However, it is enough (IMO) to show "you can do it yourself today" is not a valid argument against an infrastructure service.

So can we please instead focus on the pros and cons of having such an infrastructure service instead of using the canard "you can build it yourself" as an argument against roads, power, and sewers?

If yes, we should discuss your list of pros and cons.

HOWEVER, I think it is probably best to postpone any additional discussion until after 8.4 is released and any initial bugs are worked out so anyone interested could focus on the reasoning.

In fact, if you use an optimized/dumped autoloader, then Composer simply builds an internal giant lookup table of what class maps to what file. PSR-4 is then completely irrelevant at runtime. It's already one giant O(1) lookup map. That can be done today. That is done today.

Yes, it can be done today. It is done today. By. Developer. Managed. Apps.

Already done. I explained how an spl_autoload_map() would do exactly
that above.

When you have a proven that it's even possible to have multiple local symbol tables, we can talk. Until then, please spare us.

My one useful takeaway from your email — except that I already knew
that — was the need to figure out how PHP can handle multiple symbol
tables. Beyond that, your take your own advice and spare us (me) from
your contempt and condescension as they are not good looks on anyone.

I find it amusing that several of your responses to me saying "you could do this stupid thing but no one does that" is "WordPress does that thing." I make no comment other than to observe it.

And I observe you are imprecise when making your observation.

WordPress as an entity does not do that thing. Plugin developers do. Independently. Because they have not been given any other viable option.

But let me understand: In a thread started by Michael Morris where he explicitly said the most important thing for him is multi-version loading, you're going to insist you're talking only about moving Composer's classmap logic into core, and nothing about multi-version loading.

Where did you get that from what I wrote?

I acknowledged that a next step is doing a proof-of-concept exploration of multi-symbol tables in PHP core, not that needing to be able to load same-named symbols was something new to me.

And Michael Morris proposed spl_autoload_map(). I was just advocating for it.

If that's the case, then please be polite to Michael Morris and get out of his thread.

Once again, you make incorrect assumptions.

Also, be aware that classmap-in-core was already discussed 3 years ago and went nowhere.

https://wiki.php.net/rfc/autoload_classmap
https://externals.io/message/113545

Largely because, as Sara said then, and Rowan just said on this thread, it can be done better in user-space and is already done better in user-space... by Composer.

See the explanation to Rowan above about why doing it in user-space is not a valid argument against everything that can be done in user-space.

https://externals.io/message/113569

You even commented in that thread:

https://externals.io/message/113554

Yeah, but unlike now I was super busy and could not contribute much to that debate.

So it's not a new idea, it's an idea that's already been greeted with a general "meh".

Property Hooks were once a general "meh." too. ¯_(ツ)_/¯

Yes, most "developer managed apps" use Composer today to side-step the "bajillion autoloaders" problem. It's a solved problem.

Or said another way with, in a nod to history "Let them eat cake."

Nothing precludes Wordpress from doing the same. I admittedly have not looked at WP's core in a very long time, but I would be absolutely shocked if it wasn't pretty straightforward to build code into WP core that looks at the source directories of all plugins, finds the classes there, and builds a big index (stored in a cache directory or the database) that it can use in one single autoloader registered by WP itself. I know that can be done, because that's exactly how Drupal 7's autoloader worked. I know, because I wrote it. In 2008. (It was later modified by others, but the initial version was mine.)

That would work even with WP's "download code and drop it on disk" model. That has been possible since PHP 5.2 at least, when I wrote exactly that for Drupal. It wasn't even that hard. Literally any "user managed app" could do the same.

Why hasn't WP core done that in order to make life easier for plugin developers and avoid registering 50 separate autoloaders? I dunno, you should ask Matt Mullenweg. We have nothing to do with it.

You make a fair point there.

However, as it is a case "anyone could address," no one will.

BTW, there is a benefit beyond just WordPress developers to having a core autoload map format in PHP that can just be expected to work. With that I could publish a library to be used by ANY PHP app, regardless of what autoloader it needs. Wikipedia even has a few relevant entries:

https://en.wikipedia.org/wiki/Interoperability https://en.wikipedia.org/wiki/Interoperability
https://en.wikipedia.org/wiki/Network_effect https://en.wikipedia.org/wiki/Network_effect

That said, as I said above, I am going to go away for now and come back after 8.4 settles.

-Mike

4 months ago by Rowan Tommins [IMSoP] — view source

unread

Let me use an analogy. Envision two people on a city council of a small town. One proposes the city should
implement a water, power and sewer grid so anyone who wants to build a new home or business in the city
would be able to do so easily.

This is a terrible analogy, IMHO. Here's a more pertinent one:

You have a city, where there is a working electricity grid. [That's autoloading, provided by PHP for many years.]

Most of the city are using a particular plug and socket system; the sockets aren't fitted by the electricity company, but the shops are full of devices that use these plugs, and most people have found the sockets really easy to fit. [That's Composer]

One district runs most of its appliances on gas, and has its own way of connecting them. People want to run electric appliances, and the grid goes right through the district, but for some reason, nobody's connected it up to the houses. So everyone has multiple gas-to-electricity converters, which is annoying. [That's WordPress]

Someone comes along, and rather than approaching the district [WordPress] and offering to help wire them to the existing grid, decides that what's needed is for the city to provide them with a new type of plug [a different autoloader function], which they won't be able to use immediately anyway.

https://en.wikipedia.org/wiki/Interoperability
https://en.wikipedia.org/wiki/Network_effect

Adding something to core in the hope of it becoming a standard is exactly the opposite of the network effect. The network effect is "everyone already uses Composer, so nobody cares what PHP core does".

See also https://xkcd.com/927/ and since I used an electric plug analogy, check out the "international standard" plug, which has been adopted in exactly one country: https://en.wikipedia.org/wiki/IEC_60906-1

If you want to solve autoloading in WordPress, this is the wrong forum.

Firstly, anything you do in core will not be adopted by WordPress, because it will be 5+ years before their minimum PHP version is high enough to use it. You're going to have to write a userland polyfill anyway.

Secondly, mapping namespaces to directories isn't the hard part for WordPress. The hard part is integrating into their existing plugin installer system - which already has a system for metadata, it doesn't need a new config file - and persuading plugin authors to actually use it.

Maybe you also need Composer to make some changes to control it without the CLI, so plugins can list their requirements and have them installed centrally. That's also not a subject for this list.

Regards,

Rowan Tommins
[IMSoP]

4 months ago by Mike Schinkel — view source

unread

Let me use an analogy. Envision two people on a city council of a small town. One proposes the city should
implement a water, power and sewer grid so anyone who wants to build a new home or business in the city
would be able to do so easily.

This is a terrible analogy, IMHO. Here's a more pertinent one:

Sure, we can go back and forth and discredit each others analogies, but that's not going to change your mind nor likely mine.

The crux of our disagreement is I see value on infrastructure services and you evidently do not.

If you want to solve autoloading in WordPress, this is the wrong forum.

Firstly, anything you do in core will not be adopted by WordPress, because it will be 5+ years before their minimum PHP version is high enough to use it. You're going to have to write a userland polyfill anyway.

That is what you are missing on this point.

If PHP supported a cascading class map autoloader then a developer could build their sites and their plugins for that version of PHP without having to depend on WordPress changing to support it. And if they are distributing plugins they could build fallback mechanisms for earlier versions of PHP plus also recommend to their users that they upgrade to the newer versions of PHP.

It is about empowering individuals developers, not about empowering WordPress.

But, like I said, since anyone could no it, evidently no one will.

Secondly, mapping namespaces to directories isn't the hard part for WordPress. The hard part is integrating into their existing plugin installer system - which already has a system for metadata, it doesn't need a new config file - and persuading plugin authors to actually use it.

No, integrating into the plugin system would actually be quite easy per the class map design we've been discussing. I know the internals of that system well, I've probably debugged through it literally hundreds of times.

And sorry, the existing metadata system in WordPress has no information about plugin dependencies and is inadequate for that purpose. It only has an ini-like plugin header with basic information and is not suitable for dependencies. Also, "persuading plugin authors to actually use it" won't change a thing because they are already using it effectively, generally speaking.

Maybe you also need Composer to make some changes to control it without the CLI, so plugins can list their requirements and have them installed centrally.

That well could be.

That's also not a subject for this list.

And we disagree, yet again.

Anyway, I am trying to avoid this thread so I can instead invest time in research and proof-of-concept work. Please don't pull me back in.

-Mike

4 months ago by Larry Garfield — view source

unread

In fact, if you use an optimized/dumped autoloader, then Composer simply builds an internal giant lookup table of what class maps to what file. PSR-4 is then completely irrelevant at runtime. It's already one giant O(1) lookup map. That can be done today. That is done today.

Yes, it can be done today. It is done today. By. Developer. Managed. Apps.

Already done. I explained how an spl_autoload_map() would do exactly
that above.

When you have a proven that it's even possible to have multiple local symbol tables, we can talk. Until then, please spare us.

My one useful takeaway from your email — except that I already knew
that — was the need to figure out how PHP can handle multiple symbol
tables. Beyond that, your take your own advice and spare us (me) from
your contempt and condescension as they are not good looks on anyone.

I find it amusing that several of your responses to me saying "you could do this stupid thing but no one does that" is "WordPress does that thing." I make no comment other than to observe it.

But let me understand: In a thread started by Michael Morris where he explicitly said the most important thing for him is multi-version loading, you're going to insist you're talking only about moving Composer's classmap logic into core, and nothing about multi-version loading.

If that's the case, then please be polite to Michael Morris and get out of his thread.

Also, be aware that classmap-in-core was already discussed 3 years ago and went nowhere.

https://wiki.php.net/rfc/autoload_classmap
https://externals.io/message/113545

Largely because, as Sara said then, and Rowan just said on this thread, it can be done better in user-space and is already done better in user-space... by Composer.

https://externals.io/message/113569

You even commented in that thread:

https://externals.io/message/113554

So it's not a new idea, it's an idea that's already been greeted with a general "meh".

Yes, most "developer managed apps" use Composer today to side-step the "bajillion autoloaders" problem. It's a solved problem.

Nothing precludes Wordpress from doing the same. I admittedly have not looked at WP's core in a very long time, but I would be absolutely shocked if it wasn't pretty straightforward to build code into WP core that looks at the source directories of all plugins, finds the classes there, and builds a big index (stored in a cache directory or the database) that it can use in one single autoloader registered by WP itself. I know that can be done, because that's exactly how Drupal 7's autoloader worked. I know, because I wrote it. In 2008. (It was later modified by others, but the initial version was mine.)

That would work even with WP's "download code and drop it on disk" model. That has been possible since PHP 5.2 at least, when I wrote exactly that for Drupal. It wasn't even that hard. Literally any "user managed app" could do the same.

Why hasn't WP core done that in order to make life easier for plugin developers and avoid registering 50 separate autoloaders? I dunno, you should ask Matt Mullenweg. We have nothing to do with it.

--Larry Garfield

4 months ago by Chuck Adams — view source

unread

Hello all. Hitting reset again as the primary problem at hand has become clear. Let's recap it.

Autoloading is great for loading packages, but it can't load different versions of the same package at the same time. Why would you want to do that?

When you don't have full control of the code.

For example, consider Drupal. It is running Twig at some version of 3 at the moment. Suppose Twig 4 is introduced with significant backward compatibility breaks (Not saying the authors would do such a thing) but also wonderful features.
…[snip]...
This is why I advocate a new keyword for this - import. Import’s behavior is most similar to require_once, but it doesn't have to be the same. Since it is a new entrypoint into the engine the way the engine considers the code can be different - whether slightly different or radically different is a debate for another time. I'm going to stick with only those changes that make sense in the context of package links.

I’m seeing a lot of conflation of ‘module’ and ‘package’ in these discussions, and to me they mean different things:

A module is a sort of “first class namespace” that can export symbols and import others. Think ES5 or python modules. If you don’t want it 1-1 with files, think Perl modules.
A package is an “installable” unit that provides modules, among other things. Packages have metadata, the most important piece of which is a machine-readable version.

Certainly there’s overlap between the two, but the first is a more low-level thing that doesn’t need worry itself about versioning let alone multiple simultaneous versions. I just don’t want to see the possibility of having basic “import” and “export” functionality crushed under the bikeshed while all the fine semantics of versioning are worked out.

—c

4 months ago by Mike Schinkel — view source

unread

Hi Chuck,

Hello all. Hitting reset again as the primary problem at hand has become clear. Let's recap it.

Autoloading is great for loading packages, but it can't load different versions of the same package at the same time. Why would you want to do that?

When you don't have full control of the code.in

For example, consider Drupal. It is running Twig at some version of 3 at the moment. Suppose Twig 4 is introduced with significant backward compatibility breaks (Not saying the authors would do such a thing) but also wonderful features.
…[snip]...
This is why I advocate a new keyword for this - import. Import’s behavior is most similar to require_once, but it doesn't have to be the same. Since it is a new entrypoint into the engine the way the engine considers the code can be different - whether slightly different or radically different is a debate for another time. I'm going to stick with only those changes that make sense in the context of package links.

I’m seeing a lot of conflation of ‘module’ and ‘package’ in these discussions, and to me they mean different things:

A module is a sort of “first class namespace” that can export symbols and import others. Think ES5 or python modules. If you don’t want it 1-1 with files, think Perl modules.

A package is an “installable” unit that provides modules, among other things. Packages have metadata, the most important piece of which is a machine-readable version.

Your definitions are language-specific. For example, in Go the definitions for those terms are the opposite of how you defined them.

The point being that PHP is free to choose how they are defined with respect to PHP.

To which I will add "as long as the terms are used consistently."

-Mike

4 months ago by Chuck Adams — view source

unread

Your definitions are language-specific. For example, in Go the definitions for those terms are the opposite of how you defined them.

The point being that PHP is free to choose how they are defined with respect to PHP.

To which I will add "as long as the terms are used consistently.”

Okay, some languages may swap the terms, others like JS glom the concepts together, and in Perl 5 “package Foo” defines a module. I’d say most PHP devs are more familiar with the terms as they’re used in JS, but whatever makes sense for PHP is what’s best. I suppose I do have a dog in the fight, but I don’t much care how it’s groomed.

Then there’s GHC Haskell which has import “package-name” ModuleName.{Foo,Bar,Baz}, which seems to cover all the bases. Decent ideas from the syntax, but I don’t think I want to replicate backpack :)

—c

4 months ago by Mike Schinkel — view source

unread

The point being that PHP is free to choose how they are defined with respect to PHP.

To which I will add "as long as the terms are used consistently.”

...but whatever makes sense for PHP is what’s best.

Definitely.

-Mike

4 months ago by Dusk — view source

unread

Can PHP support multiple packages without rewriting the whole engine? I think so, but it isn't trivial, and the side effects need to be cordoned off so that those who need this complexity can have it while the beginning and intermediate coders can ignore it just like they ignore strict comparison operators and strict typing unless a library they are trying to use foists it on them.

I think that focusing on the syntax and tooling for executing these imports is starting at the wrong end of this problem. The bulk of the work for this feature is going to be whatever engine changes are required to support versioning, not the tooling around it.

To that end - consider the following. Let's say that two different files in your project import different versions of package Foo. Foo contains a definition of the FooBar class, and contains functions which return that object.

If $foobar is one of those FooBar objects, what does $foobar::class return? Is it the same as the fully qualified name of FooBar (e.g. "Foo\FooBar")? Does the result differ depending on what file contains that code?
What happens if you try to pass that string back to something like new $class() or construct a ReflectionClass for it? Does that depend on the location of the call? What if the call is through something like PDO::FETCH_CLASS which occurs within the runtime?
Within Foo, would it be true that if $x = new FooBar(), then $x::class === FooBar::class? Does this differ outside Foo (with an appropriately qualified name for FooBar)?
If those two files both create FooBar objects of their respective versions, what happens if you try to pass one of those objects to a function in the file using the "wrong" version of Foo? Does it pass type checks, and what happens if it does? If not, how does the check fail?
What shows up in the output of functions like get_declared_classes()? Are there multiple instances of FooBar in there for each version? How are they distinguished from one another?

4 months ago by Michael Morris — view source

unread

To that end - consider the following. Let's say that two different files
in your project import different versions of package Foo. Foo contains a
definition of the FooBar class, and contains functions which return that
object.

If $foobar is one of those FooBar objects, what does $foobar::class
return?

Depends on where it was imported to. The current system ALWAYS imports to
the root namespace. This new system can import there, but also to another
namespace. This was outlined in my previous email.

For @A\Foo\FooBar the import will put FooBar at \A\Foo\FooBar, and that's
what class will return. @B\Foo\FooBar will bind it to \B\Foo\FooBar and
that will be the return of the class constant within the class.

Is it the same as the fully qualified name of FooBar (e.g. "Foo\FooBar")?
Does the result differ depending on what file contains that code?

Again, it depends on which package namespace the code was imported into.
Foo\FooBar isn't a fully qualified name even in current PHP. Fully
qualified names start with , so the fully qualified name is \Foo\FooBar
provided it was included into the root namespace.

What happens if you try to pass that string back to something like new
$class() or construct a ReflectionClass for it? Does that depend on the
location of the call? What if the call is through something like
PDO::FETCH_CLASS which occurs within the runtime?

Again, which package are we in?

Within Foo, would it be true that if $x = new FooBar(), then $x::class
=== FooBar::class?

Yes.

Does this differ outside Foo (with an appropriately qualified name for
FooBar)?

That's tricker, because the namespace matters in a way that it doesn't
matter now. Given an import mapping of "@A\FooBar" then

namespace A;

$x = new FooBar();
$x::class === FooBar::class // true, however...
echo $x::class // \A\Foo\FooBar

That holds true even if FooBar's declaration file doesn't invoke any
namespace.

If those two files both create FooBar objects of their respective
versions, what happens if you try to pass one of those objects to a
function in the file using the "wrong" version of Foo? Does it pass type
checks, and what happens if it does? If not, how does the check fail?

Each package has its own Foo\FooBar. They won't be interoperable even
though they arise from the same code. If they should be or need to be
interoperable then PHP will have to gain a notion of package beyond what's
been scoped out here.

What shows up in the output of functions like get_declared_classes()?
Are there multiple instances of FooBar in there for each version? How are
they distinguished from one another?

You'll get
\A\Foo\FooBar
\B\Foo\FooBar

If you also directly load Foo\FooBar into the root namespace using the
composer autoloader you could also see \Foo\FooBar

4 months ago by Jordan LeDoux — view source

unread

Hello all. Hitting reset again as the primary problem at hand has become
clear. Let's recap it.

Autoloading is great for loading packages, but it can't load different
versions of the same package at the same time. Why would you want to do
that?

When you don't have full control of the code.

For example, consider Drupal. It is running Twig at some version of 3 at
the moment. Suppose Twig 4 is introduced with significant backward
compatibility breaks (Not saying the authors would do such a thing) but
also wonderful features.

If you're writing a Drupal extension you might want to use this new Twig.
This is possible if you are willing to monkey-type the package - that is,
have a code package traverse over the entire package and change all
instances of namespace Twig in the files to namespace NewTwig. You can
then use the package at the namespace of \NewTwig.

This is painful, but the pain factor increases if multiple extension
developers choose to do the same thing. Each extension using its own Twig
library is going to incur a performance hit.

One upshot of this is I've noted that major package distributors, like
Symfony, take BC into account with major releases - and may not develop new
features or change things in those releases out of fear of people not
wanting to upgrade.

Now don't get me wrong, changing things just because is a bad thing. If a
BC can be avoided it should be. But having a mechanism to move forward is
important.

In some ways versioning packages is like static typing variables. It
doesn't seem important at all until you are faced with a problem only it
can solve, or faced with a problem created by dynamic typing of variables.

What can be done in the engine?

Well first off, recognize that autoloading isn't going to work with a
versioned package scheme. Autoloaders, regardless of their resolution
schema be it PSR-0, PSR-4, or BobbysFirstAutoloader-Scheme can only have
one symbol per package, set by the namespace.

Can PHP support multiple packages without rewriting the whole engine? I
think so, but it isn't trivial, and the side effects need to be cordoned
off so that those who need this complexity can have it while the beginning
and intermediate coders can ignore it just like they ignore strict
comparison operators and strict typing unless a library they are trying to
use foists it on them.

This is why I advocate a new keyword for this - import. Import's behavior
is most similar to require_once, but it doesn't have to be the same. Since
it is a new entrypoint into the engine the way the engine considers the
code can be different - whether slightly different or radically different
is a debate for another time. I'm going to stick with only those changes
that make sense in the context of package links.

Let's start with the simplest problem, importing this file.

namespace A;
function foo() { echo 'Hi'; }

To review, if we require_once this file we'll find the function at
\A\foo(). If our current file uses the same namespace we can just use foo()

At its root import would do the same. import "file.php" would do the
same as a require_once assuming there's no difference between the file
structure rules for import - again there is opportunity here, but it's not
a requirement.

If that's all it does, it's pointless. However, import can alias.

import 'file.php' as B;

Now we have \B\foo(); This makes it relatively easy to have two different
versions of the package running since in our own code we can always
reference the foo in the B namespace. But while that allows limited package
versioning, it doesn't solve the multiple extensions wanting to use the new
stuff problem outlined above.

So we have to call out the version in code, like so.

import 'file.php v1.0.0';

A simple space separates the version from the file. If the filename has a
space, well \ characters aren't just for namespaces.

Now for the first real behavior difference between import and
require_once, even if we aren't doing anything fancy. Import cares about
the namespace it's invoked from. Require_once does not. To illustrate
this behavior he's some pseudocode - we are including the file.php given
earlier

namespace D;
require_once 'file.php';

\A\foo(); // Hi.

import 'file.php';

\D\A\foo(); // Hi.

See that? The namespace of the calling file is prepended to the namespace
contained in the import.

Why? What's the value here? I'll explain.

Now, let's suppose we do have two versions of file.php. So in addition to
the above, elsewhere in the code this happens

namespace C;
import 'file.php v2.0.0'

A\foo(); // Welcome, since version 2 echoes welcome. Remember your
namespace resolution rules - this import is actually at:
\C\A\foo(); Welcome, as this is the absolute path to the code we just
imported.
\A\foo(); // Hi, as the package at root was brought in by require_once()
\D\A\foo(); Hi, as that's what was imported into the D namespace.

Now for the kicker

namespace E;
import 'file.php';

A\foo(); // Hi.

The engine can be left as is and this would work, but if the engine is
altered to support symbolic links on the symbol table then the performance
hit might be avoided. That is, when a redundant import occurs that would
pull the same package the engine just quickly links up the new namespace.
Hence \E\A\foo() quietly points to \D\A\foo() as it was declared first.

What hasn't been discussed in this iteration are the following critical
points:

How the package path gets resolved in the first place. Does it work
like require and check locally then check the PHP include paths?

When does the code get downloaded from where it is downloaded?

Is a registry used like composer and npm, or are repos directly invoked
as in go (I don't remember how Python does it, but someone providing that
example might be useful)

The huge ball of wax that is the package definition file. Just look at
the properties of composer.json and package.json to get an idea of that
scope. How much of if any of this should PHP deal with.

Is import to be locked into loading other PHP files, or could it deal
with .so (Unix) or .dll (Windows) files? Phar files?

It's not like I'm not interested in any of these questions, but too many
questions at once is too much so I'd like to leave them aside for now.

And there are yet more questions as well raised in previous iterations,
but I've again left those out because they touched off controversy. While
I'm not afraid of such, I'm inclined to avoid it if possible.

A quick thank you to everyone who has participated in the thread, even the
torpedo tossers because it's forcing me to think this through entirely. And
I'm trying to take as much into consideration as possible. And yes, this
remains a brainstorm for now, but each successive brainstorm is more tight
than the one before it.

I think it's strange that this discussion has driven deep down the tangent
of versioning, as if the selling point of any kind of module/package system
in PHP core would be to do what composer does. Let compose do what composer
does, it does it well.

Instead of building out the specific features like this that honestly
shouldn't be built into the language directly, I would think it makes more
sense for the discussion to be centered around the compiler and engine
features that would LET or ENABLE software like composer to easily meet
these requirements.

Things like separating global scope between importer and importee, managed
visibility of symbols and exports from modules/packages, allowing for
separate autoloaders for things which are called or included via an import,
etc. Those are the things that the language itself can do.

All this other stuff feels like a distraction.

Jordan

4 months ago by Rowan Tommins [IMSoP] — view source

unread

I think it's strange that this discussion has driven deep down the tangent
of versioning...
[...]
Things like separating global scope between importer and importee, managed
visibility of symbols and exports from modules/packages, allowing for
separate autoloaders for things which are called or included via an import,
etc. Those are the things that the language itself can do.

All this other stuff feels like a distraction.

I agree. I wrote most of the below a couple of days ago, but I don't think it posted correctly, so apologies if some people see it twice:

Autoloading is just a way to load files later, by the engine telling you when a class is first needed. PHP does not, and should not, make any assumptions about how files are laid out on disk; an autoloader doesn't actually need to load any files at all, and if it does, it uses the same include or require statements which have been in PHP for decades.

Likewise, installing packages and defining version schemes is a completely separate problem space that can probably be served by a few small tweaks to Composer once the language provides the underlying functionality.

The core of the problem you seem to want to solve is this: if you have two files foo_1.php and foo_2.php, which both define a class \Acme\Foo, how do you load both of them, so that you end up with two differently named classes?

In JS, that's easy, because functions and object constructors (and "classes") exist as objects you can pass around as variables, they don't need to know their own name. In PHP, everything is based on the idea that functions and classes are identified by name. You can rewrite the name in the class declaration, and in direct references to it, but what about code using ::class, or constructing a name and using "new $name", and so on? How will tools using static analysis or reflection handle the renaming - e.g. how does DI autowiring work if names are in some sense dynamic?

You've also got to work out what to do with transitive dependencies - if I "import 'foo_1.php' as MyFoo", but Foo in turn has "import 'guzzle_2.php' as MyGuzzle", what namespace do all Guzzle's classes get rewritten into? What about dependencies that are specifically intended to bridge between packages, like PSR-7 RequestInterface?

My advice: start with the assumption that something has already installed all the files you need into an arbitrary directory structure, and something is going to generate a bunch of statements to load them. What happens next, in the language itself, to make them live side by side without breaking? If we get a solid solution to that (which I'm skeptical of), we can discuss how Composer, or the WordPress plugin installer, would generate whatever include/import/alias/rewrite statements we end up creating.

Regards,

Rowan Tommins
[IMSoP]
Rowan Tommins
[IMSoP]

4 months ago by Jordan LeDoux — view source

unread

On Mon, Jul 8, 2024 at 2:42 AM Rowan Tommins [IMSoP] imsop.php@rwec.co.uk
wrote:

On 8 July 2024 04:25:45 CEST, Jordan LeDoux jordan.ledoux@gmail.com
wrote:

I think it's strange that this discussion has driven deep down the tangent
of versioning...
[...]
Things like separating global scope between importer and importee, managed
visibility of symbols and exports from modules/packages, allowing for
separate autoloaders for things which are called or included via an
import,
etc. Those are the things that the language itself can do.

All this other stuff feels like a distraction.

I agree. I wrote most of the below a couple of days ago, but I don't think
it posted correctly, so apologies if some people see it twice:

Autoloading is just a way to load files later, by the engine telling you
when a class is first needed. PHP does not, and should not, make any
assumptions about how files are laid out on disk; an autoloader doesn't
actually need to load any files at all, and if it does, it uses the same
include or require statements which have been in PHP for decades.

Likewise, installing packages and defining version schemes is a completely
separate problem space that can probably be served by a few small tweaks to
Composer once the language provides the underlying functionality.

The core of the problem you seem to want to solve is this: if you have two
files foo_1.php and foo_2.php, which both define a class \Acme\Foo, how do
you load both of them, so that you end up with two differently named
classes?

In JS, that's easy, because functions and object constructors (and
"classes") exist as objects you can pass around as variables, they don't
need to know their own name. In PHP, everything is based on the idea that
functions and classes are identified by name. You can rewrite the name in
the class declaration, and in direct references to it, but what about code
using ::class, or constructing a name and using "new $name", and so on? How
will tools using static analysis or reflection handle the renaming - e.g.
how does DI autowiring work if names are in some sense dynamic?

You've also got to work out what to do with transitive dependencies - if I
"import 'foo_1.php' as MyFoo", but Foo in turn has "import 'guzzle_2.php'
as MyGuzzle", what namespace do all Guzzle's classes get rewritten into?
What about dependencies that are specifically intended to bridge between
packages, like PSR-7 RequestInterface?

My advice: start with the assumption that something has already installed
all the files you need into an arbitrary directory structure, and something
is going to generate a bunch of statements to load them. What happens next,
in the language itself, to make them live side by side without breaking? If
we get a solid solution to that (which I'm skeptical of), we can discuss
how Composer, or the WordPress plugin installer, would generate whatever
include/import/alias/rewrite statements we end up creating.

Regards,

Rowan Tommins
[IMSoP]
Rowan Tommins
[IMSoP]

I think it could be done somewhat simply (relative to the other things that
have been discussed) if the engine reserved a specific namespace for
imported symbols internally. Something like:

\__Imported\MyImportStatement

Where the \__Imported namespace is reserved and throws a parser error if
it occurs in code anywhere, and MyImportStatement corresponds to an
application importing the code using something like import MyPackage as MyImportStatement;

Then, all symbols which are loaded into the global space as a result of the
import are actually rewritten into the hidden namespace the engine actually
uses under the hood, and any uses from the import statement in the
application code which has the import would reference the symbols in the
prefixed namespace.

This would not be trivial however. The engine code which supports this
would need to keep track of a kind of "context" for each file, based on
what namespace the file was included from. For instance, if an autoload
occurs inside the package that was loaded into MyImportStatement, the
engine would need to be aware that the code being executed is defined in
that namespace, REGARDLESS of whether it was a class, function, or
statement, and load ALL symbols that are created as a result into the
rewritten namespace. It would also need to translate in the other direction
for use statements inside the package, since it would not know ahead of
time what rewritten namespace it would actually be loaded in.

However, this is the simplest solution I see that doesn't involve writing a
second PHP engine just for this sort of thing.

Jordan

PS: For those unaware, for each "symbol" (something that has a unique
referenceable name in the code, roughly), there is at least one name that
refers to ONLY that thing internally. (I'm fairly certain that there are NO
situations where one name can refer to two things at all, but I am not
enough of an expert in the C code to be completely certain about this, and
it's entirely possible this is in fact a niche common thing that I've never
encountered before). When something is namespaced, the entire namespace in
the engine is prefixed to the "name" of the thing when it is created. So a
function foo in the namespace Bar has the name "\Bar\foo". Any time you
use it as just "foo", the engine because of context knows to put "Bar" in
front of it before looking up its definition to execute it.

The global symbol space are the items which have nothing prepended, and if
a namespace was inaccessible because the parser errored on use,
namespace, new, and other similar statements that used a part of that
namespace like I outlined, the result would be that for the engine it would
treat all of the code as if it were one application that it knows some
extremely complex namespace replacement rules for (because that's what it
actually would be), but to PHP devs it would act almost like sandboxes
where code from one area cannot access or affect other areas.

This would create some edge cases, like how global behaves, or how any of
the superglobal variables could be used, etc. But those are probably easier
to nail down than writing a different engine or running a second process
and setting up some messaging between the two. Though that might be the
more "correct" way to handle something like this. However, I could be wrong
about the difficulty of this, as I've never attempted that kind of change
in the Zend engine before.

4 months ago by Mike Schinkel — view source

unread

I agree. I wrote most of the below a couple of days ago, but I don't think it posted correctly, so apologies if some people see it twice:

Autoloading is just a way to load files later, by the engine telling you when a class is first needed. PHP does not, and should not, make any assumptions about how files are laid out on disk; an autoloader doesn't actually need to load any files at all, and if it does, it uses the same include or require statements which have been in PHP for decades.

I think maybe you are replying to an earlier iteration by Michael Morris and have not seen the more recent iteration?

There he explored adding an additional function to named spl_autoload_map() where the difference from spl_autoload_register() is that while the latter uses procedural code to determine what should be loaded the former would use a declarative map to determine what should be loaded. Then Composer and/or other tools could/would generate that map for PHP.

With an spl_autoload_map() PHP would not need to make any assumptions about how files are laid out on disk. Instead, PHP would use a schema of a yet-to-be-determined data representation format to discover declaratively where files needed to be laid out on disk.

Likewise, installing packages and defining version schemes is a completely separate problem space that can probably be served by a few small tweaks to Composer once the language provides the underlying functionality.

The core of the problem you seem to want to solve is this: if you have two files foo_1.php and foo_2.php, which both define a class \Acme\Foo, how do you load both of them, so that you end up with two differently named classes?

That is one (1) of the core problems, yes.

In JS, that's easy, because functions and object constructors (and "classes") exist as objects you can pass around as variables, they don't need to know their own name. In PHP, everything is based on the idea that functions and classes are identified by name. You can rewrite the name in the class declaration, and in direct references to it, but what about code using ::class, or constructing a name and using "new $name", and so on? How will tools using static analysis or reflection handle the renaming - e.g. how does DI autowiring work if names are in some sense dynamic?

This is one of the unfortunate aspects of PHP never makes types a first-class data type. But I digress.

You've also got to work out what to do with transitive dependencies - if I "import 'foo_1.php' as MyFoo", but Foo in turn has "import 'guzzle_2.php' as MyGuzzle", what namespace do all Guzzle's classes get rewritten into? What about dependencies that are specifically intended to bridge between packages, like PSR-7 RequestInterface?

Which is a direct result of the other problem you mentioned, i.e. IOW without attempting to address the prior problem this would not be a problem. #fwiw

My advice: start with the assumption that something has already installed all the files you need into an arbitrary directory structure, and something is going to generate a bunch of statements to load them.

And this sentence is why I chose to reply to your message. That assumption itself blocks the needs of user-managed apps.

(Did you happen to read my compare-and-contrast of user-managed vs. developer-managed apps from a few days ago?)

I feel it is likely those who have never worked professionally in PHP on user-managed apps like WordPress — which I assume describes you accurately? — are probably just simply unaware of the problems that your assumptions cause for user-managed apps. And yes, some developers have no empathy for others who have different circumstances, but I honestly don't think you (Rowan) are in the category.

Developer-managed apps use a build tool to put all vendor code in a single hierarchical set of namespaces and then load needed code from there. But that really does not work for user-managed apps like WordPress or Drupal. Or at least not as they exist today, and probably not even if they were rearchitected for the start.

What works for user-managed apps is that each add-on (plugin in WordPress, module in Drupal) is stored in its own self-contained directory containing its own vendor code — where some of the vendor code could easily be duplicated in another add-on — and then the user-managed apps itself manages loading of all add-ons itself without a PSR-4 autoloader. As it exists, there are no standard for how add-on filenames and directory structures much be named nor how they are to load their dependencies so it is impossible for WordPress or Drupal to take on that role using PSR-4 for them.

Michael Morris' idea to add an spl_autoload_map() function would allow addressing the needs of user-managed apps that treat each add-on as a self-contained entity. But making the assumption that "something has already installed all the files you need into an arbitrary directory structure" is not sufficient for the problems Michael Morris and I have been trying to address.

An autoloader map schema that has enough information for PHP to understand how to manage the conflicting names, and for the user-managed apps and Composer to be able to tell PHP what names are conflicting is in fact a solid way forward.

-Mike

4 months ago by Rowan Tommins [IMSoP] — view source

unread

I think maybe you are replying to an earlier iteration by Michael Morris and have not seen the more recent iteration?

I wrote the message a few days ago, but it didn't post; but the more recent discussion still seems to be focussing on things that can be solved in userland, rather than the fundamentally hard parts.

There he explored adding an additional function to named spl_autoload_map() where the difference from spl_autoload_register() is that while the latter uses procedural code to determine what should be loaded the former would use a declarative map to determine what should be loaded. Then Composer and/or other tools could/would generate that map for PHP.

You can implement this in userland, right now. The point of calling procedural code for the autoloader is that it can do anything it likes to define the symbol - it can look it up in a table of directories, it can load some code from a database and eval() it, whatever you need. In fact, Composer already does implement such a file map, that's what its "optimize autoloader" option creates.

What you can't easily do is run different code depending on where the symbol is used - but since the autoloader is only called once per symbol, doing so wouldn't make much sense.

My advice: start with the assumption that something has already installed all the files you need into an arbitrary directory structure, and something is going to generate a bunch of statements to load them.

[...]

What works for user-managed apps is that each add-on (plugin in WordPress, module in Drupal) is stored in its own self-contained directory containing its own vendor code

Note that I said "arbitrary directory structure", not "PSR-4/Composer directory structure"; the files are on disk somewhere. PHP didn't put them there, some application did. The application knows where they are, and needs to tell PHP somehow.

— where some of the vendor code could easily be duplicated in another add-on

This is the hard part I was suggesting you focus on.

and then the user-managed apps itself manages loading of all add-ons itself without a PSR-4 autoloader. As it exists, there are no standard for how add-on filenames and directory structures much be named nor how they are to load their dependencies so it is impossible for WordPress or Drupal to take on that role using PSR-4 for them.

WordPress doesn't need PHP Internals, or even PHP-FIG, to define how plugins should be laid out on disk, and to write an autoloader for whatever they come up with.

Michael Morris' idea to add an spl_autoload_map() function would allow addressing the needs of user-managed apps that treat each add-on as a self-contained entity. But making the assumption that "something has already installed all the files you need into an arbitrary directory structure" is not sufficient for the problems Michael Morris and I have been trying to address.

It doesn't need to solve all the needs of the application, it needs to solve the parts we don't already have. WordPress already knows how to download files to disk; it could trivially design a system for plugin authors to lay out their own classes in some agreed layout and write an autoloader using the functionality that's been around since PHP 5.3.

The part it can't do is load two classes with the same fully-qualified name, because the language has no base functionality to build that on. Designing configuration files is a complete waste of time until you've designed that base functionality: when you load two classes with the same fully-qualified name, what exactly do you want the engine to do? What will need to change in the core of the language to make that possible?

Regards,
Rowan Tommins
[IMSoP]

4 months ago by Jordi Boggiano — view source

unread

And this sentence is why I chose to reply to your message. That assumption itself blocks the needs of user-managed apps.

(Did you happen to read my compare-and-contrast of user-managed vs. developer-managed apps from a few days ago?)

I feel it is likely those who have never worked professionally in PHP on user-managed apps like WordPress — which I assume describes you accurately? — are probably just simply unaware of the problems that your assumptions cause for user-managed apps. And yes, some developers have no empathy for others who have different circumstances, but I honestly don't think you (Rowan) are in the category.

Just one note here: You keep saying user-managed apps but from what I
can tell, these problems really only apply to WordPress.

There are others like Contao CMS who decided as a project they wanted
user-managed plugins but also wanted to rely on Composer and its
ecosystem of packages, and they made it happen [1].

So while I have some sympathy for all developers stuck maintaining WP
sites, and plugin authors not willing to do everything themselves and
deciding to bundle a vendor dir with Composer-installed dependencies..
It feels like you're all kinda held hostage with the choices of the
WordPress project, which sucks for sure but saying it is unfixable is
not helping.

As for the rest of the thread, I feel like everyone needs to take a few
days to chill because it's getting a bit heated around here.

Best,
Jordi

[1] https://docs.contao.org/manual/en/installation/contao-manager/

--
Jordi Boggiano
@seldaek -https://seld.be

4 months ago by Mike Schinkel — view source

unread

And this sentence is why I chose to reply to your message. That assumption itself blocks the needs of user-managed apps.

(Did you happen to read my compare-and-contrast of user-managed vs. developer-managed apps from a few days ago?)

I feel it is likely those who have never worked professionally in PHP on user-managed apps like WordPress — which I assume describes you accurately? — are probably just simply unaware of the problems that your assumptions cause for user-managed apps. And yes, some developers have no empathy for others who have different circumstances, but I honestly don't think you (Rowan) are in the category.

Just one note here: You keep saying user-managed apps but from what I can tell, these problems really only apply to WordPress.

There are numerous others, such as Joomla, Phalcon, CMS Made Simple, TextPattern, OpenCart, ExpressionEngine and ProcessWire.

But yes, the fact that WordPress has well over 50% marketshare of CMS and dwarfs all the others by well over an order of magnitude makes it appear that it only really applies to WordPress.

Which, given the number of users for each, it kinda does. But that also begs the question if dismissing the PHP app with by-far the largest user base is the smartest approach?

There are others like Contao CMS who decided as a project they wanted user-managed plugins but also wanted to rely on Composer and its ecosystem of packages, and they made it happen [1]

I just scoured the Contao website and the websites of a few of their 3rd party extensions and after quite a bit of detective work I found how Contao allows user-managed plugins, per se. They did not make it obvious on their site how to do it, though.

First you have to download and install a `.phar file called Contao Manager[1] and install it on your server. Which yeah, a developer could do for an end-user, but most end-users are going to be blocked by this.

Second, to use Contao Manager you have to have a PHP installation that allows using the functions proc_open()/proc_close() and allow_url_fopen() which AFAIK most responsible web hosts lock down on a shared server, which is the majority of WP hosting. The reason those are required is Contao Manager is just using Composer on the backend via CLI to install and manage plugins.

So while yes, Contao does allow users to manage plugins, that was not what I meant when I made a distinction between user-managed and developer-managed apps. To me, Contao is squarely in the developer-managed app category.

So while I have some sympathy for all developers stuck maintaining WP sites, and plugin authors not willing to do everything themselves and deciding to bundle a vendor dir with Composer-installed dependencies.. It feels like you're all kinda held hostage with the choices of the WordPress project, which sucks for sure but saying it is unfixable is not helping.

How are you envisioning it be fixed?

If there are ways to fix things that do not require WordPress to make a change with major BC breakage nor require users upgrade web hosts to support dedicated features I would be very happy to find out what those are.

And yes, to concur by also clarify, developers are held hostage by the combined choices of the WordPress and the PHP projects. It is kinda like being a kid with needs but also with feuding parents. ¯_(ツ)_/¯

-Mike

[1] https://docs.contao.org/manual/en/installation/contao-manager/

4 months ago by Michael Morris — view source

unread

As for the rest of the thread, I feel like everyone needs to take a few
days to chill because it's getting a bit heated around here.

People are passionate about the things they love. I've been busy with work
and on Iteration V. Details later this week, likely in the form of a
github repo hosted markdown file that will be easier to follow, as the
number of points to address is getting too lengthy to deal with in a list
conversation and, more worryingly, people are glomming onto and criticizing
the proposal for things already dropped.

In particular - I'm not talking about ditching composer. I don't want to
get in the business of building a package manager, especially when there's
an existing one.

Some highlights of what I am working on

Import Maps - These would be prepared by hand or by a package manager
like composer. An internal autoloader will work with them. Unlike autoload
functions, import maps get merged together so the actual seek operation can
remain optimized. Given Composer already can build an optimized class map
file it is already well positioned to feed information into this code. The
reason to use it is that it will be able to detect symbols the autoload
system cannot: functions and constants. It will also be able to load
packages and modules.
Packages - Packages load differently and can effectively monkey-type the
code of an existing package on the fly in much the same way that namespaces
themselves work with symbol names as a flat string replace. Existing code
can be loaded into packages, but also an outline for writing packages that
have privacy modifiers to their members - i.e. protected class SomeClass {}
Modules - Files which are code first instead of template first.

These are being prepped as at least 3 interrelated RFC drafts which can be
dealt with piecemeal, and any of the three not receiving enough support for
inclusion doesn't preclude the others from going forward.

4 months ago by Rowan Tommins [IMSoP] — view source

unread

Just to repeat a point that's been raised a few times: this is not a great time of year for this kind of discussion. If you come back after 8.4 is baked, you may get more enthusiasm. That will also give you time to make some more detailed analysis, so we don't have to argue about hypothetical difficulties.

Import Maps - These would be prepared by hand or by a package manager
like composer.

As Larry mentioned, there was a proposal for this a while ago, but not much enthusiasm, since it's so easy to implement in userland, and doing so means we don't have to include all the possible options someone might want.

it will be able to detect symbols the autoload
system cannot: functions and constants.

Autoloading functions and constants isn't blocked by autoloaders being procedural, it's blocked by the unfortunate decision made many years ago that a function call like "strlen" dynamically falls back to meaning "\strlen", rather than being resolved once at compile-time like class names.

So far, nobody's quite cracked how that should interact with autoloading. Don't expect this to be easy.

Packages - Packages load differently and can effectively monkey-type the
code of an existing package on the fly in much the same way that namespaces
themselves work with symbol names as a flat string replace.

This is an interesting - but extremely complex - problem, and the one I've been urging you to focus on if you're really up for the challenge. It probably needs quite a deep dive into how the language works to find out what assumptions it's going to break. (If you're just going to talk about configuration, and not the actual implementation, don't expect much enthusiasm.)

Existing code
can be loaded into packages, but also an outline for writing packages that
have privacy modifiers to their members - i.e. protected class SomeClass {}

This part seems interesting, as long as it's not tied heavily into other changes; existing code using /** @internal */ should ideally need minimal changes to make use of it.

Modules - Files which are code first instead of template first.

If by "template first" you mean "you have to write <?php at the top", I repeat my earlier "meh". I'm pretty sure it's also been discussed before, and dropped when it met with that general reaction.

The name "modules" implies something more, so maybe I should reserve judgement. Having both "packages" and "modules" sounds pretty confusing though.

Regards,
Rowan Tommins
[IMSoP]

4 months ago by Michael Morris — view source

unread

On Wed, Jul 10, 2024 at 3:29 PM Rowan Tommins [IMSoP] imsop.php@rwec.co.uk
wrote:

Just to repeat a point that's been raised a few times: this is not a great
time of year for this kind of discussion. If you come back after 8.4 is
baked, you may get more enthusiasm. That will also give you time to make
some more detailed analysis, so we don't have to argue about hypothetical
difficulties.

I'm in no rush - though it might not seem that way. I don't see this being
able to land before PHP 10. I'm pessimistic about the scope of these
changes. It can be done - and pieces have often been discussed before, but
they peter out. If they are to be solved some amount of stupid bulldog
tenacity will be needed. I think I'm stupid enough to provide that, but I
need to do it without being annoying.

In any event these threads have already shown me a great deal of what I
need to learn in order to get to an effective final form, whatever that is.

Import Maps - These would be prepared by hand or by a package manager
like composer.

As Larry mentioned, there was a proposal for this a while ago, but not
much enthusiasm, since it's so easy to implement in userland, and doing so
means we don't have to include all the possible options someone might want.

it will be able to detect symbols the autoload
system cannot: functions and constants.

Autoloading functions and constants isn't blocked by autoloaders being
procedural, it's blocked by the unfortunate decision made many years ago
that a function call like "strlen" dynamically falls back to meaning
"\strlen", rather than being resolved once at compile-time like class
names.

So far, nobody's quite cracked how that should interact with autoloading.
Don't expect this to be easy.

At worst this is the sort of "unfortunate decision" that can be eschewed in
the PHP module files to make them easier to work with. But I really do
need a list of these things we'd love to do but can't because of reasons.

Packages - Packages load differently and can effectively monkey-type
the
code of an existing package on the fly in much the same way that
namespaces
themselves work with symbol names as a flat string replace.

This is an interesting - but extremely complex - problem, and the one I've
been urging you to focus on if you're really up for the challenge. It
probably needs quite a deep dive into how the language works to find out
what assumptions it's going to break. (If you're just going to talk about
configuration, and not the actual implementation, don't expect much
enthusiasm.)

I need to know where to start, beyond cloning the PHP source code repo -
which I have. Any advice on where to look would be appreciated.

Existing code
can be loaded into packages, but also an outline for writing packages that
have privacy modifiers to their members - i.e. protected class SomeClass
{}

This part seems interesting, as long as it's not tied heavily into other
changes; existing code using /** @internal */ should ideally need minimal
changes to make use of it.

Modules - Files which are code first instead of template first.

If by "template first" you mean "you have to write <?php at the top", I
repeat my earlier "meh". I'm pretty sure it's also been discussed before,
and dropped when it met with that general reaction.

The name "modules" implies something more, so maybe I should reserve
judgement. Having both "packages" and "modules" sounds pretty confusing
though.

The largest thrust of modules is to step forward with changes that are
desirable but impossible to implement because of BC breaks brought on by
unfortunate design decisions like the one mentioned previously. Likely
these will be visited on a case by case basis. For another is the need of
classes to have the function keyword all over the place.

It could end up that things like package privacy can only be supported in
the modules. As to the difference, since it meanders all of the place
here's the defs I'm going with - A module is a file. A package is a
collection of files.

4 months ago by Rowan Tommins [IMSoP] — view source

unread

The largest thrust of modules is to step forward with changes that are
desirable but impossible to implement because of BC breaks brought on by
unfortunate design decisions like the one mentioned previously. Likely
these will be visited on a case by case basis. For another is the need of
classes to have the function keyword all over the place.

That sounds like a one-time chance for some fairly random changes you happen to like the idea of, at the cost of permanently forking the language into two dialects. It doesn't sound much to do with "modules", and I don't think it will be popular.

It could end up that things like package privacy can only be supported in
the modules.

Please, please, don't do that. I don't want to rewrite a bunch of code into a different flavour of the language, just to make use of a new feature that has nothing to do with those changes.

Rowan Tommins
[IMSoP]

4 months ago by Jordan LeDoux — view source

unread

I'm in no rush - though it might not seem that way. I don't see this being
able to land before PHP 10. I'm pessimistic about the scope of these
changes. It can be done - and pieces have often been discussed before, but
they peter out. If they are to be solved some amount of stupid bulldog
tenacity will be needed. I think I'm stupid enough to provide that, but I
need to do it without being annoying.

In any event these threads have already shown me a great deal of what I
need to learn in order to get to an effective final form, whatever that is.

The point of people asking multiple times to wait until any other time of
year is not because anyone is worried you are trying to get it in right
away, it's because doing this kind of freeform "I don't know what I don't
know" discussion is unkind to all of the experts on the list who CAN tell
you "well, what you don't know is X". Most of them probably won't even
respond at this time of year, even if they did read, which they probably
didn't.

Import Maps - These would be prepared by hand or by a package manager
like composer.

As Larry mentioned, there was a proposal for this a while ago, but not
much enthusiasm, since it's so easy to implement in userland, and doing so
means we don't have to include all the possible options someone might want.

it will be able to detect symbols the autoload
system cannot: functions and constants.

Autoloading functions and constants isn't blocked by autoloaders being
procedural, it's blocked by the unfortunate decision made many years ago
that a function call like "strlen" dynamically falls back to meaning
"\strlen", rather than being resolved once at compile-time like class
names.

So far, nobody's quite cracked how that should interact with autoloading.
Don't expect this to be easy.

At worst this is the sort of "unfortunate decision" that can be eschewed
in the PHP module files to make them easier to work with. But I really do
need a list of these things we'd love to do but can't because of reasons.

That's your job as the proposer. :)

Packages - Packages load differently and can effectively monkey-type
the
code of an existing package on the fly in much the same way that
namespaces
themselves work with symbol names as a flat string replace.

This is an interesting - but extremely complex - problem, and the one
I've been urging you to focus on if you're really up for the challenge. It
probably needs quite a deep dive into how the language works to find out
what assumptions it's going to break. (If you're just going to talk about
configuration, and not the actual implementation, don't expect much
enthusiasm.)

I need to know where to start, beyond cloning the PHP source code repo -
which I have. Any advice on where to look would be appreciated.

Try to change something, compile, then debug a test file with it. That's
how I went from "has only done basic C" to "wrote almost all of the
implementation of operator overloads" in a few months. Once you start on
that, you'll be able to ask more specific questions that are more likely to
get a specific and quick response. You'll want to know "when do we use this
macro" and so on.

You can also reference all the documentation that has been built for people
who are getting into learning the PHP source:

https://www.phpinternalsbook.com/index.html

Existing code
can be loaded into packages, but also an outline for writing packages
that
have privacy modifiers to their members - i.e. protected class SomeClass
{}

This part seems interesting, as long as it's not tied heavily into other
changes; existing code using /** @internal */ should ideally need minimal
changes to make use of it.

Modules - Files which are code first instead of template first.

If by "template first" you mean "you have to write <?php at the top", I
repeat my earlier "meh". I'm pretty sure it's also been discussed before,
and dropped when it met with that general reaction.

The name "modules" implies something more, so maybe I should reserve
judgement. Having both "packages" and "modules" sounds pretty confusing
though.

The largest thrust of modules is to step forward with changes that are
desirable but impossible to implement because of BC breaks brought on by
unfortunate design decisions like the one mentioned previously. Likely
these will be visited on a case by case basis. For another is the need of
classes to have the function keyword all over the place.

It could end up that things like package privacy can only be supported in
the modules. As to the difference, since it meanders all of the place
here's the defs I'm going with - A module is a file. A package is a
collection of files.

So then the purpose of modules, to you, is explicitly to provide features
that "can't be done" in PHP? Most of the ones people want are being worked
on in some way, even if they "can't be done", so I'm curious what sort of
list of features you'll come up with. That feels like it lacks the defining
and critical feature of packages and modules in literally every language
that has ever had it though:

Encapsulation/Restricted Global Scope/Local Symbols

If "modules" are not somehow separated in a controllable way, then you
aren't building "modules", you're forking PHP.

Jordan

4 months ago by Jordi Boggiano — view source

unread

On Wed, Jul 10, 2024 at 5:51 AM Jordi Boggiano j.boggiano@seld.be
wrote:
As for the rest of the thread, I feel like everyone needs to take
a few days to chill because it's getting a bit heated around here.
People are passionate about the things they love. I've been busy with
work and on Iteration V. Details later this week, likely in the form
of a github repo hosted markdown file that will be easier to follow,
as the number of points to address is getting too lengthy to deal with
in a list conversation and, more worryingly, people are glomming onto
and criticizing the proposal for things already dropped.

Maybe if people are criticizing outdated versions of the spec it's
because the first email came up exactly two weeks ago, and you're now on
a 5th rewrite, and it all got mangled in the same 130+ email thread
here. So yeah maybe someone missed some update..

That's why I said above, maybe take a breather, start a new thread with
v5 in a week or two to get a fresh start.

Best,
Jordi

--
Jordi Boggiano
@seldaek -https://seld.be

4 months ago by Stephen Reay — view source

unread

From: Stephen Reay <php-lists@koalephant.com mailto:php-lists@koalephant.com>
Sent: Wednesday, July 3, 2024 1:17 PM

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders

Can run userland code. It doesn't have to; FYI spl_autoload (https://www.php.net/manual/en/function.spl-autoload.php) has existed since php5.1 and works amazingly well.

That "standards" like psr-whatever can't (read: choose not to) use it says more about people and maintaining their little fiefdoms than anything else.

As a PHP-FIG Core Committee member, I find this characterisation of people involved in the FIG offensive. My contribution, however big or small, is intended to help the PHP community at large.

If you choose to be offended by my opinion, I can't really help that.

No, but you also don't need to air your personal grievances on the mailing list. If you don't like what FIG or any other entity in the PHP ecosystem is doing, this is NOT the place to air that grievance. Internals is for discussing changes to the runtime. Calling out entities like this here is bound to alienate folks who want to work on the engine, and who are also parts of those groups.

I'm glad we're in agreement that this list is about the runtime and not about composer or FIG. I look forward to seeing a response from you as vivid as this one, the next time someone responds to a discussion about something like function autoloading with "X isn't really a problem because composer".

It also doesn't help your argument when you're stating things that are flat out wrong as facts. You can absolutely use spl_autoload() alongside the PSR recommendations or Composer; see more below

Please re-read what I wrote, before making ironic statements about 'facts'. I never said you can't use them "along side" each other. I said that the PSR's are incapable of using the built-in functionality provided by spl_autoload. That is: you can't adhere to either PSR using the builtin autoloader alone. That you can use them along-side each other is unrelated to spl_autoload, it's a function of the stack created by spl_autoload_register.

.

To come back to spl_autoload: That function pre-dates namespaces and is highly opinionated on how to organise code. All lower-case filenames, class per-file, files in include_path, full namespace in path, you name it. If that is what projects wanted at the time, or even now, PSR-0 and the PHP-FIG would possibly not even exist.

It's less highly opinionated than either PSR, but that's my whole point: it's someone else's opinion, hence it's opposed by FIG.

That's a gross mischaracterization.

In point of fact, most frameworks that joined FIG in the beginning were leveraging spl_autoload_register(), which provides a stack of autoloaders that each provide their own logic for how to map classes to where on the filesystem they live. spl_autoload_register() came after spl_autoload(), and was introduced to add flexibility to the language, as spl_autoload is proscriptive and only allows a single approach to autoloading, and it wasn't even one that was widely used at the time it was introduced. It's not about opinions, it's about recognizing that different approaches might have merit. (Some might give better performance, some might allow pulling items out of a phar or tarball, etc.)

spl_autoload, spl_autoload_register, and spl_autoload_extensions were all added in php5.1 (compare https://3v4l.org/H60EG#v5.0.5 vs https://3v4l.org/H60EG#v5.1.0). Maybe you're thinking of __autoload, which had no default implementation before spl_autoload was added.

The configurable part for autoloading in the language is spl_autoload_register(), full stop.

spl_autoload is (and has been since inception) configurable via spl_autoload_extensions, and via the standard include path.

4 months ago by Rob Landers — view source

unread

From: Stephen Reay php-lists@koalephant.com
Sent: Wednesday, July 3, 2024 1:17 PM

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders

Can run userland code. It doesn't have to; FYI spl_autoload (https://www.php.net/manual/en/function.spl-autoload.php) has existed since php5.1 and works amazingly well.

That "standards" like psr-whatever can't (read: choose not to) use it says more about people and maintaining their little fiefdoms than anything else.

As a PHP-FIG Core Committee member, I find this characterisation of people involved in the FIG offensive. My contribution, however big or small, is intended to help the PHP community at large.

If you choose to be offended by my opinion, I can't really help that.

No, but you also don't need to air your personal grievances on the mailing list. If you don't like what FIG or any other entity in the PHP ecosystem is doing, this is NOT the place to air that grievance. Internals is for discussing changes to the runtime. Calling out entities like this here is bound to alienate folks who want to work on the engine, and who are also parts of those groups.

It also doesn't help your argument when you're stating things that are flat out wrong as facts. You can absolutely use spl_autoload() alongside the PSR recommendations or Composer; see more below.

To come back to spl_autoload: That function pre-dates namespaces and is highly opinionated on how to organise code. All lower-case filenames, class per-file, files in include_path, full namespace in path, you name it. If that is what projects wanted at the time, or even now, PSR-0 and the PHP-FIG would possibly not even exist.

It's less highly opinionated than either PSR, but that's my whole point: it's someone else's opinion, hence it's opposed by FIG.

That's a gross mischaracterization.

In point of fact, most frameworks that joined FIG in the beginning were leveraging spl_autoload_register(), which provides a stack of autoloaders that each provide their own logic for how to map classes to where on the filesystem they live. spl_autoload_register() came after spl_autoload(), and was introduced to add flexibility to the language, as spl_autoload is proscriptive and only allows a single approach to autoloading, and it wasn't even one that was widely used at the time it was introduced. It's not about opinions, it's about recognizing that different approaches might have merit. (Some might give better performance, some might allow pulling items out of a phar or tarball, etc.)

PSR-0 was created because a large number of projects were writing their own autoloaders that were doing similar things, and most of them were doing things differently than spl_autoload() due to limitations of that approach, and all were using spl_autoload_register(). Creating a standard approach allowed users of these projects to use a single autoloader to load code from each within their application, which helped improve performance and reduced autoloading conflicts. PSR-4 extended the concept, while keeping some of the core ideas in place. And, again, YOU DO NOT NEED TO FOLLOW either one.

Why?

Because Composer uses spl_autoload_register() internally, and enables multiple autoloading approaches (PSR-0, PSR-4, classmap, file, etc.) out of the box. And if you don't want to use those for your own code... you can add another autoloader to the stack using spl_autoload_register(). You can even add your own before invoking the Composer autoloader to ensure it gets precedence. Composer's then becomes primarily a tool for loading the third-party code your application depends on.

Neither of which is the point I was making - someone claimed that autoloaders are implicitly userland code. The point is they don't have to be, and there is a perfectly useable one built in to the SPL extension; if it's "too opinionated" (or the opinions are ones you don't like), it's hardly the most in-depth of functions, and it already has configurable parts, so adding in more control shouldn't exactly require a rocket scientist to add, for example, the ability to use the original case of the class name.

The configurable part for autoloading in the language is spl_autoload_register(), full stop. And this does require userland code. Yes, you can register spl_autoload() with it, and this is part of the engine, but that's the only language-level autoloader at this time. I'd argue we shouldn't add any more to the engine; the stack approach of spl_autoload_register() ensures we can reduce engine complexity and maintenance by offloading it to something that can evolve at a faster pace than the language.

I'm following the packaging threads closely, and the one thing I've failed to see a solid argument for is what problems the current approach of using namespaced code doesn't address. I can definitely see a need for marking things as package private (i.e., not part of the publicly consumable API), but that also feels like something we could address in other ways. I know Larry has asked this same question before, and it's really what I want to see answered, because packages might be the solution, but there may be other approaches we could take that also accomplish those goals.

--
Matthew Weier O'Phinney
mweierophinney@gmail.com
https://mwop.net/
he/him

Hi Mathew!

My main feedback to PSR’s is that they are fundamentally broken due to being outdated. The idea behind the standards is sound, but there are only a few PSRs that are applicable to today’s PHP. When I look at creating new libraries today, PSR’s are a good inspiration, but they probably shouldn’t be used in actual programs. Here’s a short list of outdated standards I’ve collected over the last couple of years:

• 7
• 11
• 15
• 18
• 20
Most of these breakdown if you start dealing with fibers, various new scopes introduced by runtimes, new http standards, etc.

Ergo, if you’ve run into these issues, you are likely inclined to stay as far away from PSR’s as reasonably possible and may have a negative view of them. However, for enterprise applications, PSR’s are a godsend. So they do have their uses, don’t get me wrong, but if you want to use technology newer than 2019-ish, you need to avoid them; and this is a large part of why I say PSR-XX isn’t a valid argument on this list.

All that being said, I’ve also looked into contributing over there, but my time is solidly booked up at the moment. Even on this list, I mostly only participate on the train to/from work.

— Rob

4 months ago by Ken Guest — view source

unread

My main feedback to PSR’s is that they are fundamentally broken due to
being outdated. The idea behind the standards is sound, but there are only
a few PSRs that are applicable to today’s PHP. When I look at creating new
libraries today, PSR’s are a good inspiration, but they probably shouldn’t
be used in actual programs. Here’s a short list of outdated standards I’ve
collected over the last couple of years:

7

11

15

18

20

Most of these breakdown if you start dealing with fibers, various new
scopes introduced by runtimes, new http standards, etc.

Ergo, if you’ve run into these issues, you are likely inclined to stay as
far away from PSR’s as reasonably possible and may have a negative view of
them. However, for enterprise applications, PSR’s are a godsend. So they do
have their uses, don’t get me wrong, but if you want to use technology
newer than 2019-ish, you need to avoid them; and this is a large part of
why I say PSR-XX isn’t a valid argument on this list.

This is one of the reasons why we [FIG] now have the concept of PHP
Evolving Recommendations (PERs), which can be updated/evolved over time
with multiple releases - instead of having having a succession of PSRs and
having to know which one supersedes another.

The only example of this thus far is PER-CS for Coding Standards - version
1 being the PER equivalent of PSR-12 and version 2 of PER-CS being an
update addressing syntax in PHP that was not present when PSR-12, e.g.
match, enums, attributes and all that other wonderful stuff.

There is no reason why any current PSR can't be replaced with a PER
equivalent - given a Working Group, time and focus.

Except maybe PSR-8.

--
http://about.me/kenguest/

4 months ago by Rob Landers — view source

unread

__

My main feedback to PSR’s is that they are fundamentally broken due to being outdated. The idea behind the standards is sound, but there are only a few PSRs that are applicable to today’s PHP. When I look at creating new libraries today, PSR’s are a good inspiration, but they probably shouldn’t be used in actual programs. Here’s a short list of outdated standards I’ve collected over the last couple of years:

• 7
• 11
• 15
• 18
• 20
Most of these breakdown if you start dealing with fibers, various new scopes introduced by runtimes, new http standards, etc.

Ergo, if you’ve run into these issues, you are likely inclined to stay as far away from PSR’s as reasonably possible and may have a negative view of them. However, for enterprise applications, PSR’s are a godsend. So they do have their uses, don’t get me wrong, but if you want to use technology newer than 2019-ish, you need to avoid them; and this is a large part of why I say PSR-XX isn’t a valid argument on this list.

This is one of the reasons why we [FIG] now have the concept of PHP Evolving Recommendations (PERs), which can be updated/evolved over time with multiple releases - instead of having having a succession of PSRs and having to know which one supersedes another.

The only example of this thus far is PER-CS for Coding Standards - version 1 being the PER equivalent of PSR-12 and version 2 of PER-CS being an update addressing syntax in PHP that was not present when PSR-12, e.g. match, enums, attributes and all that other wonderful stuff.

There is no reason why any current PSR can't be replaced with a PER equivalent - given a Working Group, time and focus.

Except maybe PSR-8.

--
http://about.me/kenguest/

If I could have one to be a PER first, it would be the container interface (PSR-11). When it was originally worked on, there was basically one lifecycle of an object: a request. For almost all possible SAPIs, after the request ended, everything was gone. Today, we basically have multiple ones if you are using modern runtimes:

Environment Scope: configuration that survives execution of the program
Global Scope: services/entities that can exist beyond the lifetime of a request, but may or may not (depending on runtime)
Request Scope: services/entities that should be unique for every request.
Volatile Scope: services/entities that should be created every time they are injected.
Right now, these are all mixed into one giant container, that may or may not be shared between requests, because that is the interface we have to work with. It isn't great :| but it works, barely.

— Rob

4 months ago by Larry Garfield — view source

unread

If I could have one to be a PER first, it would be the container
interface (PSR-11). When it was originally worked on, there was
basically one lifecycle of an object: a request. For almost all
possible SAPIs, after the request ended, everything was gone. Today, we
basically have multiple ones if you are using modern runtimes:

Environment Scope: configuration that survives execution of the
program

Global Scope: services/entities that can exist beyond the lifetime
of a request, but may or may not (depending on runtime)

Request Scope: services/entities that should be unique for every
request.

Volatile Scope: services/entities that should be created every time
they are injected.
Right now, these are all mixed into one giant container, that may or
may not be shared between requests, because that is the interface we
have to work with. It isn't great :| but it works, barely.

— Rob

There's been on and off discussion this year about container registration, not just retrieval. That could be done as an add-on PSR, most likely. The challenge is that there's fundamentally different ways to go about registration, and little consensus on it. (And the people most interested in talking about it haven't wanted to go to the effort of organizing a Working Group.) I suspect scoping would have similar challenges. But if someone can get a working group together around a particular direction, we'd be open to that discussion.

That would be entirely off topic for this list, though, so let's not go further down that rabbit hole.

--Larry Garfield

4 months ago by Mike Schinkel — view source

unread

Sent from my iPhone

Autoloading runs userland code. This means it has the potential conflict between different packages with different autoloaders

Can run userland code. It doesn't have to; FYI spl_autoload (https://www.php.net/manual/en/function.spl-autoload.php https://www.php.net/manual/en/function.spl-autoload.php) has existed since php5.1 and works amazingly well.

Excellent point!

It has been so long that I have seen anyone use that, however, I actually forgot it exists.

Neither of which is the point I was making - someone claimed that autoloaders are implicitly userland code. The point is they don't have to be, and there is a perfectly useable one built in to the SPL extension; if it's "too opinionated" (or the opinions are ones you don't like), it's hardly the most in-depth of functions, and it already has configurable parts, so adding in more control shouldn't exactly require a rocket scientist to add, for example, the ability to use the original case of the class name.

Me personally, the opinions that I do not like are the one-symbol-per file assumption, which is also a key issue I have with PSR-4.

#fwiw.

I'm following the packaging threads closely, and the one thing I've failed to see a solid argument for is what problems the current approach of using namespaced code doesn't address. I can definitely see a need for marking things as package private (i.e., not part of the publicly consumable API), but that also feels like something we could address in other ways.

Understanding that the thread has been a brainstorming thread more than a proposal thread — ignoring whether or not it is effective to brainstorm on this list because of interpersonal list dynamics — my two cents in answer to the question of "What problem(s) are we trying to solve?"

Side-by-side symbol loading — PHP currently makes it difficult if not impossible to use different versions of the same library as dependencies of higher-level dependencies.
Symbol encapsulation — Allowing symbols to be hidden from code that should not use them.
Multiple symbols per file — Finding an approach that would be able to gain wide adoption for multiple symbols per file— without effectively requiring an app to load all source on each page load — to better support locality of behavior.

See: https://htmx.org/essays/locality-of-behaviour/
Unified loading — Currently constants, variables, functions, are the have-nots of the autoloading realm. Providing a manner for loading them, and a unified manner across all symbols would be even better.
Community buy-in — While not a goal in and of itself, ideally there would be a solution that would gain broad support over time so the approach does not get dismissed by the majority of developers simply for reasons such as it is not what they are already familiar with. Having official PHP endorsement would go a long way to address this.

-Mike

4 months ago by Michael Morris — view source

unread

Me personally, the opinions that I do not like are the one-symbol-per file
assumption, which is also a key issue I have with PSR-4.

That's a PSR-4 issue, not an autoloader one. Autoloaders, even in composer,
can use class maps to discover which file must be loaded to make the symbol
accessible, even if multiple unused symbols in that file come along for the
ride.

Understanding that the thread has been a brainstorming thread more than a
proposal thread — ignoring whether or not it is effective to brainstorm on
this list because of interpersonal list dynamics — my two cents in answer
to the question of "What problem(s) are we trying to solve?"

Brainstorming in a large group is difficult, but not impossible. The last
time I tried something like this I did succeed - it was the introduction of
runtime assertion to Drupal back in version 8. It took 6 MONTHS of work
and debate to get that one accepted, but it has paid off well. If this
goes on for a year I wouldn't be surprised. I'm fully prepared for the
blind alleys and wrong turns ahead.

Side-by-side symbol loading — PHP currently makes it difficult if
not impossible to use different versions of the same library as
dependencies of higher-level dependencies.

Symbol encapsulation — Allowing symbols to be hidden from code that
should not use them.

Uhm, I'm not formally trained so that one got by me - could you please give
an example of this? I might have posted one already without realizing it.

Multiple symbols per file — Finding an approach that would be able
to gain wide adoption for multiple symbols per file— without effectively
requiring an app to load all source on each page load — to better
support locality of behavior.

See: https://htmx.org/essays/locality-of-behaviour/

Unified loading — Currently constants, variables, functions, are the
have-nots of the autoloading realm. Providing a manner for loading them,
and a unified manner across all symbols would be even better.

Community buy-in — While not a goal in and of itself, ideally there
would be a solution that would gain broad support over time so the approach
does not get dismissed by the majority of developers simply for reasons
such as it is not what they are already familiar with. Having official PHP
endorsement would go a long way to address this.

This last one is essential because something as core to the language must
be popular.

4 months ago by Mike Schinkel — view source

unread

Me personally, the opinions that I do not like are the one-symbol-per file assumption, which is also a key issue I have with PSR-4.

That's a PSR-4 issue, not an autoloader one. Autoloaders, even in composer, can use class maps to discover which file must be loaded to make the symbol accessible, even if multiple unused symbols in that file come along for the ride.

My statement you are commenting on was about spl_autoload and PSR-4, full stop.

Symbol encapsulation — Allowing symbols to be hidden from code that should not use them.

Uhm, I'm not formally trained so that one got by me - could you please give an example of this? I might have posted one already without realizing it.

Which part? Symbol, or Encapsulation?

If Symbols, that is just the collective name for classes, interfaces, enums, functions, constants and variables.

If Encapsulation, then that means symbol hiding. PHP offers only limited forms of symbol hiding with private and protected. Currently it is not possible in PHP to have a top-level symbol in a namespace — vs. just a function, property or constant enclosed in and as part of a class — that a developer can disallow other developers from accessing via regular access methods, e.g. instantiating the class, implementing the interface, using the constant, calling the function. etc.

Does that answer?

-Mike

4 months ago by Larry Garfield — view source

unread

So let's take another crack at this based on all the points raised in
the thread. This should also underline why I don't consider this an RFC

I am iterating until we arrive at something that may be refinable
into an RFC. And I say we because without the aid of those in this
conversation I would not have arrived at what will follow.

Before I continue I would like to apologize for being somewhat
irritable. We're all here because we enjoy using this language and want
to see it improved and prevent bad changes. Opinions will differ on
this and in the heat of the moment of arguing a point things can get
borderline.

Returning to a point I made earlier, Composer isn't used on Wordpress.
I went over to the Wordpress discussion list and read over why, because
that discussion provides clues to what kind of package management may
be adoptable. I think the largest point is that Wordpress should be
usable without ever resorting to using the command line. Yes, it does
have a command line tool - wp-cli - and it is powerful, but using it as
an administrator of a Wordpress site is not required.

The largest block to composer's inclusion in Wordpress is the inability
to run multiple versions of a module. Yes, it's a mess when this
happens, but if you're an end user, you just want your plugins to work.
If one plugin that no one has updated in a year that you're using is
consuming version 2 of a package, you're gonna be annoyed at best if
the module stops working when you install a new plugin that is using
version 3 of the same package and has a BC break in it. Composer can't
resolve this easily.

There's still a few key, fatal issues in this version of the proposal.

It appears like it expects PHP to be able to write to disk itself as part of normal operation. This is an immediate fatal flaw. Most security best practices these days recommend that the disk where code is be read-only. Some hosting companies mandate it. Any language feature that precludes that is dead in the water.
Supporting multiple versions of the same class is waaaay out of scope. You seem to imply Composer is the reason we cannot do that. That's incorrect. PHP has a single global symbol table for classes. (And a separate one for functions for not-great historical reasons.) Trying to define the same class twice will fatal the engine. While there are some screwy conditional-include games you can play, they're fragile and still would not allow WP Plugin A to use v1 of a library and WP Plugin B to use v2 of a library. I am not versed in that part of the engine, but I would be shocked if splitting up the global symbol table was possible, let alone feasible.
Using URLs as the package naming system is the dumbest thing Go ever did. Let's not replicate that. :-)

I think the core problem here is that this thread keeps trying to graft Python/Go/JS/Rust style packages onto PHP. PHP, however, is structurally closer to Java/C#, so following that package logic (which is built off of namespaces) would be far more natural, and thus far easier to migrate to.

Really, the only targets we should be looking at, IMO, are:

Package-level visibility.
Giving the compiler/optimizer/JIT a larger "scope" of code to compile/optimize at once, so it can do smarter things.

I think everything else is a distraction.

I have some thoughts on how we can far more easily accomplish 1, and maybe 2, but I will probably hold off on that for now as it would just get lost in the noise of this thread, plus the list is too busy as is these days with everyone trying to get their in-flight RFCs finalized before the feature freeze deadline. :-) (Really, this is a poor time to be having this kind of discussion. Fall is generally the better time, just logistically.)

--Larry Garfield

4 months ago by Alexander Pravdin — view source

unread

Sorry for getting into this hot topic. Just wanted to add my two cents.

Really, the only targets we should be looking at, IMO, are:

Package-level visibility.

Giving the compiler/optimizer/JIT a larger "scope" of code to compile/optimize at once, so it can do smarter things.

I think everything else is a distraction.

As a userland developer, I like one of the author's intentions - to
allow writing modern PHP code without poor legacy, without opening
tags, which will also enable the compiler to apply more magick and
more optimizations. I could see it as a new file extension, for
example, "phpx" and release it in PHP10. This will not drop BC and at
the same time, users will have the opportunity to explicitly tell the
compiler that they want to use the new format. The inclusion of
old-style files will still be allowed. In the far future, we may
feature-freeze old-style files and gradually migrate to the new ones.
I personally support the movement from the current "plain-text
template-first" language to a "coding-first" language, where files
contain code by default. We already support multiline raw strings in
PHP code so the usage in templates should not suffer. Or it can even
be simplified by introducing a better way to craft templates in the
new "coding-first" format of PHP files.

I also support the idea of having packages with private members,
autoloading constants and functions without using classes.

--
Alexander.

4 months ago by Rowan Tommins [IMSoP] — view source

unread

I personally support the movement from the current "plain-text
template-first" language to a "coding-first" language, where files
contain code by default.

I honestly don't read the <?php at the top of files as an "opening tag" any more, just as a "magic number" that indicates it's a PHP file, like the doctype at the top of an HTML file. It might as well be "I <3 PHP".

Any suggestion to add a special of avoiding those 4 characters scores a big shrug from me.

Rowan Tommins
[IMSoP]

4 months ago by Alexander Pravdin — view source

unread

On Tue, Jul 2, 2024 at 1:20 AM Rowan Tommins [IMSoP]
imsop.php@rwec.co.uk wrote:

I personally support the movement from the current "plain-text
template-first" language to a "coding-first" language, where files
contain code by default.

I honestly don't read the <?php at the top of files as an "opening tag" any more, just as a "magic number" that indicates it's a PHP file, like the doctype at the top of an HTML file. It might as well be "I <3 PHP".

Any suggestion to add a special of avoiding those 4 characters scores a big shrug from me.

The issue is that without this legacy "magic number" the PHP file
contains plain text/html/whatever. This is weird. This means PHP is a
template processor in the first place. And adding some code is a
feature on top of the template processor. I was actually talking not
about avoiding 4 chars for the sake of avoiding 4 chars, but about
making the PHP source file a programming code file, not a template.
And in addition, cut some weird legacy allowing further engine
optimizations. Historical/legacy/BC reasons pop up regularly in
discussions about something that can not be optimized/improved. What
is the percentage of files containing PHP code only and pure PHP
templates in an average project? 100% vs 0%? 99% vs 1%? Who uses plain
PHP templates today? Why programmers should care about templating
legacy in each and every PHP file? I think it's time to switch from
"templating PHP" to "programming PHP".

Alexander

4 months ago by Mike Schinkel — view source

unread

Using URLs as the package naming system is the dumbest thing Go ever did. Let's not replicate that. :-)

This is probably the wrong thread and maybe even the wrong list to discuss this, but having programmed in PHP for a decade and then half a decade also in Go I think using URLs as the package naming system to be one of the smarter things Go ever did.

So I would really like to hear objective arguments as to why you think Go using URLs for packages is dumb. Feel free to take this to another thread, to a private email, or elsewhere if not appropriate for here.

-Mike

4 months ago by Michael Morris — view source

unread

On Mon, Jul 1, 2024 at 9:02 AM Larry Garfield larry@garfieldtech.com
wrote:

Supporting multiple versions of the same class is waaaay out of
scope.

No, it's actually the heart of the problem now that I've had a few days to
think on this, and it's something an autoloader can NOT resolve.

You seem to imply Composer is the reason we cannot do that. That's
incorrect. PHP has a single global symbol table for classes. (And a
separate one for functions for not-great historical reasons.) Trying to
define the same class twice will fatal the engine. While there are some
screwy conditional-include games you can play, they're fragile and still
would not allow WP Plugin A to use v1 of a library and WP Plugin B to use
v2 of a library. I am not versed in that part of the engine, but I would
be shocked if splitting up the global symbol table was possible, let alone
feasible.

Let's assume all that's true. Nothing you said stops this

import 'package v1' as \Package
import 'package v2' as \NewPackage

The problem with the current situation is the top level namespace of
packages are baked into the files. The import statement can alias the top
level namespace. Even if the engine cannot maintain multiple symbol tables,
what it can do is apply this aliasing on the fly as the same sort of hack
to the language that the current namespace implementation is (it's a blind
string replace, which is why they switched the namespace operator from ::
to the bloody escape character).

But for the alias to work the package has to tell import what its top level
namespace name is, especially for when the file is imported without an
alias, so that name will be used to graft the code in the package onto the
symbol table.

To illustrate, a package with a single file, its contents are

class TestClass {}

Note, no namespace declared in the file itself. The minimum contents of
the package declaration file, whatever form it takes, is:

namespace TestPackage;

With just that in place there's no reason I can see for the engine to be
unable to do this.

import 'testpackage';

\TestPackage\TestClass // This is actually created symbol

import 'testpackage' as NewPackage

\NewPackage\TestClass // This is the actually created symbol.

Set aside for a moment how the import statement resolves 'testpackage' -
the rules for resolving paths, urls, and version numbers are a different
topic. But to say there is no way this can be done - bah.

What this does cut off at the knees though is allowing users who want this
flexibility to divest their control over what gets loaded for their code to
the discretion of an autoloader. Much of the time, that's ok, but in more
complex applications it backfires. Sorta like typeless variables
themselves. For beginning programmers writing simple code they're fine. I
know - I was one - and this simplicity hooked me in where I initially
couldn't get my head around datatyping with C# when I was young. But at
the opposite end of the spectrum it backfires as having little to no
control over data types creates at least as many problems as it solves,
especially for automated testing.

We can have it both ways though. Let the infrastructure in place for
autoloading to stay in place, which will include the version of the code
called out in composer.json (assuming composer handles autoloading). But
if I want to use a specific version of a package, even one that is in the
overall application I'm writing for such as WordPress or Drupal, I should
be able to do that. But that's going to require aliasing the package's name
as outlined above.

4 months ago by Michael Morris — view source

unread

First off, in 10 years of using gmail I've never had it lose an email.
Well, it happened after I spent 4 hours on this. So, this is sorta
iteration 4. I'll type this up in Visual Studio code and then paste to
gmail.

The Wordpress discussion about composer and the decision not to use it keys
in the features a package management system must meet. These are

Command Line should not be required
Plugins using version X of a package should not be affected by plugins
using version Y
Install and maintenance of the site should remain possible from the
browser alone.
Backwards compatibility must be maintained.

A core PHP package management system should be a revision of what's come
before that doesn't disturb what has come before. PHP packages are as
follows:

Extensions, written in C++
PECL Libraries, which I believe can be in PHP or C, but are server-wide
Phar archives
Composer libraries, which are always in PHP and leverage namespaces,
PSR-4 convention and the autoload system.

Some of my prior discussions have been about what I'd like to see in a new
module system, but for scope, clarity, and sanity reasons I'm going to set
that aside. (Not to mention some of the more controversial pieces have been
there)

Within the package management system there is a single new keyword -
import. I'll go into details on it in a bit but suffice to say its behavior
is different from include/require.

In this schema a PHP application is a collection of packages. Existing PHP
code that does not use the import statement or the special directories
mentioned below will not invoke anything discussed on this page and will
not need to change in any way.

Applications

The application is the root package. It is the package that imports to the
root namespace. When PHP is asked to parse a file it will look for a
.php-packages folder, first in the current working directory then in
parent directories. If it doesn't find one, business as usual. If we do
find one we follow its directives about setting up an application
environment.

The .php-packages folder is where PHP will put package related code for
the application at hand. Code written explicitly for these changes will
also put their package related files there - composer's vendor directory,
composer.json, composer.lock, and so on - rather than putting those files
in the site root. The folder is hidden to prevent web servers like nginx or
apache from serving the files directly in any way.

The .php-packages directory will have a configuration file called
php.mod. This tells the parser:

How to initialize the application
What autoloaders are to be used
How to resolve import

Let's look at what such a file might look like for Drupal. For the moment
I'm going to use go.mod's syntax. The final syntax to be used, be it ini,
yaml, toml, json, is a discussion for another time. The part to focus in on
here is what type of information do we need.

package Drupal

php 10

registry //packagist.org/packages composer

init (
composer install
)

require (
./vendor/autoload.php
)

imports (
//getcomposer.org/composer.phar
)

The directives do the following:

package sets the name of the package if it is imported. Sort of
irrelevant here, but present for consistency
registry sets up the default registry for the import statement and the
loader used to install packages
init is the command(s) to run before the application is started for the
first time.
require is the file(s) to require before running any file in the
application
imports are the imports the package needs, in this case the composer.phar
to load composer in locally.

Now, given the popularity of composer some of these directives likely could
be put in as logical defines, particularly the init, require and imports
directives just being put in place if composer is selected as a loader. I
don't rule out the possibility of a competitor to composer showing up one
day though, as yarn was introduced as an alternative to npm.

If this theoretical version of Drupal moves its composer.json and
composer.lock files into .php-packages then the autoloader doesn't have
to be required in the index.php file. Also, the application can be started
without running composer install

Import statement
Given the Drupal application package in the previous section, we can have
an extension file call out its dependencies in code rather than in config.

import "twig/twig"

But granted, there's not much point to this since Drupal already uses
Twig. But suppose for the sake of argument that Symfony releases a Twig 4
with some major BC breaks, and we're working on a Drupal extension that
wants to use the new features of that Twig before Drupal itself upgrades
Twig in core. Well, now we can do this

import "twig/twig ^4.0" as NewTwig

Note that we have to mount this package off the root using the alias syntax
above. If the new package isn't written with this system in mind then
we'll end up having NewTwig prefixed on the normal namespace path which
would end up looking like this

use NewTwig\Twig\Environment

However, if this new twig has a php-mod file then the name of the top
level namespace must be put there, and all the other files do not have to
call out that namespace at their top. This allows the alias to plug in more
seamlessly allowing for this use statement

use NewTwig\Environment

Again, running two versions of the same package is not ideal, but sometimes
it's unavoidable. Especially in large ecosystems like WordPress.

Imported packages run on their own request thread (I think that's the best
way - I'm sure the guys working on the egine know best). They don't see or
affect anything outside themselves. They don't see global variables or even
the superglobals. Namespace resolution for them is - touchy.

We could do it like this: A symbol starting without a \ resolves locally,
then goes up the chain till it hits the application root. This is most
similar to the current namespace system, but it allows for hidden
dependencies and frankly, that worries me.

I'm more inclined to isolate package so that if they want to use something
they need to import it for themselves even if the hosting application
already has the lib. This means namespace resolution in a package stops at
the root of the package. We end up with something like this

namespace MyExtension
import "twig/twig ^3.0"
use \Twig\Environment

The package manager should be able to provide this without needing to
download the package twice. Note also in this import that we specify we
want version 3 which we've tested our Extension with. If that matches the
App, great! Only one copy of the code needed. If it doesn't, it's
suboptimal but it will run.

As mentioned above, import also needs to be able to load extensions.
Currently extensions are locked into PHP - they can't advance at different
rates. This has led to several extension improvements getting abandoned
over BC concerns. However, this would be nice

import "ext://mysql ^2.0"

Architecturally this would allow the PHP suite to move to a more mono-repo
style with the various extensions having defaults for the current PHP
distribution and newer versions available. It would also stop a repeat of
having to have both mysql and mysqli functions loaded at the same time.

This is enough to chew on for now. I'll take notes from the conversation
that follows and iterate again.

4 months ago by michal.brzuchalski@gmail.com — view source

unread

Hi Michael,

pon., 1 lip 2024 o 01:18 Michael Morris tendoaki@gmail.com napisał(a):

...
Applications

The application is the root package. It is the package that imports to the
root namespace. When PHP is asked to parse a file it will look for a
.php-packages folder, first in the current working directory then in
parent directories. If it doesn't find one, business as usual. If we do
find one we follow its directives about setting up an application
environment.

The .php-packages folder is where PHP will put package related code for
the application at hand. Code written explicitly for these changes will
also put their package related files there - composer's vendor directory,
composer.json, composer.lock, and so on - rather than putting those files
in the site root. The folder is hidden to prevent web servers like nginx or
apache from serving the files directly in any way.

First, you use the term Application then Site - decide.
Not all PHP applications are HTTP Applications, consider consumers, cron
tasks other daemons, these don't need the existence of either Nginx or
Apache.
Not all PHP HTTP Applications expose files to Nginx or Apache - most of
these I know like Rest API give ZERO access to any application file to
Nginx or Apache.

The .php-packages directory will have a configuration file called
php.mod. This tells the parser:

You propose to move composer.json, composer.lock, and vendor into a
hidden folder with no good reason which simply adds more confusion.

Let's look at what such a file might look like for Drupal. For the moment
I'm going to use go.mod's syntax. The final syntax to be used, be it ini,
yaml, toml, json, is a discussion for another time. The part to focus in on
here is what type of information do we need.

package Drupal

php 10

registry //packagist.org/packages composer

init (
composer install
)

require (
./vendor/autoload.php
)

imports (
//getcomposer.org/composer.phar
)

This looks like a completely new file format which simply makes
interoperability harder, consider tools like GitHub Dependabot, PHPMetrics
or other tools that analyze dependencies - what you propose requires
implementing a parser in userland or other languages the tool uses.

The directives do the following:
...

init is the command(s) to run before the application is started for the

first time.

What is the use case for it I don't get it.
Many PHP Applications are distributed as a container image with all
dependencies already included.
How would it work with multi-threaded environments, which thread would be
responsible for running init? How do you solve concurrency problems then?

If this theoretical version of Drupal moves its composer.json and
composer.lock files into .php-packages then the autoloader doesn't have
to be required in the index.php file. Also, the application can be started
without running composer install

index.php is just a common name for PHP HTTP Applications for modern
frameworks like Symfony and others that one lays in public/ folder as an
entry point, often this is the only file in this directory.
If I understand your idea correctly you'd like a couple of I/O operations
on the filesystem to find the .php-packages directory in CWD else if not
in the parent directory and so on rather than directly pointing where it
is, right?

Cheers,
Michał Marcin Brzuchalski

4 months ago by Michael Morris — view source

unread

On Mon, Jul 1, 2024 at 1:33 AM Michał Marcin Brzuchalski <
michal.brzuchalski@gmail.com> wrote:

Hi Michael,

pon., 1 lip 2024 o 01:18 Michael Morris tendoaki@gmail.com napisał(a):

...
Applications

The application is the root package. It is the package that imports to
the root namespace. When PHP is asked to parse a file it will look for a
.php-packages folder, first in the current working directory then in
parent directories. If it doesn't find one, business as usual. If we do
find one we follow its directives about setting up an application
environment.

The .php-packages folder is where PHP will put package related code for
the application at hand. Code written explicitly for these changes will
also put their package related files there - composer's vendor directory,
composer.json, composer.lock, and so on - rather than putting those files
in the site root. The folder is hidden to prevent web servers like nginx or
apache from serving the files directly in any way.

First, you use the term Application then Site - decide.
Not all PHP applications are HTTP Applications, consider consumers, cron
tasks other daemons, these don't need the existence of either Nginx or
Apache.
Not all PHP HTTP Applications expose files to Nginx or Apache - most of
these I know like Rest API give ZERO access to any application file to
Nginx or Apache.

The .php-packages directory will have a configuration file called
php.mod. This tells the parser:

You propose to move composer.json, composer.lock, and vendor into a
hidden folder with no good reason which simply adds more confusion.

Let's look at what such a file might look like for Drupal. For the moment
I'm going to use go.mod's syntax. The final syntax to be used, be it ini,
yaml, toml, json, is a discussion for another time. The part to focus in on
here is what type of information do we need.

package Drupal

php 10

registry //packagist.org/packages composer

init (
composer install
)

require (
./vendor/autoload.php
)

imports (
//getcomposer.org/composer.phar
)

This looks like a completely new file format which simply makes
interoperability harder,

When you read my messages, I'll read yours. The fact that you blasted over
my statement of "The final syntax to be used is a discussion for another
time" betrays your intent - to torpedo this discussion and add nothing
valuable to it.

[Initial Feedback] PHP User Modules - An Adaptation of ES6 from JavaScript

-- Aleksander Machniak Kolab Groupware Developer [https://kolab.org] Roundcube Webmail Developer [https://roundcube.net]

Regards,

Regards,

Regards,

Regards,

--
Aleksander Machniak
Kolab Groupware Developer [https://kolab.org]
Roundcube Webmail Developer [https://roundcube.net]