# Modules, again. - Thread: https://externals.io/message/127277 - Original archive: https://news-web.php.net/php.internals/127277 - Started by Michael Morris on 2025-05-04T07:34:05+00:00 - Messages: 21 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #127277 — Michael Morris — 2025-05-04T07:34:05+00:00 URL: https://externals.io/message/127277#127277 It's been 9 months. Been researching, working on other projects, mulling over points raised the last time I brought this up. And at the moment I don't think PHP 8.5 is in its final weeks so this isn't a distraction for that. The previous discussion got seriously, seriously derailed and I got lost even though I started it. I'm not going to ask anyone to dig into the archives, let's just start afresh. -------------------------------------------------------------------------------- THE PROBLEM -------------------------------------------------------------------------------- PHP has no way of dealing with userland code trying to write to the same entry on the symbol tables. Namespaces are a workaround of this and combined with autoloaders they've yielded the package environments we currently have (usually composer), but at the end of the day if two code blocks want different versions of the same package invoked at the same time there's going to be a crash. This is seen most visibly in WordPress plugins that use composer, as they resort to monkey-typing the composer packages they consume to avoid such collisions. -------------------------------------------------------------------------------- PROPOSAL -------------------------------------------------------------------------------- Modules - blocks of PHP code with independent symbol tables and autoload queues. Instead of using any new keywords along with the backwards compatibility problems that creates three existing keywords will be used in a new way: "use", "require", and "yield" The first file that PHP loads will always be on the "main thread". To bring in code as a module the use require structure is used. The simplest possible version of this is as follows: PHP Code ----------------------------------------------------------------------- use require 'mymodule.php'; -------------------------------------------------------------------------------- The contents of 'mymodule.php' have two requirements. First, a namespace is *required* of a module. Second, yield statements are used to mark the functions classes and constants the module exports, and at least one such yield must be present. Hence the file may look something like this: PHP Code ----------------------------------------------------------------------- namespace MyModule; yield function sum(a, b) { return a + b; } -------------------------------------------------------------------------------- Returning to our caller, it could make use of this function as follows. PHP Code ----------------------------------------------------------------------- use require 'mymodule.php'; echo MyModule::sum(3, 4); -------------------------------------------------------------------------------- So far there is nothing here that couldn't have been done with a static class. The important difference though is in behavior. 1. The module does not affect or see variables on the main thread - including the superglobals. A module can only get to them if it receives them as an argument in some sort of setter yield function. 2. The module does not affect or see constants or functions established on the main thread or in other modules. It can see and autoload classes from the main thread if the module author opts into this (discussed below). The use case above is not typical - usually inclusions from modules are more targeted. PHP Code ----------------------------------------------------------------------- use sum require 'mymodule.php'; echo sum(3, 4); // 7 -------------------------------------------------------------------------------- Here the class is not created in the main thread, only the yielded function is. Aliases and multiple objects can be declared in the use just as is the case now. PHP Code ----------------------------------------------------------------------- use sum, difference as subtract require 'mymodule.php'; -------------------------------------------------------------------------------- And if desired the namespace of the module can be aliased on the fly using the wildcard operator PHP Code ----------------------------------------------------------------------- use * as AliasModule require 'mymodule.php'; AliasModule::sum(3,4); -------------------------------------------------------------------------------- Require continues to invoke autoloaders, even when used in the context of use require. The callback defined in spl_autoload_register will receive a second argument from this context, boolean true if the main thread is loading a module, and if a module is loading a module its namespace string will be sent. Hence PHP Code ----------------------------------------------------------------------- require('./vendor/composer/autoload.php') use require 'mymodule'; // Callback will receive args ('mymodule', true) class A; // Callback will receive args ('A', false) - the false case should preserve BC. -------------------------------------------------------------------------------- And in a module PHP Code ----------------------------------------------------------------------- namespace MyModule; /* * Mod Author can elect to use the global autoloader * by passing the string "global" instead of a callback */ spl_autoload_register('global'); use require 'othermodule'; // Callback will receive args ('othermodule', 'MyModule') class A; // Callback will receive args ('A', 'mymodule') -------------------------------------------------------------------------------- Autoload callbacks are currently required to directly load the file. For modules this *will not work* because the loading of a module file involves the setup of a new symbol table, autoload queue, and slightly different parsing rules (again - namespace is required, not optional, and at least one top level yield statement must be present). So when an autoloader is asked about a module it must return the absolute path to the module, or false if it can't resolve it (handing off to the next loader in the queue). PHP will then load the module as if the filepath had been placed in the use require statement in the first place. -------------------------------------------------------------------------------- CLOSING REMARKS -------------------------------------------------------------------------------- I want to take a moment to ruminate on what doors are opened by the above, but all that follows is NOT part of my proposal. The above gets what I feel is needed - a way to cleanly run disparate packages in an application whose authors refuse to update it to embrace composer (cough - WordPress - cough). But even for projects that do embrace composer and its package management the above makes the prospect of a large API change in a major version far less frightening. In the current setup all the extensions have to be kept current and more or less on the same page. The larger the extension authoring set becomes the less feasible this is - projects get abandoned, stagnate and if you have a site that needs such then finding a replacement or upgrading it personally can be a pain. The string that gets passed to the autoloader could be anything btw - rules for which versions the module might accept like 'mymodule@7.x' can work if the package manager is written to parse out such, but the rules for such, not to mention what the composer.json file or equivalent would need to look like is outside the scope of this proposal. It should be noted that Modules offer a much more black box behavior than the current namespaces and autoloaders can provide. The only part of a module the outside world can see is what it yields. If a class isn't yielded the outside world can't make an instance of it. This shielding of internal API's should be useful because no matter how big a note you make in the comments about how a class shouldn't be used by outside code sooner or later someone will do it and their code will break when you change the internal API. While it is their fault for doing such, it can be a concern especially if the use of such "internal" API's becomes commonplace (Drupal has several such instances of this) Finally, my previous writings on this have mused about possibly having module files possess vastly different parsing rules. I bring this up as a possibility but I won't dwell on it as it could easily become a thread derailing distraction. That said, it should be possible to use the autoload callback to signal to PHP that a block of code should be loaded using the module parsing rules if having them be different in any way is desired. That or a new require_module statement could be used to pull in code under the module parsing rules. As to why this might be desirable - I'm no expert on the engine but I'm going to guess that giving each module an independent symbol table will incur overhead, possibly significant overhead. One way to claw back performance is to fix bugs that have been unfixable for backwards compatibility reasons. There is no existing module code, so in one fell swoop this code can step away from those problems. If this is done though it has to be done right as the window closes once projects with modules start appearing. --- ## #127278 — Arvids Godjuks — 2025-05-04T11:05:43+00:00 URL: https://externals.io/message/127277#127278 · In reply to: #127277 > It's been 9 months. Been researching, working on other projects, mulling > over > points raised the last time I brought this up. And at the moment I don't > think > PHP 8.5 is in its final weeks so this isn't a distraction for that. The > previous discussion got seriously, seriously derailed and I got lost even > though > I started it. I'm not going to ask anyone to dig into the archives, let's > just > start afresh. > > *snip* > I've started reading, saw Wordpress... almost stopped right there, but decided *okay, this is too rash of me, read further*.... "My disappointment is immeasurable and my day is ruined".gif It ended exactly how my first judging brain wave thought it would go - this whole thing is about fixing WordPress by changing the language. Plain and simple. No. -- Arvīds Godjuks +371 26 851 664 arvids.godjuks@gmail.com Telegram: @psihius https://t.me/psihius --- ## #127279 — Jochen Schultz — 2025-05-04T11:57:26+00:00 URL: https://externals.io/message/127277#127279 · In reply to: #127278 --- ## #127280 — Arvids Godjuks — 2025-05-04T12:12:13+00:00 URL: https://externals.io/message/127277#127280 · In reply to: #127279 > > > Am 04.05.25, 13:07 schrieb Arvids Godjuks : > > […] Yes, we should ignore it when working on language design and trying to improve it. For whatever reason, they made a deliberate commercial decision not to do anything about it, and this is not even touching on events of recent times :) Two - I already have to deal with JavaScript at times and all it's ecosystem. I do not want another JavaScript in my PHP, thank you. For me, any concept for modules, packages, etc, etc that ignores autoloading is dead on arrival. Ideally it should be transparent to any consuming code and be done on the defining side, erroring out when attempting to load/access something internal to a namespaca/package/module. I'm also not a fan of ability to mix multiple package versions in one code base. -- Arvīds Godjuks +371 26 851 664 arvids.godjuks@gmail.com Telegram: @psihius https://t.me/psihius --- ## #127281 — Deleu — 2025-05-04T12:15:05+00:00 URL: https://externals.io/message/127277#127281 · In reply to: #127278 Hi! On Sun, May 4, 2025 at 8:08 AM Arvids Godjuks wrote: > > > > > > […] > > I've started reading, saw Wordpress... almost stopped right there, but > decided *okay, this is too rash of me, read further*.... > > "My disappointment is immeasurable and my day is ruined".gif > It ended exactly how my first judging brain wave thought it would go - > this whole thing is about fixing WordPress by changing the language. Plain > and simple. > > No. > I think that is a very short sighted statement about the proposal. I have been using PHP for 15 years and I don't know, don't use nor don't care about the Wordpress ecosystem. However, the proposal has immeasurable merits to address many things that have been discussed several times on this list over the years. 1- Private classes. Instead of the currently being discussed nested class declaration with its own set of awkwardness, Modules would allow the declaration of classes that are not globally visible. Regardless of the state of Wordpress, **at first glance** this seems a superior proposal than Nested Class. 2- Friends classes. It has been discussed a couple of times the introduction of class friendship as a way to expose internals of a class only to a specific subset of known / expected classes. This has some foundation in the fact that you don't want to declare public attributes in a global class. With modules, you could declare a class that is not exported to the global scope and use public attributes knowing full well that only classes within your own module will be able to touch them. Again, major props to this proposal over Friends classes. 3- Class autoload vs Function autoload. There have been hundreds of hours spent on discussing function autoloading. I don't quite remember why that is such a hot topic and I also don't remember what it tries to accomplish, but it seems adjacent enough to consider that maybe this proposal could be a door on that realm as well. 4- Familiarity with the Typescript ecosystem. For better or worse, browsers run on Javascript for 3 decades and not even Wasm is fully changing that. This proposal is too familiar from Typescript's import / export system and would help bridge the gap for frontend engineers to better understand the flow of PHP code. ----------- I haven't read (or don't remember) the old discussion. From a superficial look of the proposal, it does seem rather awkward to not go full in with `import / export` keywords, even if it requires a soft reserved keyword. I'm not sure the syntax would require a full hard keyword reservation? Perhaps yield is good enough instead of export, but `use require` does seem rather odd. import ... require (as opposed to import ... from) would also work better-ish. In any case, this threads in bikeshedding space. I guess I'd rather have use .... require anyway than not have it at all, specially since IDEs often end up writing it for us anyway. -- Marco Deleu --- ## #127283 — Michael Morris — 2025-05-04T14:30:28+00:00 URL: https://externals.io/message/127277#127283 · In reply to: #127278 On Sun, May 4, 2025 at 7:06 AM Arvids Godjuks wrote: > > It ended exactly how my first judging brain wave thought it would go - > this whole thing is about fixing WordPress by changing the language. Plain > and simple. > > That is a lie. Go back and read again - I discuss how this will improve Drupal, Symfony and other projects that embrace composer. --- ## #127284 — Larry Garfield — 2025-05-04T21:18:56+00:00 URL: https://externals.io/message/127277#127284 · In reply to: #127277 > It's been 9 months. Been researching, working on other projects, > mulling over > points raised the last time I brought this up. And at the moment I > don't think > PHP 8.5 is in its final weeks so this isn't a distraction for that. > The > previous discussion got seriously, seriously derailed and I got lost > even though > I started it. I'm not going to ask anyone to dig into the archives, > let's just > start afresh. This proposal requires: 1. Every module author to change their coding structure 2. Every consumer of a package to change their coding structure 3. Devs to abandon "it just works" autoloading and explicitly import packages. 4. Abandoning 16 years of PSR-0/4 file convention in favor of "module = file", which will almost certainly result in multi-thousand-line files even for clean, well-factored code. As I stated in the previous thread, this is a total nonstarter. Python/Javascript style module definition is simply not compatible with PHP. It may have been had we gone that way in 2005, but we didn't, and that ship has long since sailed. Any module proposal worth considering can require only the first point above. Requiring work from a package author to module-ify their code is fine, though it shouldn't be onerous. But it needs to "just work" with the rest of the existing ecosystem. This proposal fails at that. Arnaud and I spent some time last year exploring a different approach to modules that requires only trivial changes by package authors, and none by consumers: https://github.com/Crell/php-rfcs/blob/master/modules/spec-brainstorm.md https://github.com/arnaud-lb/php-src/pull/10 In our case, the main target was performance by giving the opcache optimizer a larger chunk of code to work with, plus the potential for namespace-private symbols. Arnaud was able to make it mostly work, and there was significant potential performance to be had, but it also ran into some very complex issues very quickly. (Circular dependencies, modules split across multiple directories, testing, etc.) We largely haven't gotten back to it as the implementation was very complex and it wasn't clear if the benefit would be worth it, though I'd still like to revisit it if possible. --Larry Garfield --- ## #127285 — Michael Morris — 2025-05-04T21:25:56+00:00 URL: https://externals.io/message/127277#127285 · In reply to: #127284 On Sun, May 4, 2025 at 5:19 PM Larry Garfield wrote: > > > […] > > This proposal requires: > > 1. Every module author to change their coding structure > 2. Every consumer of a package to change their coding structure > 3. Devs to abandon "it just works" autoloading and explicitly import > packages. > 4. Abandoning 16 years of PSR-0/4 file convention in favor of "module = > file", which will almost certainly result in multi-thousand-line files even > for clean, well-factored code. > You're 4 for 4 on falsehoods. Care to actually read what I wrote because I know you're more intelligent and better than this and very disappointed. --- ## #127286 — Larry Garfield — 2025-05-04T21:37:23+00:00 URL: https://externals.io/message/127277#127286 · In reply to: #127285 > > > […] > > You're 4 for 4 on falsehoods. Care to actually read what I wrote > because I know you're more intelligent and better than this and very > disappointed. > PHP Code ----------------------------------------------------------------------- > > namespace MyModule; > > yield function sum(a, b) { return a + b; } > > -------------------------------------------------------------------------------- Every module author changes their code. (Which is reasonable.) > PHP Code ----------------------------------------------------------------------- > > use require 'mymodule.php'; > > echo MyModule::sum(3, 4); > -------------------------------------------------------------------------------- Every caller changes their code, to use `use require` and to use the pseudo-class syntax to call the function. > PHP Code ----------------------------------------------------------------------- > > use sum require 'mymodule.php'; > > echo sum(3, 4); // 7 > -------------------------------------------------------------------------------- Removes the pseudo-class syntax, but still has the "use sum require..." change for the consumer. As described, the implication is that the millions of existing use Some\Class\Here; lines in code in the wild will... not work if the Some\Class package becomes a module? I think that's what's implied, but it's not clear. If that triggers autoloading, how does the autoloader know how to find it? It needs to know the module name, which as described is the file name of the module, not the class. Since the module name is the file name, that means if a module has 50 internal classes, it must be in the same file. Hence, giant files. If those reads of your post are inaccurate, then please show with examples how they are inaccurate, because that is how I understood your proposal text at face value. --Larry Garfield --- ## #127287 — Michael Morris — 2025-05-04T22:40:50+00:00 URL: https://externals.io/message/127277#127287 · In reply to: #127286 On Sun, May 4, 2025 at 5:38 PM Larry Garfield wrote: > > > […] > ----------------------------------------------------------------------- > > […] > -------------------------------------------------------------------------------- > > Every module author changes their code. (Which is reasonable.) > Actually no. PHP Modules are only needed if the package author wants the functionality modules provide - specifically having an independent symbol table and autoload queue. Existing packages don't have to change in any way. If a module includes one their symbols will be written into that module's symbol table. If two modules include the same package they each write it onto their own respective symbol tables. > > > […] > ----------------------------------------------------------------------- > > […] > -------------------------------------------------------------------------------- > > Every caller changes their code, to use `use require` and to use the > pseudo-class syntax to call the function. > Again, if they want module functionality. No one *has* to do this - the new syntax was carefully chosen to let the existing syntax work as it always has. > > > […] > ----------------------------------------------------------------------- > > […] > -------------------------------------------------------------------------------- > > Removes the pseudo-class syntax, but still has the "use sum require..." > change for the consumer. > > As described, the implication is that the millions of existing > > use Some\Class\Here; > > lines in code in the wild will... not work if the Some\Class package > becomes a module? This is left up to the autoloader. When you call that use statement from the main thread the autoloader receives args ("Some\Class\Here", false). When you call that use statement from a module the autoloader receives arguments ("Some\Class\Here", "RequestingModule"). If the autoloader isn't rewritten it will ignore the 2nd argument and serve out the same code for both module and main thread. HOWEVER, modules can define their own autoloaders and those will be executed independent of the autoloader on the main thread. Furthermore, the autoloader on the main thread will not be used unless the module opts into using it. > I think that's what's implied, but it's not clear. If that triggers > autoloading, how does the autoloader know how to find it? It needs to know > the module name, which as described is the file name of the module, not the > class. By convention the filename might be the module name, but that a convention of package management, autoloading, some future PSR-X that is far, far outside of scope here. The module's name is its namespace! For a module a namespace isn't just a run time string replacement, it has a very real effect. The namespace will be the 2nd argument of every autoload call made from the module or any file required/included in its scope. Existing packages don't need to be aware this is happening - but the module can have a custom autoloader that always loads version 2 of a popular package even if composer on the main thread is set to include version 3. > > Since the module name is the file name, that means if a module has 50 > internal classes, it must be in the same file. Hence, giant files. > I hope you can see why this isn't true now. The module entry file can simply yield out its public assets like so. namespace Vendor\Package; require 'autoloader.php'; yield { A, B, C } The above assuming that the autoloader knows where \Vendor\Package\A, \Vendor\Package\B and \Vendor\Package\C are located at. If an autoloader isn't used they'll have to be explicitly required before being yielded out. > > If those reads of your post are inaccurate, then please show with examples > how they are inaccurate, because that is how I understood your proposal > text at face value. > > Ok, let's go over them in turn > > […] No. Modules aren't replacing the current mechanism - just augmenting it when needed. The new syntax is only necessary when an independent symbol table is desired. Further, there's nothing stopping a package from having a standard require entry file that doesn't set up the symbol table and related protections, and one that does. > > […] That depends on the package maintainer. They can certainly rewrite the package such that consuming code must use module syntax to access it, but this isn't required of them. And as mentioned above they can get clever and provide multiple entry points to their code if they desire just as JavaScript package devs have been doing to make sure their packages work whether the consumer uses ESM import or CommonJS import. > > […] packages. I don't know where you got that from - maybe the old thread from 9 months ago? The proposal I made last night not only takes autoloading into account but also discusses how autoload functions will receive a second argument moving forward that lets them make decisions about what code to supply to modules based on whatever logic they have which is well beyond the scope of this proposal. I will admit that while use doesn't really do anything and is handled at compile time, use - require evals at run time. The specified symbols come out of the module. The module can be located with an autoloader if desired, but it's not the same mechanism because module creation involves the creation of a new symbol table that will be used by the module file and EVERY file it requires and every file THEY require. This is why the autoloader is to return a string when handling module requests for classes because PHP should have the responsibility of attaching the code to the correct symbol table. The autoloader is told what module is about to receive the code. If the autoloader requires it's going to affect the symbol table it was declared in, which is ok if the module's own autoloader is handling the request - but if the global autoloader on the main thread does a require it's going to affect the symbol table of the main thread. > > […] file", which will almost certainly result in multi-thousand-line files even for clean, well-factored code. Again, I don't see this, at all. And I'm a little insulted you'd think that I'm dumb enough to propose something that outrageous. Modules simply have their own symbol table scope. They can import any package as they exist today without any modification into that scope. They can therefore pull a different version of the same package, which is desirable if they have a compatibility issue with the latest version of the package. Those autoloader are important, but out of scope. Another way to look at this is in JavaScript the ESM module convention was introduced without any package management ability and worked with raw absolute file paths. It would be a decade before import-maps were introduced and true integration with npm started and it's still not finished. I'm starting from the same place, use (some symbols) require and if PHP can't find the path it queries the autoloader and gives it a chance to return a path for PHP to use here. All existing code stays as is, and can even evolve as is when they don't need to worry with version mismatches. This syntax only really becomes useful when you want to "black box" the package from the rest of the application and allow the module to do what it needs without fear of affecting the rest of the application. Over time it might eclipse the old approach, but --- ## #127288 — Larry Garfield — 2025-05-05T04:43:29+00:00 URL: https://externals.io/message/127277#127288 · In reply to: #127287 > > > […] > > Actually no. PHP Modules are only needed if the package author wants > the functionality modules provide - specifically having an independent > symbol table and autoload queue. Existing packages don't have to change > in any way. If a module includes one their symbols will be written > into that module's symbol table. If two modules include the same > package they each write it onto their own respective symbol tables. I thought it self-evident that I was referring to "every package author that wants to expose their package as a module", not literally every owner of a package on Packagist. (Though presumably over time those two circles will increasingly overlap.) But let me make that explicit. And, as noted, requiring the author of a package to introduce new syntax in order to turn the package into a module is a reasonable expectation. I have no issue with that. > > […] > > Again, if they want module functionality. No one *has* to do this - the > new syntax was carefully chosen to let the existing syntax work as it > always has. If my library is structured as a module, with yield and stuff, then how exactly would someone use it without using the `use require` syntax at least? > > […] > > This is left up to the autoloader. When you call that use statement > from the main thread the autoloader receives args ("Some\Class\Here", > false). When you call that use statement from a module the autoloader > receives arguments ("Some\Class\Here", "RequestingModule"). If the > autoloader isn't rewritten it will ignore the 2nd argument and serve > out the same code for both module and main thread. I believe you are grossly misusing the term "thread" here. There is no separate execution thread involved, just scoped symbol tables. See note further down about the rest of the points raised here. > HOWEVER, modules can define their own autoloaders and those will be > executed independent of the autoloader on the main thread. Furthermore, > the autoloader on the main thread will not be used unless the module > opts into using it. Nothing in your initial post discussed any of this, or even implied it. > > […] > > By convention the filename might be the module name, but that a > convention of package management, autoloading, some future PSR-X that > is far, far outside of scope here. The module's name is its namespace! > For a module a namespace isn't just a run time string replacement, it > has a very real effect. The namespace will be the 2nd argument of every > autoload call made from the module or any file required/included in its > scope. Existing packages don't need to be aware this is happening - > but the module can have a custom autoloader that always loads version 2 > of a popular package even if composer on the main thread is set to > include version 3. From the proposal: > use require 'mymodule.php'; > > echo MyModule::sum(3, 4); How exactly does MyModule get defined? That is not at all clear. > > […] > > I hope you can see why this isn't true now. The module entry file can > simply yield out its public assets like so. > > namespace Vendor\Package; > require 'autoloader.php'; > yield { A, B, C } ... Nothing in your initial proposal suggested that in the slightest. It did not even include yield with brackets to list multiple options. > The above assuming that the autoloader knows where \Vendor\Package\A, > \Vendor\Package\B and \Vendor\Package\C are located at. If an > autoloader isn't used they'll have to be explicitly required before > being yielded out. > > > […] > > Ok, let's go over them in turn > > > […] > > No. Modules aren't replacing the current mechanism - just augmenting it > when needed. The new syntax is only necessary when an independent > symbol table is desired. Further, there's nothing stopping a package > from having a standard require entry file that doesn't set up the > symbol table and related protections, and one that does. See above. > > […] > > That depends on the package maintainer. They can certainly rewrite the > package such that consuming code must use module syntax to access it, > but this isn't required of them. And as mentioned above they can get > clever and provide multiple entry points to their code if they desire > just as JavaScript package devs have been doing to make sure their > packages work whether the consumer uses ESM import or CommonJS import. > > > […] > > I don't know where you got that from - maybe the old thread from 9 > months ago? The proposal I made last night not only takes autoloading > into account but also discusses how autoload functions will receive a > second argument moving forward that lets them make decisions about what > code to supply to modules based on whatever logic they have which is > well beyond the scope of this proposal. It came from the examples in the original proposal at the top of this thread: > use sum require 'mymodule.php'; > > echo sum(3, 4); // 7 So if I want to call sum(), I have to explicitly `use sum require` it. I cannot just let the autoloader sort it out. (Assuming we were talking about a class or function autoloading ever happens, of course.) > > […] > > Again, I don't see this, at all. And I'm a little insulted you'd think > that I'm dumb enough to propose something that outrageous. Modules > simply have their own symbol table scope. They can import any package > as they exist today without any modification into that scope. They can > therefore pull a different version of the same package, which is > desirable if they have a compatibility issue with the latest version of > the package. Those autoloader are important, but out of scope. The implication from your original post is that if I want to import 3 symbols from a module, I require a single file. That file has one or more symbols that are `yield` ed. Implication: If I want 3 symbols in a module together, they have to be in the same file. Your response here about having a fronting file that yields symbols it doesn't define, on the assumption that an autoloader will figure it out in the backend, was never stated nor implied anywhere in your proposal. Rather, it states that modules look like this: > yield function sum(a, b) { return a + b; } IE, like Javascript or Python libraries. That is the plain reading of your original post. And naturally means that whoever is authoring that file needs to make changes to it, to at minimum add the "yield" keyword. So before you get indignant and call me a liar ("You're 4 for 4 on falsehoods"), perhaps consider how someone you know is "more intelligent" than this can come away from your post understanding it so utterly differently than what you intended. From your follow up statements, it seems that what you are proposing is not "modules" in any sense used in any language I am familiar with. Rather, you're proposing a symbol table hack to get around PHP's inability to load two symbols with the same name, and using module terminology to inaccurately describe it. (And thread terminology as well.) That is, it seems your intent is that I can import arbitrary code from an arbitrary package and shunt it off to a separate symbol table. But it's not clear if you intend for the package author to have to do anything to enable that. Your answers seem contradictory. For a concrete example, I'll offer my AttributeUtils library: https://github.com/Crell/AttributeUtils/tree/master/src It has a whole bunch of interfaces, plus a few classes. We'll focus on a subset of them for the moment. Analyzer.php (this is the main class someone uses) AttributeParser.php (this is an internal class used by Analyzer, you should never use it directly) FromReflectionClass.php (an interface that other libraries implement to trigger certain functionality in Analyzer) Say I want to modularize this library, under your proposal. What would I, as the library author, do exactly? Anything? How can I mark AttributeParser as internal-only and thou-shalt-not-use-if-you're-not-me? (That's the most common value of module systems, and the one most people in PHP seem to think of, from what I've seen.) That library is used extensively by Serde (another of my libraries). If I modularize AttributeUtils, then what changes does Serde need to make? What would it get out of doing so? Code examples, please. --Larry Garfield --- ## #127289 — Stephen Reay — 2025-05-05T07:02:16+00:00 URL: https://externals.io/message/127277#127289 · In reply to: #127284 > > > > > […] > > This proposal requires: > > 1. Every module author to change their coding structure > 2. Every consumer of a package to change their coding structure > 3. Devs to abandon "it just works" autoloading and explicitly import packages. > 4. Abandoning 16 years of PSR-0/4 file convention in favor of "module = file", which will almost certainly result in multi-thousand-line files even for clean, well-factored code. > > As I stated in the previous thread, this is a total nonstarter. Python/Javascript style module definition is simply not compatible with PHP. It may have been had we gone that way in 2005, but we didn't, and that ship has long since sailed. > > Any module proposal worth considering can require only the first point above. Requiring work from a package author to module-ify their code is fine, though it shouldn't be onerous. But it needs to "just work" with the rest of the existing ecosystem. This proposal fails at that. > > Arnaud and I spent some time last year exploring a different approach to modules that requires only trivial changes by package authors, and none by consumers: > > https://github.com/Crell/php-rfcs/blob/master/modules/spec-brainstorm.md > https://github.com/arnaud-lb/php-src/pull/10 > > In our case, the main target was performance by giving the opcache optimizer a larger chunk of code to work with, plus the potential for namespace-private symbols. Arnaud was able to make it mostly work, and there was significant potential performance to be had, but it also ran into some very complex issues very quickly. (Circular dependencies, modules split across multiple directories, testing, etc.) We largely haven't gotten back to it as the implementation was very complex and it wasn't clear if the benefit would be worth it, though I'd still like to revisit it if possible. > > --Larry Garfield > Just to add my $0.02 as a userland/library developer: I agree with Larry's points - it needs to be transparent for the *consumers*, and it absolutely needs to align with namespaces, without any expectation about how symbols relates to files. I personally think initially the goal should be a more minimal approach that simply introduces the concept of a module/package/whatever-term-you-like as something that exists, with an implicit relationship to a namespace. Future RFC's could build on that - even within the same release cycle - but initially agreeing on a basic way to define "this namespace is also a " provides a building block for both language/runtime internal enhancements and external tooling. Cheers Stephen --- ## #127290 — Arvids Godjuks — 2025-05-05T09:25:57+00:00 URL: https://externals.io/message/127277#127290 · In reply to: #127283 > > > On Sun, May 4, 2025 at 7:06 AM Arvids Godjuks > wrote: > > > […] > That is a lie. Go back and read again - I discuss how this will improve > Drupal, Symfony and other projects that embrace composer. > As Larry has already extensively covered, it seems that about 80% of the whole thing lives somewhere outside the emails you sent. That's just a bad proposal, either rushed or hasn't been worked on past 1st version of a draft. It is how it reads to me. Accounting for that (and I read through your correspondence) - there is nothing in the clarifications that didn't make me reconsider; if anything, it turned me off even more. I will repeat myself: I do not want a typescript/javascript/python/rails/whatever export/import type going into PHP. This is just going back to pre-autoloading times, where you needed to require/include files. Yes, the autoloading part is better, but then we have a whole mess of autoloader importing vs require/include at which point this whole thing is a ball of spaghetti that makes no sense and I am confused to a point I cannot make heads or tails of this whole thing. -- Arvīds Godjuks +371 26 851 664 arvids.godjuks@gmail.com Telegram: @psihius https://t.me/psihius --- ## #127291 — Edmond Dantes — 2025-05-05T18:08:06+00:00 URL: https://externals.io/message/127277#127291 · In reply to: #127277 Hello, Michael. Thank you very much for your work. It's very interesting. Let me look at this idea from an architectural perspective. Did I understand correctly from the text that: * Modules control which entities are public and which are not? * During import, we specify which entities we want to bind to which namespace? * Could this solution help with separating Internal and Public classes? In other words, the information about what is public and what is private is part of the module. And the information about how the module is integrated into the project lies outside the module. Could this task be reduced to using `use as`? For example: ```php use \MyClass as MyClass2; ``` This situation is very similar to creating aliases for namespaces, but the difference is that the alias is created not for the code being used, but for the code being loaded. Technically, this is quite possible to implement: 1. We specify that files at a certain path will be loaded into a different namespace. 2. During loading, all namespaces are inserted into the symbol table with the desired alias, as if a person had manually renamed all the modules. But there is one problem here that cannot be solved at all. `PHP` lacks an abstraction over symbols. It doesn’t have a dedicated `Class` or `Namespace` type — strings are used in code for that purpose. And that means any code that tries to load a class using a string constant will break. To solve this, `PHP` would need to introduce a new data type. That’s not a problem, but it does require an `RFC`. In addition, a new `PSR` for packages and autoloading would be needed. You have a great idea, but I think it requires some groundwork in these areas first. P.S. I consider WordPress the best example of what rapid programming can look like — and where it can lead. And I don’t hate it. :) Best Regards, Ed --- ## #127294 — Michael Morris — 2025-05-06T06:33:41+00:00 URL: https://externals.io/message/127277#127294 · In reply to: #127288 Resetting and moving the proposal writeup to a github hosted markdown file here: https://github.com/michael-lloyd-morris/php-modules-rfc/blob/main/php-modules.md On Mon, May 5, 2025 at 12:44 AM Larry Garfield wrote: > > > So before you get indignant and call me a liar ("You're 4 for 4 on > falsehoods"), perhaps consider how someone you know is "more intelligent" > than this can come away from your post understanding it so utterly > differently than what you intended. > I'm sorry for doing that. > > From your follow up statements, it seems that what you are proposing is > not "modules" in any sense used in any language I am familiar with. > Rather, you're proposing a symbol table hack to get around PHP's inability > to load two symbols with the same name, and using module terminology to > inaccurately describe it. (And thread terminology as well.) That is, it > seems your intent is that I can import arbitrary code from an arbitrary > package and shunt it off to a separate symbol table. But it's not clear if > you intend for the package author to have to do anything to enable that. > Your answers seem contradictory. > > For a concrete example, I'll offer my AttributeUtils library: > https://github.com/Crell/AttributeUtils/tree/master/src > > It has a whole bunch of interfaces, plus a few classes. We'll focus on a > subset of them for the moment. > > Analyzer.php (this is the main class someone uses) > AttributeParser.php (this is an internal class used by Analyzer, you > should never use it directly) > FromReflectionClass.php (an interface that other libraries implement to > trigger certain functionality in Analyzer) > > In answer to your question - assuming the module file is at the root of your src directory. > Say I want to modularize this library, under your proposal. What would I, > as the library author, do exactly? Anything? How can I mark > AttributeParser as internal-only and thou-shalt-not-use-if-you're-not-me? > (That's the most common value of module systems, and the one most people in > PHP seem to think of, from what I've seen.) > > That library is used extensively by Serde (another of my libraries). If I > modularize AttributeUtils, then what changes does Serde need to make? On that end ; $analyzer = new \Crell\AnalyzerModule::Analyzer(); ?> If the autoloader gets updated to identify and properly load modules (outside the scope of this RFC) the require_module statement can be dropped. If a use statement is desired it looks like this > What would it get out of doing so? > Nothing much - it just wouldn't be able to see as deeply into the AttributeUtils code. --- ## #127301 — Larry Garfield — 2025-05-07T14:57:22+00:00 URL: https://externals.io/message/127277#127301 · In reply to: #127294 > Resetting and moving the proposal writeup to a github hosted markdown > file here: > https://github.com/michael-lloyd-morris/php-modules-rfc/blob/main/php-modules.md > > […] > > > In answer to your question - assuming the module file is at the root of > your src directory. > > module \Crell\AnalyzerModule; > > export [ > \Crell\Analyzer, > \Crell\FromReflectionClass > ]; > ?> > > > […] > > On that end > > require_module ; > > $analyzer = new \Crell\AnalyzerModule::Analyzer(); > ?> > > If the autoloader gets updated to identify and properly load modules > (outside the scope of this RFC) the require_module statement can be > dropped. If a use statement is desired it looks like this > > use \Crell\AnalyzerModule::Analyzer; // This is legal for modules, not > for static classes. > > $analyzer = new Analyzer(); > ?> > > > > […] > > Nothing much - it just wouldn't be able to see as deeply into the > AttributeUtils code. So it's not really giving private symbols. It's not even blocking access to anything, since it can still just be included or autoloaded. What you're proposing is really just an optional loading facade (the real kind of facade) that lets you pull random symbols into a separate symbol table, accessed via separate syntax. (Either a class-like AnalyzerModule::Analyzer, or use-require.) Putting aside for the moment the question of whether that is a good idea or not, it is definitely not "modules" in any typical sense. I really don't think calling it "modules" is doing your proposal any favors, as it's just leading to confusion. What you're describing is a name-mangling facade, or symbol table facade, or something like that. (I don't know that there is a standard name.) I strongly recommend you recast your terminology around that, rather than "modules." That would remove a lot of confusion and allow people to review and respond to the idea you're actually proposing, now what they think a "module" should be. --Larry Garfield --- ## #127306 — Michael Morris — 2025-05-07T18:18:44+00:00 URL: https://externals.io/message/127277#127306 · In reply to: #127301 On Wed, May 7, 2025 at 10:59 AM Larry Garfield wrote: > > So it's not really giving private symbols. It's not even blocking access > to anything, since it can still just be included or autoloaded. What > you're proposing is really just an optional loading facade (the real kind > of facade) that lets you pull random symbols into a separate symbol table, > accessed via separate syntax. (Either a class-like > AnalyzerModule::Analyzer, or use-require.) > > Putting aside for the moment the question of whether that is a good idea > or not, it is definitely not "modules" in any typical sense. I really > don't think calling it "modules" is doing your proposal any favors, as it's > just leading to confusion. What you're describing is a name-mangling > facade, or symbol table facade, or something like that. (I don't know that > there is a standard name.) > > I strongly recommend you recast your terminology around that, rather than > "modules." That would remove a lot of confusion and allow people to review > and respond to the idea you're actually proposing, now what they think a > "module" should be. > > --Larry Garfield > No proposal that replaces or even significantly alters how require et al works will be taken seriously, much less gain traction. As long as a package has PHP files that require can parse then clever users can direct require the files of a package distributed this way. The only way to stop it is to truly make files meant to be used with this unreadable to require - and the simplest way to do that is to assume code first and drop support for in these files. I'm in favor of that since I believe the engine can parse the files faster because it should simplify the token parsing process quite a bit. But I don't think a majority, much less the required supermajority would be in favor of such a change, at least not without other compelling reasons. Rather than fight the co-existence with require I think it would be better to harness and work with it. This would also make the changeover easier. As for the point of whether this counts as a module - Ask 10 programmers what a module is and expect 12 different answers. I'm patterning this on the module pattern used in JavaScript. I'm sorry if that doesn't meet your definition of module. I am open to any other name - component? library? Calling it a module until a better name can be found will have to do. But I'm not going to insult my own proposal by calling it "name-mangling facade" Part of the problem is PHP deals with code on a per file basis, not on a per directory one. Speaking of facades, the PSR-4 loading recommendation of organizing classes puts a module/package facade onto PHP, but it in no way ever has worked that way. Modules and Packages traditionally do work on the directory level, but how do you affix such to a language that has never had it and prevent older mechanisms of the language, such as require, from doing an end run. This problem hasn't been solved for years for a reason - it's not simple. It is an easy one to give up on, as most have. I'm just too stupid to give up I suppose. --- ## #127308 — Rowan Tommins [IMSoP] — 2025-05-07T19:24:24+00:00 URL: https://externals.io/message/127277#127308 · In reply to: #127277 > PHP has no way of dealing with userland code trying to write to the > same entry > on the symbol tables.  Namespaces are a workaround of this and > combined with > autoloaders they've yielded the package environments we currently have > (usually > composer), but at the end of the day if two code blocks want different > versions > of the same package invoked at the same time there's going to be a > crash. This > is seen most visibly in WordPress plugins that use composer, as they > resort to > monkey-typing the composer packages they consume to avoid such collisions. I think there are two different problems being muddled here: 1) How can two *completely unrelated* libraries avoid *accidentally* declaring something with the same name? 2) How can two versions of *the same* library be used in different parts of the same running program, despite overlapping names? Namespaces are not a workaround to problem number 2, they are a solution to problem number 1: every library chooses a prefix, and avoids declaring symbols with prefixes used by other libraries. Problem number 2 is what you seem to be trying to address. In languages like EcmaScript/JavaScript/TypeScript, the solution (to both problems) is to say that declarations don't have a canonical name. They're values that you can pass around, and their name is whatever a local scope says it is. This has major implications on both the language and its ecosystem: - contracts between libraries need to be structural, because there is no name that all libraries agree on for an interface/protocol/whatever - package managers don't resolve the graph of dependencies to a consistent set, they install multiple copies of the same library, for use in different scopes - package authors are free to depend on mutually incompatible concrete implementations, rather than agreed or de facto standards and interfaces NPM is notorious for creating a giant directory of tiny modules, because that's what the language design encourages. That's not how PHP works, and it never will be. In PHP (and other languages like Java, and C#), declarations have a single fully-qualified name. If you want two different versions of the same library, you will need to find a way to make their declarations use different fully-qualified names. Most applications do not have this problem. In fact, I would say that most applications would actively suffer from having multiple copies of the same library installed. The main exception, as you have pointed out, is plugin architectures like WordPress, where the plugin might want to "privately" use some library without impacting the host application and other plugins. For those, userland solutions already exist - a random search turned up this one, which lists some alternatives: https://github.com/BrianHenryIE/strauss?tab=readme-ov-file#alternatives I *do* think PHP would benefit from some native concept of "module" for other reasons, e.g. marking internal components, optimising across multiple files. I *do not* think that ES/JS/TS is a suitable model to look at, because it is starting from a fundamentally different concept of what a declaration is. I also *do not* think that allowing multiple versions of the same library should be a core requirement of any native module support; enabling userland to achieve it efficiently would be a nice to have. -- Rowan Tommins [IMSoP] --- ## #127313 — Michael Morris — 2025-05-07T20:51:29+00:00 URL: https://externals.io/message/127277#127313 · In reply to: #127308 On Wed, May 7, 2025 at 3:24 PM Rowan Tommins [IMSoP] wrote: > > Problem number 2 is what you seem to be trying to address. > The main thing, yes. This solution incidentally solves class privacy issues several unrelated and failed RFC's have addressed, including Nested Classes which is currently not doing so well in its vote. > > NPM is notorious for creating a giant directory of tiny modules, because > that's what the language design encourages. > > Indeed. Definitely the bathwater to be thrown out. > > That's not how PHP works, and it never will be. In PHP (and other > languages like Java, and C#), declarations have a single fully-qualified > name. If you want two different versions of the same library, you will > need to find a way to make their declarations use different > fully-qualified names. > Other libraries have the means to import into a namespace because their namespaces aren't just a quick and dirty string replacement. I've even proposed such a long, long time ago - allowing namespace to be the second argument of require, only to have that shot down as "require isn't a function." or some such. > The main exception, as you have pointed out, is plugin architectures > like WordPress, where the plugin might want to "privately" use some > library without impacting the host application and other plugins. It's not just WordPress. Drupal and other CMS's have extensions, but their reliance on composer means abandoned plugins that need an old library can block updating the whole system. The larger and more popular the application is the more likely this becomes. And like it or not WordPress haters on the board, for as much as WordPress sucks (and I will agree, it sucks hard) it still have a 50% - 80% market share of PHP websites depending on who you ask. A lot of this stems from the fact that end users just want their site to work - they don't give a rat's ass about dependencies and the like - and needing to drop a plugin to upgrade a site will put a bad taste in their mouth at best. > For > those, userland solutions already exist - a random search turned up this > one, which lists some alternatives: > https://github.com/BrianHenryIE/strauss?tab=readme-ov-file#alternatives > > I've seen Strauss in action. It's called monkey-typing. Other languages support this sort of monkey typing to some degree - PHP can only do it by doing an expensive and error prone programmatic keyword search of the source files and transforming them. The end result has to be shipped with the module, severing its dependencies from getting security updates. I *do* think PHP would benefit from some native concept of "module" for > other reasons, e.g. marking internal components, optimising across > multiple files. I *do not* think that ES/JS/TS is a suitable model to > look at, because it is starting from a fundamentally different concept > of what a declaration is. > So what do you suggest then? Go's model? Rusts? Gradle or Maven from Java? Shooting down stuff is trivial, finding solutions is the real work. --- ## #127315 — Rowan Tommins [IMSoP] — 2025-05-07T22:23:55+00:00 URL: https://externals.io/message/127277#127315 · In reply to: #127313 >On Wed, May 7, 2025 at 3:24 PM Rowan Tommins [IMSoP] >wrote: > >Other libraries have the means to import into a namespace because their >namespaces aren't just a quick and dirty string replacement. I've even >proposed such a long, long time ago - allowing namespace to be the second >argument of require, only to have that shot down as "require isn't a >function." or some such. I've no idea what you're saying here, sorry. I never mentioned autoloading in my message, because it's largely irrelevant. What's relevant is how you identify something that's been declared (e.g. a class). > > […] > > >It's not just WordPress. That is why I said "plugin architectures like WordPress". I am absolutely aware that it is an example, not something unique. >I've seen Strauss in action. It's called monkey-typing. Other languages >support this sort of monkey typing to some degree - PHP can only do it by >doing an expensive and error prone programmatic keyword search of the >source files and transforming them. The end result has to be shipped with >the module, severing its dependencies from getting security updates. So, you don't like the first example I happened to pick in a 10-second search... More interestingly: Firstly, can PHP do something to make the rewriting easier? I've no idea, but if so, let's talk about it. Secondly, can the rewriting be done at install time rather than in advance? I see no reason why not; seems like something to tinker with in a Composer plugin, off-topic on this list. >I *do* think PHP would benefit from some native concept of "module" for > > […] > >So what do you suggest then? Go's model? Rusts? Gradle or Maven from >Java? Shooting down stuff is trivial, finding solutions is the real work. I suggest starting with what we already have: build modules on top of namespaces, not as something new and orthogonal; and leave package *management* (version resolution, installation, etc) to userlan, while recognising that Composer is overwhelmingly the most popular implementation. That probably means that rewriting namespace prefixes remains the only way to isolate "private" versions of a dependency, and I am totally fine with that. Rowan Tommins [IMSoP] --- ## #127321 — Larry Garfield — 2025-05-08T14:29:03+00:00 URL: https://externals.io/message/127277#127321 · In reply to: #127313 > > […] > > It's not just WordPress. Drupal and other CMS's have extensions, but > their reliance on composer means abandoned plugins that need an old > library can block updating the whole system. The larger and more > popular the application is the more likely this becomes. And like it > or not WordPress haters on the board, for as much as WordPress sucks > (and I will agree, it sucks hard) it still have a 50% - 80% market > share of PHP websites depending on who you ask. A lot of this stems > from the fact that end users just want their site to work - they don't > give a rat's ass about dependencies and the like - and needing to drop > a plugin to upgrade a site will put a bad taste in their mouth at best. If we're going to cite WordPress as a design factor, then we should also recognize that WordPress has virtually no involvement in PHP Internals, and its version requirement history means that even if PHP 8.5 were to ship with the perfect multi-version-loading system, it would be around 10 years before WordPress even supported that PHP version. As a practical matter, it's just not a meaningful factor. > > […] > > So what do you suggest then? Go's model? Rusts? Gradle or Maven from > Java? Shooting down stuff is trivial, finding solutions is the real > work. Quoting from my first post in this thread: > Arnaud and I spent some time last year exploring a different approach > to modules that requires only trivial changes by package authors, and > none by consumers: > > https://github.com/Crell/php-rfcs/blob/master/modules/spec-brainstorm.md > https://github.com/arnaud-lb/php-src/pull/10 > > In our case, the main target was performance by giving the opcache > optimizer a larger chunk of code to work with, plus the potential for > namespace-private symbols. Arnaud was able to make it mostly work, and > there was significant potential performance to be had, but it also ran > into some very complex issues very quickly. (Circular dependencies, > modules split across multiple directories, testing, etc.) We largely > haven't gotten back to it as the implementation was very complex and it > wasn't clear if the benefit would be worth it, though I'd still like to > revisit it if possible. I don't believe that is a perfect proposal, but it is a different approach that Arnaud's experimentation shows has the potential for as much as a 20% performance improvement in some situations. That's nothing to sneeze at. There's still challenges with it, both implementation and DX, but let's not pretend that aping Python is the only option that exists. I believe that qualifies as "real work." --Larry Garfield